Re: Datum values consistency within one query

From: Paul Ramsey <pramsey(at)cleverelephant(dot)ca>
To: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
Cc: PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>
Subject: Re: Datum values consistency within one query
Date: 2020-04-03 17:23:39
Message-ID: 38984788-C397-4EC3-A5D8-91DC031D2965@cleverelephant.ca
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

> On Apr 2, 2020, at 4:30 PM, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
>
> Paul Ramsey <pramsey(at)cleverelephant(dot)ca> writes:
>> Getting the datum value is really fast, so I can have a cache that
>> keeps the latest detoasted object around, and update it when the datum
>> changes, and store the cache information in the parent context. Like
>> so:
>
> Jeez, no, not like that. You're just testing a pointer.
> ...
> The case where this would actually be worth doing, probably, is where
> you are receiving a toasted-out-of-line datum. In that case you could
> legitimately use the toast pointer ID values (va_valueid + va_toastrelid)
> as a lookup key for a cache, as long as it had a lifespan of a statement
> or less. You'd have to get a bit in bed with the details of toast
> pointers, but it's not like those are going anywhere.

So, if I tested for VARATT_IS_EXTENDED(), and then for VARATT_IS_EXTERNAL_ONDISK(attr) and then did VARATT_EXTERNAL_GET_POINTER(toast_pointer, attr), I could use va_valueid + va_toastrelid as keys in the cache for things that passed that filter?
What about large const values that haven't been stored in a table yet? (eg, ST_Buffer(ST_MakePoint(0, 0), 100, 10000)) is there a stable key I can use for them?

> It would be interesting to tie that into the "expanded object"
> infrastructure, perhaps, especially if the contents of the objects
> you're interested in aren't just flat blobs of data.

Yeah, I'm wrestling with the right place to do this stuff, it's not just the detoasting going on, I also build in-memory trees on large objects and hold them around for as long as the object keeps showing repeatedly up in the query, I just test the cache right now by using memcmp on the previous value and that's really pricey.

P

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Corey Huinker 2020-04-03 17:34:17 Re: Add A Glossary
Previous Message Tom Lane 2020-04-03 17:18:12 Re: [PATCH] Fix for slow GIN index queries when "gin_fuzzy_search_limit" setting is relatively small for large tables