Re: [PATCH] Infinite loop while acquiring new TOAST Oid

Hi!

I'm working on this issue according to the plan Tom proposed above -

>I agree that we can't simply widen varatt_external to use 8 bytes for
>the toast ID in all cases. Also, I now get the point about avoiding
>use of globally assigned OIDs here: if the counter starts from zero
>for each table, then a variable-width varatt_external could actually
>be smaller than currently for many cases. However, that bit is somewhat
>orthogonal, and it's certainly not required for fixing the basic problem.

Have I understood correctly that you suppose using an individual counter
for each TOAST table? I'm working on this approach, so we store counters
in cache, but I see an issue with the first insert - when there is no
counter
in cache so we have to loop through the table with increasing steps to find
available one (i.e. after restart). Or we still use single global counter,
but
64-bit with a wraparound?

>So it seems like the plan of attack ought to be:

>1. Invent a new form or forms of varatt_external to allow different
>widths of the toast ID. Use the narrowest width possible for any
>given ID value.

I'm using the VARTAG field - there are plenty of available values, so there
is no problem in distinguishing regular toast pointer with 'short' value id
(4 bytes) from long (8 bytes).

varatt_external currently is 32-bit aligned, so there is no reason in using
narrower type for value ids up to 16 bits.Or is it?

>2. Allow TOAST tables/indexes to store either 4-byte or 8-byte IDs.
>(Conversion could be done as a side effect of table-rewrite
>operations, perhaps.)

Still on toast/detoast part, would get to that later.

>3. Localize ID selection so that tables can have small toast IDs
>even when other tables have many IDs. (Optional, could do later.)

>
Thank you!

--
Regards,
Nikita Malakhov
Postgres Professional
https://postgrespro.ru/