Re: RFI: Extending the TOAST Pointer

Hi,

> No, that's inaccurate. The complete on-disk representation of a varatt is
>
> {
> uint8 va_header; /* Always 0x80 or 0x01 */
> uint8 va_tag; /* Type of datum */
> char va_data[FLEXIBLE_ARRAY_MEMBER]; /* Type-dependent
> data, for toasted values that's currently only a varatt_external */
> } varattrib_1b_e;
>
> With va_tag being filled with one of the vartag_external values:
>
> typedef enum vartag_external
> {
> VARTAG_INDIRECT = 1,
> VARTAG_EXPANDED_RO = 2,
> VARTAG_EXPANDED_RW = 3,
> VARTAG_ONDISK = 18
> } vartag_external;
>
> This enum still has many options to go before it exceeds the maximum
> of the uint8 va_tag field. Therefore, I don't think we have no disk
> representations left, nor do I think we'll need to add another option
> to the ToastCompressionId enum.
> As an example, we can add another VARTAG option for dictionary-enabled
> external toast; like what the pluggable toast patch worked on. I think
> we can salvage some ideas from that patch, even if the main idea got
> stuck.

The problem here is that the comments are ambiguous regarding what to
call "TOAST pointer" exactly. I proposed a patch for this but it was
not accepted [1].

So the exact on-disk representation of a TOAST'ed value (for
little-endian machines) is:

0b00000001, 18 (va_tag), (varatt_external here)

Where 18 is sizeof(varatt_external) + 2, because the length includes
the length of the header.

I agree that va_tag can have another use. But since we are going to
make varatt_external variable in size (otherwise I don't see how it
could be really **extendable**) I don't think this is the right
approach.

Also I agree that this particular statement is incorrect:

> This will allow us to extend the pointers indefinitely.

varatt_external is going to be limited to 255. But it seems to be a
reasonable limitation for the nearest 10-20 years or so.

[1]: https://commitfest.postgresql.org/39/3820/

--
Best regards,
Aleksander Alekseev