Re: tableam vs. TOAST

On 2019-10-04 20:32, Robert Haas wrote:
> Here's the last patch back, rebased over that renaming. Although I
> think that Andres (and Tom) are probably right that there's room for
> improvement here, I currently don't see a way around the issues I
> wrote about inhttp://postgr.es/m/CA+Tgmoa0zFcaCpOJCsSpOLLGpzTVfSyvcVB-USS8YoKzMO51Yw@mail.gmail.com
> -- so not quite sure where to go next. Hopefully Andres or someone
> else will give me a quick whack with the cluebat if I'm missing
> something obvious.

This patch seems sound as far as the API restructuring goes.

If I may summarize the remaining discussion: This patch adds a field
toast_max_chunk_size to TableAmRoutine, to take the place of the
hardcoded TOAST_MAX_CHUNK_SIZE. The heapam_methods implementation then
sets this to TOAST_MAX_CHUNK_SIZE, thus preserving existing behavior.
Other table AMs can set this to some other value that they find
suitable. Currently, TOAST_MAX_CHUNK_SIZE is computed based on
heap-specific values and assumptions, so it's likely that other AMs
won't want to use that value. (Side note: Maybe rename
TOAST_MAX_CHUNK_SIZE then.) The concern was raised that while
TOAST_MAX_CHUNK_SIZE is stored in pg_control, values chosen by other
table AMs won't be, and so they won't have any safe-guards against
starting a server with incompatible disk layout. Then, various ways to
detect or check the TOAST chunk size at run time were discussed, but
none seemed satisfactory.

I think AMs are probably going to need a general mechanism to store
pg_control-like data somewhere. There are going to be chunk sizes,
block sizes, segment sizes, and so on. This one is just a particular
case of that.

This particular patch doesn't need to be held up by that, though.
Providing that mechanism can be a separate subproject of pluggable storage.

--
Peter Eisentraut http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services