Re: Pluggable storage

On Tue, Jun 27, 2017 at 4:08 PM, Amit Kapila <amit(dot)kapila16(at)gmail(dot)com>
wrote:

> On Thu, Jun 22, 2017 at 8:00 PM, Alexander Korotkov
> <a(dot)korotkov(at)postgrespro(dot)ru> wrote:
> > On Wed, Jun 21, 2017 at 10:47 PM, Robert Haas <robertmhaas(at)gmail(dot)com>
> wrote:
> >>
> >> On Mon, Jun 12, 2017 at 9:50 PM, Haribabu Kommi
> >> <kommi(dot)haribabu(at)gmail(dot)com> wrote:
> >> > Open Items:
> >> >
> >> > 1. The BitmapHeapScan and TableSampleScan are tightly coupled with
> >> > HeapTuple and HeapScanDesc, So these scans are directly operating
> >> > on those structures and providing the result.
> >> >
> >> > These scan types may not be applicable to different storage formats.
> >> > So how to handle them?
> >>
> >> I think that BitmapHeapScan, at least, is applicable to any table AM
> >> that has TIDs. It seems to me that in general we can imagine three
> >> kinds of table AMs:
> >>
> >> 1. Table AMs where a tuple can be efficiently located by a real TID.
> >> By a real TID, I mean that the block number part is really a block
> >> number and the item ID is really a location within the block. These
> >> are necessarily quite similar to our current heap, but they can change
> >> the tuple format and page format to some degree, and it seems like in
> >> many cases it should be possible to plug them into our existing index
> >> AMs without too much heartache. Both index scans and bitmap index
> >> scans ought to work.
> >
> >
> > If #1 is only about changing tuple and page formats, then could be much
> > simpler than the patch upthread? We can implement "page format access
> > methods" with routines for insertion, update, pruning and deletion of
> tuples
> > *in particular page*. There is no need to redefine high-level logic for
> > scanning heap, doing updates and so on...
>
> If you are changing tuple format then you do need to worry about
> places wherever we are using HeapTuple like TupleTableSlots,
> Visibility routines, etc. (just think if somebody changes tuple
> header, then all such places are susceptible to change).

Agree. I think that we can consider pluggable tuple format as an
independent feature which is desirable to have before pluggable storages.
For example, I believe some FDWs could already have benefit from pluggable
tuple format.

> Similarly,
> if the page format is changed you need to consider all page scan API's
> like heapgettup_pagemode/heapgetpage.

If page format is changed, then heapgettup_pagemode/heapgetpage should use
appropriate API functions for manipulating page items. I'm very afraid of
overriding whole heapgettup_pagemode/heapgetpage and monstrous functions
like heap_update without understanding of clear use-case. It's definitely
not needed for changing heap page format.

------
Alexander Korotkov
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company