HOT - whats next ?

Hi All,

The version 4.0 of HOT patch is very close to the state where
we can start considering it for testing for correctness as well
as benchmarking, if there is sufficient interest to give it a
chance for 8.3

I have very little clue about what community thinks about
HOT and the patch, but I am ready to do whatever it needs
to increase quality of the patch. But I need reviews and
feedback to do that.

IMHO there are two things that need to be done to make HOT
feature complete. They are,

- Support for VACUUM FULL
- Support for CREATE INDEX

Simon has interest working on these items. I have
some ideas for supporting CREATE INDEX, but would wait
for Simon's thoughts on this.

Then there are several optimization and tuning work that
needs to be done. I am planning to start work on the following
two items and need suggestions/comments to make sure that
I am following the right path:

- With Tom/Heikki's recent patch of tracking free line pointers,
we have some bits available in the page header for book-keeping.
I plan to use one bit to track if there are any LP_DELETEd items
on the page. This information would help us to quickly check
if its worth searching through the line pointers to find a
LP_DELETED item. The flag will be set whenever a tuple is marked
LP_DELETEd and reset in page-VACUUM and whenever we fail to
find a LP_DELETEd item for reuse.

- Another problem with the current HOT patch is that it generates
tuple level fragmentation while reusing LP_DELETEd items when
the new tuple is of smaller size than the original one. Heikki
supported using best-fit strategy to reduce the fragmentation
and thats worth trying. But ISTM that we can also correct
row-level defragmentation whenever we run out of free space
and LP_DELETEd tuples while doing UPDATE. Since this does not
require moving tuples around, we can do this by a simple EXCLUSIVE
lock on the page. A bit to track row-level fragmentation would
help.

These would be just hint-bits and changes need not be WAL logged.
We could have done better with some sort of counters, but that
would require heap-page specific data and that may not
necessarily be a good idea.

Comments ?

Thanks,
Pavan