Nested transactions

Hackers,

I attach my current nested transactions patch, plus the subtrans code
(src/backend/access/transam/subtrans.c and
src/include/access/subtrans.h)

I got somewhat further with the visibility issues and I think it's
"mostly there". I have successfully executed some concurrent tests with
inserting into a table repeatedly with a concurrent vacuum, but there's
one thing I'm stuck into.

The problem is that the heap tuple headers compress some info about
"Xmin is Xmax," and I'm not sure the visibility can be done right
without reverting this change. The following example fails:

begin;
insert into baz values (3);
begin;
delete from baz where a=3;
rollback;
delete from baz where a=3;
commit;

The failure happens because the first time the tuple is deleted, it gets
XMAX_IS_INVALID. The second time, HeapTupleSatisfiesSnapshot executes
the following:

else if (TransactionIdIsCurrentTransactionId(HeapTupleHeaderGetXmin(tuple)))
{
if (HeapTupleHeaderGetCmin(tuple) >= snapshot->curcid)
return false; /* inserted after scan started */

The problem is that the HeapTuple's Cmin has been overlapped with Xmax,
which in my tests was around 19000, and the snapshot's Cid is 4 or 5 so
the tests fails. The return result is incorrect, and the transaction
finishes with the tuple being visible which is, of course, also incorrect.

So I don't see how to do this. Maybe the XMIN_IS_XMAX trick can be
changed to use a test based on the transaction tree instead of
TransactionIdEquals, but I'm not sure if this is correct or what other
effects it may have.

I'm not sure I'm correctly stating the problem, and of course I'm not
sure it's the only problem left.

Manfred, Tom, any comments?

--
Alvaro Herrera (<alvherre[a]dcc.uchile.cl>)
Essentially, you're proposing Kevlar shoes as a solution for the problem
that you want to walk around carrying a loaded gun aimed at your foot.
(Tom Lane)