[WIP PATCH] Lazily assign xids for toplevel Transactions

From: "Florian G(dot) Pflug" <fgp(at)phlo(dot)org>
To: Postgresql-Hackers <pgsql-hackers(at)postgresql(dot)org>, Simon Riggs <simon(at)2ndquadrant(dot)com>
Subject: [WIP PATCH] Lazily assign xids for toplevel Transactions
Date: 2007-08-25 21:45:08
Message-ID: 46D0A2E4.20405@phlo.org
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

I'm resending this without that actual patch attached, since my first
mail didn't get through. The patch can be found here:
http://soc.phlo.org/lazyxidassign.patch

Hi

Lately, there was some interest in assigning XIDs for toplevel Transactions
only when the transaction actually needs one (that is, it writes something
to the disk), like we currently do for subtransactions. This is obviously
also a crucial part of my work on read-only transactions on PITR slaves -
since there is no way to assign "real" XIDs on a PITR slave.

I've spent the last few days factoring out that work, and turning it into
a general solution. The result is this patch, which basically does the following

.) It defines a special TemporaryTransactionId that is used as an xact's xid
until the xact calls GetCurrentTransactionId / GetTopTransactionId.
.) It introduces a new macro TransactionIdIsPermanent, which tells if an
XID is valid, and not equal to TemporaryTransactionId.
.) It lets GetTopTransactionId assign an XID on-demand, similar to
how GetCurrentTransactionId handles that for subtransactions.
.) Each transaction get an "rid" (ResourceOwnerId) assigned when it starts, and
obtains a lock on that rid, similar to how the xid is locked. This can
be used to wait for a transaction's toplevel resource owner to release all
it's locks, and serves as a unique identifier for a running transaction.
This is needed for concurrent index builds to wait until all transactions
holding a ShareLock on the target relation have ended.

The patch passes the regression test, though there are currently two issues
that need to be resolved.
1) The second waiting phase of concurrent index builds fail to wait for xacts
that haven't been assigned an XID when the reference shapshot was taken.
The "rid" doesn't help here, because it's not currently store in the
snapshot.
2) I'm not entirely sure yet how to handle two flags MyXactMadeTempRelUpdates,
MyXactMadeXLogEntry and the MyRecPtr variable. Those seems to be made
partly redundant by this patch - checking if an xact has a
permanent xid assigned already tells if the transaction made any writes.

(1) could be easiy solves by querying the list of currently active RIDs after
taking the reference snapshot. But since AFAIK HOT introduces a new method for
guaranteeing that a transaction won't use an index that might not contain
all tuples that xact is interested in, I wanted to get feedback on how HOT
currently handles this.

It's probably not the best time to come up with new patches, since everybody
seems to be busy working on getting 8.3 out. But this patch is a quite natural
fallout of my work on read-only queries on PITR slaves, and I'd be very
interested to know if the general direction this patch takes is deemed
acceptable.

greetings, Florian Pflug

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Alvaro Herrera 2007-08-25 22:21:50 Re: reindexdb hangs
Previous Message Tom Lane 2007-08-25 19:34:01 Re: Best implementation of PATRICIA