Re: Logical Decoding and HeapTupleSatisfiesVacuum assumptions

Hi,

> Having this as responsibility of plugin sounds interesting. It certainly
> narrows the scope for which we need to solve the abort issue. For 2PC
> that may be okay as we need to somehow interact with transaction manager
> as Simon noted. I am not sure if this helps streaming use-case though as
> there is not going to be any external transaction management involved there.
>
> In any case all this interlocking could potentially be made less
> impact-full by only doing it when we know the transaction did catalog
> changes prior to currently decoded change (which we do during decoding)
> since that's the only time we are interested in if it aborted or not.
>
> This all leads me to another idea. What if logical decoding provided API
> for "locking/unlocking" the currently decoded transaction against abort.
> This function would then be called by both decoding and output plugin
> before any catalog read. The function can be smart enough to be NOOP if
> the transaction is not running (ie we are not doing 2PC decoding or
> streaming) or when the transaction didn't do any catalog modifications
> (we already have that info easily accessible as bool).
>
> That would mean we'd never do any kind of heavy locking for prolonged
> periods of time (ie network calls) but only during catalog access and
> only when needed. It would also solve this for both 2PC and streaming
> and it would be easy to use by plugin authors. Just document that some
> call should be done before catalog access when in output plugin, can
> even be Asserted that the call was done probably.
>
> Thoughts?
>

Yeah, this might work. We already have SET_LOCKTAG_TRANSACTION() via
which every transaction takes an exclusive lock on its own
transactionid when it starts, for example. We ideally want a single
solution to handle 2PC and ongoing (streaming) transactions. We could
introduce a new SET_LOCKTAG_LOGICALTRANSACTION(). The logical decoding
process could take a SHARED lock on this, check if the XID is still ok
to decode, read the catalog and unlock. Abort/Commit transaction
processing could take this in EXCLUSIVE mode.

As mentioned above, the plugin API which takes this lock will be smart
enough to be a NOOP if the transaction is not running (i.e we are not
doing 2PC decoding or streaming) or when the transaction didn't do any
catalog modifications.

Thoughts?

Regards,
Nikhils
--
Nikhil Sontakke http://www.2ndQuadrant.com/
PostgreSQL/Postgres-XL Development, 24x7 Support, Training & Services