From: | Stas Kelvich <s(dot)kelvich(at)postgrespro(dot)ru> |
---|---|
To: | Craig Ringer <craig(at)2ndquadrant(dot)com> |
Cc: | Petr Jelinek <petr(dot)jelinek(at)2ndquadrant(dot)com>, PostgreSQL Hackers <pgsql-hackers(at)postgresql(dot)org>, Robert Haas <robertmhaas(at)gmail(dot)com> |
Subject: | Re: logical decoding of two-phase transactions |
Date: | 2017-03-27 09:53:01 |
Message-ID: | DFC9E270-4554-4E46-B2F8-FD864C29610D@postgrespro.ru |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-hackers |
> On 27 Mar 2017, at 12:26, Craig Ringer <craig(at)2ndquadrant(dot)com> wrote:
>
> On 27 March 2017 at 09:31, Craig Ringer <craig(at)2ndquadrant(dot)com> wrote:
>
>> We're in the last week of the CF. If you have a patch that's nearly
>> ready or getting there, now would be a good time to post it for help
>> and input from others.
>>
>> I would really like to get this in, but we're running out of time.
>>
>> Even if you just post your snapshot management work, with the cosmetic
>> changes discussed above, that would be a valuable start.
>
> I'm going to pick up the last patch and:
I’m heavily underestimated amount of changes there, but almost finished
and will send updated patch in several hours.
> * Ensure we only add the GID to xact records for 2pc commits and aborts
And only during wal_level >= logical. Done.
Also patch adds origin info to prepares and aborts.
> * Add separate callbacks for prepare, abort prepared, and commit
> prepared (of xacts already processed during prepare), so we aren't
> overloading the "commit" callback and don't have to create fake empty
> transactions to pass to the commit callback;
Done.
> * Add another callback to determine whether an xact should be
> processed at PREPARE TRANSACTION or COMMIT PREPARED time.
Also done.
> * Rename the snapshot builder faux-commit stuff in the current patch
> so it's clearer what's going on.
Hm. Okay, i’ll leave that part to you.
> * Write tests covering DDL, abort-during-decode, etc
I’ve extended test, but it is good to have some more.
> Some special care is needed for the callback that decides whether to
> process a given xact as 2PC or not. It's called before PREPARE
> TRANSACTION to decide whether to decode any given xact at prepare time
> or wait until it commits. It's called again at COMMIT PREPARED time if
> we crashed after we processed PREPARE TRANSACTION and advanced our
> confirmed_flush_lsn such that we won't re-process the PREPARE
> TRANSACTION again. Our restart_lsn might've advanced past it so we
> never even decode it, so we can't rely on seeing it at all. It has
> access to the xid, gid and invalidations, all of which we have at both
> prepare and commit time, to make its decision from. It must have the
> same result at prepare and commit time for any given xact. We can
> probably use a cache in the reorder buffer to avoid the 2nd call on
> commit prepared if we haven't crashed/reconnected between the two.
Good point. Didn’t think about restart_lsn in case when we are skipping this
particular prepare (filter_prepared() -> true, in my terms). I think that should
work properly as it use the same code path as it was before, but I’ll look at it.
> This proposal does not provide a way to safely decode a 2pc xact that
> made catalog changes which may be aborted while being decoded. The
> plugin must lock such an xact so that it can't be aborted while being
> processed, or defer decoding until commit prepared. It can use the
> invalidations for the commit to decide.
I had played with that two-pass catalog scan and it seems to be
working but after some time I realised that it is not useful for the main
case when commit/abort is generated after receiver side will answer to
prepares. Also that two-pass scan is a massive change in relcache.c and
genam.c (FWIW there were no problems with cache, but some problems
with index scan and handling one-to-many queries to catalog, e.g. table
with it fields)
Finally i decided to throw it and switched to filter_prepare callback and
passed there txn structure to allow access to has_catalog_changes field.
Stas Kelvich
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company
From | Date | Subject | |
---|---|---|---|
Next Message | Kyotaro HORIGUCHI | 2017-03-27 10:05:43 | Re: Radix tree for character conversion |
Previous Message | Ashutosh Sharma | 2017-03-27 09:39:25 | Re: [sqlsmith] Failed assertion in _hash_kill_items/MarkBufferDirtyHint |