Re: logical decoding of two-phase transactions

On 28 March 2017 at 15:38, Andres Freund <andres(at)anarazel(dot)de> wrote:
> On 2017-03-28 15:32:49 +0100, Simon Riggs wrote:
>> On 28 March 2017 at 03:53, Craig Ringer <craig(at)2ndquadrant(dot)com> wrote:
>> > On 28 March 2017 at 09:25, Andres Freund <andres(at)anarazel(dot)de> wrote:
>> >
>> >> If you actually need separate decoding of 2PC, then you want to wait for
>> >> the PREPARE to be replicated. If that replication has to wait for the
>> >> to-be-replicated prepared transaction to commit prepared, and commit
>> >> prepare will only happen once replication happened...
>> >
>> > In other words, the output plugin cannot decode a transaction at
>> > PREPARE TRANSACTION time if that xact holds an AccessExclusiveLock on
>> > a catalog relation we must be able to read in order to decode the
>> > xact.
>>
>> Yes, I understand.
>>
>> The decoding plugin can choose to enable lock_timeout, or it can
>> choose to wait for manual resolution, or it could automatically abort
>> such a transaction to avoid needing to decode it.
>
> That doesn't solve the problem. You still left with replication that
> can't progress. I think that's completely unacceptable. We need a
> proper solution to this, not throw our hands up in the air and hope that
> it's not going to hurt a whole lot of peopel.

Nobody is throwing their hands in the air, nobody is just hoping. The
concern raised is real and needs to be handled somewhere; the only
point of discussion is where and how.

>> I don't think its for us to say what the plugin is allowed to do. We
>> decided on a plugin architecture, so we have to trust that the plugin
>> author resolves the issues. We can document them so those choices are
>> clear.
>
> I don't think this is "plugin architecture" related. The output pluging
> can't do right here, this has to be solved at a higher level.

That assertion is obviously false... the plugin can resolve this in
various ways, if we allow it.

You can say that in your opinion you prefer to see this handled in
some higher level way, though it would be good to hear why and how.

Bottom line here is we shouldn't reject this patch on this point,
especially since any resource issue found during decoding could
similarly prevent progress with decoding.

--
Simon Riggs http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services