Re: Stopping logical replication protocol

in which release can be include first part?

2016-05-10 15:15 GMT+03:00 Craig Ringer <craig(at)2ndquadrant(dot)com>:

> On 10 May 2016 at 19:41, Vladimir Gordiychuk <folyga(at)gmail(dot)com> wrote:
>
>>
>> Fair enough. Though I don't understand why you'd be doing this often
>>> enough that you'd care about reopening connections. What is the problem you
>>> are trying to solve with this? The underlying reason you need this change?
>>>
>>
>> First reason it clear API in pgjdc. Second reason it ability fast enough
>> rollack to one of the previous LSN and repeat it. My use case for second
>> reason - I have logical decoding extension that prepare only data as
>> key-value pair without information about (insert, update, delete) for
>> example if it delete as key I use primary key for table and as value null. Via
>> pgjdc by replication protocol connects receiver data, consumer group
>> changes to batch and send it to Kafka. If some problem occurs during
>> delivery to kafka consumer, I should stop current replication, go back to
>> success LSN and repeat all messages from success LSN. I think it operation
>> can be quite common, but reopen connection or not stopping replication
>> will increase latency.
>>
>
> It will, but not tons. The biggest cost (at least if there are any long
> running xacts) will be replaying since the restart_lsn when you restart the
> decoding session, which will happen whether you reconnect or just stop
> decoding and return to command mode.
>
>
>
>> Anyway, here's a draft patch that does the parts other than the reorder
>>> buffer processing stop. It passes 'make check' and the src/test/recovery
>>> tests, but I haven't written anything to verify the client-initiated abort
>>> handling. You have test code for this and I'd be interested to see the
>>> results.
>>>
>>
>> What about keepAlive package when we already send/receive CopyDone? Is It
>> really necessary?
>>
>
> No, I don't think it is, and I applied the change you made to suppress it.
>
>
>> I think it's worth making the next step, where you allow reorder buffer
>>> commit processing to be interrupted, into a separate patch on top of this
>>> one. They're two separate changes IMO.
>>>
>>
>> We will continue in the current thread, or new? I interesting in both
>> patch for my solution and pgjbc driver.
>>
>
> Same thread, I just think these are two somewhat separate changes. One is
> just in the walsender and allows return to command mode during waiting for
> WAL. The other is more intrusive into the reorder buffer etc and allows
> aborting decoding during commit processing. So two separate patches make
> sense here IMO, one on top of the other.
>
> I use git and just "git format-patch -2" to prepare a stack of two patches
> from two separate commits.
>
> --
> Craig Ringer http://www.2ndQuadrant.com/
> PostgreSQL Development, 24x7 Support, Training & Services
>