Re: Review of "pg_basebackup and pg_receivexlog to use non-blocking socket communication", was: Re: Re: [BUGS] BUG #7534: walreceiver takes long time to detect n/w breakdown

On Wednesday, January 16, 2013 4:02 PM Heikki Linnakangas wrote:
> On 07.01.2013 16:23, Boszormenyi Zoltan wrote:
> > Since my other patch against pg_basebackup is now committed,
> > this patch doesn't apply cleanly, patch rejects 2 hunks.
> > The fixed up patch is attached.
>
> Now that I look at this a high-level perspective, why are we only
> worried about timeouts in the Copy-mode and when connecting? The
> initial
> checkpoint could take a long time too, and if the server turns into a
> black hole while the checkpoint is running, pg_basebackup will still
> hang. Then again, a short timeout on that phase would be a bad idea,
> because the checkpoint can indeed take a long time.

True, but IMO, if somebody want to take basebackup, he should do that when
the server is not loaded.

> In streaming replication, the keep-alive messages carry additional
> information, the timestamps and WAL locations, so a keepalive makes
> sense at that level. But otherwise, aren't we just trying to
> reimplement
> TCP keepalives? TCP keepalives are not perfect, but if we want to have
> an application level timeout, it should be implemented in the FE/BE
> protocol.
>
> I don't think we need to do anything specific to pg_basebackup. The
> user
> can simply specify TCP keepalive settings in the connection string,
> like
> with any libpq program.

I think currently user has no way to specify TCP keepalive settings from
pg_basebackup, please let me know if there is any such existing way?

I think specifying TCP settings is very cumbersome for most users, that's
the reason most standard interfaces (ODBC/JDBC) have such application level
timeout mechanism.

By implementing in FE/BE protocol (do you mean to say that make such
non-blocking behavior inside Libpq or something else), it might be generic
and can be used for others as well but it might need few interface changes.

IMHO if by having such less impact changes for pg_basebackup, it makes
pg_basebackup network sensitive, the current approach can also be
considered.

With Regards,
Amit Kapila.