Skip site navigation (1) Skip section navigation (2)

Re: pg_basebackup blocking all queries with horrible performance

From: Fujii Masao <masao(dot)fujii(at)gmail(dot)com>
To: Magnus Hagander <magnus(at)hagander(dot)net>
Cc: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>, Lonni J Friedman <netllama(at)gmail(dot)com>, Craig Ringer <ringerc(at)ringerc(dot)id(dot)au>, Jerry Sievers <gsievers19(at)comcast(dot)net>, pgsql-admin(at)postgresql(dot)org
Subject: Re: pg_basebackup blocking all queries with horrible performance
Date: 2012-06-10 12:25:38
Message-ID: CAHGQGwHaLwySPXNb+vcEL2Vay1UN36b8NkURLwV54d_4QriHzA@mail.gmail.com (view raw or flat)
Thread:
Lists: pgsql-adminpgsql-hackers
On Sun, Jun 10, 2012 at 7:43 PM, Magnus Hagander <magnus(at)hagander(dot)net> wrote:
> On Sat, Jun 9, 2012 at 2:51 PM, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
>> Fujii Masao <masao(dot)fujii(at)gmail(dot)com> writes:
>>> This seems a bug. I think we should prevent pg_basebackup from
>>> becoming synchronous standby. Thought?
>>
>> Absolutely.  If we have replication clients that are not actually
>> capable of being standbys, there *must* be a way for the master
>> to know that.
>
> I thought we fixed this already by sending InvalidXlogRecPtr as flush
> location? And that this only applied in 9.2?
>
> Are you saying we picked pg_basebackup *in backup mode* (not log
> streaming) as synchronous standby?

Yes.

> If so then yes, that is
> *definitely* a bug that should be fixed. We should never select a
> connection that's not even streaming log as standby!

Agreed. Attached patch prevents pg_basebackup from becoming sync
standby. Also this patch fixes another problem: currently only walsender
which reaches STREAMING state can become sync walsender. OTOH,
sync walsender thinks that walsender with higher priority will be sync one
whether its state is STREAMING, and switches to potential sync walsender.
So when the standby with higher priority connects to the master, we
might have no sync standby until it reaches the STREAMING state.
To fix this problem, the patch switches walsender's state from sync to
potential *after* walsender with higher priority has reached the
STREAMING state.

We also should not select (1) background stream process forked from
pg_basebackup and (2) pg_receivexlog as sync standby because they
don't send back replication progress. To address this, I'm thinking to
introduce new option "NOSYNC" in "START_REPLICATION" command
as follows, and to change (1) and (2) so that they specify NOSYNC.

    START_REPLICATION XXX/XXX [NOSYNC]

If the standby specifies NOSYNC option, it's never assigned as sync
standby even if its name is in synchronous_standby_names. Thought?

BTW, we are discussing about changing pg_receivexlog so that it sends
back replication progress, in another thread. So if this change will have
been applied, probably we don't need to change pg_receivexlog so that
it uses NOSYNC option.

Regards,

-- 
Fujii Masao

Attachment: prevent_pgbasebackup_from_becoming_sync_standby_v1.patch
Description: application/octet-stream (403 bytes)

In response to

Responses

pgsql-hackers by date

Next:From: Heikki LinnakangasDate: 2012-06-10 12:33:06
Subject: Re: Boyer-Moore, no less
Previous:From: Magnus HaganderDate: 2012-06-10 11:43:42
Subject: Re: pg_basebackup --xlog compatibility break

pgsql-admin by date

Next:From: Fujii MasaoDate: 2012-06-10 13:34:06
Subject: Re: pg_basebackup blocking all queries with horrible performance
Previous:From: Magnus HaganderDate: 2012-06-10 10:43:32
Subject: Re: pg_basebackup blocking all queries with horrible performance

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group