Re: pg_basebackup may fail to send feedbacks.

From: Kyotaro HORIGUCHI <horiguchi(dot)kyotaro(at)lab(dot)ntt(dot)co(dot)jp>
To: masao(dot)fujii(at)gmail(dot)com
Cc: pgsql-hackers(at)postgresql(dot)org
Subject: Re: pg_basebackup may fail to send feedbacks.
Date: 2015-02-24 09:44:29
Message-ID: 20150224.184429.125139246.horiguchi.kyotaro@lab.ntt.co.jp
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hello, the attached is the v4 patch that checks feedback timing
every WAL segments boundary.

At Fri, 20 Feb 2015 17:29:14 +0900 (Tokyo Standard Time), Kyotaro HORIGUCHI <horiguchi(dot)kyotaro(at)lab(dot)ntt(dot)co(dot)jp> wrote in <20150220(dot)172914(dot)241732690(dot)horiguchi(dot)kyotaro(at)lab(dot)ntt(dot)co(dot)jp>
> > Some users may complain about the performance impact by such
> > frequent calls and we may want to get rid of them from
> > walreceiver loop in the future. If we adopt your idea now,
> > I'm afraid that it would tie our hands in that case.
> >
> > How much impact can such frequent calls of gettimeofday()
> > have on replication performance? If it's not negligible,
> > probably we should remove them at first and find out another
> > idea to fix the problem you pointed. ISTM that it's not so
> > difficult to remove them. Thought? Do you have any numbers
> > which can prove that such frequent gettimeofday() has only
> > ignorable impact on the performance?
>
> The attached patch is 'the more sober' version of SIGLARM patch.

I said that checking whether to send feedback every boundary
between WAL segments seemed too long but after some rethinking, I
changed my mind.

- The most large possible delay source in the busy-receive loop
is fsyncing at closing WAL segment file just written, this can
take several seconds. Freezing longer than the timeout
interval could not be saved and is not worth saving anyway.

- 16M bytes-disk-writes intervals between gettimeofday() seems
to be gentle enough even on platforms where gettimeofday() is
rather heavy.

regards,

--
Kyotaro Horiguchi
NTT Open Source Software Center

Attachment Content-Type Size
basebackup_reply_fix_mst_v4_WALSEG.patch text/x-patch 3.3 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Syed, Rahila 2015-02-24 09:46:22 Re: [REVIEW] Re: Compression of full-page-writes
Previous Message Vladimir Borodin 2015-02-24 09:13:17 Re: pg_upgrade and rsync