Re: pg_basebackup -F t fails when fsync spends more time than tcp_user_timeout

From: Michael Paquier <michael(at)paquier(dot)xyz>
To: "r(dot)takahashi_2(at)fujitsu(dot)com" <r(dot)takahashi_2(at)fujitsu(dot)com>
Cc: PostgreSQL Hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: pg_basebackup -F t fails when fsync spends more time than tcp_user_timeout
Date: 2019-09-03 01:58:18
Message-ID: 20190903015818.GB3765@paquier.xyz
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Mon, Sep 02, 2019 at 05:38:56PM +0900, Michael Paquier wrote:
> Thinking wider, don't we have the same problem with wal_sender_timeout
> in the case where a sync request takes longer than the time it would
> take the backend to terminate the connection?

I have been able to work more on that, and that can indeed happen with
wal_sender_timeout. While reviewing the code, I have noticed that
there is little point to enable do_sync when fetching WAL segments.
This actually led to too many fsyncs done for the plain format as each
WAL segment is fsync'd first by walmethods.c, then fsync'd again by
fsync_pgdata() in pg_wal/.

Attached is an updated patch, which needs to go down to v10.
--
Michael

Attachment Content-Type Size
v2-0001-Delay-fsync-of-pg_basebackup-tar-format-until-the.patch text/x-diff 3.5 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Tsunakawa, Takayuki 2019-09-03 02:42:19 RE: SIGQUIT on archiver child processes maybe not such a hot idea?
Previous Message Peter Geoghegan 2019-09-03 01:53:09 Re: [HACKERS] [WIP] Effective storage of duplicates in B-tree index.