Re: pg_basebackup -F t fails when fsync spends more time than tcp_user_timeout

From: Michael Paquier <michael(at)paquier(dot)xyz>
To: "r(dot)takahashi_2(at)fujitsu(dot)com" <r(dot)takahashi_2(at)fujitsu(dot)com>
Cc: PostgreSQL Hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: pg_basebackup -F t fails when fsync spends more time than tcp_user_timeout
Date: 2019-09-02 06:40:57
Message-ID: 20190902064057.GD1841@paquier.xyz
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Mon, Sep 02, 2019 at 04:42:55AM +0000, r(dot)takahashi_2(at)fujitsu(dot)com wrote:
> I think fsync() for each tablespace is not necessary.
> Like pg_basebackup -F p, I think fsync() is necessary only once at the end.

Yes, I agree that we overlooked that part when introducing
tcp_user_timeout. It is possible to sync all the contents of
pg_basebackup's tar format once at the end with fsync_dir_recurse().

Looking at the original discussion that brought bc34223b, the proposed
patches did what we have now on HEAD but we did not really exchange
about doing a fsync() just at the end with all the result base
directory contents:
https://www.postgresql.org/message-id/CAB7nPqQL0fCp0eDcVD6+3+Je24xeApU14vKz_pBpNA0sTPwLgQ@mail.gmail.com

Attached is a patch to do that, which should go down to v12 where
tcp_user_timeout has been introduced. Takahashi-san, what do you
think?
--
Michael

Attachment Content-Type Size
pgbasebackup-sync-end.patch text/x-diff 1.3 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Michael Paquier 2019-09-02 06:51:34 Re: SIGQUIT on archiver child processes maybe not such a hot idea?
Previous Message Fabien COELHO 2019-09-02 06:34:56 Re: pg_dump --exclude-* options documentation