Re: Reliable WAL file shipping over unreliable network

From: Dianne Skoll <dfs(at)roaringpenguin(dot)com>
To: Rui DeSousa <rui(dot)desousa(at)icloud(dot)com>
Cc: pgsql-admin(at)lists(dot)postgresql(dot)org
Subject: Re: Reliable WAL file shipping over unreliable network
Date: 2018-03-01 01:54:58
Message-ID: 7E076BDF-33E4-45CA-97AE-C933F315B603@roaringpenguin.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-admin

Wow, that's good sleuthing! Are you going to report the bug to the rsync maintainers?

You can work around it by not having your script trust rsync, but to do a sha1 hash on both ends after rsync exits. We do this in production because we once encountered a server with bad memory that was mysteriously corrupting the files; we ended up never removing the sha1-comparison code.

Regards,

Dianne.

On February 28, 2018 8:10:59 PM EST, Rui DeSousa <rui(dot)desousa(at)icloud(dot)com> wrote:
>
>I’ve tested this and it seems that there is a still a bug in rsync
>(rsync version 3.1.2 protocol version 31). I used a 1GB archive
>filesytem to allow for an out of space test case. Not sure of the
>actual cause as it seems to work a few times; however, it then fails
>leaving a truncated file and returning a success code.

In response to

Browse pgsql-admin by date

  From Date Subject
Next Message Mark Kirkwood 2018-03-01 01:59:14 Re: Reliable WAL file shipping over unreliable network
Previous Message Rui DeSousa 2018-03-01 01:10:59 Re: Reliable WAL file shipping over unreliable network