Re: Change pgarch_readyXlog() to return .history files first

From: David Steele <david(at)pgmasters(dot)net>
To: Michael Paquier <michael(at)paquier(dot)xyz>
Cc: Pg Hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: Change pgarch_readyXlog() to return .history files first
Date: 2018-12-20 11:57:30
Message-ID: 35420aa0-9e8b-66a2-3d69-d02cd4f8f6cf@pgmasters.net
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On 12/15/18 2:10 AM, Michael Paquier wrote:
> On Fri, Dec 14, 2018 at 08:43:20AM -0500, David Steele wrote:
>> On 12/13/18 7:15 PM, Michael Paquier wrote:

>>> Or you could just use IsTLHistoryFileName here?
>>
>> We'd have to truncate .ready off the string to make that work, which
>> seems easy enough. Is that what you were thinking?
>
> Yes, that's the idea. pgarch_readyXlog returns the segment name which
> should be archived, so you could just compute it after detecting a
> .ready file.
>
>> One thing to consider is the check above is more efficient than
>> IsTLHistoryFileName() and it potentially gets run a lot.
>
> This check misses strspn(), so any file finishing with .history would
> get eaten even if that's unlikely to happen.

Good point. The new patch uses IsTLHistoryFileName().

>>> If one wants to simply check this code, you can just create dummy orphan
>>> files in archive_status and see in which order they get removed.
>>
>> Seems awfully racy. Are there currently any tests like this for the
>> archiver that I can look at extending?
>
> Sorry for the confusion, I was referring to manual testing here.

Ah, I see. Yes, that's exactly how I tested it, in addition to doing
real promotions.

> Thinking about it, we could have an automatic test to check for the file
> order pattern by creating dummy files, starting the server with archiver
> enabled, and then parse the logs as orphan .ready files would get
> removed in the order their archiving is attempted with one WARNING entry
> generated for each. I am not sure if that is worth a test though.

Yes, parsing the logs was the best thing I could think of, too.

--
-David
david(at)pgmasters(dot)net

Attachment Content-Type Size
history-files-first-v3.patch text/plain 3.6 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message David Steele 2018-12-20 12:13:01 Re: Add timeline to partial WAL segments
Previous Message David Rowley 2018-12-20 11:49:58 Re: Tid scan improvements