Re: increasing the default WAL segment size

From: David Steele <david(at)pgmasters(dot)net>
To: Robert Haas <robertmhaas(at)gmail(dot)com>, Stephen Frost <sfrost(at)snowman(dot)net>, Beena Emerson <memissemerson(at)gmail(dot)com>, Peter Eisentraut <peter(dot)eisentraut(at)2ndquadrant(dot)com>
Cc: tushar <tushar(dot)ahuja(at)enterprisedb(dot)com>, Prabhat Sahu <prabhat(dot)sahu(at)enterprisedb(dot)com>, Ashutosh Sharma <ashu(dot)coek88(at)gmail(dot)com>, Jim Nasby <Jim(dot)Nasby(at)bluetreble(dot)com>, Kuntal Ghosh <kuntalghosh(dot)2007(at)gmail(dot)com>, Andres Freund <andres(at)anarazel(dot)de>, "pgsql-hackers(at)postgresql(dot)org" <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: increasing the default WAL segment size
Date: 2017-03-24 23:13:21
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

Hi Robert,

On 3/24/17 3:00 PM, Robert Haas wrote:
> On Wed, Mar 22, 2017 at 6:05 PM, David Steele <david(at)pgmasters(dot)net> wrote:
>>> Wait, really? I thought you abandoned this approach because there's
>>> then no principled way to handle WAL segments of less than the default
>>> size.
>> I did say that, but I thought I had hit on a compromise.
>> But, as I originally pointed out the hex characters in the filename are not
>> aligned correctly for > 8 bits (< 16MB segments) and using different
>> alignments just made it less consistent.
> I don't think I understand what the compromise is. Are you saying we
> should have one rule for segments < 16MB and another rule for segments
>> 16MB? I think using two different rules for file naming depending
> on the segment size will be a negative for both tool authors and
> ordinary users.

Sorry for the confusion, I meant to say that if we want to keep LSNs in
the filenames and not change alignment for the current default, then we
would need to drop support for segment sizes < 16MB (more or less what I
said below). Bad editing on my part.

>> It would be OK if we were willing to drop the 1,2,4,8 segment sizes because
>> then the alignment would make sense and not change the current 16MB
>> sequence.
> Well, that is true. But the thing I'm trying to do here is to keep
> this patch down to what actually needs to be changed in order to
> accomplish the original purchase, not squeeze more and more changes
> into it.

Attached is a patch to be applied on top of Beena's v8 patch that
preserves LSNs in the file naming for all segment sizes. It's not quite
complete because it doesn't modify the lower size limit everywhere, but
I think it's enough so you can see what I'm getting at. This passes
check-world and I've poked at it in other segment sizes as well manually.

Behavior for the current default of 16MB is unchanged, and all other
sizes go through a logical progression.






I believe that maintaining an easy correspondence between LSN and
filename is important. The cluster will not always be up to help with
these calculations and tools that do the job may not be present or may
have issues.

I'm happy to merge this with Beena's patch (and tidy my patch up) if
this looks like an improvement to everyone.


Attachment Content-Type Size
wal_filename_lsn_match.patch text/plain 3.0 KB

In response to


Browse pgsql-hackers by date

  From Date Subject
Next Message Michael Paquier 2017-03-24 23:15:18 Re: Potential data loss of 2PC files
Previous Message Corey Huinker 2017-03-24 22:53:38 Re: \if, \elseif, \else, \endif (was Re: PSQL commands: \quit_if, \quit_unless)