Re: increasing the default WAL segment size

On 3/21/17 3:22 PM, Robert Haas wrote:
> On Tue, Mar 21, 2017 at 9:04 AM, Stephen Frost <sfrost(at)snowman(dot)net> wrote:
>> In short, I'm also concerned about this change to make WAL file names no
>> longer match up with LSNs and also about the odd stepping that you get
>> as a result of this change when it comes to WAL file names.
>
> OK, that's a bit surprising to me, but what do you want to do about
> it? If you take the approach that Beena did, then you lose the
> correspondence with LSNs, which is admittedly not great but there are
> already helper functions available to deal with LSN -> filename
> mappings and I assume those will continue to work. If you take the
> opposite approach, then WAL filenames stop being consecutive, which
> seems to me to be far worse in terms of user and tool confusion.

They are already non-consecutive. Does 000000010000000200000000 really
logically follow 0000000100000001000000FF? Yeah, sort of, if you know
the rules.

> Also
> note that, both currently and with the patch, you can also reduce the
> WAL segment size. David's proposed naming scheme doesn't handle that
> case, I think, and I think it would be all kinds of a bad idea to use
> one file-naming approach for segments < 16MB and a separate approach
> for segments > 16MB. That's not making anything easier for users or
> tool authors.

I believe it does handle that case, actually. The minimum WAL segment
size is 1MB so they would increase like:

000000010000000100000000
000000010000000100100000
000000010000000100200000
...
0000000100000001FFF00000

You could always calculate the next WAL file by adding
(wal_seg_size_in_mb << 20) to the previous WAL file's LSN. This would
even work for WAL segments > 4GB. Overall, I think this would make
calculating WAL ranges simpler than it is now.

The biggest downside I can see is that this would change the naming
scheme for the default of 16MB compared to previous versions of
Postgres. However, for all other wal-seg-size values changes would need
to be made anyway.

--
-David
david(at)pgmasters(dot)net