Re: increasing the default WAL segment size

Hello,
PFA the updated patches.

On Fri, Jan 27, 2017 at 2:17 PM, Beena Emerson <memissemerson(at)gmail(dot)com>
wrote:

> Hello Andres,
>
> Thank you for your review.
>
> On Fri, Jan 27, 2017 at 12:39 AM, Andres Freund <andres(at)anarazel(dot)de>
> wrote:
>
>> Hi,
>>
>> On 2017-01-23 11:35:11 +0530, Beena Emerson wrote:
>> > Please find attached an updated WIP patch. I have incorporated almost
>> all
>> > comments. This is to be applied over Robert's patches. I will post
>> > performance results later on.
>> >
>> > 1. shift (>>) and AND (&) operations: The assign hook of
>> wal_segment_size
>> > sets the WalModMask and WalShiftBit. All the modulo and division
>> operations
>> > using XLogSegSize has been replaced with these. However, there are many
>> > preprocessors which divide with XLogSegSize in xlog_internal.h. I have
>> not
>> > changed these because it would mean I will have to reassign the
>> WalShiftBit
>> > along with XLogSegSize in all the modules which use these macros. That
>> does
>> > not seem to be a good idea. Also, this means shift operator can be used
>> > only in couple of places.
>>
>> I think it'd be better not to have XLogSegSize anymore. Silently
>> changing a macros behaviour from being a compile time constant to
>> something runtime configurable is a bad idea.
>>
>
> I dont think I understood u clearly. You mean convert the macros using
> XLogSegSize to functions?
>

I have moved the ModMask related changes to a separate patch
01-add-XLogSegmentOffset-macro.patch. This creates the macro
XLogSegmentOffset and replaces all "% XLogSegSize" and "% XLOG_SEG_SIZE"
with this macro.
I have not included the shift operator because the changes only apply to
about 4 lines did not give any performance boost or such.

Hm. Are GUC hooks a good way to compute the masks? Interdependent GUCs
>> are unfortunately not working well, and several GUCs might end up
>> depending on these. I think it might be better to assign the variables
>> somewhere early in StartupXLOG() or such.
>>
>
> I am not sure about these interdependent GUCs. I need to study this better
> and make changes as required.
>
>
The process flow is such thatthe Control File which sets the XLogSegSIze is
read after the GUC options are initialized. StartupXLOG is called by
StartupProcessMain() which restores the XLOG and then exits. Hence he value
initialised here are not persistent throughout the postgres run. It throws
error during pg_ctl stop.
The XLogSegSize adjustment in assign hooks have been removed and a new
macro ConvertToXSegs is used to convert the min and max wal_size to the
segment count when required. wal_segment_size set from ReadControlFile also
affects the Checkpointsegment value and hence the assign_wal_segment_size
calls CalculateCheckpointSegments.

Documentation is updated

Performance Tests:

I ran pgbench tests for different wal segment size on database of scale
factor 300 with shared_buffers of 8GB. Each of the tests ran for 10 min and
a median of 3 readings were considered. The following table shows the
performance of the patch wrt the HEAD for different client count for
various wal-segsize value. We could say that there is not performance
difference.

16 32 64 128

8MB -1.36 0.02 0.43 -0.24
16MB -0.38 0.18 -0.09 0.4
32MB -0.52 0.29 0.39 0.59
64MB -0.15 0.04 0.52 0.38

--
Thank you,

Beena Emerson

Have a Great Day!