Re: increasing the default WAL segment size

Hello Andres,

On Tue, Dec 20, 2016 at 1:58 PM, Andres Freund <andres(at)anarazel(dot)de> wrote:

> Hi,
>
> On 2016-12-19 15:14:50 +0530, Beena Emerson wrote:
> > The attached patch removes --with-wal-segsize configure option and adds a
> > new initdb option --wal-segsize. The module initdb passes the wal-segsize
> > value into an environment variable which is used to overwrite the guc
> value
> > wal_ segment_size and set the internal variables : XLogSegSize and
> > XLOG_SEG_SIZE (xlog_internal.h). The default wal_segment_size is not
> > changed but I have increased the maximum size to 1GB.
> >
> > Since XLOG_SEG_SIZE is now variable, it could not be used directly in
> > src/bin modules and few macros and few changes had to be made:
>
> I do think this has the potential for negative performance
> implications. Consider code like
> /* skip over the page header */
> if (CurrPos % XLogSegSize == 0)
> {
> CurrPos += SizeOfXLogLongPHD;
> currpos += SizeOfXLogLongPHD;
> }
> else
> right now that's doable in an efficient manner, because XLogSegSize is
> constant. If it's a variable and the compiler doesn't even know it's a
> power-of-two, it'll have to do a full "div" - and that's quite easily
> noticeable in a lot of cases.
>
> Now it could entirely be that the costs of this will be swamped by
> everything else, but I'd not want to rely on it.
>
> I think we need tests with concurrent large-file copies. And then also
> look at the profile to see whether the relevant places become new
> hotspots (not that we introduce something that's just hidden for now).

> We might be able to do a bit better, efficency wise, by storing
> XLogSegSize as a "shift factor". I.e. the 16M setting would be 24
> (i.e. XLogSegSize would be defined as 1 << 24).
>
>
Thank you.
I did not realize we could face such problems. I will perform the tests to
check the performance and do the required changes.

--

Beena Emerson

Have a Great Day!