Re: Scaling XLog insertion (was Re: Moving more work outside WALInsertLock)

I was trying to understand this patch and had few doubts:

1. In PerformXLogInsert(), why there is need to check freespace when already
during ReserveXLogInsertLocation(),
the space is reserved.
Is it possible that the record size is more than actually calculted in
ReserveXLogInsertLocation(), if so in that
case what I understand is it is moving to next page to write, however
isn't it possible that some other backend had
already reserved that space.

2. In function WaitForXLogInsertionSlotToBecomeFree(), chances are there
such that when nextslot equals lastslot, all new
backends try to reserve a slot will start waiting on same last slot
which can lead to serialization for those
backends and can impact latency.

3. GetXlogBuffer - This will get called twice, once for normal buffer,
second time for when there is not enough space in
current page, and both times it can lead to I/O whereas in earlier
algorithm, the chances of I/O is only once.

-----Original Message-----
From: pgsql-hackers-owner(at)postgresql(dot)org
[mailto:pgsql-hackers-owner(at)postgresql(dot)org] On Behalf Of Heikki Linnakangas
Sent: Friday, February 17, 2012 9:07 PM
To: Fujii Masao
Cc: Jeff Janes; Robert Haas; PostgreSQL-development
Subject: Re: Scaling XLog insertion (was Re: [HACKERS] Moving more work
outside WALInsertLock)

On 17.02.2012 07:27, Fujii Masao wrote:
> Got another problem: when I ran pg_stop_backup to take an online
> backup, it got stuck until I had generated new WAL record. This
> happens because, in the patch, when pg_stop_backup forces a switch to
> new WAL file, old WAL file is not marked as archivable until next new
> WAL record has been inserted, but pg_stop_backup keeps waiting for that
WAL file to be archived.
> OTOH, without the patch, WAL file is marked as archivable as soon as
> WAL file switch occurs.
>
> So, in short, the patch seems to handle the WAL file switch logic
incorrectly.

Yep. For a WAL-switch record, XLogInsert returns the location of the end of
the record, not the end of the empty padding space. So when the caller
flushed up to that point, it didn't flush the empty space and therefore
didn't notify the archiver.

Attached is a new version, fixing that, and off-by-one bug you pointed out
in the slot wraparound handling. I also moved code around a bit, I think
this new division of labor between the XLogInsert subroutines is more
readable.

Thanks for the testing!

--
Heikki Linnakangas
EnterpriseDB http://www.enterprisedb.com