Re: WAL format and API changes (9.5)

On Tue, Nov 18, 2014 at 4:31 AM, Heikki Linnakangas <hlinnakangas(at)vmware(dot)com>
wrote:
> WAL insertion performance
> =========================
> To measure the performance of generating WAL, I ran the
> wal-update-testsuite.sh that Amit also ran earlier. The cluster was
> initialized with:
>
> shared_buffers=512MB
> checkpoint_segments=30
> fsync=off
> autovacuum=off
> full_page_writes=off
>
> [results]
> Summary: No meaningful difference in runtime.
If I am seeing that correctly, WAL generated is reduced for all the tests,
except for the case of "hundred tiny fields" where more WAL is generated.
Now the duration time seems to be generally reduced, some noise (?) making
it sometimes higher.

> WAL replay performance
> ======================
>
> To test WAL replay performance, I ran pgbench with WAL archiving enabled,
> and timed the replay of the generated WAL. I used the attached script,
> replay-perf-test.sh for that. full_page_writes were disabled, because
> replaying full page images is quite different from replaying other
records.
> (Performance of full-page images is interesting too, but it's not expected
> that these WAL format changes make much difference to that).
>
> In summary, there is no significant difference in replay performance. The
> amount of WAL generated is much smaller with the patch.

> This concludes my performance testing, until someone wants to see some
other
> scenario being tested. I'm happy with the results.
I think you can, that's a great study, and this proves to be a gain on many
fields.

If this goes in, it is going to be one of the largest patches committed
ever.
$ git diff --stat | tail -n1
91 files changed, 3895 insertions(+), 4305 deletions(-)

There are still some XXX blocks here and there in the code.. But nothing
really huge, like here:
- * checks could go wrong too.
+ * checks could go wrong too. XXX does this comment still
make sense?
*/
- Assert(xldata->blkno != xldata->blknoNew);
+ Assert(blkno != blknoNew);

Btw, did you do a run with the buffer capture facility and checked for page
differences?

Except that, after going through the code once again, ISTM that the patch
is in a nice state. It may be better to wait for some input from Andres, he
may catch some issues I haven't spotted.
Regards,
--
Michael