Re: Full page writes improvement, code update

On Tue, 2007-03-27 at 11:52 +0900, Koichi Suzuki wrote:

> Here's an update of a code to improve full page writes as proposed in
>
> http://archives.postgresql.org/pgsql-hackers/2007-01/msg01491.php
> and
> http://archives.postgresql.org/pgsql-patches/2007-01/msg00607.php
>
> Update includes some modification for error handling in archiver and
> restoration command.
>
> In the previous threads, I posted several evaluation and shown that we
> can keep all the full page writes needed for full XLOG crash recovery,
> while compressing archive log size considerably better than gzip, with
> less CPU consumption. I've found no further objection for this proposal
> but still would like to hear comments/opinions/advices.

Koichi-san,

Looks interesting. I like the small amount of code to do this.

A few thoughts:

- Not sure why we need "full_page_compress", why not just mark them
always? That harms noone. (Did someone else ask for that? If so, keep
it)

- OTOH I'd like to see an explicit parameter set during recovery since
you're asking the main recovery path to act differently in case a single
bit is set/unset. If you are using that form of recovery, we should say
so explicitly, to keep everybody else safe.

- I'd rather mark just the nonremovable blocks. But no real difference

- We definitely don't want an normal elog in a XLogInsert critical
section, especially at DEBUG1 level

- diff -c format is easier and the standard

- pg_logarchive and pg_logrestore should be called by a name that
reflects what they actually do. Possibly pg_compresslog and
pg_decompresslog etc.. I've not reviewed those programs, but:

- Not sure why we have to compress away page headers. Touch as little as
you can has always been my thinking with recovery code.

- I'm very uncomfortable with touching the LSN. Maybe I misunderstand?

- Have you thought about how pg_standby would integrate with this
option? Can you please?

- I'll do some docs for this after Freeze, if you'd like. I have some
other changes to make there, so I can do this at the same time.

--
Simon Riggs
EnterpriseDB http://www.enterprisedb.com