Re: Read-ahead and parallelism in redo recovery

* Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> [080229 15:49]:
>
> If that isn't entirely useless, you need a better kernel. The system
> should *certainly* be bright enough to do read-ahead for our reads of
> the source xlog file. The fetches that are likely to be problematic are
> the ones for pages in the data area, which will be a lot less regular
> for typical workloads.

How difficult is it to parse the WAL logs with enough knowledge to know
what heap page (file/offset) a wal record contains (I haven't looked
into any wal code)?

There are "compression/decompression" archive_command/restore_command
programs with rudimentary knowledge of the WAL record formats. Would a
"restore_command" be able to parse the wal records as it copies them
over noting which file pages need to be read, and the just before it
exits, fork() and read each page in order.

This child doesn't need to do anything with the blocks it reads - it
just needs to read them to "pre-warm" the kernel buffer cache... If the
restoration is doing any writing, this dumb reader would hopefully be
able to keep a block ahead... And since it's separated enough from the
backend, any experiments in async_io/fadvise could easily be done.

--
Aidan Van Dyk Create like a god,
aidan(at)highrise(dot)ca command like a king,
http://www.highrise.ca/ work like a slave.