Re: Compression and on-disk sorting

On Mon, May 15, 2006 at 03:02:07PM -0500, Jim C. Nasby wrote:
> The problem is that it seems like there's never enough ability to clue
> the OS in on what the application is trying to accomplish. For a long
> time we didn't have a background writer, because the OS should be able
> to flush things out on it's own before checkpoint. Now there's talk of a
> background reader, because backends keep stalling on waiting on disk IO.

Hmm, I thought the background writer was created to reduce the cost of
checkpoint which has to write out modified pages in the buffers. The
background writer tries to keep the number of dirty pages down.

I don't know about Oracle, but with or without an OS, backends are
going to block on I/O and you'd still need a background reader in both
cases for precisely the same reason. We might be able to do without a
background reader if we did asyncronous i/o, but that can't be done
portably.

> In this case the problem is that we want to tell the OS "Hey, if this
> stuff is actually going to go out to the spindles then compress it. And
> by the way, we won't be doing any random access on it, either." But
> AFAIK there's no option like that in fopen... :)

posix_fadvise(). We don't use it and many OSes don't support it, but it
is there.

The O/S is some overhead, but the benefits outweigh the costs IMHO.

Have a nice day,
--
Martijn van Oosterhout <kleptog(at)svana(dot)org> http://svana.org/kleptog/
> From each according to his ability. To each according to his ability to litigate.