| From: | Martijn van Oosterhout <kleptog(at)svana(dot)org> |
|---|---|
| To: | "Jim C(dot) Nasby" <jnasby(at)pervasive(dot)com> |
| Cc: | Zeugswetter Andreas DCP SD <ZeugswetterA(at)spardat(dot)at>, Greg Stark <gsstark(at)mit(dot)edu>, Andrew Dunstan <andrew(at)dunslane(dot)net>, Rod Taylor <pg(at)rbt(dot)ca>, "Bort, Paul" <pbort(at)tmwsystems(dot)com>, pgsql-hackers(at)postgresql(dot)org |
| Subject: | Re: Compression and on-disk sorting |
| Date: | 2006-05-18 18:32:10 |
| Message-ID: | 20060518183210.GD4359@svana.org |
| Views: | Raw Message | Whole Thread | Download mbox | Resend email |
| Thread: | |
| Lists: | pgsql-hackers |
On Thu, May 18, 2006 at 11:22:46AM -0500, Jim C. Nasby wrote:
> AFAIK logtape currently reads in much less than 256k blocks. Of course
> if you get lucky you'll read from one tape for some time before
> switching to another, which should have a sort-of similar effect if the
> drives aren't very busy with other things.
Logtape works with BLCKSZ blocks. Whether it's advisable or not, I
don't know. One thing I'm noticing with this compression-in-logtape is
that the memory cost per tape increases considerably. Currently we rely
heavily on the OS to cache for us.
Lets say we buffer 256KB per tape, and we assume a large sort with many
tapes, you're going to blow all your work_mem on buffers. If using
compression uses more memory so that you can't have as many tapes and
thus need multiple passes, well, we need to test if this is still a
win.
Have a nice day,
--
Martijn van Oosterhout <kleptog(at)svana(dot)org> http://svana.org/kleptog/
> From each according to his ability. To each according to his ability to litigate.
| From | Date | Subject | |
|---|---|---|---|
| Next Message | Joshua D. Drake | 2006-05-18 18:32:30 | Re: [OT] MySQL is bad, but THIS bad? |
| Previous Message | Jim C. Nasby | 2006-05-18 18:29:56 | Re: [OT] MySQL is bad, but THIS bad? |