Re: PostgreSQL 8.4 performance tuning questions

Scott Carey <scott(at)richrelevance(dot)com> writes:
> On 7/30/09 2:53 PM, "Tom Lane" <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
>> Scott Carey <scott(at)richrelevance(dot)com> writes:
>>> Gzip does have some quirky performance behavior depending on the chunk size
>>> of data you stream into it.
>>
>> Can you enlarge on that comment? I'm not sure that pg_dump is aware
>> that there's anything to worry about there.

> For whatever reason, some other internals of gzip tend to perform much
> better if submitting say, 4k or 8k or 16k chunks rather than little bits at
> a time. But I'm sure some of that also depends on what library you're using
> since they all vary somewhat.

AFAIK there is only one widely-used implementation of zlib, and it
hasn't changed much in a long time.

I did some tracing and verified that pg_dump passes data to deflate()
one table row at a time. I'm not sure about the performance
implications of that, but it does seem like it might be something to
look into.

regards, tom lane