|From:||Hans-Juergen Schoenig <postgres(at)cybertec(dot)at>|
|To:||pgsql-patches <pgsql-patches(at)postgresql(dot)org>, Zoltan Boszormenyi <zb(at)cybertec(dot)at>|
|Subject:||posix advises ...|
|Views:||Raw Message | Whole Thread | Download mbox | Resend email|
recently we had a bit of a nightmare with some kernels and concurrent
the thing we encountered was the following: a single "SELECT COUNT(*)
FROM table" on a big table (50 gb) gave us constant 350 mb / sec I/O. as
soon as a second scan dropped in speed dropped to 2 mb / sec. first i
thought that some random I/O dropped in but synchronous scans worked
fine. we found out that there is some madness in some linux kernel /
controller combinations causing this issue.
it did some tests on my local boxes which was clearly not affected by
this problem and I have seen a single SATA disks dropping from 65 mb /
sec to around 45. this is not good.
i found a patch by grep stark implementing posix_fadvise for bitmap
scans. i quickly hacked in suggestions to issue the same advises when a
seq scan is done.
the impact was surprisingly high. single scans went up from 65 mb / sec
to something around 70. concurrent scans are basically at steady, high
speed - no dropping I/O speed anymore until something like 16 scans or so.
even the broken controller when up from "350mb -> 2mb" to "350 -> 50mb".
by replacing the kernel and the driver we see steady behavior here as
maybe it is worth to discuss posix_fadvise.
we hacked up a simple patch based on greg's work which nicely fixed the
problem for us (brute force).
we also made some simple autoconf hack to check for broken posix_fadvise.
maybe people want to test if they see similar performance differences.
if a patch like that is likely to be accepted we would hack up some more
|Next Message||Hans-Juergen Schoenig||2008-05-11 10:05:44||XIDs and big boxes again ...|
|Previous Message||Tino Wildenhain||2008-05-11 09:13:05||Re: Setting a pre-existing index as a primary key|
|Next Message||daveg||2008-05-11 11:30:47||pg_dump lock timeout|
|Previous Message||Nikhils||2008-05-11 07:57:49||Re: [firstname.lastname@example.org: Re: [BUGS] Problem identifying constraints which should not be inherited]|