Re: Are random writes optimized sequentially by Linux kernel?

OK, thank you.

Now - PostgreSQL-related question. If the system reorders writes to minimize
seeking, I suppose that in heavy write-loaded PostgreSQL instalation dstat
(or iostat) realtime write statistics should be close to the maximum
possible value reported by bonnie++ (or simple dd) utility.

So, if, for example, I have in a heavy-loaded PostgreSQL installation:
- with a 50MB/s write speed limit reported by bonnie++ or dd (on a clean
system),
- under a heavy PostgreSQL load the write throughput is only 10MB/s
(synchronous_commit is off, checkpoint is executed every 10 minutes or even
more),
- writeback buffer (accordingly to /proc/meminfo) is not fully filled,
- sometimes INSERTs or UPDATEs slow down in 10 second and more with no
explicit correlation with checkpoints

then - something goes wrong?

What I am trying to understand - why does the system fall to a writing
bottleneck (e.g. 10MB/s) much before it achieves the maximum disk throughput
(e.g. 50MB/s). How could it happen if the Linux IO scheduler reorders write
operations, so time for seeking is minimal?

Or, better, I can reformulate the question. In which cases PostgreSQL may
stall on INSERT/UPDATE operation if synchronous_commit is off and there are
no locking between transactions? In which cases these operations lost their
deterministic time (in theory) and may slowdown in 100-1000 times?

On Wed, Jan 7, 2009 at 10:54 PM, <david(at)lang(dot)hm> wrote:

> On Wed, 7 Jan 2009, Dmitry Koterov wrote:
>
> Hello.
>>
>> Suppose I perform 1000 RANDOM writes into a file. These writes are saved
>> into Linux writeback buffer and are flushed to the disc asynchronously,
>> that's OK.
>>
>> The question is: will physical writes be performed later in the sequence
>> of
>> physical SECTOR position on the disc (minimizing head seeking)? Or Linux
>> background writer knows nothing about physical on-disc placement and
>> flushes
>> data in order it is saved in the RAM?
>>
>> E.g., if I write in the application:
>>
>> a) block 835
>> b) block 136
>> c) block 956
>> d) block 549
>> e) block 942
>>
>> dows the Linux background writer save flush them e.g. in physical order
>> "136
>> - 549 - 835 - 942 - 956" or not?
>>
>
> yes, the linux IO scheduler will combine and re-order write requests.
>
> they may end up being done 835-942-956-549-136 if the system thinks the
> head happens to be past 549 and moving up when the requests hit the IO
> system.
>
> David Lang
>
> --
> Sent via pgsql-performance mailing list (pgsql-performance(at)postgresql(dot)org)
> To make changes to your subscription:
> http://www.postgresql.org/mailpref/pgsql-performance
>