Re: Block at a time ...

From: "Pierre C" <lists(at)peufeu(dot)com>
To: "Scott Carey" <scott(at)richrelevance(dot)com>, "Craig James" <craig_james(at)emolecules(dot)com>
Cc: "pgsql-performance(at)postgresql(dot)org" <pgsql-performance(at)postgresql(dot)org>
Subject: Re: Block at a time ...
Date: 2010-03-22 20:55:27
Message-ID: op.u9zlqp0reorkce@localhost
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-performance


>> This is one of the more-or-less solved problems in Unix/Linux. Ext*
>> file systems have a "reserve" usually of 10% of the disk space that
>> nobody except root can use. It's not for root, it's because with 10%
>> of the disk free, you can almost always do a decent job of allocating
>> contiguous blocks and get good performance. Unless Postgres has some
>> weird problem that Linux has never seen before (and that wouldn't be
>> unprecedented...), there's probably no need to fool with
>> file-allocation strategies.
>>
>> Craig
>
> Its fairly easy to break. Just do a parallel import with say, 16
> concurrent tables being written to at once. Result? Fragmented tables.

Delayed allocation (ext4, XFS) helps a lot for concurrent writing at a
medium-high rate (a few megabytes per second and up) when lots of data can
sit in the cache and be flushed/allocated as big contiguous chunks. I'm
pretty sure ext4/XFS would pass your parallel import test.

However if you have files like tables (and indexes) or logs that grow
slowly over time (something like a few megabytes per hour or less), after
a few days/weeks/months, horrible fragmentation is an almost guaranteed
result on many filesystems (NTFS being perhaps the absolute worst).

In response to

Responses

Browse pgsql-performance by date

  From Date Subject
Next Message Dave Crooke 2010-03-22 21:06:00 Re: Block at a time ...
Previous Message Tom Lane 2010-03-22 19:21:35 Re: PostgreSQL upgraded to 8.2 but forcing index scan on query produces faster