Block at a time ...

From: Dave Crooke <dcrooke(at)gmail(dot)com>
To: Alvaro Herrera <alvherre(at)commandprompt(dot)com>
Cc: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>, Greg Stark <gsstark(at)mit(dot)edu>, Pierre C <lists(at)peufeu(dot)com>, Greg Smith <greg(at)2ndquadrant(dot)com>, Paul McGarry <paul(at)paulmcgarry(dot)com>, pgsql-performance(at)postgresql(dot)org
Subject: Block at a time ...
Date: 2010-03-16 23:58:50
Message-ID: ca24673e1003161658g6a0fa30dg2c158db395ac08@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-performance

I agree with Tom, any reordering attempt is at best second guessing the
filesystem and underlying storage.

However, having the ability to control the extent size would be a worthwhile
improvement for systems that walk and chew gum (write to lots of tables)
concurrently.

I'm thinking of Oracle's AUTOEXTEND settings for tablespace datafiles .... I
think the ideal way to do it for PG would be to make the equivalent
configurable in postgresql.conf system wide, and allow specific per-table
settings in the SQL metadata, similar to auto-vacuum.

An awesomely simple alternative is to just specify the extension as e.g. 5%
of the existing table size .... it starts by adding one block at a time for
tiny tables, and once your table is over 20GB, it ends up adding a whole 1GB
file and pre-allocating it. Very little wasteage.

Cheers
Dave

On Tue, Mar 16, 2010 at 4:49 PM, Alvaro Herrera
<alvherre(at)commandprompt(dot)com>wrote:

> Tom Lane escribió:
> > Alvaro Herrera <alvherre(at)commandprompt(dot)com> writes:
> > > Maybe it would make more sense to try to reorder the fsync calls
> > > instead.
> >
> > Reorder to what, though? You still have the problem that we don't know
> > much about the physical layout on-disk.
>
> Well, to block numbers as a first step.
>
> However, this reminds me that sometimes we take the block-at-a-time
> extension policy too seriously. We had a customer that had a
> performance problem because they were inserting lots of data to TOAST
> tables, causing very frequent extensions. I kept wondering whether an
> allocation policy that allocated several new blocks at a time could be
> useful (but I didn't try it). This would also alleviate fragmentation,
> thus helping the physical layout be more similar to logical block
> numbers.
>
> --
> Alvaro Herrera
> http://www.CommandPrompt.com/
> PostgreSQL Replication, Consulting, Custom Development, 24x7 support
>

Responses

Browse pgsql-performance by date

  From Date Subject
Next Message Alvaro Herrera 2010-03-17 00:14:05 Re: Block at a time ...
Previous Message Greg Smith 2010-03-16 23:54:52 Re: shared_buffers advice