Skip site navigation (1) Skip section navigation (2)

pgsql: Use O_DIRECT if available when using O_SYNC for wal_sync_method.

From: momjian(at)svr1(dot)postgresql(dot)org (Bruce Momjian)
To: pgsql-committers(at)postgresql(dot)org
Subject: pgsql: Use O_DIRECT if available when using O_SYNC for wal_sync_method.
Date: 2005-07-29 03:22:36
Message-ID: 20050729032236.1909B5299C@svr1.postgresql.org (view raw or flat)
Thread:
Lists: pgsql-committers
Log Message:
-----------

Use O_DIRECT if available when using O_SYNC for wal_sync_method.

Also, write multiple WAL buffers out in one write() operation.

ITAGAKI Takahiro

---------------------------------------------------------------------------

> If we disable writeback-cache and use open_sync, the per-page writing
> behavior in WAL module will show up as bad result. O_DIRECT is similar
> to O_DSYNC (at least on linux), so that the benefit of it will disappear
> behind the slow disk revolution.
>
> In the current source, WAL is written as:
>     for (i = 0; i < N; i++) { write(&buffers[i], BLCKSZ); }
> Is this intentional? Can we rewrite it as follows?
>    write(&buffers[0], N * BLCKSZ);
>
> In order to achieve it, I wrote a 'gather-write' patch (xlog.gw.diff).
> Aside from this, I'll also send the fixed direct io patch (xlog.dio.diff).
> These two patches are independent, so they can be applied either or both.
>
>
> I tested them on my machine and the results as follows. It shows that
> direct-io and gather-write is the best choice when writeback-cache is off.
> Are these two patches worth trying if they are used together?
>
>
>             | writeback | fsync= | fdata | open_ | fsync_ | open_
> patch       | cache     |  false |  sync |  sync | direct | direct
> ------------+-----------+--------+-------+-------+--------+---------
> direct io   | off       |  124.2 | 105.7 |  48.3 |   48.3 |  48.2
> direct io   | on        |  129.1 | 112.3 | 114.1 |  142.9 | 144.5
> gather-write| off       |  124.3 | 108.7 | 105.4 |  (N/A) | (N/A)
> both        | off       |  131.5 | 115.5 | 114.4 |  145.4 | 145.2
>
> - 20runs * pgbench -s 100 -c 50 -t 200
>    - with tuning (wal_buffers=64, commit_delay=500, checkpoint_segments=8)
> - using 2 ATA disks:
>    - hda(reiserfs) includes system and wal.
>    - hdc(jfs) includes database files. writeback-cache is always on.
>
> ---
> ITAGAKI Takahiro

Modified Files:
--------------
    pgsql/src/backend/access/transam:
        xlog.c (r1.210 -> r1.211)
        (http://developer.postgresql.org/cvsweb.cgi/pgsql/src/backend/access/transam/xlog.c.diff?r1=1.210&r2=1.211)

Responses

pgsql-committers by date

Next:From: Bruce MomjianDate: 2005-07-29 03:23:04
Subject: pgsql: Done: < * Consider use of open/fcntl(O_DIRECT) to minimize OS
Previous:From: Bruce MomjianDate: 2005-07-29 03:18:03
Subject: pgsql: Thank you for applying patch --- regexp_replace.

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group