Re: Why Wal_buffer is 64KB

Hi Pierre,

First of all , I Thank all for sharing the information on this Issue.

On Thu, Mar 25, 2010 at 11:44 PM, Pierre C <lists(at)peufeu(dot)com> wrote:
>

> If you do large transactions, which emits large quantities of xlog, be
> aware that while the previous xlog segment is being fsynced, no new writes
> happen to the next segment. If you use large wal_buffers (more than 16 MB)
> these buffers can absorb xlog data while the previous segment is being
> fsynced, which allows a higher throughput. However, large wal_buffers also
> mean the COMMIT of small transactions might find lots of data in the buffers
> that noone has written/synced yet, which isn't good. If you use dedicated
> spindle(s) for the xlog, you can set the walwriter to be extremely
> aggressive (write every 5 ms for instance) and use fdatasync. This way, at
> almost every rotation of the disk, xlog gets written. I've found this
> configuration gives increased throughput, while not compromising latency,
> but you need to test it for yourself, it depends on your whole system.

Small testing is done from my end. I have created a "test" table with one
row and done insertion into it(10,00,000- rows). I have turned off fsync and
syncronous_commit. I saw there is fast insert if i do so, but if i turn it
on then there is latency.

Before fsync / syncronous_commit on
============================
postgres=# explain analyze insert into test
values(generate_series(1,1000000));
QUERY PLAN
---------------------------------------------------------------------------------------------
Result (cost=0.00..0.01 rows=1 width=0) (actual time=0.015..6293.674
rows=1000000 loops=1)
Total runtime: *37406.012 ms*
(2 rows)

After fsync/syncronous_commit off
=========================
postgres=# explain analyze insert into test
values(generate_series(1,1000000));
QUERY PLAN
---------------------------------------------------------------------------------------------
Result (cost=0.00..0.01 rows=1 width=0) (actual time=0.154..5801.584
rows=1000000 loops=1)
Total runtime: *29378.626 ms
*(2 rows)

I request to know here is, what would be xlog files with wal_buffer. Does
xlog will recycle or grow in creating one more for this particular
transaction. Could you explain here, when wal_buffer is 64kb which is very
small, and everything is in xlog files written, so wt happens if we increase
the wal_buffer here?

Regards
Raghav