Re: Vectored I/O in bulk_write.c

From: Thomas Munro <thomas(dot)munro(at)gmail(dot)com>
To: PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>
Cc: Heikki Linnakangas <hlinnaka(at)iki(dot)fi>
Subject: Re: Vectored I/O in bulk_write.c
Date: 2024-03-10 23:42:26
Message-ID: CA+hUKGKOsSvGcwDQP3sCmRT-wbwmxOfo53O2VhVbnk8xpuXwWQ@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Here also is a first attempt at improving the memory allocation and
memory layout.

I wonder if bulk logging should trigger larger WAL writes in the "Have
to write it ourselves" case in AdvanceXLInsertBuffer(), since writing
8kB of WAL at a time seems like an unnecessarily degraded level of
performance, especially with wal_sync_method=open_datasync. Of course
the real answer is "make sure wal_buffers is high enough for your
workload" (usually indirectly by automatically scaling from
shared_buffers), but this problem jumps out when tracing bulk_writes.c
with default settings. We write out the index 128kB at a time, but
the WAL 8kB at a time.

Attachment Content-Type Size
v3-0001-Provide-vectored-variant-of-smgrextend.patch text/x-patch 12.6 KB
v3-0002-Use-vectored-I-O-for-bulk-writes.patch text/x-patch 4.3 KB
v3-0003-Improve-bulk_write.c-memory-management.patch text/x-patch 5.7 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Michael Paquier 2024-03-10 23:42:39 Re: Hooking into ExplainOneQuery() complicated by missing standard_ExplainOneQuery
Previous Message jian he 2024-03-10 23:40:00 Re: Adding OLD/NEW support to RETURNING