Map WAL segment files on PMEM as WAL buffers

From: Takashi Menjo <takashi(dot)menjo(at)gmail(dot)com>
To: pgsql-hackers(at)postgresql(dot)org
Cc: Tomas Vondra <tomas(dot)vondra(at)enterprisedb(dot)com>, Masahiko Sawada <sawada(dot)mshk(at)gmail(dot)com>, Heikki Linnakangas <hlinnaka(at)iki(dot)fi>, tsunakawa(dot)takay(at)fujitsu(dot)com, "Deng, Gang" <gang(dot)deng(at)intel(dot)com>, Takashi Menjo <takashi(dot)menjou(dot)vg(at)hco(dot)ntt(dot)co(dot)jp>
Subject: Map WAL segment files on PMEM as WAL buffers
Date: 2021-05-19 01:25:45
Message-ID: CAOwnP3PHDOW=-BqutjJPN0qEdF8ocqBuOiE9bpe-_c-1bnQ1yA@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hi hackers,

In response to PMEM-related discussions in the previous thread [1],
especially Tomas' performance report [2], I have worked for the way
that maps WAL segment files on PMEM as WAL buffers. I start this new
thread to go that way since the previous one has focused on another
patchset that I called "Non-volatile WAL buffer."

The patchset using WAL segment files is attached to this mail. Note
that it is tested on 8e4b332 (Mar 22, 2021) and cannot be applied to
the latest master. Also note that it has a known issue related to
checkpoint request (see Section 1.4 in the attached PDF for details).
I'm rebasing and fixing it, so please be patient for an update.

This mail also has a performance report PDF comparing PMEM patchsets
including ones that I have posted to pgsql-hackers ever, and such
zipped and rebased patchsets for reproducibility. The report covers
how to build and configure PostgreSQL with the patchsets, so please
see it before you use them.

Regards,
Takashi

[1] https://www.postgresql.org/message-id/flat/002f01d5d28d%2423c01430%246b403c90%24%40hco.ntt.co.jp_1
[2] https://www.postgresql.org/message-id/9beaac79-2375-8bfc-489b-eb62bd8d4020@enterprisedb.com

--
Takashi Menjo <takashi(dot)menjo(at)gmail(dot)com>

Attachment Content-Type Size
0001-Add-with-libpmem-option-for-PMEM-support.patch application/octet-stream 5.3 KB
0002-Add-wal_pmem_map-to-GUC.patch application/octet-stream 4.6 KB
0003-Export-InstallXLogFileSegment.patch application/octet-stream 2.1 KB
0004-Map-WAL-segment-files-as-WAL-buffers.patch application/octet-stream 16.8 KB
Performance-of-PMEM-patchsets-20210519.pdf application/pdf 1.4 MB
Applying-PMDK-to-WAL-operations-for-persistent-memory-20210322.zip application/x-zip-compressed 16.2 KB
Non-volatile-WAL-buffer-20210322.zip application/x-zip-compressed 37.7 KB

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Peter Geoghegan 2021-05-19 01:59:49 Re: MaxOffsetNumber for Table AMs
Previous Message Kyotaro Horiguchi 2021-05-19 00:53:08 Re: pg_get_wal_replay_pause_state() should not return 'paused' while a promotion is ongoing.