RE: [PoC] Non-volatile WAL buffer

From: "tsunakawa(dot)takay(at)fujitsu(dot)com" <tsunakawa(dot)takay(at)fujitsu(dot)com>
To: 'Takashi Menjo' <takashi(dot)menjo(at)gmail(dot)com>, Tomas Vondra <tomas(dot)vondra(at)enterprisedb(dot)com>
Cc: Masahiko Sawada <sawada(dot)mshk(at)gmail(dot)com>, Heikki Linnakangas <hlinnaka(at)iki(dot)fi>, Takashi Menjo <takashi(dot)menjou(dot)vg(at)hco(dot)ntt(dot)co(dot)jp>, "Deng, Gang" <gang(dot)deng(at)intel(dot)com>, "pgsql-hackers(at)postgresql(dot)org" <pgsql-hackers(at)postgresql(dot)org>
Subject: RE: [PoC] Non-volatile WAL buffer
Date: 2021-03-09 07:23:41
Message-ID: TYAPR01MB2990888B2966C7B3FCF3EEEDFE929@TYAPR01MB2990.jpnprd01.prod.outlook.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

From: Takashi Menjo <takashi(dot)menjo(at)gmail(dot)com>
> > The other question is whether simply placing WAL on DAX (without any
> > code changes) is safe. If it's not, then all the "speedups" are
> > computed with respect to unsafe configuration and so are useless. And
> > BTT should be used instead, which would of course produce very different
> results.
>
> I think it's safe, thanks to the checksum in the header of WAL record (xl_crc in
> struct XLogRecord). In DAX mode, user data (WAL record
> here) is written to the PMEM device by a smaller unit (probably a byte or a
> cache line) than the traditional 512-byte disk sector. So a torn-write such that
> "some bytes in a sector persist, other bytes not"
> can occur when crash. AFAICS, however, the checksum for WAL records can
> also support such a torn-write case.

I'm afraid I may be misunderstanding, so let me ask a naive question.

I understood "simply placing WAL on DAX (without any code changes)" means placing WAL files on DAX-aware filesystems such as ext4 and xfs, withoug modifying Postgres source code. That is, use the PMEM as a high performance storage device. Is this correct?

Second, does it what you represented as "master" in your test results?

I'd simply like to know what percentage of performance improvement we can expect by utilizing PMDK and modifying Postgres source code, and how much improvement we consider worthwhile.

Regards
Takayuki Tsunakawa

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Noah Misch 2021-03-09 07:25:47 Re: pg_upgrade fails with non-standard ACL
Previous Message Justin Pryzby 2021-03-09 07:16:55 Re: [HACKERS] Custom compression methods