RE: [PoC] Non-volatile WAL buffer

From: "tsunakawa(dot)takay(at)fujitsu(dot)com" <tsunakawa(dot)takay(at)fujitsu(dot)com>
To: 'Tomas Vondra' <tomas(dot)vondra(at)enterprisedb(dot)com>, Takashi Menjo <takashi(dot)menjo(at)gmail(dot)com>, Heikki Linnakangas <hlinnaka(at)iki(dot)fi>
Cc: Takashi Menjo <takashi(dot)menjou(dot)vg(at)hco(dot)ntt(dot)co(dot)jp>, "Deng, Gang" <gang(dot)deng(at)intel(dot)com>, "pgsql-hackers(at)postgresql(dot)org" <pgsql-hackers(at)postgresql(dot)org>
Subject: RE: [PoC] Non-volatile WAL buffer
Date: 2020-11-25 00:27:20
Message-ID: TYAPR01MB2990B44B0817FD881CD94177FEFA0@TYAPR01MB2990.jpnprd01.prod.outlook.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

From: Tomas Vondra <tomas(dot)vondra(at)enterprisedb(dot)com>
> It's interesting that they only place the tail of the log on PMEM, i.e.
> the PMEM buffer has limited size, and the rest of the log is not on
> PMEM. It's a bit as if we inserted a PMEM buffer between our wal buffers
> and the WAL segments, and kept the WAL segments on regular storage. That
> could work, but I'd bet they did that because at that time the NV
> devices were much smaller, and placing the whole log on PMEM was not
> quite possible. So it might be unnecessarily complicated, considering
> the PMEM device capacity is much higher now.
>
> So I'd suggest we simply try this:
>
> clients -> buffers (DRAM) -> wal segments (PMEM)
>
> I plan to do some hacking and maybe hack together some simple tools to
> benchmarks various approaches.

I'm in favor of your approach. Yes, Intel PMEM were available in 128/256/512 GB when I checked last year. That's more than enough to place all WAL segments, so a small PMEM wal buffer is not necessary. I'm excited to see Postgres gain more power.

Regards
Takayuki Tsunakawa

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Bossart, Nathan 2020-11-25 00:48:26 Re: A few new options for CHECKPOINT
Previous Message Andy Fan 2020-11-25 00:10:21 Re: About adding a new filed to a struct in primnodes.h