RE: Disable WAL logging to speed up data loading

From: "osumi(dot)takamichi(at)fujitsu(dot)com" <osumi(dot)takamichi(at)fujitsu(dot)com>
To: 'Laurenz Albe' <laurenz(dot)albe(at)cybertec(dot)at>, "tsunakawa(dot)takay(at)fujitsu(dot)com" <tsunakawa(dot)takay(at)fujitsu(dot)com>, 'Fujii Masao' <masao(dot)fujii(at)oss(dot)nttdata(dot)com>, Kyotaro Horiguchi <horikyota(dot)ntt(at)gmail(dot)com>
Cc: "ashutosh(dot)bapat(dot)oss(at)gmail(dot)com" <ashutosh(dot)bapat(dot)oss(at)gmail(dot)com>, "pgsql-hackers(at)lists(dot)postgresql(dot)org" <pgsql-hackers(at)lists(dot)postgresql(dot)org>
Subject: RE: Disable WAL logging to speed up data loading
Date: 2020-10-28 09:55:45
Message-ID: OSBPR01MB48885361D758639102DCED7BED170@OSBPR01MB4888.jpnprd01.prod.outlook.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hi, Laurenz

> > I wrote and attached the first patch to disable WAL logging.
> > This patch passes the regression test of check-world already and is
> > formatted by pgindent.
>
> Without reading the code, I have my doubts about that feature.
>
> While it clearly will improve performance, it opens the door to data loss.
Therefore, this feature must avoid that
that kind of inconsistent server starts up again.
This has been discussed in this thread already.

> People will use it to speed up their data loads and then be unhappy if they
> cannot use their backups to recover from a problem.
>
> What happens if you try to do archive recovery across a time where wal_level
> was "none"? Will the recovery process fail, as it should, or will you end up
> with data corruption?
>
> We already have a performance-related footgun in the shape of fsync = off.
> Do we want to add another one?
Further, in this thread, we discuss that
this feature is intended to serve under
some specific opportunities like DBA wants
to load data as soon as possible and/or the operation itself is easily *repeatable*.
So, before and after the change of wal_level, DBA needs to take a full backup to
prepare the unexpected crash.

But anyway, I'll fix and enrich the documents. Thanks.

Best,
Takamichi Osumi

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Masahiko Sawada 2020-10-28 10:03:14 Re: shared tempfile was not removed on statement_timeout
Previous Message Oleksandr Shulgin 2020-10-28 09:55:18 Re: cutting down the TODO list thread