Re: Disable WAL logging to speed up data loading

From: Laurenz Albe <laurenz(dot)albe(at)cybertec(dot)at>
To: "osumi(dot)takamichi(at)fujitsu(dot)com" <osumi(dot)takamichi(at)fujitsu(dot)com>, "tsunakawa(dot)takay(at)fujitsu(dot)com" <tsunakawa(dot)takay(at)fujitsu(dot)com>, 'Stephen Frost' <sfrost(at)snowman(dot)net>, Magnus Hagander <magnus(at)hagander(dot)net>, Robert Haas <robertmhaas(at)gmail(dot)com>
Cc: Fujii Masao <masao(dot)fujii(at)oss(dot)nttdata(dot)com>, Kyotaro Horiguchi <horikyota(dot)ntt(at)gmail(dot)com>, "ashutosh(dot)bapat(dot)oss(at)gmail(dot)com" <ashutosh(dot)bapat(dot)oss(at)gmail(dot)com>, "pgsql-hackers(at)lists(dot)postgresql(dot)org" <pgsql-hackers(at)lists(dot)postgresql(dot)org>
Subject: Re: Disable WAL logging to speed up data loading
Date: 2020-11-19 07:50:24
Message-ID: c597fe056dffaf1d977540dfda8d7271f53dcfb0.camel@cybertec.at
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Thu, 2020-11-19 at 05:24 +0000, osumi(dot)takamichi(at)fujitsu(dot)com wrote:
> > > > ereport(WARNING,
> > > > (errmsg("WAL was generated with wal_level=minimal, data may
> > > > be missing"),
> > > > errhint("This happens if you temporarily set
> > > > wal_level=minimal without taking a new base backup.")));
> > > > There's definitely a question about if a WARNING there is really
> > > > sufficient or not, considering that you could end up with 'logged'
> > > > tables on the replica that are missing data, but I'm not sure that
> > > > inventing a new, independent, mechanism for checking WAL level
> > > > changes makes
> > > sense.
> >
> > I don't know why WARNING was chosen. I think it should be FATAL,
> > resulting in the standby shutdown, disabling restarting it, and urging the user
> > to rebuild the standby. (I guess that's overreaction because the user may
> > not perform operations that lack WAL while wal_level is minimal.)
>
> Yeah, I agree that WARNING is not sufficient.

I missed that this is only a warning when I looked at it before.
Yes, it should be a fatal error.

I think that there should two patches: one that turns this warning into
a FATAL and should be backpatched. If you change the test to

ControlFile->wal_level <= WAL_LEVEL_MINIMAL

it will automatically work for your new feature too.

Then your new wal_level would be a second patch only for HEAD.

With that, the only remaining consideration with this patch is the danger
that enabling wal_level=none without taking a backup before can lead to
data loss. But that is intended, so I think that an unmistakable warning
in the documentation would be good enough.

Yours,
Laurenz Albe

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message tsunakawa.takay@fujitsu.com 2020-11-19 07:50:46 RE: Disable WAL logging to speed up data loading
Previous Message japin 2020-11-19 07:39:51 Use macros for calculating LWLock offset