Re: Disable WAL logging to speed up data loading

From: Kyotaro Horiguchi <horikyota(dot)ntt(at)gmail(dot)com>
To: osumi(dot)takamichi(at)fujitsu(dot)com
Cc: tsunakawa(dot)takay(at)fujitsu(dot)com, sawada(dot)mshk(at)gmail(dot)com, robertmhaas(at)gmail(dot)com, masao(dot)fujii(at)oss(dot)nttdata(dot)com, laurenz(dot)albe(at)cybertec(dot)at, ashutosh(dot)bapat(dot)oss(at)gmail(dot)com, pgsql-hackers(at)lists(dot)postgresql(dot)org
Subject: Re: Disable WAL logging to speed up data loading
Date: 2021-01-13 02:07:06
Message-ID: 20210113.110706.198230270559275306.horikyota.ntt@gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

At Tue, 12 Jan 2021 07:09:28 +0000, "osumi(dot)takamichi(at)fujitsu(dot)com" <osumi(dot)takamichi(at)fujitsu(dot)com> wrote in
> On Tuesday, January 12, 2021 12:52 PM Takayuki/綱川 貴之 <tsunakawa(dot)takay(at)fujitsu(dot)com> wrote:
> > From: Osumi, Takamichi/大墨 昂道 <osumi(dot)takamichi(at)fujitsu(dot)com>
> > > I updated the patch following this discussion, and fixed the
> > > documentation as well.
> >
> >
> > + (rmid == RM_GIST_ID && info == RM_GIST_ID) ||
> > + (rmid == RM_GENERIC_ID)))
> >
> > info is wrong for GiST, and one pair of parenthesis is unnecessary. The above
> > would be:
> >
> > + (rmid == RM_GIST_ID && info ==
> > XLOG_GIST_ASSIGN_LSN) ||
> > + rmid == RM_GENERIC_ID))
> Oops, sorry for this careless mistake.
> Fixed. And again, regression test produces no failure.

@@ -449,6 +450,18 @@ XLogInsert(RmgrId rmid, uint8 info)
return EndPos;
}

+ /* Issues only limited types of WAL when wal logging is disabled */
+ if (wal_level == WAL_LEVEL_NONE &&
+ !((rmid == RM_XLOG_ID && info == XLOG_CHECKPOINT_SHUTDOWN) ||
+ (rmid == RM_XLOG_ID && info == XLOG_PARAMETER_CHANGE) ||
+ (rmid == RM_XACT_ID && info == XLOG_XACT_PREPARE) ||
+ (rmid == RM_GIST_ID && info == XLOG_GIST_ASSIGN_LSN) ||
+ rmid == RM_GENERIC_ID))

As Sawada-san mentioned, this prevents future index AMs from being
pluggable. Providing an interface would work but seems a bit too
invasive. The record insertion flags is there for this very purpose.

XLogBeginInsert();
XLogSetRecrodFlags(XLOG_MARK_ESSENTIAL); # new flag value
XLOGInsert(....);

regards.

--
Kyotaro Horiguchi
NTT Open Source Software Center

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Kyotaro Horiguchi 2021-01-13 02:09:14 Re: [Patch] Optimize dropping of relation buffers using dlist
Previous Message Hou, Zhijie 2021-01-13 02:06:58 RE: Single transaction in the tablesync worker?