RE: Disable WAL logging to speed up data loading

From: "tsunakawa(dot)takay(at)fujitsu(dot)com" <tsunakawa(dot)takay(at)fujitsu(dot)com>
To: 'Kyotaro Horiguchi' <horikyota(dot)ntt(at)gmail(dot)com>, "osumi(dot)takamichi(at)fujitsu(dot)com" <osumi(dot)takamichi(at)fujitsu(dot)com>
Cc: "sawada(dot)mshk(at)gmail(dot)com" <sawada(dot)mshk(at)gmail(dot)com>, "robertmhaas(at)gmail(dot)com" <robertmhaas(at)gmail(dot)com>, "masao(dot)fujii(at)oss(dot)nttdata(dot)com" <masao(dot)fujii(at)oss(dot)nttdata(dot)com>, "laurenz(dot)albe(at)cybertec(dot)at" <laurenz(dot)albe(at)cybertec(dot)at>, "ashutosh(dot)bapat(dot)oss(at)gmail(dot)com" <ashutosh(dot)bapat(dot)oss(at)gmail(dot)com>, "pgsql-hackers(at)lists(dot)postgresql(dot)org" <pgsql-hackers(at)lists(dot)postgresql(dot)org>
Subject: RE: Disable WAL logging to speed up data loading
Date: 2021-01-13 06:01:34
Message-ID: TYAPR01MB29903B614F328CF5D246DAA7FEA90@TYAPR01MB2990.jpnprd01.prod.outlook.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

From: Kyotaro Horiguchi <horikyota(dot)ntt(at)gmail(dot)com>
> XLogBeginInsert();
> XLogSetRecrodFlags(XLOG_MARK_ESSENTIAL); # new flag value
> XLOGInsert(....);

Oh, sounds like a nice idea. That's more flexible by allowing WAL-emitting modules to specify which WAL records are mandatory even when wal_level is none.

For example, gistXLogAssignLSN() adds the above flag like this:

XLogBeginInsert();
XLogSetRecordFlags(XLOG_MARK_UNIMPORTANT | XLOG_MARK_ESSENTIAL);
XLogRegisterData((char *) &dummy, sizeof(dummy));

(Here's a word play - unimportant but essential, what's that?)

And the filter in XLogInsert() becomes:

+ if (wal_level == WAL_LEVEL_NONE &&
+ !((rmid == RM_XLOG_ID && info == XLOG_CHECKPOINT_SHUTDOWN) ||
+ (rmid == RM_XLOG_ID && info == XLOG_PARAMETER_CHANGE) ||
+ (rmid == RM_XACT_ID && info == XLOG_XACT_PREPARE) ||
+ (curinsert_flags & XLOG_MARK_ESSENTIAL)))

Or,

+ if (wal_level == WAL_LEVEL_NONE &&
+ !(curinsert_flags & XLOG_MARK_ESSENTIAL))

and add the new flag when emitting XLOG_CHECKPOINT_ONLINE, XLOG_PARAMETER_CHANGE and XLOG_PREPARE records. I think both have good reasons: the former centralizes the handling of XACT and XLOG RM WAL records (as the current XLOG module does already), and the latter delegates the decision to each module. Which would you prefer? (I kind of like the former, but this is a weak opinion.)

Regards
Takayuki Tsunakawa

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Amit Kapila 2021-01-13 06:10:33 Re: remove unneeded pstrdup in fetch_table_list
Previous Message Amit Kapila 2021-01-13 05:59:20 Re: Logical Replication - behavior of ALTER PUBLICATION .. DROP TABLE and ALTER SUBSCRIPTION .. REFRESH PUBLICATION