Re: Disable WAL logging to speed up data loading

At Fri, 27 Nov 2020 13:34:49 +0000, "osumi(dot)takamichi(at)fujitsu(dot)com" <osumi(dot)takamichi(at)fujitsu(dot)com> wrote in
> Thank you, Horiguchi-San
>
> > I haven't seen a criteria of whether a record is emitted or not for
> > wal_leve=none.
> >
> > We're emitting only redo logs. So I think theoretically we don't need anything
> > other than the shutdown checkpoint record because we don't perform
> > recovery and checkpoint record is required at startup.
> >
> > RM_XLOG_ID:
> > XLOG_FPI_FOR_HINT - not needed?
> > XLOG_FPI - not needed?
> >
> > XLOG_CHECKPOINT_SHUTDOWN - must have
> >
> > So how about the followings?
> > XLOG_CHECKPOINT_ONLINE
> > XLOG_NOOP
> > XLOG_NEXTOID
> > XLOG_SWITCH
> > XLOG_BACKUP_END
> > XLOG_PARAMETER_CHANGE
> > XLOG_RESTORE_POINT
> > XLOG_FPW_CHANGE
> > XLOG_END_OF_RECOVERY
> >
> >
> > RM_XACT_ID:
> > XLOG_XACT_COMMIT
> > XLOG_XACT_PREPARE
> > XLOG_XACT_ABORT
> > XLOG_XACT_COMMIT_PREPARED
> > XLOG_XACT_ABORT_PREPARED
> > XLOG_XACT_ASSIGNMENT
> > XLOG_XACT_INVALIDATIONS
> >
> > Do we need all of these?
> No. Strictly speaking, you are right.
> We still have types of WAL that are not necessarily needed.
> For example, XLOG_END_OF_RECOVERY is not useful
> because wal_level=none doesn't recover from any accidents.
> Or, XLOG_CHECKPOINT_ONLINE is used when we execute CHECKPOINT
> not for shutting down. Thus we could eliminate more.

Yeah, although it's enough only to restrict non-harmful records
practically, if we find that only a few kinds of records are needed,
maybe it's cleaner to allow only required record type(s).

> > And, currenly what decides whether to emit a wal record according to
> > wal_level is the caller of XLogInsert.
> Yes.
>
> > So doing this at XLogInsert-level means
> > that we bring the criteria of the necessity of wal-record into xlog layer only for
> > wal_level=none. I'm not sure it is the right direction.
> I'm sorry. I didn't understand what "doing this" and "xlog layer" meant.

"doing this" meant filtering record types.

> Did you mean that fixing the caller side of XLogInsert (e.g. CreateCheckPoint)
> is not the right direction ? Or, fixing the function of XLogInsert is not the right direction ?

Maybe it's right that if we can filter-out records looking only rmid,
since the xlog facility doesn't need to know about record types of a
resource manager. But if we need to finer-grained control on the
record types, I'm afraid that that's wrong. However, if we need only
the XLOG_CHECKPOINT_SHUTDOWN record, it might be better to let
XLogInsert filter records rather than inserting that filtering code to
all the caller sites.

> > At Fri, 27 Nov 2020 07:01:16 +0000, "tsunakawa(dot)takay(at)fujitsu(dot)com"
> > <tsunakawa(dot)takay(at)fujitsu(dot)com> wrote in
> > > I'm afraid "none" doesn't represent the behavior because RM_XLOG_ID and
> > RM_XACT_ID WAL records, except for XLOG_FPI_*, are emitted. What's the
> > good name? IIUC, "minimal" is named after the fact that the minimal
> > amount of WAL necessary for crash recovery is generated. "norecovery" or
> > "unrecoverable"?
> Lastly, I found another name which expresses the essential characteristic of this wal_level.
> How about the name of wal_level="crash_unsafe" ?
> What did you think ?

I don't dislike "none" since it seems to me practically "none". It
seems rather correct if we actually need only the shutdown checkpoint
record.

"unrecoverable" is apparently misleading. "crash_unsafe" is precise
but seems somewhat alien being among "logical", "replica" and
"minimal".

regards.

--
Kyotaro Horiguchi
NTT Open Source Software Center