Re: 011_crash_recovery.pl intermittently fails

From: Kyotaro Horiguchi <horikyota(dot)ntt(at)gmail(dot)com>
To: tgl(at)sss(dot)pgh(dot)pa(dot)us
Cc: thomas(dot)munro(at)gmail(dot)com, pg(at)bowt(dot)ie, pgsql-hackers(at)lists(dot)postgresql(dot)org, craig(dot)ringer(at)enterprisedb(dot)com, robertmhaas(at)gmail(dot)com
Subject: Re: 011_crash_recovery.pl intermittently fails
Date: 2021-03-08 08:32:42
Message-ID: 20210308.173242.463790587797836129.horikyota.ntt@gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

At Sun, 07 Mar 2021 20:09:33 -0500, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote in
> Thomas Munro <thomas(dot)munro(at)gmail(dot)com> writes:
> > Thanks! I'm afraid I wouldn't get around to it for a few weeks, so if
> > you have time, please do. (I'm not sure if it's strictly necessary to
> > log *this* xid, if a higher xid has already been logged, considering
> > that the goal is just to avoid getting confused about an xid that is
> > recycled after crash recovery, but coordinating that might be more
> > complicated; I don't know.)
>
> Yeah, ideally the patch wouldn't add any unnecessary WAL flush,
> if there's some cheap way to determine that our XID must already
> have been written out. But I'm not sure that it's worth adding
> any great amount of complexity to avoid that. For sure I would
> not advocate adding any new bookkeeping overhead in the mainline
> code paths to support it.

We need to *write* an additional record if the current transaction
haven't yet written one (EnsureTopTransactionIdLogged()). One
annoyance is the possibly most-common usage of calling
pg_current_xact_id() at the beginning of a transaction, which leads to
an additional 8 byte-long log of XLOG_XACT_ASSIGNMENT. We could also
avoid that by detecting any larger xid is already flushed out.

I haven't find a simple and clean way to tracking the maximum
flushed-out XID. The new cooperation between xlog.c and xact.c
related to XID and LSN happen on shared variable makes things
complex...

So the attached doesn't contain the max-flushed-xid tracking feature.

regards.

--
Kyotaro Horiguchi
NTT Open Source Software Center

Attachment Content-Type Size
v1-0001-Run-011_crash_recovery.pl-with-wal_level-minimal.patch text/x-patch 889 bytes
v1-0002-Make-sure-published-XIDs-are-persistent.patch text/x-patch 7.1 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Greg Nancarrow 2021-03-08 08:42:40 Re: Parallel INSERT (INTO ... SELECT ...)
Previous Message Amit Kapila 2021-03-08 08:31:53 Re: Parallel INSERT (INTO ... SELECT ...)