Re: pgsql: Avoid duplicate XIDs at recovery when building initial snapshot

From: Dmitry Dolgov <9erthalion6(at)gmail(dot)com>
To: Michael Paquier <michael(at)paquier(dot)xyz>
Cc: Alvaro Herrera <alvherre(at)2ndquadrant(dot)com>, Andres Freund <andres(at)anarazel(dot)de>, PostgreSQL Developers <pgsql-hackers(at)lists(dot)postgresql(dot)org>
Subject: Re: pgsql: Avoid duplicate XIDs at recovery when building initial snapshot
Date: 2018-11-30 13:54:04
Message-ID: CA+q6zcXzUV3V9O5XQrATL9=W6SnjRvaX4yGd=syMUdyjOhnDEw@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-committers pgsql-hackers

> On Thu, Nov 1, 2018 at 7:09 AM Michael Paquier <michael(at)paquier(dot)xyz> wrote:
>
> On Tue, Oct 23, 2018 at 10:43:38AM +0900, Michael Paquier wrote:
> > Well, following the same kind of thoughts, txid_current_snapshot() uses
> > sort_snapshot() to remove all the duplicates after fetching its data
> > from GetSnapshotData(), so wouldn't we want to do something about
> > removal of duplicates if dummy PGXACT entries are found while scanning
> > the ProcArray also in this case? What I would think we should do is not
> > only to patch GetRunningTransactionData() but also GetSnapshotData() so
> > as we don't have duplicates also in this case, and do things in such a
> > way that both code paths use the same logic, and that we don't need to
> > have sort_snapshot() anymore. That would be more costly though...
>
> My apologies it took a bit longer than I thought. I got a patch on my
> desk for a couple of days, and finally took the time to finish something
> which would address the concerns raised here. As long as we don't reach
> more than hundreds of thousands of entries, there is not going to be any
> performance impact. So what I do in the attached is to revert 1df21ddb,
> and then have GetRunningTransactionData() sort the XIDs in the snapshot
> and remove duplicates only if at least one dummy proc entry is found
> while scanning, for xids and subxids. This way, there is no need to
> impact most of the instance deployments with the extra sort/removal
> phase as most don't use two-phase transactions. The sorting done at
> recovery when initializing the standby snapshot still needs to happen of
> course.
>
> The patch is added to the upcoming CF for review.

Unfortunately, patch has some conflict with the current master. Could you
please post a rebased version?

In response to

Browse pgsql-committers by date

  From Date Subject
Next Message Simon Riggs 2018-11-30 14:55:47 Re: pgsql: Avoid duplicate XIDs at recovery when building initial snapshot
Previous Message Alvaro Herrera 2018-11-30 13:22:22 pgsql: Silence compiler warning

Browse pgsql-hackers by date

  From Date Subject
Next Message Dmitry Dolgov 2018-11-30 14:08:39 Re: [HACKERS] Custom compression methods
Previous Message Dmitry Dolgov 2018-11-30 13:51:27 Re: [HACKERS] Can ICU be used for a database's default sort order?