Re: XID formatting and SLRU refactorings (was: Add 64-bit XIDs into PostgreSQL 15)

From: Maxim Orlov <orlovmg(at)gmail(dot)com>
To: Andrey Borodin <amborodin86(at)gmail(dot)com>
Cc: Aleksander Alekseev <aleksander(at)timescale(dot)com>, Postgres hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>, Andres Freund <andres(at)anarazel(dot)de>, Peter Geoghegan <pg(at)bowt(dot)ie>, Michael Paquier <michael(at)paquier(dot)xyz>, Ian Lawrence Barwick <barwick(at)gmail(dot)com>, Simon Riggs <simon(dot)riggs(at)enterprisedb(dot)com>, Hamid Akhtar <hamid(dot)akhtar(at)percona(dot)com>, Pavel Borisov <pashkin(dot)elfe(at)gmail(dot)com>, Justin Pryzby <pryzby(at)telsasoft(dot)com>, Peter Eisentraut <peter(dot)eisentraut(at)enterprisedb(dot)com>, Japin Li <japinli(at)hotmail(dot)com>, Alexander Korotkov <aekorotkov(at)gmail(dot)com>, Kyotaro Horiguchi <horikyota(dot)ntt(at)gmail(dot)com>, Ilya Anfimov <ilan(at)tzirechnoy(dot)com>
Subject: Re: XID formatting and SLRU refactorings (was: Add 64-bit XIDs into PostgreSQL 15)
Date: 2023-01-09 13:29:01
Message-ID: CACG=ezZsCaAEPJKra_cbvV9_2YG5HpBBFuSDM11qPEOwShS73g@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

>
> Do I get it right that the proposed v51 patchset only changes the SLRU
> filenames and type of pageno representation? Is SLRU wraparound still
> exactly there after 0xFFFFFFFF byte?
>
After applying the whole patch set, SLRU will become 64–bit without a
wraparound. Thus, no wraparound
should be there.

0001 - should make SLRU internally 64–bit, no effects from "outside"
0002 - should make SLRU callers 64–bit, SLRU segment files naming are
changed
0003 - make upgrade from previous versions feasible

> Also, I do not understand what is the reason for splitting 1st and 2nd
> steps. Certainly, there must be some logic behind it, but I just can't
> grasp it...
>
As we did discuss somewhere in the beginning of the discussion, we try to
make every commit as independent as possible.
Thus, it is much easier to review and commit. I see no problem to meld
these commits into one, if consensus will be reached.

> And the purpose of the 3rd step with pg_upgrade changes is a complete
> mystery for me. Please excuse my incompetence in the topic, but maybe
> some commit message or comments would help. What kind of xact segments
> conversion we do? Why is it only necessary for xacts, but not other
> SLRUs?
>
The purpose of the third patch is to make upgrade feasible. Since we've
change pg_xact files naming,
Postgres could not read status of "old" transactions from "old" pg_xact
files. So, we have to convert those files.
The major problem here is that we must handle possible segment wraparound
(in "old" cluster). The whole idea
for an upgrade is to read SLRU pages for pg_xact one by one and write it in
a "new" filename.

Maybe, It's just a little bit complicated, since the algorithm is intended
to deal with different SLRU pages per segment
in "new" and "old" clusters. But, on the other hand, it is already created
in original patch set of 64–bit XIDs and will be useful
in the future. AFAICS, arguably, any variant of 64–bit XIDs should lead to
increase of an amount of SLRU pages per segment.

And as for other SLRUs, they cannot survive pg_upgrade mostly by the fact,
that cluster must be stopped upon upgrade.
Thus, no conversion needed.

--
Best regards,
Maxim Orlov.

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Aleksander Alekseev 2023-01-09 13:30:47 Re: XID formatting and SLRU refactorings (was: Add 64-bit XIDs into PostgreSQL 15)
Previous Message Richard Guo 2023-01-09 13:28:16 Re: Allow DISTINCT to use Incremental Sort