pgsql: Increase width of RelFileNumbers from 32 bits to 56 bits.

From: Robert Haas <rhaas(at)postgresql(dot)org>
To: pgsql-committers(at)lists(dot)postgresql(dot)org
Subject: pgsql: Increase width of RelFileNumbers from 32 bits to 56 bits.
Date: 2022-09-27 17:32:35
Message-ID: E1odERe-002H6A-LM@gemulon.postgresql.org
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-committers pgsql-hackers

Increase width of RelFileNumbers from 32 bits to 56 bits.

RelFileNumbers are now assigned using a separate counter, instead of
being assigned from the OID counter. This counter never wraps around:
if all 2^56 possible RelFileNumbers are used, an internal error
occurs. As the cluster is limited to 2^64 total bytes of WAL, this
limitation should not cause a problem in practice.

If the counter were 64 bits wide rather than 56 bits wide, we would
need to increase the width of the BufferTag, which might adversely
impact buffer lookup performance. Also, this lets us use bigint for
pg_class.relfilenode and other places where these values are exposed
at the SQL level without worrying about overflow.

This should remove the need to keep "tombstone" files around until
the next checkpoint when relations are removed. We do that to keep
RelFileNumbers from being recycled, but now that won't happen
anyway. However, this patch doesn't actually change anything in
this area; it just makes it possible for a future patch to do so.

Dilip Kumar, based on an idea from Andres Freund, who also reviewed
some earlier versions of the patch. Further review and some
wordsmithing by me. Also reviewed at various points by Ashutosh
Sharma, Vignesh C, Amul Sul, Álvaro Herrera, and Tom Lane.

Discussion: http://postgr.es/m/CA+Tgmobp7+7kmi4gkq7Y+4AM9fTvL+O1oQ4-5gFTT+6Ng-dQ=g@mail.gmail.com

Branch
------
master

Details
-------
https://git.postgresql.org/pg/commitdiff/05d4cbf9b6ba708858984b01ca0fc56d59d4ec7c

Modified Files
--------------
contrib/pg_buffercache/Makefile | 4 +-
.../pg_buffercache/pg_buffercache--1.3--1.4.sql | 30 +++
contrib/pg_buffercache/pg_buffercache.control | 2 +-
contrib/pg_buffercache/pg_buffercache_pages.c | 39 +++-
contrib/pg_prewarm/autoprewarm.c | 4 +-
contrib/pg_walinspect/expected/pg_walinspect.out | 4 +-
contrib/pg_walinspect/sql/pg_walinspect.sql | 4 +-
doc/src/sgml/catalogs.sgml | 2 +-
doc/src/sgml/func.sgml | 5 +
doc/src/sgml/pgbuffercache.sgml | 2 +-
doc/src/sgml/storage.sgml | 11 +-
src/backend/access/gin/ginxlog.c | 2 +-
src/backend/access/rmgrdesc/gistdesc.c | 2 +-
src/backend/access/rmgrdesc/heapdesc.c | 2 +-
src/backend/access/rmgrdesc/nbtdesc.c | 2 +-
src/backend/access/rmgrdesc/seqdesc.c | 2 +-
src/backend/access/rmgrdesc/xlogdesc.c | 21 ++-
src/backend/access/transam/README | 5 +-
src/backend/access/transam/varsup.c | 209 ++++++++++++++++++++-
src/backend/access/transam/xlog.c | 60 ++++++
src/backend/access/transam/xlogprefetcher.c | 14 +-
src/backend/access/transam/xlogrecovery.c | 6 +-
src/backend/access/transam/xlogutils.c | 6 +-
src/backend/backup/basebackup.c | 2 +-
src/backend/catalog/catalog.c | 95 ----------
src/backend/catalog/heap.c | 27 +--
src/backend/catalog/index.c | 11 +-
src/backend/catalog/storage.c | 8 +
src/backend/commands/tablecmds.c | 12 +-
src/backend/commands/tablespace.c | 2 +-
src/backend/nodes/gen_node_support.pl | 4 +-
src/backend/replication/logical/decode.c | 1 +
src/backend/replication/logical/reorderbuffer.c | 2 +-
src/backend/storage/file/reinit.c | 28 +--
src/backend/storage/freespace/fsmpage.c | 2 +-
src/backend/storage/lmgr/lwlocknames.txt | 1 +
src/backend/storage/smgr/md.c | 7 +
src/backend/storage/smgr/smgr.c | 2 +-
src/backend/utils/adt/dbsize.c | 7 +-
src/backend/utils/adt/pg_upgrade_support.c | 13 +-
src/backend/utils/cache/relcache.c | 2 +-
src/backend/utils/cache/relfilenumbermap.c | 4 +-
src/backend/utils/misc/pg_controldata.c | 9 +-
src/bin/pg_checksums/pg_checksums.c | 4 +-
src/bin/pg_controldata/pg_controldata.c | 2 +
src/bin/pg_dump/pg_dump.c | 26 +--
src/bin/pg_rewind/filemap.c | 6 +-
src/bin/pg_upgrade/info.c | 3 +-
src/bin/pg_upgrade/pg_upgrade.c | 6 +-
src/bin/pg_upgrade/relfilenumber.c | 4 +-
src/bin/pg_waldump/pg_waldump.c | 2 +-
src/bin/scripts/t/090_reindexdb.pl | 2 +-
src/common/relpath.c | 20 +-
src/fe_utils/option_utils.c | 40 ++++
src/include/access/transam.h | 40 ++++
src/include/access/xlog.h | 1 +
src/include/catalog/catalog.h | 3 -
src/include/catalog/catversion.h | 2 +-
src/include/catalog/pg_class.h | 16 +-
src/include/catalog/pg_control.h | 2 +
src/include/catalog/pg_proc.dat | 10 +-
src/include/common/relpath.h | 7 +-
src/include/fe_utils/option_utils.h | 2 +
src/include/storage/buf_internals.h | 55 +++++-
src/include/storage/relfilelocator.h | 12 +-
src/test/regress/expected/alter_table.out | 24 ++-
src/test/regress/expected/fast_default.out | 4 +-
src/test/regress/expected/oidjoins.out | 2 +-
src/test/regress/sql/alter_table.sql | 8 +-
src/test/regress/sql/fast_default.sql | 4 +-
70 files changed, 694 insertions(+), 290 deletions(-)

Responses

Browse pgsql-committers by date

  From Date Subject
Next Message Robert Haas 2022-09-27 17:42:21 pgsql: Include common/relpath.h in utils/relfilenumbermap.h
Previous Message Robert Haas 2022-09-27 16:11:27 pgsql: Move RelFileNumber declarations to common/relpath.h.

Browse pgsql-hackers by date

  From Date Subject
Next Message Robert Haas 2022-09-27 17:54:17 Re: making relfilenodes 56 bits
Previous Message Ashwin Agrawal 2022-09-27 16:31:24 Re: Backends stalled in 'startup' state