Re: pgsql: Remove the restriction that the relmap must be 512 bytes.

From: Michael Paquier <michael(at)paquier(dot)xyz>
To: Robert Haas <rhaas(at)postgresql(dot)org>
Cc: pgsql-committers(at)lists(dot)postgresql(dot)org
Subject: Re: pgsql: Remove the restriction that the relmap must be 512 bytes.
Date: 2022-07-27 04:34:48
Message-ID: YuDAaGhZMOIL1FGs@paquier.xyz
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-committers pgsql-hackers

Hi Robert,

On Tue, Jul 26, 2022 at 07:10:22PM +0000, Robert Haas wrote:
> Remove the restriction that the relmap must be 512 bytes.
>
> Instead of relying on the ability to atomically overwrite the
> entire relmap file in one shot, write a new one and durably
> rename it into place. Removing the struct padding and the
> calculation showing why the map is exactly 512 bytes, and change
> the maximum number of entries to a nearby round number.
>
> Patch by me, reviewed by Andres Freund and Dilip Kumar.
>
> Discussion: http://postgr.es/m/CA+TgmoZq5%3DLWDK7kHaUbmWXxcaTuw_QwafgG9dr-BaPym_U8WQ%40mail.gmail.com
> Discussion: http://postgr.es/m/CAFiTN-ttOXLX75k_WzRo9ar=VvxFhrHi+rJxns997F+yvkm==A@mail.gmail.com

The CI on Windows is blowing up here and there after something that
looks to come from this commit, as of this backtrace:
00000000`007fe300 00000001`405c62dd postgres!errfinish(
char * filename = 0x00000001`40bf1513 "fd.c",
int lineno = 0n756,
char * funcname = 0x00000001`40bf14e0 "durable_rename")+0x41b
[c:\cirrus\src\backend\utils\error\elog.c @ 683]
00000000`007fe360 00000001`4081647b postgres!durable_rename(
char * oldfile = 0x00000000`007fe430 "base/16384/pg_filenode.map.tmp",
char * newfile = 0x00000000`007fe830 "base/16384/pg_filenode.map",
int elevel = 0n21)+0x22d [c:\cirrus\src\backend\storage\file\fd.c @
753]
00000000`007fe3b0 00000001`408166c9 postgres!write_relmap_file(
struct RelMapFile * newmap = 0x00000000`007fecb0,
bool write_wal = true,
bool send_sinval = true,
bool preserve_files = true,
unsigned int dbid = 0x4000,
unsigned int tsid = 0x67f,
char * dbpath = 0x00000000`0090b1c0 "base/16384")+0x38b
[c:\cirrus\src\backend\utils\cache\relmapper.c @ 971]

Here is one of them, kicked by the CF bot, but I have seen similar
crashes with some of my own things (see the txt file in crashlog, in a
manual VACUUM):
https://cirrus-ci.com/task/5240408958566400

Thanks,
--
Michael

In response to

Responses

Browse pgsql-committers by date

  From Date Subject
Next Message Michael Paquier 2022-07-27 04:37:00 pgsql: Add overflow protection for block-related data in WAL records
Previous Message Andres Freund 2022-07-26 20:03:54 Re: Unstable tests for recovery conflict handling

Browse pgsql-hackers by date

  From Date Subject
Next Message Amit Kapila 2022-07-27 04:36:14 Re: Perform streaming logical transactions by background workers and parallel apply
Previous Message Peter Smith 2022-07-27 03:37:06 Re: Perform streaming logical transactions by background workers and parallel apply