| From: | Ayush Tiwari <ayushtiwari(dot)slg01(at)gmail(dot)com> |
|---|---|
| To: | Bharath Rupireddy <bharath(dot)rupireddyforpostgres(at)gmail(dot)com> |
| Cc: | Nazir Bilal Yavuz <byavuz81(at)gmail(dot)com>, solai v <solai(dot)cdac(at)gmail(dot)com>, Ian Lawrence Barwick <barwick(at)gmail(dot)com>, PostgreSQL Hackers <pgsql-hackers(at)postgresql(dot)org> |
| Subject: | Re: Add a pg_wal_preallocate() SQL function to eagerly create future WAL segments |
| Date: | 2026-11-20 17:15:07 |
| Message-ID: | CAJTYsWXWXfEc8bxV6kiFqU=Vgve3N0Sf4f8Vr2ZFbLU4RJJLcg@mail.gmail.com |
| Views: | Whole Thread | Raw Message | Download mbox | Resend email |
| Thread: | |
| Lists: | pgsql-hackers |
On Fri, 16 Oct 2026 at 19:28, Ayush Tiwari <ayushtiwari(dot)slg01(at)gmail(dot)com> wrote:
>
> Hi,
>
> On Sun, 11 Oct 2026 at 18:45, Ayush Tiwari <ayushtiwari(dot)slg01(at)gmail(dot)com> wrote:
> > On Fri, 25 Sept 2026 at 20:54, Ayush Tiwari <ayushtiwari(dot)slg01(at)gmail(dot)com> wrote:
> > > On Thu, 27 Aug 2026 at 17:48, Ayush Tiwari <ayushtiwari(dot)slg01(at)gmail(dot)com> wrote:
> > > >
> > > > On Mon, 17 Aug 2026 at 05:08, Bharath Rupireddy <bharath(dot)rupireddyforpostgres(at)gmail(dot)com> wrote:
> > > >>
> > > >> Hi,
> > > >>
> > > >> On Sun, Aug 16, 2026 at 8:09 AM Ayush Tiwari
> > > >> <ayushtiwari(dot)slg01(at)gmail(dot)com> wrote:
> > > >> >
> > > >> > Upon thinking more, I've removed the force option for now.
> > > >> > We can revisit/add it if and when need be, adding something like that
> > > >> > is easier than removing it later.
> > > >> >
> > > >> > v6 attached. Thoughts?
> > > >>
> > > >> Thanks for the new patch. It mostly looks good to me with a few minor
> > > >> comments (at the end). I marked the CF entry as RfC
> > > >> (https://commitfest.postgresql.org/patch/7052/) pgindent, make check,
> > > >> make check-world with both debug and release builds run fine. I did
> > > >> the following two tests to ensure it works as expected.
> > > >>
> > > >> 1/ Concurrent WAL file creation. A TAP test with an injection point
> > > >> where one backend is paused right before preallocating a segment while
> > > >> another creates it first with an INSERT query. The backend that
> > > >> preallocates creates fewer files than requested, returns without
> > > >> error, and the file ends up created only once.
> > > >>
> > > >> 2/ Preallocated 100 WAL files on the primary, then failed over to the
> > > >> standby and ran pg_rewind on the old primary to bring it back as a
> > > >> standby of the newly promoted primary. pg_rewind removed them [1]
> > > >> since they are beyond the divergence point.
> > > >>
> > > >> A few minor comments:
> > > >>
> > > >> 1/
> > > >> + nsegsadded = PreallocNXlogFiles(nsegs);
> > > >> +
> > > >> + PG_RETURN_INT64(nsegsadded);
> > > >>
> > > >> It might be good to return the WAL file names that were created for
> > > >> the record, but I don't see any strong use for that now. We can add it
> > > >> in future if needed.
> > > >>
> > > >> 2/
> > > >> + if (PG_ARGISNULL(0))
> > > >> + bytes = (int64) min_wal_size_mb * 1024 * 1024;
> > > >> + else
> > > >> + {
> > > >> + bytes = PG_GETARG_INT64(0);
> > > >> +
> > > >> + if (bytes < 0)
> > > >> + ereport(ERROR,
> > > >> + (errcode(ERRCODE_NUMERIC_VALUE_OUT_OF_RANGE),
> > > >> + errmsg("number of bytes to preallocate must not be negative")));
> > > >> + }
> > > >> +
> > > >>
> > > >> I'm fine with this, however, I will leave it to others and the
> > > >> committer to decide whether to auto-calculate from min_wal_size when
> > > >> user input is NULL.
> > > >>
> > > >> [1]
> > > >> pg_rewind: pg_wal/00000001000000000000006F (REMOVE)
> > > >> pg_rewind: pg_wal/00000001000000000000006E (REMOVE)
> > > >> ...
> > > >> ...
> > > >> pg_rewind: pg_wal/00000001000000000000000E (REMOVE)
> > > >> pg_rewind: pg_wal/00000001000000000000000D (REMOVE)
Rebased.
Regards,
Ayush
| Attachment | Content-Type | Size |
|---|---|---|
| v11-0001-Add-pg_wal_preallocate-to-eagerly-create-future-WAL-segments.patch | application/octet-stream | 14.0 KB |
| From | Date | Subject | |
|---|---|---|---|
| Previous Message | Andres Freund | 2026-11-20 17:09:20 | Re: Backpatch for fix simplehash to preserve consistency in case of OOM |