Re: Changing shared_buffers without restart

From: Yuhang Qiu <iamqyh(at)gmail(dot)com>
To: Palak Chaturvedi <chaturvedipalak1911(at)gmail(dot)com>
Cc: Ashutosh Bapat <ashutosh(dot)bapat(dot)oss(at)gmail(dot)com>, pgsql-hackers(at)postgresql(dot)org, Heikki Linnakangas <heikki(dot)linnakangas(at)databricks(dot)com>, Haoyu Huang <haoyu(dot)huang(at)databricks(dot)com>, Tomas Vondra <tomas(at)vondra(dot)me>, Peter Eisentraut <peter(at)eisentraut(dot)org>, Thomas Munro <thomas(dot)munro(at)gmail(dot)com>, Dmitry Dolgov <9erthalion6(at)gmail(dot)com>, Robert Haas <robertmhaas(at)gmail(dot)com>, Andres Freund <andres(at)anarazel(dot)de>, Jakub Wartak <jakub(dot)wartak(at)enterprisedb(dot)com>
Subject: Re: Changing shared_buffers without restart
Date: 2026-08-27 09:01:35
Message-ID: 78DD860A-DD0E-4B70-A0CA-EE5CCA3E0E60@gmail.com
Views: Whole Thread | Raw Message | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hi Palak,

0010 and 0011 both look good to me.

I missed the existing `HAVE_RESIZABLE_SHMEM` guard. Ignore that comment.

> It seems whether this actually PANICs depends on the kernel. On my box
> (Linux 6.17, huge_pages=off) those over-wide MADV_REMOVE calls all
> return 0, so I don't hit the PANIC locally.

I reproduced it on Linux 5.10 with `huge_pages=off`. The code calls
`MADV_REMOVE` from the new end to `maximum_size`, crossing from the RW
area into the existing `PROT_NONE` tail. The call returns EACCES and the
resize PANICs.

I tested this fix:
```c
char *current_end = (char *) TYPEALIGN(page_size,
(char *) result->location + result->size);
char *reserved_end = (char *) TYPEALIGN_DOWN(page_size,
(char *) result->location + result->maximum_size);
char *max_end = Min(current_end, reserved_end);
```

The first bound stops at the current allocation; the second preserves the
last page when it is shared with the next structure. Linux 6.7 changed
the check from `VM_WRITE` to `VM_MAYWRITE` [1], which explains why the
over-wide call succeeds on 6.17.

> I did try to reproduce it, including widening the window artificially,
> but couldn't catch it. It seems to need AIO in the mix.

The race is:
1. A backend takes a buffer above the shrink target before processing the
new-allocation barrier.
2. It publishes the mapping entry (`BM_TAG_VALID` is set), while `BM_VALID`
is still clear.
3. It processes the barrier before the read completes. This can also
happen on the synchronous path between `StartReadBuffers()` and
`WaitReadBuffers()`.
4. `EvictExtraBuffers()` sees `BM_VALID` clear and skips the buffer, leaving
the mapping entry behind after the shrink.

To force it, hold the read completion after step 2, let the backend process
the barrier, and then start the shrink.

Both the precheck and the locked assertion should use `BM_TAG_VALID`, and
the latter should check the state returned by `LockBufHdr()`. The shrink
will then roll back instead of leaving an orphaned entry.

Running `autoconf` will update `configure` with new `configure.ac`.

> Could you share the error you saw, or how to reproduce it?
I tested it with `huge_pages = off` on Linux 5.10. The error is EACCES which
I explained above.

Best regards,
Yuhang Qiu

[1] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=e8e17ee90eaf650c855adb0a3e5e965fd6692ff1

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Peter Eisentraut 2026-08-27 09:03:56 Re: [PATCH] SQL/PGQ: Fix inferred property graph keys with INCLUDE columns
Previous Message Heikki Linnakangas 2026-08-27 08:58:41 Re: pg_upgrade silently truncates nextMultiOffset to 32 bits