Re: Buffer locking is special (hints, checksums, AIO writes)

From: Chao Li <li(dot)evan(dot)chao(at)gmail(dot)com>
To: Andres Freund <andres(at)anarazel(dot)de>
Cc: Kirill Reshke <reshkekirill(at)gmail(dot)com>, Heikki Linnakangas <hlinnaka(at)iki(dot)fi>, Melanie Plageman <melanieplageman(at)gmail(dot)com>, Matthias van de Meent <boekewurm+postgres(at)gmail(dot)com>, pgsql-hackers(at)postgresql(dot)org, Thomas Munro <thomas(dot)munro(at)gmail(dot)com>, Noah Misch <noah(at)leadboat(dot)com>, Robert Haas <robertmhaas(at)gmail(dot)com>, Michael Paquier <michael(dot)paquier(at)gmail(dot)com>
Subject: Re: Buffer locking is special (hints, checksums, AIO writes)
Date: 2026-01-15 00:04:21
Message-ID: 58821140-0182-4FF3-9A4F-5B168DDB9243@gmail.com
Views: Whole Thread | Raw Message | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

> On Jan 15, 2026, at 07:37, Andres Freund <andres(at)anarazel(dot)de> wrote:
>
> Hi,
>
> On 2026-01-15 07:20:27 +0800, Chao Li wrote:
>>> On Jan 15, 2026, at 00:30, Andres Freund <andres(at)anarazel(dot)de> wrote:
>>> On 2026-01-14 11:41:19 +0800, Chao Li wrote:
>>>> Basically, code changes in 0003 is straightforward, just a couple of small comments:
>>>>
>>>> 1
>>>> ```
>>>> - * refcounts in buf_internals.h. This limitation could be lifted by using a
>>>> - * 64bit state; but it's unlikely to be worthwhile as 2^18-1 backends exceed
>>>> - * currently realistic configurations. Even if that limitation were removed,
>>>> - * we still could not a) exceed 2^23-1 because inval.c stores the ProcNumber
>>>> - * as a 3-byte signed integer, b) INT_MAX/4 because some places compute
>>>> - * 4*MaxBackends without any overflow check. We check that the configured
>>>> - * number of backends does not exceed MAX_BACKENDS in InitializeMaxBackends().
>>>> + * refcounts in buf_internals.h. This limitation could be lifted, but it's
>>>> ```
>>>>
>>>> Before this patch, there was room for lifting the limitation. With this
>>>> patch, state is 64bit already, but the significant 32bit will be used for
>>>> buffer locking as stated in buf_internals.h, in other words, there is no
>>>> room for lifting the limitation now. If that’s true, then I think we can
>>>> remove the statements about lifting limitation.
>>>
>>> I'm not following - there's plenty space for more bits if we need that:
>>>
>>> * State of the buffer itself (in order):
>>> * - 18 bits refcount
>>> * - 4 bits usage count
>>> * - 12 bits of flags
>>> * - 18 bits share-lock count
>>> * - 1 bit share-exclusive locked
>>> * - 1 bit exclusive locked
>>>
>>> That's 54 bits in total. Which part is in the lower and which in the upper
>>> 32bit isn't relevant for anything afaict?
>>
>> Because I saw the comment in buf_internals.h:
>> ```
>> * NB: A future commit will use a significant portion of the remaining bits to
>> * implement buffer locking as part of the state variable.
>> ```
>> That seems to indicate all the significant 32 bits will be used for buffer locking.
>
> A significant portion != all. As the above excerpt from the comment shows, the
> locking uses 20 bits. We could increase max backends by 5 bits without running
> out of bits (we'd need space both in the refcount bitspace as well as the
> share-lock bitspace).

Make sense. I think I misread the comment.

>
>
>> Also, there is an assert that concretes the impression:
>> ```
>> StaticAssertDecl(BUF_REFCOUNT_BITS + BUF_USAGECOUNT_BITS + BUF_FLAG_BITS == 32,
>> "parts of buffer state space need to equal 32");
>> ```
>
> You can see that being relaxed in the subsequent commit, when we start to use
> more bits.
>

Sure. I plan to review 0003-0005 today. I believe I will get better understanding.

So, 0001 and 0002 LGTM now.

Best regards,
--
Chao Li (Evan)
HighGo Software Co., Ltd.
https://www.highgo.com/

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Amit Langote 2026-01-15 00:12:35 Re: Segmentation fault on proc exit after dshash_find_or_insert
Previous Message Andres Freund 2026-01-15 00:01:38 Re: Adding basic NUMA awareness