Re: increase size of pg_commit_ts buffers

From: Tomas Vondra <tomas(dot)vondra(at)enterprisedb(dot)com>
To: Thomas Munro <thomas(dot)munro(at)gmail(dot)com>, Andrey Borodin <x4mmm(at)yandex-team(dot)ru>
Cc: Alvaro Herrera <alvherre(at)alvh(dot)no-ip(dot)org>, Pg Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>
Subject: Re: increase size of pg_commit_ts buffers
Date: 2021-11-12 17:39:13
Message-ID: 620512a0-2f5a-bf79-b780-0002aedece0d@enterprisedb.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hi,

I think this is ready to go. I was wondering why it merely doubles the
number of buffers, as described by previous comments. That seems like a
very tiny increase, so considering how much the hardware grew over the
last few years it'd probably fail to help some of the large boxes.

But it turns out that's not what the patch does. The change is this

> - return Min(16, Max(4, NBuffers / 1024));
> + return Min(256, Max(4, NBuffers / 512));

So it does two things - (a) it increases the maximum from 16 to 256 (so
16x), and (b) it doubles the speed how fast we get there. Until now we
add 1 buffer per 1024 shared buffers, and the maximum would be reached
with 128MB. The patch lowers the steps to 512, and the maximum to 1GB.

So this actually increases the number of commit_ts buffers 16x, not 2x.
That seems reasonable, I guess. The increase may be smaller for systems
with less that 1GB shared buffers, but IMO that's a tiny minority of
production systems busy enough for this patch to make a difference.

The other question is of course what overhead could this change have on
workload that does not have issues with commit_ts buffers (i.e. it's
using commit_ts, but would be fine with just the 16 buffers). But my
guess is this is negligible, based on how simple the SLRU code is and my
previous experiments with SLRU.

So +1 to just get this committed, as it is.

regards

--
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Alvaro Herrera 2021-11-12 17:57:57 Re: support for MERGE
Previous Message Justin Pryzby 2021-11-12 17:31:02 Re: Parallel Append can break run-time partition pruning