Re: SLRU optimization - configurable buffer pool and partitioning the SLRU lock

From: Dilip Kumar <dilipbalaut(at)gmail(dot)com>
To: Amul Sul <sulamul(at)gmail(dot)com>
Cc: Alvaro Herrera <alvherre(at)alvh(dot)no-ip(dot)org>, tender wang <tndrwang(at)gmail(dot)com>, PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>, Andrey Borodin <x4mmm(at)yandex-team(dot)ru>
Subject: Re: SLRU optimization - configurable buffer pool and partitioning the SLRU lock
Date: 2023-12-15 05:15:36
Message-ID: CAFiTN-t+W+uuSy4FYUQSEfA0raVH1f3Jywb6D394QhR-zpavWQ@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Thu, Dec 14, 2023 at 1:53 PM Dilip Kumar <dilipbalaut(at)gmail(dot)com> wrote:
>
> On Thu, Dec 14, 2023 at 8:43 AM Amul Sul <sulamul(at)gmail(dot)com> wrote:
> >
> > On Mon, Dec 11, 2023 at 10:42 AM Dilip Kumar <dilipbalaut(at)gmail(dot)com> wrote:
> >>
> >> On Thu, Nov 30, 2023 at 3:30 PM Dilip Kumar <dilipbalaut(at)gmail(dot)com> wrote:
> >> >
> >> > On Wed, Nov 29, 2023 at 4:58 PM Dilip Kumar <dilipbalaut(at)gmail(dot)com> wrote:
> >>
> >> Here is the updated patch based on some comments by tender wang (those
> >> comments were sent only to me)
> >
> >
> > few nitpicks:
> >
> > +
> > + /*
> > + * Mask for slotno banks, considering 1GB SLRU buffer pool size and the
> > + * SLRU_BANK_SIZE bits16 should be sufficient for the bank mask.
> > + */
> > + bits16 bank_mask;
> > } SlruCtlData;
> >
> > ...
> > ...
> >
> > + int bankno = pageno & ctl->bank_mask;
> >
> > I am a bit uncomfortable seeing it as a mask, why can't it be simply a number
> > of banks (num_banks) and get the bank number through modulus op (pageno %
> > num_banks) instead of bitwise & operation (pageno & ctl->bank_mask) which is a
> > bit difficult to read compared to modulus op which is quite simple,
> > straightforward and much common practice in hashing.
> >
> > Are there any advantages of using & over % ?
>
> I am not sure either but since this change in 0002 is by Andrey, I
> will let him comment on this before we change or take any decision.
>
> > Also, a few places in 0002 and 0003 patch, need the bank number, it is better
> > to have a macro for that.
> > ---
> >
> > extern bool SlruScanDirCbDeleteAll(SlruCtl ctl, char *filename, int64 segpage,
> > void *data);
> > -
> > +extern bool check_slru_buffers(const char *name, int *newval);
> > #endif /* SLRU_H */
> >
> >
> > Add an empty line after the declaration, in 0002 patch.
> > ---
> >
> > -TransactionIdSetStatusBit(TransactionId xid, XidStatus status, XLogRecPtr lsn, int slotno)
> > +TransactionIdSetStatusBit(TransactionId xid, XidStatus status, XLogRecPtr lsn,
> > + int slotno)
> >
> > Unrelated change for 0003 patch.
>
> Fixed
>
> Thanks for your review, PFA updated version.
>
> I have added @Amit Kapila to the list to view his opinion about
> whether anything can break in the clog group update with our changes
> of bank-wise SLRU lock.

Updated the comments about group commit safety based on the off-list
suggestion by Alvaro.

--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com

Attachment Content-Type Size
v12-0001-Make-all-SLRU-buffer-sizes-configurable.patch application/octet-stream 23.9 KB
v12-0002-Divide-SLRU-buffers-into-banks.patch application/octet-stream 13.6 KB
v12-0003-Remove-the-centralized-control-lock-and-LRU-coun.patch application/octet-stream 76.7 KB

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message shveta malik 2023-12-15 05:32:23 Re: Synchronizing slots from primary to standby
Previous Message Sutou Kouhei 2023-12-15 04:45:31 Re: Make COPY format extendable: Extract COPY TO format implementations