RE: [Patch] Optimize dropping of relation buffers using dlist

From: "k(dot)jamison(at)fujitsu(dot)com" <k(dot)jamison(at)fujitsu(dot)com>
To: "tsunakawa(dot)takay(at)fujitsu(dot)com" <tsunakawa(dot)takay(at)fujitsu(dot)com>
Cc: 'Amit Kapila' <amit(dot)kapila16(at)gmail(dot)com>, Kyotaro Horiguchi <horikyota(dot)ntt(at)gmail(dot)com>, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>, Andres Freund <andres(at)anarazel(dot)de>, Robert Haas <robertmhaas(at)gmail(dot)com>, Tomas Vondra <tomas(dot)vondra(at)2ndquadrant(dot)com>, pgsql-hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: RE: [Patch] Optimize dropping of relation buffers using dlist
Date: 2020-09-28 08:57:36
Message-ID: OSBPR01MB23416098E23E8B9A7A499AC8EF350@OSBPR01MB2341.jpnprd01.prod.outlook.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Monday, September 28, 2020 5:08 PM, Tsunakawa-san wrote:

> From: Jamison, Kirk/ジャミソン カーク <k(dot)jamison(at)fujitsu(dot)com>
> > Is my understanding above correct?
>
> No. I simply meant DropRelFileNodeBuffers() calls the following function,
> and avoids the optimization if it returns InvalidBlockNumber.
>
>
> BlockNumber
> smgrcachednblocks(SMgrRelation reln, ForkNumber forknum) {
> return reln->smgr_cached_nblocks[forknum];
> }

Thank you for clarifying.

So in the new function, it goes something like:
if (InRecovery)
{
if (reln->smgr_cached_nblocks[forknum] != InvalidBlockNumber)
return reln->smgr_cached_nblocks[forknum];
else
return InvalidBlockNumber;
}

I've revised the patch and added the new function accordingly in the attached file.
I also did not remove the duplicate code from smgrnblocks because Amit-san mentioned
that when the caching for non-recovery cases is implemented, we can use it
for non-recovery cases as well.

Although I am not sure if the way it's written in DropRelFileNodeBuffers is okay.
BlockNumberIsValid(nTotalBlocks)

nTotalBlocks = smgrcachednblocks(smgr_reln, forkNum[j]);
nBlocksToInvalidate = nTotalBlocks - firstDelBlock[j];

if (BlockNumberIsValid(nTotalBlocks) &&
nBlocksToInvalidate < BUF_DROP_FULLSCAN_THRESHOLD)
{
//enter optimization loop
}
else
{
//full scan for each fork
}

Regards,
Kirk Jamison

Attachment Content-Type Size
v17-Optimize-DropRelFileNodeBuffers-during-recovery.patch application/octet-stream 10.1 KB
v1-Prevent-invalidating-blocks-in-smgrextend-during-recovery.patch application/octet-stream 1.1 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Ajin Cherian 2020-09-28 09:01:35 Re: [HACKERS] logical decoding of two-phase transactions
Previous Message Amit Kapila 2020-09-28 08:52:07 Re: Logical replication CPU-bound with TRUNCATE/DROP/CREATE many tables