RE: [PATCH] Speedup truncates of relation forks

From: "Jamison, Kirk" <k(dot)jamison(at)jp(dot)fujitsu(dot)com>
To: 'Alvaro Herrera from 2ndQuadrant' <alvherre(at)alvh(dot)no-ip(dot)org>
Cc: 'Fujii Masao' <masao(dot)fujii(at)gmail(dot)com>, Thomas Munro <thomas(dot)munro(at)gmail(dot)com>, Masahiko Sawada <sawada(dot)mshk(at)gmail(dot)com>, "Tsunakawa, Takayuki" <tsunakawa(dot)takay(at)jp(dot)fujitsu(dot)com>, Tomas Vondra <tomas(dot)vondra(at)2ndquadrant(dot)com>, "pgsql-hackers(at)postgresql(dot)org" <pgsql-hackers(at)postgresql(dot)org>, Adrien Nayrat <adrien(dot)nayrat(at)anayrat(dot)info>
Subject: RE: [PATCH] Speedup truncates of relation forks
Date: 2019-09-09 06:52:03
Message-ID: D09B13F772D2274BB348A310EE3027C656BFED@g01jpexmbkw24
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

On Friday, September 6, 2019 11:51 PM (GMT+9), Alvaro Herrera wrote:

Hi Alvaro,
Thank you very much for the review!

> On 2019-Sep-05, Jamison, Kirk wrote:
> > I also mentioned it from my first post if we can just remove this dead code.
> > If not, it would require to modify the function because it would also
> > need nforks as input argument when calling DropRelFileNodeBuffers. I
> > kept my changes in the latest patch.
> > So should I remove the function now or keep my changes?
> Please add a preliminary patch that removes the function. Dead code is good,
> as long as it is gone. We can get it pushed ahead of the rest of this.

Alright. I've attached a separate patch removing the smgrdounlinkfork.

> What does it mean to "mark" a dirty page in FSM? We don't have the concept
> of marking pages as far as I know (and I don't see that the patch does any
> sort of mark). Do you mean to find where it is and return its block number?

Yes. "Mark" is probably not a proper way to describe it, so I temporarily
changed it to "locate" and renamed the function to FreeSpaceMapLocateBlock().
If anyone could suggest a more appropriate name, that'd be appreciated.

> If so, I wonder how this handles concurrent table extension: are we keeping
> some sort of lock that prevents it?
> (... or would we lose any newly added pages that receive tuples while this
> truncation is ongoing?)

I moved the the description about acquiring AccessExclusiveLock
from FreeSpaceMapLocateBlock() and visibilitymap_truncate_prepare() to the
smgrtruncate description instead.
IIUC, in lazy_truncate_heap() we still acquire AccessExclusiveLock for the relation
before calling RelationTruncate(), which then calls smgrtruncate().
While holding the exclusive lock, the following are also called to check
if rel has not extended and verify that end pages contain no tuples while
we were vacuuming (with non-exclusive lock).
new_rel_pages = RelationGetNumberOfBlocks(onerel);
new_rel_pages = count_nondeletable_pages(onerel, vacrelstats);
I assume that the above would update the correct number of pages.
We then release the exclusive lock as soon as we have truncated the pages.

> I think the new API of smgrtruncate() is fairly confusing. Would it be better
> to define a new struct { ForkNum forknum; BlockNumber blkno; } and pass an
> array of those?

This is for readbility, right? However, I think there's no need to define a
new structure for it, so I kept my changes.
smgrtruncate(SMgrRelation reln, ForkNumber *forknum, int nforks, BlockNumber *nblocks).
I also declared *forkNum and nforks next to each other as suggested by Sawada-san.

What do you think about these changes?

Kirk Jamison

Attachment Content-Type Size
v1-0001-Remove-deadcode-smgrdounlinkfork.patch application/octet-stream 3.5 KB
v7-0001-Speedup-truncates-of-relation-forks.patch application/octet-stream 18.9 KB

In response to


Browse pgsql-hackers by date

  From Date Subject
Next Message Amit Kapila 2019-09-09 07:20:26 Re: [HACKERS] PATCH: Batch/pipelining support for libpq
Previous Message Michael Paquier 2019-09-09 05:28:14 Re: refactoring - share str2*int64 functions