Re: Avoiding smgrimmedsync() during nbtree index builds

From: Melanie Plageman <melanieplageman(at)gmail(dot)com>
To: Andres Freund <andres(at)anarazel(dot)de>
Cc: Heikki Linnakangas <hlinnaka(at)iki(dot)fi>, Pg Hackers <pgsql-hackers(at)postgresql(dot)org>, Peter Geoghegan <pg(at)bowt(dot)ie>, Thomas Munro <thomas(dot)munro(at)gmail(dot)com>
Subject: Re: Avoiding smgrimmedsync() during nbtree index builds
Date: 2021-11-19 20:11:57
Message-ID: CAAKRu_ZmkSg1_ZuvA_jK1uda9T1Jj4BrkBGTOFWq9ys9THtC3A@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Mon, May 3, 2021 at 5:24 PM Melanie Plageman
<melanieplageman(at)gmail(dot)com> wrote:
>
> So, I've written a patch which avoids doing the immediate fsync for
> index builds either by using shared buffers or by queueing sync requests
> for the checkpointer. If a checkpoint starts during the index build and
> the backend is not using shared buffers for the index build, it will
> need to do the fsync.

I've attached a rebased version of the patch (old patch doesn't apply).

With the patch applied (compiled at O2), creating twenty empty tables in
a transaction with a text column and an index on another column (like in
the attached SQL [make a test_idx schema first]) results in a fairly
consistent 15-30% speedup on my laptop (timings still in tens of ms -
avg 50 ms to avg 65 ms so run variation affects the % a lot).
Reducing the number of fsync calls from 40 to 1 was what likely causes
this difference.

- Melanie

Attachment Content-Type Size
v2-0001-Index-build-avoids-immed-fsync.patch application/octet-stream 18.5 KB
index_create.sql application/octet-stream 1.4 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Thomas Munro 2021-11-19 20:18:37 Re: [PATCH] Make ENOSPC not fatal in semaphore creation
Previous Message Marcos Pegoraro 2021-11-19 20:09:36 Re: update with no changes