Re: Avoiding smgrimmedsync() during nbtree index builds

From: Justin Pryzby <pryzby(at)telsasoft(dot)com>
To: Melanie Plageman <melanieplageman(at)gmail(dot)com>
Cc: Andres Freund <andres(at)anarazel(dot)de>, Heikki Linnakangas <hlinnaka(at)iki(dot)fi>, pgsql-hackers(at)postgresql(dot)org, Peter Geoghegan <pg(at)bowt(dot)ie>, Thomas Munro <thomas(dot)munro(at)gmail(dot)com>
Subject: Re: Avoiding smgrimmedsync() during nbtree index builds
Date: 2022-01-16 20:25:59
Message-ID: 20220116202559.GW14051@telsasoft.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Thu, Jan 13, 2022 at 09:52:55AM -0600, Justin Pryzby wrote:
> This is failing on windows CI when I use initdb --data-checksums, as attached.
>
> https://cirrus-ci.com/task/5612464120266752
> https://api.cirrus-ci.com/v1/artifact/task/5612464120266752/regress_diffs/src/test/regress/regression.diffs
>
> +++ c:/cirrus/src/test/regress/results/bitmapops.out 2022-01-13 00:47:46.704621200 +0000
> ..
> +ERROR: could not read block 0 in file "base/16384/30310": read only 0 of 8192 bytes

The failure isn't consistent, so I double checked my report. I have some more
details:

The problem occurs maybe only ~25% of the time.

The issue is in the 0001 patch.

data-checksums isn't necessary to hit the issue.

errlocation says: LOCATION: mdread, md.c:686 (the only place the error
exists)

With Andres' windows crash patch, I obtained a backtrace - attached.
https://cirrus-ci.com/task/5978171861368832
https://api.cirrus-ci.com/v1/artifact/task/5978171861368832/crashlog/crashlog-postgres.exe_0fa8_2022-01-16_02-54-35-291.txt

Maybe its a race condition or synchronization problem that nowhere else tends
to hit.

Separate from this issue, I wonder if it'd be useful to write a DEBUG log
showing when btree uses shared_buffers vs fsync. And a regression test which
first SETs client_min_messages=debug to capture the debug log to demonstrate
when/that new code path is being hit. I'm not sure if that would be good to
merge, but it may be useful for now.

--
Justin

Attachment Content-Type Size
crashlog-postgres.exe_0fa8_2022-01-16_02-54-35-291.txt text/plain 18.2 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Noah Misch 2022-01-16 21:02:41 Re: XLogReadRecord() error in XlogReadTwoPhaseData()
Previous Message Tom Lane 2022-01-16 18:30:08 Re: Isn't wait_for_catchup slightly broken?