Re: Unlogged bloom indexes break after crash recovery

From: Narek Galstyan <narek(dot)galstyan(at)enterprisedb(dot)com>
To: Heikki Linnakangas <hlinnaka(at)iki(dot)fi>, pgsql-hackers(at)postgresql(dot)org
Subject: Re: Unlogged bloom indexes break after crash recovery
Date: 2026-11-04 21:30:30
Message-ID: CAHsn6_yrpwjna4nGCcWYdAe5Uy-d6RqeJwKZaxSU9aLqBv4dHQ@mail.gmail.com
Views: Whole Thread | Raw Message | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

I agree your patch is better!

Two suggestions, attached as patches, to be fixup-ed into yours:

1. Flush init fork pages in generic_redo to avoid the kind of corruption
reported in 8e684ce11dd, where shared buffers would have an updated version
of init fork, and the standby would copy the stale on-disk version for
their main fork.
2. Update generic-wal docs, which currently claim Generic Xlog does not
log anything for unlogged tables.

And a note:

Extensions in the wild (e.g. pgvector here
<https://github.com/pgvector/pgvector/blob/b0665dca96f0a186d438adbd1c35945d71da5490/src/ivfbuild.c#L1047-L1049>
and
a few private ones I know) assume GenericXLog does not log the init fork
and they manually log the init fork with logic similar to the linked
pgvector ivfflat. This means that for pgvector ivfflat, the potentially
substantial ivfflat init fork (~8MB for lists=1000, dim = 1536) will be
WAL-logged twice. This is pretty harmless. Still, probably worth addressing
in pgvector. Notably, pgvector hnsw does not have the double logging
problem because it does not use GenericXLog for the initial build.

In any case I think should not be a blocker for your patch.

Narek
--

On Wed, Nov 4, 2026 at 1:47 PM Heikki Linnakangas <hlinnaka(at)iki(dot)fi> wrote:

> On 03/11/2026 03:06, Narek Galstyan wrote:
> > Hi all,
> >
> > I noticed that bloom indexes are not properly WAL-looged on unlogged
> > tables and attached is my attempt at fixing it
> >
> > Since commit ccadf73163c <https://github.com/postgres/postgres/commit/
> > ccadf73163ca88bdaa74b8223d4dde05d17f550b> (discussion: https://
> > www.postgresql.org/message-id/6e5bbc08-cdfc-
> > b2b3-9e23-1a914b9850a9(at)iki(dot)fi <https://www.postgresql.org/message-
> > id/6e5bbc08-cdfc-b2b3-9e23-1a914b9850a9(at)iki(dot)fi>), `contrib/bloom` index
> > AM's blbuildempty() wrote the metapage of the init fork through generic
> > WAL, which emits no WAL record for an unlogged relation.
> >
> > The init fork must be WAL-logged as it is used to recreate main fork of
> > an unlogged relation after a crash.
> >
> > Without this fix, after a crash, an unlogged and bloom-indexed relation
> > gets a zero-ed out index meta page. As a result, insertions and index
> > scans fail with "Relation is not a bloom index" or "could not read
> > blocks 0..0", until an index rebuild.
>
> Yep, my fault with ccadf73163c.
>
> Your proposed fix fixes it, but only for contrib/bloom. Any other
> indexam using generic WAL would also need to know that they cannot use
> generic WAL for the init fork. How about the attached, to teach
> GenericXLogFinish() to WAL log the init fork, instead?
>
> - Heikki
>

Attachment Content-Type Size
0002-Document-that-generic-WAL-logs-init-fork-pages.patch application/octet-stream 1.5 KB
0001-Flush-init-fork-pages-after-generic-WAL-delta-redo.patch application/octet-stream 1.7 KB

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Jelte Fennema-Nio 2026-11-04 21:38:37 Re: Don't use the deprecated and insecure PQcancel in our frontend tools anymore
Previous Message Jelte Fennema-Nio 2026-11-04 21:09:55 Re: Bump soft open file limit (RLIMIT_NOFILE) to hard limit on startup