Re: Trying to break online checksums with LLMs

From: Daniel Gustafsson <daniel(at)yesql(dot)se>
To: Robert Haas <robertmhaas(at)gmail(dot)com>
Cc: PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>
Subject: Re: Trying to break online checksums with LLMs
Date: 2026-09-10 21:50:33
Message-ID: 3108345D-DC76-4FDA-9F9E-7583AEC81CC3@yesql.se
Views: Whole Thread | Raw Message | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

> On 10 Sep 2026, at 23:35, Robert Haas <robertmhaas(at)gmail(dot)com> wrote:
>
> On Thu, Sep 10, 2026 at 5:08 PM Daniel Gustafsson <daniel(at)yesql(dot)se> wrote:
>> I have a feeling this had been brought up in one of all the postcommit threads,
>> but I fail to find it now (perhaps it was a basebackup and not checksums
>> thread?). The gist of that discussion was that there is somewhere between very
>> little add nothing we can do to protect against orphaned/external files in the
>> datadir when using the catalog as the main source of truth for what we expect
>> it to contain. I wonder if the best we can do is to document exactly how to
>> verify the state of the page when getting a backup page verification failure?
>
> I think that it is fine to disclaim external files -- if you add
> random files to the data directory, stuff may break, and that's sad,
> but oh well. But I think orphaned files are a completely different
> ball game. That's just a normal outcome of running PostgreSQL for a
> period of time under real-world conditions, and to my knowledge this
> would be the first feature that doesn't work with them. I'd like to
> hear what others think, but my first reaction is to think that's
> probably not OK. If we don't remove the files automatically, and don't
> even provide any tools to remove the files manually, I think it
> follows logically that everything we ship has to cope with their
> possible existence. Otherwise, it seems like we're shipping a feature
> that sometimes doesn't work and for which we can't even provide
> reliable steps to fix it when it doesn't.

It is too close to midnight for investigation, so I am mostly asking an open
question as a TODO marker for myself in the morning: Are we guaranteed that a
create relation that crash will end up with orphan files containing correct
checksums? That is, if you have a cluster initdb'd with checksums on, and
never use neither online checksums or pg_checksums, can you end with an orphan
file from CREATE TABLE crashing which cause a backup page verification failure?

--
Daniel Gustafsson

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Sehrope Sarkuni 2026-09-10 21:50:37 Re: Proposal to allow setting cursor options on Portals
Previous Message Nathan Bossart 2026-09-10 21:41:54 fixes for a few GUC descriptions