| From: | Daniel Gustafsson <daniel(at)yesql(dot)se> |
|---|---|
| To: | Robert Haas <robertmhaas(at)gmail(dot)com> |
| Cc: | PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>, Heikki Linnakangas <hlinnaka(at)iki(dot)fi> |
| Subject: | Re: Trying to break online checksums with LLMs |
| Date: | 2026-09-14 13:47:28 |
| Message-ID: | 110EE07B-5BD9-4AF3-A6DD-F396878F25FC@yesql.se |
| Views: | Whole Thread | Raw Message | Download mbox | Resend email |
| Thread: | |
| Lists: | pgsql-hackers |
> On 10 Sep 2026, at 23:35, Robert Haas <robertmhaas(at)gmail(dot)com> wrote:
> I think that it is fine to disclaim external files -- if you add
> random files to the data directory, stuff may break, and that's sad,
> but oh well. But I think orphaned files are a completely different
> ball game. That's just a normal outcome of running PostgreSQL for a
> period of time under real-world conditions, and to my knowledge this
> would be the first feature that doesn't work with them. I'd like to
> hear what others think, but my first reaction is to think that's
> probably not OK. If we don't remove the files automatically, and don't
> even provide any tools to remove the files manually, I think it
> follows logically that everything we ship has to cope with their
> possible existence. Otherwise, it seems like we're shipping a feature
> that sometimes doesn't work and for which we can't even provide
> reliable steps to fix it when it doesn't.
Thinking about this over the weekend I think there is really only one possible
option. When enabling checksums online the data checksums launcher will have
to scan the datadir for orphan files, and abort processing if any are found
(and log which files it found). While we don't have a tool for clearning, I
don't think online checksums can, or should, handle the orphan files in any way
other than bailing out. This needs adding net new code, which this really
isn't a great time to be doing. CC:ing Heikki from RMT since I know he has
eyes on this already.
I ended up pushing the patchset for the other open item to master in the
meantime (after off-list +1 from RMT), but held off from backpatching to 19 at
the time of pushing till the BF has had a few builds of it. If we decide to
take online checksums out of 19 due to the above it's also less to back out.
--
Daniel Gustafsson
| From | Date | Subject | |
|---|---|---|---|
| Next Message | Nathan Bossart | 2026-09-14 13:48:22 | Re: Fix unnecessary shared memory page allocation in CalculateShmemSize() |
| Previous Message | 2026-09-14 13:43:45 | AW: Does postgresql have a diff tool? |