pg16: invalid page/page verification failed

On an instance running pg16.0:

log_time | 2023-10-05 10:03:00.014-05
backend_type | autovacuum worker
left | page verification failed, calculated checksum 5074 but expected 5050
context | while scanning block 119 of relation "public.postgres_log_2023_10_05_0900"

This is the only error I've seen so far, and for all I know there's a
issue on the storage behind the VM, or a cosmic ray hit. But I moved
the table out of the way and saved a copy of get_raw_page() in case
someone wants to ask about it.

public | BROKEN_postgres_log_2023_10_05_0900 | table | postgres | permanent | heap | 1664 kB

This table is what it sounds like: a partition into which CSV logs are
COPY'ed. It would've been created around 8am. There's no special
params set for the table nor for autovacuum.

Although we have a ZFS tablespace, these tables aren't on it, and
full_page_writes=on. There's no crashes, and the instance has been up
since it was pg_upgraded from v15.4 on sep25.

pg_stat_all_tables indicates that the table was never (successfully)
vacuumed.

This was compiled to RPM on centos7, and might include a few commits
made since v16.0.

postgres=# SELECT * FROM heap_page_item_attrs(get_raw_page(801594131::regclass::text, 119), 801594131);
lp | lp_off | lp_flags | lp_len | t_xmin | t_xmax | t_field3 | t_ctid | t_infomask2 | t_infomask | t_hoff | t_bits | t_oid | t_attrs
1 | 2304 | 1 | 16 | | | | | | | | | |
2 | 8160 | 1 | 16 | | | | | | | | | |
3 | 8144 | 1 | 16 | | | | | | | | | |
...all the same except for lp_off...
365 | 2352 | 1 | 16 | | | | | | | | | |
366 | 2336 | 1 | 16 | | | | | | | | | |
367 | 2320 | 1 | 16 | | | | | | | | | |

postgres=# SELECT FROM (SELECT tuple_data_split(801594131, t_data, t_infomask, t_infomask2, t_bits) a FROM heap_page_items(get_raw_page(801594131::regclass::text, 119))) WHERE a IS NOT NULL;
WARNING: page verification failed, calculated checksum 5074 but expected 5050
(0 rows)

--
Justin