Re: Perl modules for testing/viewing/corrupting/repairing your heap files

On Wed, Apr 15, 2020 at 7:22 AM Mark Dilger
<mark(dot)dilger(at)enterprisedb(dot)com> wrote:
> I steered away from software released under the GPL, such as pg_hexedit, owing to difficulties in getting anything I develop accepted. (That's a hard enough problem without licensing issues.). I'm not taking a political stand for or against the GPL here, just a pragmatic position that I wouldn't be able to integrate pg_hexedit into a postgres submission.
>
> (Thanks for writing pg_hexedit, BTW. I'm not criticizing it.)

The only reason that pg_hexedit is under the GPL is that it's derived
from pg_filedump, which was and is also GPL 2. Note that pg_filedump
is hosted on community resources, and is something that index access
methods know about and try not to break (grep for pg_filedump in the
Postgres source code). pg_hexedit supports all index access methods
with the core distribution, including even the unpopular ones, like
SP-GiST.

> That's hard to do from a TAP test without modules like this, as you have to calculate by hand the offsets where you're going to write the corruption, and the bit pattern you are going to write to that location. Even if you do all that, nobody else is likely going to be able to read and maintain your tests.

Logical corruption is almost inherently a once-off thing. I think that
a tool like pg_hexedit is useful for seeing how the system behaves
with certain novel kinds of logical corruption, which it will tolerate
to varying degrees and with diverse symptoms. Pretty much for
investigating on a once-off basis.

I have occasionally wished for an SQL-like interface to bufpage.c
routines like PageIndexTupleDelete(), PageRepairFragmentation(), etc.
That would probably be a great deal more maintainable than what you
propose to do. It's not really equivalent, of course, but it would
give tests a way to dynamically manipulate/damage pages at the
"logical level". That seems like the thing that's hard to simulate
right now.

--
Peter Geoghegan