Re: Rewriting the test of pg_upgrade as a TAP test - take three - remastered set

On 5/16/21 9:55 PM, Michael Paquier wrote:
> On Sat, May 15, 2021 at 02:22:24PM -0400, Andrew Dunstan wrote:
>> PostgresNode is currently able to create nodes suitable for upgrade down
>> to release 10. If we add '-w' to the 'pg_ctl start' flags that can
>> extend down to release 9.5. (Just tested) I think we should do that
>> forthwith. '-w' is now the default, but having it there explicitly does
>> no harm.
> Agreed. When testing manually, I have personally never worked on any
> patches that required binaries older than 9.4, so I would be fine if
> the TAP tests are able to work easily down to 9.5, even if pg_upgrade
> is supported down to 8.4.
>
>> If people are interested in what's incompatible on older versions, they
>> can look at
>> <https://gitlab.com/adunstan/postgresnodeng/-/blob/master/PostgresNode.pm>
>> starting at about line 2764.
> We should really have adjust_conf() at some point in the in-core
> module.

Yes, I'm going to be proposing a series of smallish patches including
these when the tree is branched (which I hope will be in a few weeks).

>> I don't think this will work, though, unless there is enough data to
>> exercise pg_upgrade fairly thoroughly. The approach taken by both
>> test.sh and (somewhat more comprehensively) by the buildfarm cross
>> version upgrade module is to test a cluster where the regression tests
>> have been run.
> Yeah, that's what my patch is doing with pg_regress, FWIW. This
> requires regress.so from the old version.
>
>> That might be more difficult when testing against older
>> versions, so I have published a snapshot of the dumps of each of the
>> versions we tests against in the buildfarm animal crake. These could be
>> loaded into PostgresNode instances and then an upgrade attempted. See
>> <https://gitlab.com/adunstan/pg-old-bin/-/tree/master/data>. The data
>> goes back to 9.2. These compressed dumps are a couple of megabytes each,
>> not huge.
> I agree that this can be simpler in some cases. In your experience,
> how much of an issue is it when it becomes necessary to keep around
> binaries that rely on libraries older than what a system can support?
> It is easy to counter issues in this area with OpenSSL and
> non-necessary things, but we had in the past also cases where we had
> code that conflicted with the kernel, aka 3e68686.

That one at least isn't an issue. Old versions of postgres didn't have
pg_rewind.

>
> At the end of this exercise, what I think we should achieve is to:
> 1) Reduce the diff between the buildfarm code and the in-core code.
> 2) Get rid of test.sh.
> 3) Be able to run easily upgrade tests across major versions for
> developers.
>
> As of now, 3) requires some extra facilities depending on if this is
> done by the buildfarm or the in-core tests:
> 1) Path to the source code of the old version. This is required once
> it becomes necessary to find out src/test/regress/ for the schedule,
> the tests to run and its regress.so. There is no need to do that if
> you have a dump of the old instance.
> 2) Path to a custom dump to replace the run with pg_regress from 1).
> 3) Location of the old binaries, for pg_upgrade. When it comes to
> PostgresNode, we require an install path, so we cannot use directly
> the location of the binaries.
>
> Looking at the code of the buildfarm, its code does something smarter
> than what my patch or HEAD's test.sh does now, as these require the
> path for the old source. The buildfarm code first scans for the
> probin's used in the regression database and then updates any
> references. What test.sh and my patch do is using the path to the old
> source code and run a single UPDATE. The approach taken by the
> buildfarm code is more portable, as a path to the old source code
> becomes necessary only if running pg_regress manually. So, what about
> doing the following thing?
> 1) Update the TAP test so as probin entries are updated in the same way
> as the buildfarm.
> 2) Allow one to specify a path to a custom dump, or a path to the old
> source code for pg_regress.
>
> If we do that, then it should be possible to reduce the code footprint
> in the buildfarm code, while still allowing people to test major
> upgrades in the same old-fashioned way, right? That's assuming that
> PostgresNode is made compatible down to 9.2, of course, as a first
> step, as that's the range of the dumps you are keeping around for the
> buildfarm.
>

I'm intending to add some older dumps. -) But for now 9.2 is a good target.

cheers

andrew

--
Andrew Dunstan
EDB: https://www.enterprisedb.com