Re: Rewriting the test of pg_upgrade as a TAP test - take three - remastered set

On 5/14/21 10:26 PM, Michael Paquier wrote:
> Hi,
> (Adding Andrew in CC for the buildfarm and PostgresNode parts.)
>
> $subject has been around for a couple of years now, with the following
> threads:
> https://www.postgresql.org/message-id/20180126080026.GI17847@paquier.xyz
> https://www.postgresql.org/message-id/CAB7nPqRdaN1A1YNjxNL9T1jUEWct8ttqq29dNv8W_o37+e8wfA@mail.gmail.com
>
> An advantage of moving to TAP is that we can then remove the support
> for upgrades within the MSVC scripts, and also remove pg_upgrade's
> test.sh that has accumulated tweaks that are solved by the TAP tests,
> resulting in cleanup:
> 8 files changed, 230 insertions(+), 403 deletions(-)
>
> Based on the past discussions, there were two obstacles preventing to
> do this switch:
> - Support for tests with older versions, something where the gap as
> been closed thanks to Andrew's work in 4c4eaf3d.
> - Buildfarm support, and I am not sure how things need to be extended
> there.
>
> Another thing to note is that HEAD uses oldbindir, bindir and libdir
> to track the location of the old and new libraries and binaries. With
> the infrastructure in place, once can define only an install path for
> a PostgresNode, so this version uses only two variables:
> - oldinstall, for the installation path of the version to upgrade
> from.
> - oldsrc, to point to the source of the old version.
>
> It is not difficult to switch to one approach or the other, but
> reducing the logic to a minimum number of variables is a good deal to
> take IMO.
>
> I have been testing this patch a bit with older versions, down to 12,
> and that was logically working, and PostgresNode may need more to be
> able to work with ~11, as documented in get_new_node(). And I have
> not tested that with MSVC yet. Anyway, attached is a new patch to
> make the discussion move on. Even if there is still work to be done
> here, would people here still support this switch?

PostgresNode is currently able to create nodes suitable for upgrade down
to release 10. If we add '-w' to the 'pg_ctl start' flags that can
extend down to release 9.5. (Just tested) I think we should do that
forthwith. '-w' is now the default, but having it there explicitly does
no harm.

If people are interested in what's incompatible on older versions, they
can look at
<https://gitlab.com/adunstan/postgresnodeng/-/blob/master/PostgresNode.pm>
starting at about line 2764.

I don't think this will work, though, unless there is enough data to
exercise pg_upgrade fairly thoroughly. The approach taken by both
test.sh and (somewhat more comprehensively) by the buildfarm cross
version upgrade module is to test a cluster where the regression tests
have been run. That might be more difficult when testing against older
versions, so I have published a snapshot of the dumps of each of the
versions we tests against in the buildfarm animal crake. These could be
loaded into PostgresNode instances and then an upgrade attempted. See
<https://gitlab.com/adunstan/pg-old-bin/-/tree/master/data>. The data
goes back to 9.2. These compressed dumps are a couple of megabytes each,
not huge.

cheers

andrew

--
Andrew Dunstan
EDB: https://www.enterprisedb.com