Re: Improve TAP tests of pg_upgrade for cross-version tests

From: Michael Paquier <michael(at)paquier(dot)xyz>
To: Andres Freund <andres(at)anarazel(dot)de>
Cc: Justin Pryzby <pryzby(at)telsasoft(dot)com>, pgsql-hackers(at)lists(dot)postgresql(dot)org, Andrew Dunstan <andrew(at)dunslane(dot)net>, Robert Haas <robertmhaas(at)gmail(dot)com>
Subject: Re: Improve TAP tests of pg_upgrade for cross-version tests
Date: 2022-10-03 02:23:11
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

On Sun, Oct 02, 2022 at 10:02:37AM -0700, Andres Freund wrote:
> This fails tests widely, and has so for a while:
> Note that it causes timeouts, which end up chewing up a cfbot "slot" for an
> hour...

Sorry for kicking the can down the road for a too-long time. Attached
is an updated patch that I have strimmed down to the minimum that
addresses all the issues I want fixed as of the scope of this thread:
- The filtering of the dumps is reduced to a minimum, removing only
comments and empty lines.
- The configuration of the old and new nodes is tweaked so as it is
abvle to handle upgrade from nodes older than 10.
- pg_dumpall is tweaked to use --extra-float-digits=0 for the old
nodes older than 11 to minimize the amount of diffs generated.

That's quite nice in itself, as it becomes possible to use much more
dump patterns loaded as part of the tests. More filtering rules could
be used, like the part about procedures and functions that I have sent
in the previous versions, but what I have here is enough to make the
test complete with all the versions supported by I am
thinking to get this stuff applied soon before moving on to the PATH

There is still one issue reported upthread by Justin about the fact
that we can use unexpected commands depending on the node involved.
For example, something like that would build a PATH so as psql from
the old node is used, not from the new node:
$newnode->command_ok(['psql', '-X', '-f', 'blah.sql']);

So with the current, we could fail upgrade_adapt.sql if the
version upgraded from does not support psql's \if, and I don't think
that we should use a full path to the binary either. I am not
completely done analyzing that and this deserves a separate thread, as
it impacts all the commands used in TAP tests manipulating nodes from
multiple versions.

Attachment Content-Type Size
v4-0001-Add-more-filtering-capabilities-in-the-dumps.patch text/plain 6.8 KB

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Masahiko Sawada 2022-10-03 02:58:33 Re: [PoC] Improve dead tuple storage for lazy vacuum
Previous Message Justin Pryzby 2022-10-03 02:15:41 Re: CI and test improvements