Re: multi-install PostgresNode fails with older postgres versions

From: Mark Dilger <mark(dot)dilger(at)enterprisedb(dot)com>
To: Andrew Dunstan <andrew(at)dunslane(dot)net>
Cc: PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: multi-install PostgresNode fails with older postgres versions
Date: 2021-03-30 21:33:36
Message-ID: 748CB63C-2265-4BEB-9D4F-04B268CB39DF@enterprisedb.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

> On Mar 30, 2021, at 10:39 AM, Mark Dilger <mark(dot)dilger(at)enterprisedb(dot)com> wrote:
>
> Andrew,
>
> While developing some cross version tests, I noticed that PostgresNode::init fails for postgres versions older than 9.3, like so:
>
> # Checking port 52814
> # Found port 52814
> Name: 9.2.24
> Data directory: /Users/mark.dilger/hydra/postgresnode.review/src/test/modules/test_cross_version/tmp_check/t_001_verify_paths_9.2.24_data/pgdata
> Backup directory: /Users/mark.dilger/hydra/postgresnode.review/src/test/modules/test_cross_version/tmp_check/t_001_verify_paths_9.2.24_data/backup
> Archive directory: /Users/mark.dilger/hydra/postgresnode.review/src/test/modules/test_cross_version/tmp_check/t_001_verify_paths_9.2.24_data/archives
> Connection string: port=52814 host=/var/folders/6n/3f4vwbnn7fz5qk0xqhgbjrkw0000gp/T/L_A2w1x7qb
> Log file: /Users/mark.dilger/hydra/postgresnode.review/src/test/modules/test_cross_version/tmp_check/log/001_verify_paths_9.2.24.log
> # Running: initdb -D /Users/mark.dilger/hydra/postgresnode.review/src/test/modules/test_cross_version/tmp_check/t_001_verify_paths_9.2.24_data/pgdata -A trust -N
> initdb: invalid option -- N
> Try "initdb --help" for more information.
> Bail out! system initdb failed
>
> The problem is clear enough; -N/--nosync was added in 9.3, and PostgresNode::init is passing -N to initdb unconditionally. I wonder if during PostgresNode::new a call should be made to pg_config and the version information grep'd out so that version specific options to various functions (init, backup, etc) could hinge on the version of postgres being used?
>
> You could also just remove the -N option, but that would slow down all tests for everybody, so I'm not keen to do that. Or you could remove -N in cases where $node->{_install_path} is defined, which would be far more acceptable. I'm leaning towards using the output of pg_config, though, since this problem is likely to come up again with other options/commands.
>
> Thoughts?

I fixed this largely as outlined above, introducing a few new functions which ease test development and using one of them to condition the behavior of init() on the postgres version.

In the tests I have been developing (not included), the developer (or some buildfarm script) has to list all postgresql installations in a configuration file, like so:

/Users/mark.dilger/install/8.4
/Users/mark.dilger/install/9.0.23
/Users/mark.dilger/install/9.1.24
/Users/mark.dilger/install/9.2.24
/Users/mark.dilger/install/9.3.25
/Users/mark.dilger/install/9.4.26
/Users/mark.dilger/install/9.5.25
/Users/mark.dilger/install/9.6
/Users/mark.dilger/install/10
/Users/mark.dilger/install/11
/Users/mark.dilger/install/12
/Users/mark.dilger/install/13

The tests can't be hardcoded to know anything about which specific postgres versions will be installed, or what version of postgres exists in any particular install directory. It makes the tests easier to maintain if they can do stuff like:

$node{$_} = PostgresNode->get_new_node(...) for (@installed_versions);

if ($node{a}->newer_than_version($node{b}))
{
# check that newer version A's whatever can connect to and work with older server B
....
}

I therefore included functions of that sort in the patch along with the $node->at_least_version(version) function that the fix uses.

Attachment Content-Type Size
v1-0001-Extending-PostgresNode-cross-version-functionalit.patch.WIP application/octet-stream 5.2 KB

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Isaac Morland 2021-03-30 21:42:30 Trouble with initdb trying to run regression tests
Previous Message Merlin Moncure 2021-03-30 21:17:03 unconstrained memory growth in long running procedure stored procedure after upgrading 11-12