Re: ssl tests aren't concurrency safe due to get_free_port()

From: Andrew Dunstan <andrew(at)dunslane(dot)net>
To: Andres Freund <andres(at)anarazel(dot)de>, pgsql-hackers(at)postgresql(dot)org, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
Subject: Re: ssl tests aren't concurrency safe due to get_free_port()
Date: 2022-11-03 18:16:51
Message-ID: b8f5d215-cd90-9ea3-88d8-03a2c864ebb1@dunslane.net
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers


On 2022-11-02 We 15:09, Andrew Dunstan wrote:
> On 2022-10-17 Mo 10:59, Andrew Dunstan wrote:
>> On 2022-10-04 Tu 01:39, Andrew Dunstan wrote:
>>> On 2022-10-02 Su 12:49, Andres Freund wrote:
>>>> 2) Use a lockfile containing a pid to protect the choice of a port within a
>>>> build directory. Before accepting a port get_free_port() would check if the
>>>> a lockfile exists for the port and if so, if the test using it is still
>>>> alive. That will protect against racyness between multiple tests inside a
>>>> build directory, but won't protect against races between multiple builds
>>>> running concurrently on a machine (e.g. on a buildfarm host)
>>>>
>>>>
>>> I think this is the right solution. To deal with the last issue, the
>>> lockdir should be overrideable, like this:
>>>
>>>
>>>   my $port_lockdir = $ENV{PG_PORT_LOCKDIR} || $build_dir;
>>>
>>>
>>> Buildfarm animals could set this, probably to the global lockdir (see
>>> run_branches.pl). Prior to that, buildfarm owners could do that manually.
>>>
>>>
>> The problem here is that Cluster.pm doesn't have any idea where the
>> build directory is, or even if there is one present at all.
>>
>> meson does appear to let us know that, however, with MESON_BUILD_ROOT,
>> so probably the best thing would be to use PG_PORT_LOCKDIR if it's set,
>> otherwise MESON_BUILD_ROOT if it's set, otherwise the tmp_check
>> directory. If we want to backport to the make system we could export
>> top_builddir somewhere.
>>
>>
> Here's a patch which I think does the right thing.
>
>
>

Updated with a couple of thinkos fixed.

cheers

andrew

--
Andrew Dunstan
EDB: https://www.enterprisedb.com

Attachment Content-Type Size
prevent-concurrent-free-port-v2.patch text/x-patch 2.9 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Jonathan S. Katz 2022-11-03 18:21:18 Re: heavily contended lwlocks with long wait queues scale badly
Previous Message Justin Pryzby 2022-11-03 18:11:18 Re: Fix proposal for comparaison bugs in PostgreSQL::Version