Re: Binding Postgres to port 0 for testing

Thanks Michael and Erik for the help, I really appreciate it!

Thanks for explaining the context why PostgreSQL doesn't allow binding
against port 0.

I somehow didn't consider looking at the postgres tests, though it makes
sense that they need to solve this problem. If I read the perl code
correctly though it seems that this could, in theory, cause a race? The
script checks first whether the port has been assigned to a test, then
binds a socket to check whether it is used by someone else, closes this
test socker, and then starts a server process. I guess it's unlikely
enough, but isn't there a risk that some other process (that isn't
controlled by this perl script) binds to the found port right after this
test bind but right before postgres calls bind? I guess it should be rare
enough so that it wouldn't cause flaky tests.

I decided to implement the following (this strategy works, though it might
be a bit brittle if PostgreSQL changes the error output format in the
future):
1. Loop, starting from port 5432, incrementing each iteration
2. Start postgres with the given port
3. Parse the output to check whether postgres either writes a line that
ends with "could not create any TCP/IP sockets" (in which case I continue)
or with "database system is ready to accept connections" (in which case I
break).

This is definitely not the most elegant solution, but it should do for now.
At the moment I want to be able to set up everything in one process. In my
experience this makes debugging problems a bit easier but comes at the cost
of a more complex test driver (I recognize that it is a bit weird that the
application layer initializes the runtime environment in this case).

Also, this is a hobby-project and I am more interested in fun learning than
reducing work :) Generally I would agree that reusing existing and testing
code to run this would be better unless there's a really good reason not to
do that.

On Sun, Mar 26, 2023 at 7:27 PM Michael Paquier <michael(at)paquier(dot)xyz> wrote:

> On Sat, Mar 25, 2023 at 11:01:33AM -0600, Markus Pilman wrote:
> > Now the problem is that I need to find a TCP port for each running
> postgres
> > instance. There's multiple ways to do this, but by far the easiest one I
> > know is to bind to port 0. So my plan was to start postgres with "-p 0"
> and
> > then parse stdout to figure out which port it actually uses. But that
> > doesn't seem to work:
>
> Note that you can find some inspiration about that in the code tree
> within src/test/perl/PostgreSQL/Test/Cluster.pm, particularly
> get_free_port(), where we have now accumulated a couple of years of
> experience in designing something that's rather safe, even if it comes
> with its own limits. It is in perl so perhaps you could just reuse it
> rather than reinvent the wheel? Of course, still it should not be
> complicated to translate that in a different language, but there may
> be no need to reinvent the wheel. And seeing your first message with
> the requirements you list, this does what you are looking for:
> - Create an empty cluster.
> - Freely create databases, tablespaces, queries, etc.
> - Wipe out the whole.
>
> The test cases around src/test/recovery/t/ could be a good starting
> point, as well.
> --
> Michael
>