ProcArrayStruct->pgprocnos vs ->maxProcs vs PGPROC ordering

Hi,

I just tried to add a, seemingly innocuous, assertion to
ProcArrayAdd()/Remove() that proc->pgprocno is < arrayP->maxProcs. That
quickly fails.

The reason for that is that PGPROC are created in the following order
1) MaxBackends normal (*[1]) backends
2) NUM_AUXILIARY_PROCS auxiliary processes
3) max_prepared_xacts prepared transactions.

and pgprocnos are assigned sequentially - they are needed to index into
ProcGlobal->allProcs.

In contrast to that procarray.c initializes maxProcs to
#define PROCARRAY_MAXPROCS (MaxBackends + max_prepared_xacts)
i.e. without the aux processes.

Which means that some of the prepared transactions have pgprocnos that are
bigger than ProcArrayStruct->maxProcs. Hence my assertion failure.

This is obviously not a bug, but is quite hard to understand / confusing. I
think I made a similar mistake before.

I'd at least like to add a comment with a warning somewhere in ProcArrayStruct
or such.

An alternative approach would be to change the PGPROC order to instead be 1)
aux, b) normal backends, 3) prepared xacts and give aux processes a negative
or invalid pgprocno.

One small advantage of that would be that we'd not need to "skip" over the
"aux process hole" between normal and prepared xact PGPROCs in various
procarray.c routines that iterate over procs.

Greetings,

Andres Freund

[1] well, kinda. It's user backends followed by autovacuum worker, launcher,
worker processes and wal senders.