[PATCH] RFC: new_oid_hook, to let an extension propose catalog OIDs

Hi hackers,

The attached patch adds a hook through which an extension may propose
the OID of a new catalog row. I'd like to hear whether a hook like this
could be acceptable, before I spend more time on it.

Motivation
----------

I am working on an extension that runs a query across several
PostgreSQL servers: a coordinator plans it and sends parts of the plan
to the other servers. A plan refers to tables, types, functions and
operators by OID, so the same object must have the same OID on every
server. Sending SQL instead of plans avoids that, but it means
planning again on every server, and getting the same plan everywhere is
not guaranteed.

DDL runs on all servers, so they create the same objects in the same
order, but nothing makes the OID counters agree: any temporary table or
other object created on one server only moves its counter. Greenplum
and its forks solve this by patching the server so that the coordinator
dispatches the OIDs it assigned together with the DDL; the segments
take them instead of generating their own. An extension cannot do that
today.

The closest existing mechanism is binary_upgrade_set_next_*_oid(), but
it works only in binary-upgrade mode and covers only some catalogs.

The patch
---------

GetNewOidWithIndex() calls new_oid_hook, if it is set, before
generating an OID:

typedef Oid (*new_oid_hook_type) (Relation relation, Oid indexId,
AttrNumber oidcolumn);

InvalidOid leaves the choice to the server, as now. A valid OID is
checked like any other candidate: it must not be in use (the same index
scan with SnapshotAny as the loop does), and it must not be below
FirstNormalObjectId. Unlike a generated candidate, a proposed OID that
fails either check is an error rather than a reason to try another one,
since the extension would accept no other. So the hook cannot create
duplicate OIDs or OIDs in the range of system objects.

The hook is not called in bootstrap mode, during binary upgrade (which
assigns OIDs of its own), or for TOAST value OIDs, which are not
catalog rows.

There is one more case where a relation OID can be in use: a file of
that number may exist already, as a relfilenumber is not always equal
to the OID of its relation (e.g. after TRUNCATE). GetNewRelFileNumber()
keeps a proposed OID in that case, and NewOidNeedsRelFileNumber() tells
heap_create_with_catalog() and index_create() to take a separate
relfilenumber from a second call. The relation then just starts with
relfilenode <> oid, which is a state it would reach anyway after the
first rewrite.

NEW_OID_HOOK_VERSION lets an extension check at compile time whether
the hook exists, so it can fall back to something else when built
against a server without it.

The core part is about 60 lines plus comments. A new test module,
src/test/modules/test_new_oid_hook, proposes OIDs for pg_class and
pg_namespace and checks the error cases and the file collision for a
table and an index.

Questions
---------

- Is a hook here acceptable at all, or would you rather extend the
binary-upgrade mechanism ("the next OID for this catalog is X") to
normal operation?
- The hook gets the catalog but not the object being created. That is
enough for my case, since the coordinator knows which OIDs to expect
in which order, but a richer interface (e.g. the new tuple) is
possible.
- Should a proposed OID in use be an error, as now, or should the
server fall back to a generated one and let the extension notice?

The patch applies to master as of 4848fd6752a and passes check-world on
Linux, macOS and Windows (MSVC and MinGW) in the GitHub Actions CI that
comes with the tree.

Thanks,
Georgy Shelkovy