PostgreSQL Weekly News - September 12, 2021

From: PWN via PostgreSQL Announce <announce-noreply(at)postgresql(dot)org>
To: PostgreSQL Announce <pgsql-announce(at)lists(dot)postgresql(dot)org>
Subject: PostgreSQL Weekly News - September 12, 2021
Date: 2021-09-13 11:03:50
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-announce

# PostgreSQL Weekly News - September 12, 2021

# PostgreSQL Product News

pg_dumpbinary 2.5, a program used to dump a PostgreSQL database in binary
format, [released](

pgBadger v11.6, a PostgreSQL log analyzer and graph tool written in
Perl, [released](

[pgagroal 1.3.0, a high-performance protocol-native connection pool for
PostgreSQL, [released](

# PostgreSQL Jobs for September


# PostgreSQL in the News

Planet PostgreSQL: [](

PostgreSQL Weekly News is brought to you this week by David Fetter

Submit news and announcements by Sunday at 3:00pm PST8PDT to david(at)fetter(dot)org(dot)

# Applied Patches

Michaël Paquier pushed:

- Remove some unused variables in TAP tests. Author: Amul Sul Discussion:

- Move the shared memory size calculation to its own function. This change
refactors the shared memory size calculation in
CreateSharedMemoryAndSemaphores() to its own function. This is intended for
use in a future change related to the setup of huge pages and shared memory
with some GUCs, while useful on its own for extensions. Author: Nathan
Bossart Discussion:

- Clean up some code using "(expr) ? true : false". All the code paths
simplified here were already using a boolean or used an expression that led to
zero or one, making the extra bits unnecessary. Author: Justin Pryzby
Reviewed-by: Tom Lane, Michael Paquier, Peter Smith Discussion:

- Introduce GUC shared_memory_size. This runtime-computed GUC shows the size of
the server's main shared memory area, taking into account the amount of shared
memory allocated by extensions as this is calculated after processing
shared_preload_libraries. Author: Nathan Bossart Discussion:

- Fix compilation warning in ipci.c. A Size had better use %zu when printed.
Oversight in bd17880, per buildfarm member lapwing.

- Make shared_memory_size a preset option. bd17880 set up that as a memory
parameter, but the docs told a different story. A preset parameter is adapted
here, as this option is compiled at startup time. Reported-by: Fujii Masao

Peter Eisentraut pushed:

- Improve fix pkg-config files for static linking. Amend
4c2eab3a0dec2eae40892fb525830a5947a398c7 to link against the libraries without
the `"_shlib"` suffix, since this is meant for static linking.

- Fix incorrect format placeholders.

- Disable anonymous record hash support except in special cases. Commit
01e658fa74 added hash support for row types. This also added support for
hashing anonymous record types, using the same approach that the type cache
uses for comparison support for record types: It just reports that it works,
but it might fail at run time if a component type doesn't actually support the
operation. We get away with that for comparison because most types support
that. But some types don't support hashing, so the current state can result
in failures at run time where the planner chooses hashing over sorting,
whereas that previously worked if only sorting was an option. We do, however,
want the record hashing support for path tracking in recursive unions, and the
SEARCH and CYCLE clauses built on that. In that case, hashing is the only
plan option. So enable that, this commit implements the following approach:
The type cache does not report that hashing is available for the record type.
This undoes that part of 01e658fa74. Instead, callers that require hashing no
matter what can override that result themselves. This patch only touches the
callers to make the aforementioned recursive query cases work, namely the
parse analysis of unions, as well as the hash_array() function. Reported-by:
Sait Talha Nisanci <sait(dot)nisanci(at)microsoft(dot)com> Bug: #17158 Discussion:

- Fix typo.

- Remove useless casts. Casting the argument of strVal() to (`Value *`) is
useless, since strVal() already does that. Most code didn't do that anyway;
this was apparently just a style that snuck into certain files. Reviewed-by:
Dagfinn Ilmari Mannsåker <ilmari(at)ilmari(dot)org> Reviewed-by: Kyotaro Horiguchi
<horikyota(dot)ntt(at)gmail(dot)com> Discussion:

- Remove Value node struct. The Value node struct is a weird construct. It is
its own node type, but most of the time, it actually has a node type of
Integer, Float, String, or BitString. As a consequence, the struct name and
the node type don't match most of the time, and so it has to be treated
specially a lot. There doesn't seem to be any value in the special construct.
There is very little code that wants to accept all Value variants but nothing
else (and even if it did, this doesn't provide any convenient way to check
it), and most code wants either just one particular node type (usually
String), or it accepts a broader set of node types besides just Value. This
change removes the Value struct and node type and replaces them by separate
Integer, Float, String, and BitString node types that are proper node types
and structs of their own and behave mostly like normal node types. Also, this
removes the T_Null node tag, which was previously also a possible variant of
Value but wasn't actually used outside of the Value contained in A_Const.
Replace that by an isnull field in A_Const. Reviewed-by: Dagfinn Ilmari
Mannsåker <ilmari(at)ilmari(dot)org> Reviewed-by: Kyotaro Horiguchi
<horikyota(dot)ntt(at)gmail(dot)com> Discussion:

- Fix `_equalA_Const`. 639a86e36aaecb84faaf941dcd0b183ba0aba9e9 neglected to make
the necessary adjustments to `_equalA_Const`. Found only via

Fujii Masao pushed:

- Fix typo in comments. Author: Hou Zhijie Discussion:

- postgres_fdw: Allow application_name of remote connection to be set via GUC.
This commit adds postgres_fdw.application_name GUC which specifies a value for
application_name configuration parameter used when postgres_fdw establishes a
connection to a foreign server. This GUC setting always overrides
application_name option of the foreign server object. This GUC is useful when
we want to specify our own application_name per remote connection. Previously
application_name of a remote connection could be set basically only via
options of a server object. But which meant that every session connecting to
the same foreign server basically should use the same application_name. Also
if we want to change the setting, we had to execute "ALTER SERVER ... OPTIONS
..." command. It was inconvenient. Author: Hayato Kuroda Reviewed-by:
Masahiro Ikeda, Fujii Masao Discussion:

- postgres_fdw: Revert unstable tests for postgres_fdw.application_name. Commit
449ab63505 added the tests that check that postgres_fdw.application_name GUC
works as expected. But they were unstable and caused some buildfarm members to
report the failure. This commit reverts those unstable tests. Reported-by:
Tom Lane as per buildfarm Discussion:

- Fix issue with WAL archiving in standby. Previously, walreceiver always closed
the currently-opened WAL segment and created its archive notification file,
after it finished writing the current segment up and received any WAL data
that should be written into the next segment. If walreceiver exited just
before any WAL data in the next segment arrived at standby, it did not create
the archive notification file of the current segment even though that's known
completed. This behavior could cause WAL archiving of the segment to be
delayed until subsequent restartpoints or checkpoints created its notification
file. To fix the issue, this commit changes walreceiver so that it creates an
archive notification file of a current WAL segment immediately if that's known
completed before receiving next WAL data. Back-patch to all supported
branches. Reported-by: Kyotaro Horiguchi Author: Fujii Masao Reviewed-by:
Kyotaro Horiguchi Discussion:

- pgbench: Stop counting skipped transactions as soon as timer is exceeded. When
throttling is used, transactions that lag behind schedule by more than the
latency limit are counted and reported as skipped. Previously, there was the
case where pgbench counted all skipped transactions even if the timer
specified in -T option was exceeded. This could take a very long time to do
that especially when unrealistically high rate setting in -R option caused
quite a lot of transactions that lagged behind schedule. This could prevent
pgbench from ending immediately, and so pgbench could look like it got stuck
to users. To fix the issue, this commit changes pgbench so that it stops
counting skipped transactions as soon as the timer is exceeded. The timer can
make pgbench end soon even when there are lots of skipped transactions that
have not been counted yet. Note that there is no guarantee that all skipped
transactions are counted under -T though there is under -t. This is OK in
practice because it's very unlikely to happen with realistic setting. Also
this is not the issue that this commit newly introdues. There used to be the
case where pgbench ended without counting all skipped transactions since
before. Back-patch to v14. Per discussion, we decided not to bother
back-patch to the stable branches because it's hard to imagine the issue
happens in practice (with realistic setting). Author: Yugo Nagata, Fabien
COELHO Reviewed-by: Greg Sabino Mullane, Fujii Masao Discussion:

Tom Lane pushed:

- Make timetz_zone() stable, and correct a bug for DYNTZ abbreviations.
Historically, timetz_zone() has used time(NULL) as the reference point for
deciding whether DST is active. That means its result can change
intra-statement, requiring it to be marked VOLATILE (cf. 35979e6c3). But that
definition is pretty inconsistent with the way we deal with timestamps
elsewhere. Let's make it use the transaction start time ("now()") as the
reference point instead. That lets it be marked STABLE, and also saves a
kernel call per invocation. While at it, remove the function's use of
pg_time_t and pg_localtime. Those are inconsistent with the other code in this
area, which indeed created a bug: timetz_zone() delivered completely wrong
answers if the zone was specified by a dynamic TZ abbreviation. (We need to
do something about that in the back branches, but the fix will look different
from this.) Aleksander Alekseev and Tom Lane Discussion:

- Fix actively-misleading comments about the contents of struct pg_tm. pgtime.h
documented the PG interpretation of tm_mon right alongside the POSIX
interpretation of tm_year, with no hint that neither comment was correct
throughout our code. Perhaps someday we ought to switch to using two separate
struct definitions to provide a clearer indication of which semantics are in
use where. But I fear the tedium-versus-safety-gain tradeoff would not be
very good. Discussion:

- Further fix psql query-cancel test. The query to wait for pg_sleep to be
running did no such thing, because the regex pattern it used could match
itself. Report:

- Fix rewriter to set hasModifyingCTE correctly on rewritten queries. If we copy
data-modifying CTEs from the original query to a replacement query (from a DO
INSTEAD rule), we must set hasModifyingCTE properly in the replacement query.
Failure to do this can cause various unpleasantness, such as unsafe usage of
parallel plans. The code also neglected to propagate hasRecursive, though
that's only cosmetic at the moment. A difficulty arises if the rule action is
an INSERT...SELECT. We attach the original query's RTEs and CTEs to the
sub-SELECT Query, but data-modifying CTEs are only allowed to appear in the
topmost Query. For the moment, throw an error in such cases. It would
probably be possible to avoid this error by attaching the CTEs to the top
INSERT Query instead; but that would require a bunch of new code to adjust
ctelevelsup references. Given the narrowness of the use-case, and the need to
back-patch this fix, it does not seem worth the trouble for now. We can
revisit this if we get field complaints. Per report from Greg Nancarrow.
Back-patch to all supported branches. (The test case added here does not fail
before v10, but there are plenty of places checking top-level hasModifyingCTE
in 9.6, so I have no doubt that this code change is necessary there too.)
Greg Nancarrow and Tom Lane Discussion:

- In psql tab completion, offer spelled-out commands not abbreviations. Various
psql backslash commands have both single-letter and long forms, for example \e
and \edit. Previously, tab completion generally offered the single-letter
form but not the long form. It seems more sensible to offer the long form,
because (a) no useful completion can happen when you've already typed the
single letter, and (b) if you're not so familiar with the command set as to
know that, the long form is likely to be less confusing. Haiying Tang,
reviewed by Dagfinn Ilmari Mannsåker and myself Discussion:

- Fix misleading comments about TOAST access macros. Seems to have been my error
in commit aeb1631ed. Noted by Christoph Berg. Discussion:

- Avoid useless malloc/free traffic around getFormattedTypeName(). Coverity
complained that one caller of getFormattedTypeName() failed to free the
returned string. Which is true, but rather than fixing that one, let's get
rid of this tedious and error-prone requirement. Now that
getFormattedTypeName() caches its result, strdup'ing that result and expecting
the caller to free it accomplishes little except to waste cycles. We do
create a leak in the case where getTypes didn't make a TypeInfo for the type,
but that basically shouldn't ever happen. Back-patch, as commit 6c450a861
was. This isn't a particularly interesting bug fix, but the API change seems
like a hazard for future back-patching activity if we don't back-patch it.

- Check for relation length overrun soon enough. We don't allow relations to
exceed 2^32-1 blocks, because block numbers are 32 bits and the last possible
block number is reserved to mean InvalidBlockNumber. There is a check for
this in mdextend, but that's really way too late, because the smgr API
requires us to create a buffer for the block-to-be-added, and we do not want
to have any buffer with blocknum InvalidBlockNumber. (Such a case can trigger
assertions in bufmgr.c, plus I think it might confuse ReadBuffer's logic for
data-past-EOF later on.) So put the check into ReadBuffer. Per report from
Christoph Berg. It's been like this forever, so back-patch to all supported
branches. Discussion:

- Avoid fetching from an already-terminated plan. Some plan node types don't
react well to being called again after they've already returned NULL.
PortalRunSelect() has long dealt with this by calling the executor with
NoMovementScanDirection if it sees that we've already run the portal to the
end. However, commit ba2c6d6ce overlooked this point, so that persisting an
already-fully-fetched cursor would fail if it had such a plan. Per report
from Tomas Barton. Back-patch to v11, as the faulty commit was. (I've
omitted a test case because the type of plan that causes a problem isn't all
that stable.) Discussion:

- Fix some anomalies with NO SCROLL cursors. We have long forbidden fetching
backwards from a NO SCROLL cursor, but the prohibition didn't extend to cases
in which we rewind the query altogether and then re-fetch forwards. I think
the reason is that this logic was mainly meant to protect plan nodes that
can't be run in the reverse direction. However, re-reading the query output
is problematic if the query is volatile (which includes SELECT FOR UPDATE, not
just queries with volatile functions): the re-read can produce different
results, which confuses the cursor navigation logic completely. Another
reason for disliking this approach is that some code paths will either fetch
backwards or rewind-and-fetch-forwards depending on the distance to the target
row; so that seemingly identical use-cases may or may not draw the "cursor can
only scan forward" error. Hence, let's clean things up by disallowing rewind
as well as fetch-backwards in a NO SCROLL cursor. Ordinarily we'd only make
such a definitional change in HEAD, but there is a third reason to consider
this change now. Commit ba2c6d6ce created some new user-visible anomalies for
non-scrollable cursors WITH HOLD, in that navigation in the cursor result got
confused if the cursor had been partially read before committing. The only
good way to resolve those anomalies is to forbid rewinding such a cursor,
which allows removal of the incorrect cursor state manipulations that
ba2c6d6ce added to PersistHoldablePortal. To minimize the behavioral change
in the back branches (including v14), refuse to rewind a NO SCROLL cursor only
when it has a holdStore, ie has been held over from a previous transaction due
to WITH HOLD. This should avoid breaking most applications that have been
sloppy about whether to declare cursors as scrollable. We'll enforce the
prohibition across-the-board beginning in v15. Back-patch to v11, as
ba2c6d6ce was. Discussion:

- Make pg_regexec() robust against out-of-range search_start. If search_start is
greater than the length of the string, we should just return REG_NOMATCH
immediately. (Note that the equality case should *not* be rejected, since the
pattern might be able to match zero characters.) This guards various internal
assumptions that the min of a range of string positions is not more than the
max. Violation of those assumptions could allow an attempt to fetch
string[search_start-1], possibly causing a crash. Jaime Casanova pointed out
that this situation is reachable with the new regexp_xxx functions that accept
a user-specified start position. I don't believe it's reachable via any
in-core call site in v14 and below. However, extensions could possibly call
pg_regexec with an out-of-range search_start, so let's back-patch the fix
anyway. Discussion:

Álvaro Herrera pushed:

- Add PublicationTable and PublicationRelInfo structs. These encapsulate a
relation when referred from replication DDL. Currently they don't do anything
useful (they're just wrappers around RangeVar and Relation respectively) but
in the future they'll be used to carry column lists. Extracted from a larger
patch by Rahila Syed. Author: Rahila Syed <rahilasyed90(at)gmail(dot)com>
Reviewed-by: Álvaro Herrera <alvherre(at)alvh(dot)no-ip(dot)org> Reviewed-by: Tomas
Vondra <tomas(dot)vondra(at)enterprisedb(dot)com> Reviewed-by: Amit Kapila
<amit(dot)kapila16(at)gmail(dot)com> Discussion:

Noah Misch pushed:

- AIX: Fix missing libpq symbols by respecting SHLIB_EXPORTS. We make each AIX
shared library export all globals found in .o files that originate in the
library. That doesn't include symbols acquired by -lpgcommon_shlib. That is
good on average, but it became a problem for libpq when commit
e6afa8918c461c1dd80c5063a950518fa4e950cd moved five official libpq API symbols
into src/common. Fix this by implementing the SHLIB_EXPORTS mechanism for
AIX, so affected libraries export the same symbols that they export on Linux.
This reintroduces symbols pg_encoding_to_char, pg_utf_mblen,
pg_char_to_encoding, pg_valid_server_encoding, and
pg_valid_server_encoding_id. Back-patch to v13, where the aforementioned
commit first appeared. While a minor release is usually the wrong time to add
or remove symbol exports in libpq or libecpg, we should expect users to want
each documented symbol. Tony Reix Discussion:

- Revoke PUBLIC CREATE from public schema, now owned by pg_database_owner. This
switches the default ACL to what the documentation has recommended since
CVE-2018-1058. Upgrades will carry forward any old ownership and ACL. Sites
that declined the 2018 recommendation should take a fresh look. Recipes for
commissioning a new database cluster from scratch may need to create a schema,
grant more privileges, etc. Out-of-tree test suites may require such updates.
Reviewed by Peter Eisentraut. Discussion:

- Update src/test/kerberos to account for previous commit.

Amit Kapila pushed:

- Log new catalog xmin candidate in LogicalIncreaseXminForSlot(). Similar to
LogicalIncreaseRestartDecodingForSlot() add a debug message to
LogicalIncreaseXminForSlot() reporting a new catalog_xmin candidate. This
just adds additional diagnostic information during logical decoding that can
aid debugging. Author: Ashutosh Bapat Reviewed-by: Masahiko Sawada, Amit
Kapila Discussion:

- Invalidate relcache for publications defined for all tables. Updates/Deletes
on a relation were allowed even without replica identity after we define the
publication for all tables. This would later lead to an error on subscribers.
The reason was that for such publications we were not invalidating the
relcache and the publication information for relations was not getting
rebuilt. Similarly, we were not invalidating the relcache after dropping of
such publications which will prohibit Updates/Deletes without replica identity
even without any publication. Author: Vignesh C and Hou Zhijie Reviewed-by:
Hou Zhijie, Kyotaro Horiguchi, Amit Kapila Backpatch-through: 10, where it was
introduced Discussion:

Heikki Linnakangas pushed:

- Fix missing words in comment. Introduced by commit c3928b467a, backpatch to
v14 like that one. Author: Amit Langote Discussion:

Andres Freund pushed:

- windows: Only consider us to be running as service if stderr is invalid.
Previously pgwin32_is_service() would falsely return true when postgres is
started from somewhere within a service, but not as a service. That is e.g.
always the case with windows docker containers, which some CI services use to
run windows tests in. When postgres falsely thinks its running as a service,
no messages are writting to stdout / stderr. That can be very confusing and
causes a few tests to fail. To fix additionally check if stderr is invalid in
pgwin32_is_service(). For that to work in backend processes, pg_ctl is changed
to pass down handles so that postgres can do the same check (otherwise
"default" handles are created). While this problem exists in all branches,
there have been no reports by users, the prospective CI usage currently is
only for master, and I am not a windows expert. So doing the change in only
master for now seems the sanest approach. Author: Andres Freund
<andres(at)anarazel(dot)de> Reviewed-By: Magnus Hagander <magnus(at)hagander(dot)net>

Magnus Hagander pushed:

- Consistently use read-only instead of "read only". This affects one message
and some documentation that used the format "read only", unlike everything
else that used read-only. Backpatch-through: 14 Discussion:

Daniel Gustafsson pushed:

- Consistently use "superuser" instead of "super user". The correct nomenclature
for the highest privileged user is superuser and not "super user", this
replaces the few instances where that was used erroneously. No user-visible
changes are done as all changes are in comments, so no back-patching. Author:
Bharath Rupireddy <bharath(dot)rupireddyforpostgres(at)gmail(dot)com> Discussion:

Browse pgsql-announce by date

  From Date Subject
Next Message PostgreSQL Global Development Group 2021-09-13 14:26:27 Trademark Actions Against the PostgreSQL Community
Previous Message Gilles Darold via PostgreSQL Announce 2021-09-09 13:40:46 pg_dumpbinary v2.5 released