PostgreSQL Weekly News - December 20, 2020

From: PWN via PostgreSQL Announce <announce-noreply(at)postgresql(dot)org>
To: PostgreSQL Announce <pgsql-announce(at)lists(dot)postgresql(dot)org>
Subject: PostgreSQL Weekly News - December 20, 2020
Date: 2020-12-20 23:49:52
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-announce

# PostgreSQL Weekly News - December 20, 2020

FOSDEM PGDay 2021,will be held on line, on Feb 6-7, 2021.

The Code of Conduct Committee welcomes new member Michael Goldberg

Person of the week:

# PostgreSQL Product News

PoWA 4.1, a performance tool for PostgreSQL, released.

Database .NET v31.5, a multi-database management tool, now with support for
PostgreSQL, released.

pgAdmin4 4.29, a web- and native GUI control center for PostgreSQL, released.

# PostgreSQL Jobs for December

# PostgreSQL in the News

Planet PostgreSQL:

PostgreSQL Weekly News is brought to you this week by David Fetter

Submit news and announcements by Sunday at 3:00pm PST8PDT to david(at)fetter(dot)org(dot)

# Applied Patches

Michaël Paquier pushed:

- Add some checkpoint/restartpoint status to ps display. This is done for
end-of-recovery and shutdown checkpoints/restartpoints (end-of-recovery
restartpoints don't exist) rather than all types of checkpoints, in cases
where it may not be possible to rely on pg_stat_activity to get a status from
the startup or checkpointer processes. For example, at the end of a crash
recovery, this is useful to know if a checkpoint is running in the startup
process, while previously the ps display may only show some information about
"recovering" something, that can be confusing while a checkpoint runs.
Author: Justin Pryzby Reviewed-by: Nathan Bossart, Kirk Jamison, Fujii Masao,
Michael Paquier Discussion:

- Improve some code around cryptohash functions. This adjusts some code related
to recent changes for cryptohash functions: - Add a variable in md5.h to track
down the size of a computed result, moved from pgcrypto. Note that
pg_md5_hash() assumed a result of this size already. - Call explicit_bzero()
on the hashed data when freeing the context for fallback implementations. For
MD5, particularly, it would be annoying to leave some non-zeroed data around.
- Clean up some code related to recent changes of uuid-ossp. .gitignore still
included md5.c and a comment was incorrect. Discussion:

- doc: Fix explanation related to pg_shmem_allocations. Offsets are shown as
NULL only for anonymous allocations. Author: Benoit Lobréau Reviewed-by:
Kyotaro Horiguchi Discussion:
Backpatch-through: 13

Jeff Davis pushed:

- Revert "Cannot use WL_SOCKET_WRITEABLE without WL_SOCKET_READABLE.". This
reverts commit 3a9e64aa0d96c8ffb6c682b082d0f72b1d373327. Commit 4bad60e3
fixed the root of the problem that 3a9e64aa worked around. This enables
proper pipelining of commands after terminating replication, eliminating an
undocumented limitation. Discussion:
Backpatch-through: 9.5

Tom Lane pushed:

- Improve hash_create()'s API for some added robustness. Invent a new flag bit
HASH_STRINGS to specify C-string hashing, which was formerly the default; and
add assertions insisting that exactly one of the bits HASH_STRINGS,
HASH_BLOBS, and HASH_FUNCTION be set. This is in hopes of preventing
recurrences of the type of oversight fixed in commit a1b8aa1e4 (i.e.,
mistakenly omitting HASH_BLOBS). Also, when HASH_STRINGS is specified, insist
that the keysize be more than 8 bytes. This is a heuristic, but it should
catch accidental use of HASH_STRINGS for integer or pointer keys. (Nearly all
existing use-cases set the keysize to NAMEDATALEN or more, so there's little
reason to think this restriction should be problematic.) Tweak hash_create()
to insist that the HASH_ELEM flag be set, and remove the defaults it had for
keysize and entrysize. Since those defaults were undocumented and basically
useless, no callers omitted HASH_ELEM anyway. Also, remove memset's zeroing
the HASHCTL parameter struct from those callers that had one. This has never
been really necessary, and while it wasn't a bad coding convention it was
confusing that some callers did it and some did not. We might as well save a
few cycles by standardizing on "not". Also improve the documentation for
hash_create(). In passing, improve reinit.c's usage of a hash table by
storing the key as a binary Oid rather than a string; and, since that's a
temporary hash table, allocate it in CurrentMemoryContext for neatness.

- Fix varchar_2.out to match reality in cs_CZ locale. Seems to be a
copy-and-pasteo in c06d6aa4c. Per buildfarm.

- Avoid memcpy() with same source and destination during relmapper init. A
narrow reading of the C standard says that memcpy(x,x,n) is undefined,
although it's hard to envision an implementation that would really misbehave.
However, analysis tools such as valgrind might whine about this; accordingly,
let's band-aid relmapper.c to not do it. See also 5b630501e, d3f4e8a8a,
ad7b48ea0, and other similar fixes. Apparently, none of those folk tried
valgrinding initdb? This has been like this for long enough that I'm
surprised it hasn't been reported before. Back-patch, just in case anybody
wants to use a back branch on a platform that complains about this; we
back-patched those earlier fixes too. Discussion:

- Add a couple of missed .gitignore entries. Any subdirectory that's ignoring
/output_iso/ should also ignore /tmp_check_iso/, which could be left behind by
a failed pg_isolation_regress_check run. I think these have been wrong for
awhile, but it doesn't seem important to fix in back branches.

- Remove now-useless ALWAYS_SUBDIRS entry in src/test/Makefile. Commit 257836a75
added the "locale" subdirectory to SUBDIRS, but neglected to remove it from
ALWAYS_SUBDIRS. This oversight had no functional effect because the
filter-out function would remove it anyway. Still, it's confusing to readers
to list a subdirectory in both places, especially because it makes the
associated comment into a partial lie.

- Avoid memcpy() with same source and destination in pgstat_recv_replslot. Same
type of issue as in commit 53d4f5fef and earlier fixes; also found by
apparently-more-picky-than-the-buildfarm valgrind testing. This one is an
oversight in commit 986816750. Since that's new in HEAD, no need for a

- Doc: improve description of pgbench script weights. Point out the workaround
to be used if you want to write a script file name that includes "@". Clean
up the text a little. Fabien Coelho, additional wordsmithing by me

Peter Eisentraut pushed:

- Clean up ancient test style. Many older tests where written in a style like
`SELECT '' AS two, i.* FROM INT2_TBL` where the first column indicated the
number of expected result rows. This has gotten increasingly out of date, as
the test data fixtures have expanded, so a lot of these were wrong and
misleading. Moreover, this style isn't really necessary, since the psql
output already shows the number of result rows. To clean this up, remove all
those extra columns. Discussion:

Tomáš Vondra pushed:

- Error out when Gather Merge input is not sorted. To build Gather Merge path,
the input needs to be sufficiently sorted. Ensuring this is the responsibility
of the code constructing the paths, but create_gather_merge_plan tried to
handle unsorted paths by adding an explicit Sort. In light of the recent
issues related to Incremental Sort, this is rather fragile. Some of the
expressions may be volatile or parallel unsafe, in which case we can't add the
Sort here. We could do more checks and add the Sort in at least some cases,
but it seems cleaner to just error out and make it clear this is a bug in code
constructing those paths. Author: James Coleman Reviewed-by: Tomas Vondra

Álvaro Herrera pushed:

- Remove useless variable stores. Mistakenly introduced in 4cbe3ac3e867; bug
repaired in 148e632c0541 but the stores were accidentally.

Peter Geoghegan pushed:

- Remove obsolete btrescan() comment. "Ordering stuff" refered to a _bt_first()
call to _bt_orderkeys(). However, the _bt_orderkeys() function was renamed to
_bt_preprocess_keys() by commit fa5c8a055a0. _bt_preprocess_keys() is
directly referenced just after the removed comment already, which seems

Bruce Momjian pushed:

- doc: clarify COPY TO for partitioning/inheritance. It was not clear how COPY
TO behaved with partitioning/inheritance because the paragraphs were so far
apart. Also reword to simplify. Discussion:
Author: Justin Pryzby Backpatch-through: 10

Fujii Masao pushed:

- Revert "Get rid of the dedicated latch for signaling the startup process".
Revert ac22929a26, as well as the followup fix 113d3591b8. Because it broke
the assumption that the startup process waiting for the recovery conflict on
buffer pin should be waken up only by buffer unpin or the timeout enabled in
ResolveRecoveryConflictWithBufferPin(). It caused, for example, SIGHUP signal
handler or walreceiver process to wake that startup process up unnecessarily
frequently. Additionally, add the comments about why that dedicated latch
that the reverted patch tried to get rid of should not be removed. Thanks to
Kyotaro Horiguchi for the discussion. Author: Fujii Masao Discussion:

- pg_stat_statements: Track time at which all statistics were last reset. This
commit adds "stats_reset" column into the pg_stat_statements_info view. This
column indicates the time at which all statistics in the pg_stat_statements
view were last reset. Per discussion, this commit also changes
pg_stat_statements_info code so that "dealloc" column is reset at the same
time as "stats_reset" is reset, i.e., whenever all pg_stat_statements entries
are removed, for the sake of consistency. Previously "dealloc" was reset only
when pg_stat_statements_reset(0, 0, 0) is called and was not reset when
pg_stat_statements_reset() with non-zero value argument discards all entries.
This was confusing. Author: Naoki Nakamichi, Yuki Seino Reviewed-by: Yuki
Seino, Kyotaro Horiguchi, Li Japin, Fujii Masao Discussion:

Amit Kapila pushed:

- Update comment atop of ReorderBufferQueueMessage(). The comments atop of this
function describes behaviour in case of a transactional WAL message only, but
it accepts both transactional and non-transactional WAL messages. Update the
comments to describe behaviour in case of non-transactional WAL message as
well. Ashutosh Bapat, rephrased by Amit Kapila Discussion:

Alexander Korotkov pushed:

- Multirange datatypes. Multiranges are basically sorted arrays of
non-overlapping ranges with set-theoretic operations defined over them. Since
v14, each range type automatically gets a corresponding multirange datatype.
There are both manual and automatic mechanisms for naming multirange types.
Once can specify multirange type name using multirange_type_name attribute in
CREATE TYPE.  Otherwise, a multirange type name is generated automatically.
If the range type name contains "range" then we change that to "multirange".
Otherwise, we add "_multirange" to the end. Implementation of multiranges
comes with a space-efficient internal representation format, which evades
extra paddings and duplicated storage of oids.  Altogether this format allows
fetching a particular range by its index in O(n). Statistic gathering and
selectivity estimation are implemented for multiranges. For this purpose,
stored multirange is approximated as union range without gaps. This field will
likely need improvements in the future. Catversion is bumped. Discussion:
Author: Paul Jungwirth, revised by me Reviewed-by: David Fetter, Corey
Huinker, Jeff Davis, Pavel Stehule Reviewed-by: Alvaro Herrera, Tom Lane,
Isaac Morland, David G. Johnston Reviewed-by: Zhihong Yu, Alexander Korotkov

- Fixes for pg_dump.c regarding multiranges. This commit fixes two wrong version
number checks and one wrong check for null.

- Fix compiler warning introduced in 6df7a9698b.

# Pending Patches

Michaël Paquier sent in another revision of a patch to introduce SHA1 cryptohash

Andrey Borodin sent in another revision of a patch to use a shared lock in
GetMultiXactIdMembers for offsets and members, make the MultiXact local cache
size configurable, add a condition variable to wait for next MultXact offset in
a corner case, and add GUCs to tune MultiXact SLRUs.

Kyotaro HORIGUCHI sent in a patch to ensure that invalid-page-header error
messages are cancelled only when appropriate, i.e. only on non-standby nodes.

Andrey V. Lepikhov sent in another revision of a patch to move the multi-insert
decision logic into the executor, and use same along with three new methods to
the FDW API: BeginForeignCopy, ExecForeignCopy, and EndForeignCopy, to speed up
COPY FROM as applied to tables with remote partitions.

Fujii Masao sent in two more revisions of a patch to add a
log_recovery_conflict_waits GUC that does what it says on the label.

Tom Lane sent in two revisions of a patch to fix the way PL/pgsql does

Bharath Rupireddy and Hou Zhijie traded patches to make it possible to use
parallel INSERTs in CTAS.

Nathan Bossart and Kyotaro HORIGUCHI traded patches to intended to fix a bug
that manifested as archive status ".ready" files may be created too early.

Kirk Jamison sent in another revision of a patch to make dropping relation
buffers more efficient using dlist.

Amit Kapila sent in three more revisions of a patch to implement logical
decoding of two-phase transactions.

Yuzuko Hosoya sent in another revision of a patch to make autoanalyze work
better on partitioned tables.

Atsushi Torikoshi sent in a patch to add a wait_start colunmn to the pg_locks

Iwata Aya sent in another revision of a patch to add tracing to libpq.

Laurenz Albe sent in two more revisions of a patch to add session statistics to

Peter Eisentraut sent in another revision of a patch to change return type of
EXTRACT to numeric.

Fabrízio de Royes Mello sent in another revision of a patch to implement minimal
logical decoding on standbys.

Justin Pryzby sent in two more revisions of a patch to allow CLUSTER, VACUUM
FULL and REINDEX to change tablespace on the fly.

Jeff Davis sent in a patch to enforce ASCII restore point names.

Bharath Rupireddy sent in a patch intended to fix a bug that manifested as -
cached connection leaks from the postgres_fdw if the associated user mapping is
dropped by moving the place to close all the invalid connections to

Peter Smith sent in two more revisions of a patch to make it possible to use
multiple workers on tablesync.

Bruce Momjian and Neil Chen traded patches to implement key management.

Tomáš Vondra sent in another revision of a patch to consider unsorted paths in
generate_useful_gather_paths, check parallel safety in
generate_useful_gather_paths, disallow SRFs when considering sorts below Gather
Merge, avoid searching for volatile expr in find_em_expr_usable_for_sorting_rel,
and improve find_em_expr_usable_for_sorting_rel comment by clarifying the
relationship between find_em_expr_usable_for_sorting_rel and
prepare_sort_from_pathkeys, i.e. what restrictions need to be shared between
those two places.

Laurenz Albe sent in another revision of a patch to fix a large violation of
POLA in psql's \e family of functions, namely that if the editor was exited
before any changes were made, the previous query would be executed. The fix is
to clear the query buffer.

Heikki Linnakangas sent in another revision of a patch to make resowners more
easily extensible by using a single array and hash instead of one for each
object kind.

Justin Pryzby sent in another revision of a patch to pg_upgrade/ which
makes some changes needed to allow testing upgrade from v11.

Heikki Linnakangas sent in a patch to do COPY FROM encoding
conversion/verification in larger chunks.

Fujii Masao sent in three revisions of a patch to intended to fix a bug that
manifested as failure to detect a deadlock between backend and recovery by
adding a check for that situation.

Michaël Paquier sent in two revisions of a patch to refactor the HMAC

Konstantin Knizhnik sent in two more revisions of a patch to add a way to fire
triggers on login.

David Fetter sent in a patch to implement \gsetenv, analogous to \gset, only for
environment variables in psql.

Önder Kalacı sent in another revision of a patch to implement row filtering for
logical replication.

Denis Smirnov and Konstantin Knizhnik traded patches to implement compression
for libpq.

Peter Eisentraut sent in a patch to hange seconds argument of `make_*()` functions
to numeric.

Fujii Masao sent in a patch to intended to fix a bug that manifested as
STANDBY_LOCK_TIMEOUT may not interrupt ProcWaitForSignal() by ensuring that
StandbyLockTimeoutHandler() sets a latch.

Bharath Rupireddy sent in another revision of a patch to fail fast in CTAS/CMV
if the relation already exists.

Gilles Darold sent in another revision of a patch to fix a race condition in
DDL where a function's namespace can disappear while leaving the function in

Michaël Paquier sent in a patch to remove duplicate checks of ASCII-ness and
make the one left more efficient.

Justin Pryzby sent in two more revisions of a patch to allow psql's \dti+ to
look at `pg_toast.*`.

David CARLIER sent in a patch to implements SPIN_LOCK on ARM.

Michaël Paquier and Heikki Linnakangas traded patches to intended to fix a bug
that manifested as incorrect allocation handling for cryptohash functions with

Tom Lane and Alexander Lakhin traded patches to show examples of using Valgrind
for TAP tests.

Craig Ringer sent in a patch to identify LWLocks in tracepoints by fixing a
bogus tracepoint where an lwlock__acquire event would be fired from
LWLockWaitForVar despite that function never actually acquiring the lock, adding
the tranche id and lock pointer for each trace hit, adding a single-path
tracepoint for all lock acquires and releases, and adding some comments on
LWLock tranches to clarify matters for future readers.

Konstantin Knizhnik sent in a patch to reimplement BufferAlloc, removing double
locking in the process.

Pavel Stěhule sent in another revision of a patch to implement schema variables.

Tom Lane sent in a patch to fix a corner case bug in jsonb_concat().

Thomas Munro sent in two revisions of a patch to add pg_preadv() and
pg_pwritev(), which provide synchronous scatter/gather I/O routines, and then
use same to use vectored I/O to zero WAL segments.

Tomáš Vondra sent in another revision of a patch to implement BRIN multi-range

Browse pgsql-announce by date

  From Date Subject
Next Message Zalando SE via PostgreSQL Announce 2020-12-26 06:15:20 Postgres Operator v1.6.0
Previous Message PoWA via PostgreSQL Announce 2020-12-18 18:05:27 PoWA 4.1 is out!