== PostgreSQL Weekly News - May 10, 2020 ==

From: David Fetter <david(at)fetter(dot)org>
To: PostgreSQL Announce <pgsql-announce(at)postgresql(dot)org>
Subject: == PostgreSQL Weekly News - May 10, 2020 ==
Date: 2020-05-10 19:42:26
Message-ID: 20200510194226.GA1167@fetter.org
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-announce

== PostgreSQL Weekly News - May 10, 2020 ==

PostgreSQL Conference Europe 2020 will be held on October 20-23, 2020 in Berlin,
Germany. The CfP is open through July 31, 2020 at https://2020.pgconf.eu/callforpapers

The apt repository has been extended to cover the arm64 architecture.

PGConf London has been cancelled.

== PostgreSQL Product News ==

pgMustard 3.0, a user interface for 'explain analyze' which provides performance
tips, released.

Crunchy PostgreSQL Operator 4.3.0, a system for deploying and managing open
source PostgreSQL clusters on Kubernetes, released.

pg_probackup 2.3.0, a utility to manage backup and recovery of PostgreSQL
database clusters, released.

pg_activity 1.6, a top-like application for PostgreSQL server activity
monitoring, released.

== PostgreSQL Local ==

PGCon 2020 will take place online on May 26-29, 2020.

PG Day Russia will take place in Saint Petersburg on July 10, 2020.

FOSS4G 2020, will take place in Calgary, Alberta, Canada August 24-29 2020.
the Call for Papers is currently open at https://2020.foss4g.org/speakers/

PGDay Ukraine will take place September 5th, 2020 in Lviv at the Bank Hotel.

pgDay Israel 2020 will take place on September 10, 2020 in Tel Aviv.

PGDay Austria will take place September 18, 2020 at Schloss Schoenbrunn
(Apothekertrakt) in Vienna. The CfP is open through May 17, 2020.

== PostgreSQL in the News ==

Planet PostgreSQL: http://planet.postgresql.org/

PostgreSQL Weekly News is brought to you this week by David Fetter

Submit news and announcements by Sunday at 3:00pm PST8PDT to david(at)fetter(dot)org(dot)

== Applied Patches ==

Tom Lane pushed:

- Doc: update sections 9.22 - 9.30 for new function table layout. With the usual
quota of minor and less-minor editorial changes.

- Doc: standardize markup a bit more. We had a mishmash of <replaceable>,
<replaceable class="parameter">, and <parameter> markup for operator/function
arguments. Use <parameter> consistently for things that are in fact names of
parameters (including OUT parameters), reserving <replaceable> for things that
aren't. The latter class includes some made-up-by-the-docs type class names,
like "numeric_type", as well as placeholders for arguments that don't have
well-defined types. Possibly we could do better with those categories as
well, but for the moment I'm content not to have parameter names marked up in
different ways in different places. (This commit aligns the earlier sections
of chapter 9 with a policy that I'd arrived at while working on commit
1ad23335f, which is why the last few sections need no changes.)

- Doc: improve PDF presentation of some tables by adjusting column widths. The
PDF toolchain defaults to laying out all columns of a table with equal widths,
in contrast to the HTML rendering which automatically varies the column widths
to fit the data. In many places, this results in very badly laid-out tables,
with lots of useless whitespace in some places and text that overruns its cell
in other places. For tables that have reasonably static content, we can
improve matters by adding <colspec> entries to hand-assign the column widths.
This commit does that for a few of the tables that were worst off; it
eliminates close to 200 "contents ... exceed the available area" warnings in
an A4 PDF build. I also forced align="left" in these tables, overriding the
PDF toolchain's default which is evidently "justify". (The HTML toolchain
seems to default to that already.) Anyplace where things are tight enough
that we need to worry about this, forced justification tends to look truly

- Fix severe memory leaks in GSSAPI encryption support. Both the backend and
libpq leaked buffers containing encrypted data to be transmitted, so that the
process size would grow roughly as the total amount of data sent. There were
also far-less-critical leaks of the same sort in GSSAPI session establishment.
Oversight in commit b0b39f72b, which I failed to notice while reviewing the
code in 2c0cdc818. Per complaint from pmc(at)citylink(dot) Back-patch to v12 where
this code was introduced. Discussion:

- Doc: warn that timezone abbreviations don't work in recovery_target_time.
Moving this setting into the main configuration file was ill-considered,
perhaps, because that typically causes it to be set before
timezone_abbreviations has been set. Which in turn means that zone
abbreviations don't work, only full zone names. We could imagine hacking
things so that such cases do work, but the stability of the hack would be
questionable, and the value isn't really that high. Instead just document
that you should use a numeric zone offset or a full zone name. Per bug #16404
from Reijo Suhonen. Back-patch to v12 where this was changed. Discussion:

- Doc: further fooling-about with rendering of tables in PDF output. I concluded
that we really just ought to force all tables in PDF output to default to
"left" alignment (instead of "justify"); that is what the HTML toolchain does
and that's what most people have been designing the tables to look good with.
There are few if any places where "justify" produces better-looking output,
and there are many where it looks horrible. So change stylesheet-fo.xsl to
make that true. Also tweak column widths in a few more tables to make them
look better and avoid "exceed the available area" warnings. This commit fixes
basically everything that can be fixed through that approach. The remaining
tables that give warnings either are scheduled for redesign as per recent
discussions, or need a fundamental rethink because they Just Don't Work in a
narrow view.

- Doc: remove now-redundant align specifications in colspecs. In the wake of
commit f21599311, we don't need to set table columns' align specs retail.
Undo a few such settings I'd added in commit 5545b69ae. (The column width
adjustments stay, though.)

- Doc: update remaining tables of functions/operators for new layout. This
converts the contrib documentation to the new style, and mops up a couple of
function tables that were outside chapter 9 in the main docs. A few contrib
modules choose not to present their functions in the standard tabular format.
There might be room to rethink those decisions now that the standard format is
more friendly to verbose descriptions. But I have not undertaken to do that
here; I just converted existing tables.

- Fix YA text phrase search bug. checkcondition_str() failed to report multiple
matches for a prefix pattern correctly: it would dutifully merge the match
positions, but then after exiting that loop, if the last prefix-matching word
had had no suitable positions, it would report there were no matches. The
upshot would be failing to recognize a match that the query should match. It
looks like you need all of these conditions to see the bug: * a phrase search
(else we don't ask for match position details) * a prefix search item (else we
don't get to this code) * a weight restriction (else checkclass_str won't
fail) Noted while investigating a problem report from Pavel Borisov, though
this is distinct from the issue he was on about. Back-patch to 9.6 where
phrase search was added.

- Update oidjoins regression test for v13. We seem to have forgotten to do this
in the v12 cycle, so add it as a task in the RELEASE_CHANGES list, in hopes we
won't forget again. While here, fix findoidjoins.c so that it actually works
in the new dispensation where OID is a regular column, and change it to only
consider system relations (this avoids being fooled by the OID column in the
brintest test table). Also tweak the largeobject test so that the
somewhat-recently-added manual creation of a LO with an OID in the system
range doesn't fool findoidjoins.c. For the moment I just made that use an
unused OID, but we might have to find a more robust solution someday.

- Fix findoidjoins to recognize oidvector columns. Somehow we'd never noticed
this oversight, even though it means that such basic columns as
pg_proc.proargtypes were not being validated by the oidjoins test. Correct
the query and update the test script with the newly-found dependencies.

- Doc: fix assorted misstatements of fact in catalog & system view docs. I made
up a very crude hack to compare the docs with reality (as embodied in the
system catalogs) ... and indeed they don't match everywhere. Missing oid
columns, wrong data types, wrong "references" links, columns listed in the
wrong order. None of this seems quite important enough to back-patch.

Peter Geoghegan pushed:

- Add posting list tuple amcheck test case. Add a test case to contrib/amcheck
that creates coverage of code paths that are used to verify posting list
tuples (tuples created when deduplication merges together existing tuples to
avoid a leaf page split).

- Remove obsolete amcheck comment. Oversight in commit d114cc53.

- Normalize _bt_findsplitloc() argument names. Oversight in commit bc3087b626d.

- Refactor nbtree deletion INCOMPLETE_SPLIT check. Factor out code common to
_bt_lock_branch_parent() and _bt_pagedel() into a new utility function. This
new function is used to check that the left sibling of a deletion target page
does not have the INCOMPLETE_SPLIT page flag set. If it is set then deletion
is unsafe; there won't be a usable pivot tuple (with a downlink) in the parent
page that points to the deletion target page. The page deletion algorithm is
not prepared to deal with that. Also restructure an existing, related utility
function that checks if the right sibling of the target page has the
ISHALFDEAD page flag set. This organization highlights the symmetry between
the two cases. The goal is to make the design of page deletion clearer. Both
functions involve a sibling page with a flag that indicates that there was an
interrupted operation (a page split or a page deletion) that resulted in a
page pointed to by sibling pages, but not pointed to in the parent. And, both
functions indicate if page deletion is unsafe due to the absence of a
particular downlink in the parent page.

Amit Kapila pushed:

- Change the display of WAL usage statistics in Explain. In commit 33e05f89c5,
we have added the option to display WAL usage statistics in Explain and
auto_explain. The display format used two spaces between each field which is
inconsistent with Buffer usage statistics which is using one space between
each field. Change the format to make WAL usage statistics consistent with
Buffer usage statistics. This commit also changed the usage of "full page
writes" to "full page images" for WAL usage statistics to make it consistent
with other parts of code and docs. Author: Julien Rouhaud, Amit Kapila
Reviewed-by: Justin Pryzby, Kyotaro Horiguchi and Amit Kapila Discussion:

- Fix inconsistency in pg_buffercache docs. Commit 6e654546fb avoids locking
bufmgr partitions to make pg_buffercache less disruptive on production systems
but forgot to update the docs. Reported-by: Sawada Masahiko Author: Sawada
Masahiko Reviewed-by: Amit Kapila Backpatch-through: 10 Discussion:

Bruce Momjian pushed:

- doc: first draft of PG 13 release notes. This still needs markup, indenting,
and word wrap.

- doc: update PG 13 release notes after first draft. Minor corrections from

- doc: Fix PG 13 release note markup.

- doc: update PG 13 release notes for glossary and NO DEPENDS.

- doc: normalize contributor names in PG 13 release notes.

- doc: PG 13 retnote adjustments. Reported-by: Justin Pryzby

- doc: PG 13 release note adjustments, Justin Pryzby v2. Reported-by: Justin

- doc: PG 13 relnotes, add attributions and wording changes.

- doc: PG 13 relnotes, add pgbench script item.

- doc: PG 13 renotes: adjust attribution and pgbench item.

- doc: PG 13 relnotes, fix markup.

- pgbench: document that the default data loading is client-side. Reported-by:
Fabien COELHO Discussion:

- doc: PG 13 relnotes, update TOAST item to mention decompression. Reported-by:
Andrey M. Borodin Discussion:

- doc: PG 13 relnotes: adjust partition items and attributions. This merges
three partition publication items into two.

- doc: PG 13 relnotes: adjust wording and Unicode item.

- doc: PG 13 relnotes: adjust partitioning items. Reported-by: Amit Langote

Michaël Paquier pushed:

- Doc: Outline REPLICATION before SUPERUSER privilege. The following docs are
updated: - High-availaility section - pg_basebackup - pg_receivewal Per the
principle of least privilege, we want to encourage users to interact with
those areas using roles that have replication rights, but superusers were
mentioned first. Author: Daniel Gustafsson Reviewed-by: Fujii Masao, Michael
Paquier Discussion:

- Handle spaces for Python install location in MSVC scripts. Attempting to use
an installation path of Python that includes spaces caused the MSVC builds to
fail. This fixes the issue by using the same quoting method as ad7595b for
OpenSSL. Author: Victor Wagner Discussion:
Backpatch-through: 9.5

- Remove smgrdounlink() in smgr.c from the code tree. The last caller of this
routine was removed in b416691, and as a wise man said one day, dead code
tends to silently break. Per discussion between Fujii Masao, Peter Geoghegan,
Vignesh C and me. Reported-by: Peter Geoghegan Discussion:

Peter Eisentraut pushed:

- Fix capitalization of messages, per style guide.

- Propagate ALTER TABLE ... SET STORAGE to indexes. When creating a new index,
the attstorage setting of the table column is copied to regular
(non-expression) index columns. But a later ALTER TABLE ... SET STORAGE is
not propagated to indexes, thus creating an inconsistent and undumpable state.

- Fix several DDL issues of generated columns versus inheritance. Several
combinations of generated columns and inheritance in CREATE TABLE were not
handled correctly. Specifically: - Disallow a child column specifying a
generation expression if the parent column is a generated column. The child
column definition must be unadorned and the parent column's generation
expression will be copied. - Prohibit a child column of a generated parent
column specifying default values or identity. - Allow a child column of a
not-generated parent column specifying itself as a generated column. This
previously did not work, but it was possible to arrive at the state via
other means (involving ALTER TABLE), so it seems sensible to support it.
Add tests for each case. Also add documentation about the rules involving
generated columns and inheritance. Discussion:

- Unify find_other_exec() error messages. There were a few different ways to
line-wrap the error messages. Make them all the same, and use placeholders
for the actual program names, to save translation work.

- Improve use of prepositions in messages. *in* database, *in* cluster, *on*
server; and some related fixes

Álvaro Herrera pushed:

- Heed lock protocol in DROP OWNED BY. We were acquiring object locks then
deleting objects one by one, instead of acquiring all object locks first,
ignoring those that did not exist, and then deleting all objects together.
The latter is the correct protocol to use, and what this commits changes to
code to do. Failing to follow that leads to "cache lookup failed for relation
XYZ" error reports when DROP OWNED runs concurrently with other DDL -- for
example, a session termination that removes some temp tables. Author: Álvaro
Herrera Reported-by: Mithun Chicklore Yogendra (Mithun CY) Reviewed-by: Ahsan
Hadi, Tom Lane Discussion:

- Rework XLogReader callback system. Code review for 0dc8ead46363, prompted by a
bug closed by 91c40548d5f7. XLogReader's system for opening and closing
segments had gotten too complicated, with callbacks being passed at both the
XLogReaderAllocate level (read_page) as well as at the WALRead level
(segment_open). This was confusing and hard to follow, so restructure things
so that these callbacks are passed together at XLogReaderAllocate time, and
add another callback to the set (segment_close) to make it a coherent whole.
Also, ensure XLogReaderState is an argument to all the callbacks, so that they
can grab at the ->private data if necessary. Document the whole arrangement
more clearly. Author: Álvaro Herrera <alvherre(at)alvh(dot)no-ip(dot)org> Reviewed-by:
Kyotaro Horiguchi <horikyota(dot)ntt(at)gmail(dot)com> Discussion:

- pg_restore: Provide file name with one failure message. Almost all error
messages already include file name where relevant, but this one had been
overlooked. Repair. Backpatch to 9.5. Author: Euler Taveira
<euler(dot)taveira(at)2ndquadrant(dot)com> Discussion:
Reviewed-by: Álvaro Herrera <alvherre(at)alvh(dot)no-ip(dot)org>

Fujii Masao pushed:

- Report missing wait event for timeline history file. TimelineHistoryRead and
TimelineHistoryWrite wait events are reported during waiting for a read and
write of a timeline history file, respectively. However, previously,
TimelineHistoryRead wait event was not reported while readTimeLineHistory()
was reading a timeline history file. Also TimelineHistoryWrite was not
reported while writeTimeLineHistory() was writing one line with the details of
the timeline split, at the end. This commit fixes these issues. Back-patch to
v10 where wait events for a timeline history file was added. Author: Masahiro
Ikeda Reviewed-by: Michael Paquier, Fujii Masao Discussion:

Tomáš Vondra pushed:

- Fix handling of REWIND/MARK/BACKWARD in incremental sort. The executor flags
were not handled entirely correctly, although the bugs were mostly harmless
and it was mostly comment inaccuracy. We don't need to strip any of the flags
for child nodes. Incremental sort does not support backward scans of
mark/restore, so MARK/BACKWARDS flags should not be possible. So we simply
ensure this using an assert, and we don't bother removing them when
initializing the child node. With REWIND it's a bit less clear - incremental
sort does not support REWIND, but there is no way to signal this - it's legal
to just ignore the flag. We however continue passing the flag to child nodes,
because they might be useful to leverage that. Reported-by: Michael Paquier
Author: James Coleman Reviewed-by: Tomas Vondra Discussion:

- Do no reset bounded before incremental sort rescan. ExecReScanIncrementalSort
was resetting bounded=false, which means the optimization would be disabled on
all rescans. This happens because ExecSetTupleBound is called before the
rescan, not after it. Author: James Coleman Reviewed-by: Tomas Vondra
Discussion: https://postgr.es/m/20200414065336.GI1492@paquier.xyz

- Simplify show_incremental_sort_info a bit. Incremental sort always processes
at least one full group group before switching to prefix groups, so it's
enough to check just the number of full groups. There was no risk of division
by zero due to the extra condition, but it made the code harder to understand.
Reported-by: Ranier Vilela Discussion:

== Pending Patches ==

Nathan Bossart sent in a patch to prevent race condition when writing

Peter Eisentraut sent in another revision of a patch to unify the drop-by-OID

Juan José Santamaría Flecha and Amit Kapila traded patches to fix the MSVC build
for versions 2015 and later.

David Kimura sent in another revision of a patch to implement fallback of batch
0 for serial adaptive hash join.

Ranier Vilela sent in a patch to fix a possible overflow in postingoff.

Julien Rouhaud sent in another revision of a patch to implement collation

Dilip Kumar sent in three more revisions of a patch to fix an infelicity between
logical_work_mem and logical streaming of large in-progress transactions.

Masahiko Sawada sent in a patch to fix index cleanup.

Robert Haas sent in two more revisions of a patch to stop exporting
basebackup.c's sendTablespace(), and do a minor code cleanup for

Tom Lane and Álvaro Herrera traded patches to fix the catalog description
formatting in the docs.

Fujii Masao sent in a patch to ensure that SLRUStats is initialized to 0 in

Erik Nordström sent in a patch to add a table expansion hook.

Justin Pryzby sent in another revision of a patch to make the pg_ls_* functions
show directories and shared filesets.

Pavel Borisov sent in a patch to fix a bug in GIN queries that could cause false

Fujii Masao sent in another revision of a patch to report wait events about
timeline history.

Andy Fan sent in another revision of a patch to track uniqueness and use this
information to avoid doing unneeded work when possible.

Fujii Masao sent in two more revisions of a patch to add operators which add and
subtract numerics on the pg_lsn type.

Movead Li sent in another revision of a patch to implement CSN-based snapshots.

Álvaro Herrera sent in two more revisions of a patch to fix the WALRead API to
take seg/segcxt from XLogReaderState, and Simplify XLogReader's open_segment API
by installiing it directly in XLogReaderState->seg.ws_file instead of returning
the file descriptor.

Robert Haas sent in a patch to refactor basebackup.c

Heikki Linnakangas sent in a patch to fix a bug wherein pendingOps table was not
cleared with fsync=off.

Cary Huang sent in another revision of a patch to replicate sequence updates in
the logical WAL stream.

Melanie Plageman sent in another revision of a patch to implement adaptive
hashjoin, and provisionally detach unless last worker.

Ranier Vilela sent in another revision of a patch to fix a possible division by
zero in explain.c.

Andres Freund sent in a patch to add -Wold-style-definition to CFLAGS, modifying
the few functions it would have warned on so they no longer do.

Justin Pryzby sent in a patch to make INSERT SELECT use a BulkInsertState.

Alexander Lakhin sent in a patch to fix a bug in the gcov tests where coverage
data missed some information when a postgres node was stopped in 'immediate'

Browse pgsql-announce by date

  From Date Subject
Next Message Christoph Berg 2020-05-11 07:53:07 Re: arm64 on apt.postgresql.org
Previous Message David Steele 2020-05-08 20:30:55 Re: arm64 on apt.postgresql.org