== PostgreSQL Weekly News - November 4, 2018 ==

From: David Fetter <david(at)fetter(dot)org>
To: PostgreSQL Announce <pgsql-announce(at)postgresql(dot)org>
Subject: == PostgreSQL Weekly News - November 4, 2018 ==
Date: 2018-11-04 23:51:51
Message-ID: 20181104235151.GA4030@fetter.org
Views: Raw Message | Whole Thread | Download mbox
Lists: pgsql-announce

== PostgreSQL Weekly News - November 4, 2018 ==

FOSDEM PGDay 2019, a one day conference held before the main FOSDEM event will
be held in Brussels, Belgium, on Feb 1st, 2019. The CfP is open until November
9th, 2018 at https://2019.fosdempgday.org/callforpapers/ .

PGConf India 2019 will be on February 13-15, 2019 in Bengaluru, Karnataka.
Proposals are due via https://goo.gl/forms/F9hRjOIsaNasVOAz2 by October 31st, 2017.

Prague PostgreSQL Developer Day 2019 (P2D2 2019) is a two-day
conference that will be held on February 13-14, 2019 in Prague, Czech Republic.
The CfP is open until January 4, 2018 at https://p2d2.cz/callforpapers

== PostgreSQL Product News ==

pgpool-II 4.0.1, 3.7.6, 3.6.13, 3.5.17, and 3.4.20 released.

Postgres-XL 10R1, a fork of PostgreSQL featuring massively parallel
processing, released.

sqlite_fdw 1.1.0 released.

pglogical 2.2.1, a logical-WAL-based replication system for PostgreSQL, released.

repmgr 4.2, a replication manager for PostgreSQL, released.

pgmetrics 1.5, a command-line tool for PostgreSQL metrics, released.

pgconfigurator, a tool for estimating initial parameters for PostgreSQL
instances, released.

pgAdmin4 3.5, a web- and native GUI control center for PostgreSQL, released.

== PostgreSQL Jobs for November ==


== PostgreSQL Local ==

pgday.Seoul 2018 will be held in Seoul, South Korea on November 3, 2018 in
Seoul. Korean language information is here:

PGDay Argentina 2018 will be held in Buenos Aires on November 21, 2018.

2Q PGConf will be on December 4-5, 2018 in Chicago, IL.

PGConf.ASIA 2018 will take place on December 10-12, 2018 in Akihabara, Tokyo,

pgDay Paris 2019 will be held in Paris, France on March 12, 2019
at 199bis rue Saint-Martin. The CfP is open until November 30, 2018.

PGConf APAC 2019 will be held in Singapore March 19-21, 2019. The CfP is open
at http://2019.pgconfapac.org/cfp through November 16, 2018.

PGDay.IT 2019 will take place May 16th and May 17th in Bologna, Italy. The CfP
is open at https://2019.pgday.it/en/blog/cfp and the Call for Workshops is at
https://2019.pgday.it/en/blog/cfw until January 15, 2019.

== PostgreSQL in the News ==

Planet PostgreSQL: http://planet.postgresql.org/

PostgreSQL Weekly News is brought to you this week by David Fetter

Submit news and announcements by Sunday at 3:00pm PST8PDT to david(at)fetter(dot)org(dot)

== Applied Patches ==

Thomas Munro pushed:

- Remove incorrect comment in dshash.c. Back-patch to 11. Author: Antonin
Houska Discussion: https://postgr.es/m/8726.1540553521%40localhost

- Fix NULL handling in multi-batch Parallel Hash Left Join. NULL keys in left
joins were skipped when building batch files. Repair, by making the
keep_nulls argument to ExecHashGetHashValue() depend on whether this is a left
outer join, as we do in other paths. Bug #15475. Thinko in 1804284042e.
Back-patch to 11. Reported-by: Paul Schaap Diagnosed-by: Andrew Gierth
Dicussion: https://postgr.es/m/15475-11a7a783fed72a36%40postgresql.org

Michaël Paquier pushed:

- Improve description of pg_attrdef in documentation. The reference to
pg_attribute is switched to a link, which is more useful for the html
documentation. The conditions under which a default value is defined for a
given column are made more general. Author: Daniel Gustafsson Reviewed-by:
Tom Lane Discussion:

- Add pg_partition_tree to display information about partitions. This new
function is useful to display a full tree of partitions with a partitioned
table given in output, and avoids the need of any complex WITH RECURSIVE query
when looking at partition trees which are deep multiple levels. It returns a
set of records, one for each partition, containing the partition's name, its
immediate parent's name, a boolean value telling if the relation is a leaf in
the tree and an integer telling its level in the partition tree with given
table considered as root, beginning at zero for the root, and incrementing by
one each time the scan goes one level down. Author: Amit Langote Reviewed-by:
Jesper Pedersen, Michael Paquier, Robert Haas Discussion:

- Consolidate cross-option checks in pg_restore. This moves one check for
conflicting options from the archive restore code to the main function where
other similar checks are performed. Also reword the error message to be
consistent with other messages. The only option combination impacted is
--create specified with --single-transaction, and informing the caller at an
early step saves from opening the archive worked on. A TAP test is added for
this combination. Author: Daniel Gustafsson Reviewed-by: Fabien Coelho
Discussion: https://postgr.es/m/616808BD-4B59-4E6C-97A9-7317F62D5570@yesql.se

- Lower error level from PANIC to FATAL when restoring slots at startup. When
restoring slot information from disk at startup and filling in shared memory
information, the startup process would issue a PANIC message if more slots are
found than what max_replication_slots allows, and then Postgres generates a
core dump, recommending to increase max_replication_slots. This gives users a
switch to crash Postgres at will by creating slots, lower the configuration to
not support it, and then restart it. Making Postgres crash hard in this case
is overdoing it just to give a recommendation to users. So instead use a
FATAL, which makes Postgres fail to start without crashing, still giving the
recommendation. This is more consistent with what happens for prepared
transactions for example. Author: Michael Paquier Reviewed-by: Andres Freund
Discussion: https://postgr.es/m/20181030025109.GD1644@paquier.xyz

Peter Eisentraut pushed:

- pg_restore: Augment documentation for -N option. This was forgotten when the
option was added. Author: Michael Banck <michael(dot)banck(at)credativ(dot)de>

- Exclude temporary directories from pgindent. Exclude tmp_check and
tmp_install from pgindent. In a fully-built tree, pgindent would spend a lot
of time digging through these directories and ends up re-indenting installed
header files.

- pg_rewind: Remove unused macro. This has never been used while pg_rewind was
in the tree (possibly once copied from pg_upgrade).

- Remove obsolete pg_constraint.consrc column. This has been deprecated and
effectively unused for a long time. Reviewed-by: Daniel Gustafsson

- Remove obsolete pg_attrdef.adsrc column. This has been deprecated and
effectively unused for a long time. Reviewed-by: Alvaro Herrera
<alvherre(at)2ndquadrant(dot)com> Reviewed-by: Daniel Gustafsson <daniel(at)yesql(dot)se>

Magnus Hagander pushed:

- Fix missing whitespace in pg_dump ref page. Author: Daniel Gustafsson

- Fix some spelling errors in the documentation. Author: Daniel Gustafsson

- Fix spelling errors and typos in comments. Author: Daniel Gustafsson

Tom Lane pushed:

- Fix interaction of CASE and ArrayCoerceExpr. An array-type coercion appearing
within a CASE that has a constant (after const-folding) test expression was
mangled by the planner, causing all the elements of the resulting array to be
equal to the coerced value of the CASE's test expression. This is my
oversight in commit c12d570fa: that changed ArrayCoerceExpr to use a
subexpression involving a CaseTestExpr, and I didn't notice that
eval_const_expressions needed an adjustment to keep from folding such a
CaseTestExpr to a constant when it's inside a suitable CASE. This is another
in what's getting to be a depressingly long line of bugs associated with
misidentification of the referent of a CaseTestExpr. We're overdue to
redesign that mechanism; but any such fix is unlikely to be back-patchable
into v11. As a stopgap, fix eval_const_expressions to do what it must here.
Also add a bunch of comments pointing out the restrictions and assumptions
that are needed to make this work at all. Also fix a related oversight:
contain_context_dependent_node() was not aware of the relationship of
ArrayCoerceExpr to CaseTestExpr. That was somewhat fail-soft, in that the
outcome of a wrong answer would be to prevent optimizations that could have
been made, but let's fix it while we're at it. Per bug #15471 from Matt
Williams. Back-patch to v11 where the faulty logic came in. Discussion:

- Update time zone data files to tzdata release 2018g. DST law changes in
Morocco (with, effectively, zero notice). Historical corrections for Hawaii.

- Sync our copy of the timezone library with IANA release tzcode2018g. This
patch absorbs an upstream fix to "zic" for a recently-introduced bug that made
it output data that some 32-bit clients couldn't read. Given the current
source data, the bug only manifests in zones with leap seconds, which we don't
generate, so that there's no actual change in our installed timezone data
files from this. Still, in case somebody uses our copy of "zic" to do
something else, it seems best to apply the fix promptly. Also, update the
README's notes about converting upstream code to our conventions.

- Fix memory leak in repeated SPGIST index scans. spgendscan neglected to pfree
all the memory allocated by spgbeginscan. It's possible to get away with that
in most normal queries, since the memory is allocated in the executor's
per-query context which is about to get deleted anyway; but it causes severe
memory leakage during creation or filling of large exclusion-constraint
indexes. Also, document that amendscan is supposed to free what ambeginscan
allocates. The docs' lack of clarity on that point probably caused this bug
to begin with. (There is discussion of changing that API spec going forward,
but I don't think it'd be appropriate for the back branches.) Per report from
Bruno Wolff. It's been like this since the beginning, so back-patch to all
active branches. In HEAD, also fix an independent leak caused by commit
2a6368343 (allocating memory during spgrescan instead of spgbeginscan, which
might be all right if it got cleaned up, but it didn't). And do a bit of code
beautification on that commit, too. Discussion:

- Yet further rethinking of build changes for macOS Mojave. The solution
arrived at in commit e74dd00f5 presumes that the compiler has a suitable
default -isysroot setting ... but further experience shows that in many
combinations of macOS version, XCode version, Xcode command line tools
version, and phase of the moon, Apple's compiler will *not* supply a default
-isysroot value. We could potentially go back to the approach used in commit
68fc227dd, but I don't have a lot of faith in the reliability or life
expectancy of that either. Let's just revert to the approach already shipped
in 11.0, namely specifying an -isysroot switch globally. As a partial
response to the concerns raised by Jakob Egger, adjust the contents of
Makefile.global to look like CPPFLAGS = -isysroot $(PG_SYSROOT) ...
PG_SYSROOT = /path/to/sysroot This allows overriding the sysroot path at build
time in a relatively painless way. Add documentation to installation.sgml
about how to use the PG_SYSROOT option. I also took the opportunity to
document how to work around macOS's "System Integrity Protection" feature. As
before, back-patch to all supported versions. Discussion:

- First-draft release notes for 11.1. As usual, the release notes for other
branches will be made by cutting these down, but put them up for community
review first. Note that a fair percentage of the entries apply only to prior
branches because their issue was already fixed in 11.0.

- Make ts_locale.c's character-type functions cope with UTF-16. On Windows, in
UTF8 database encoding, what char2wchar() produces is UTF16 not UTF32, ie,
characters above U+FFFF will be represented by surrogate pairs. t_isdigit()
and siblings did not account for this and failed to provide a large enough
result buffer. That in turn led to bogus "invalid multibyte character for
locale" errors, because contrary to what you might think from char2wchar()'s
documentation, its Windows code path doesn't cope sanely with buffer overflow.
The solution for t_isdigit() and siblings is pretty clear: provide a 3-wchar_t
result buffer not 2. char2wchar() also needs some work to provide more
consistent, and more accurately documented, buffer overrun behavior. But
that's a bigger job and it doesn't actually have any immediate payoff, so
leave it for later. Per bug #15476 from Kenji Uno, who deserves credit for
identifying the cause of the problem. Back-patch to all active branches.
Discussion: https://postgr.es/m/15476-4314f480acf0f114@postgresql.org

- Fix unused-variable warning. Discussion:

- Fix bugs in plpgsql's handling of CALL argument lists. exec_stmt_call() tried
to extract information out of a CALL statement's argument list without using
expand_function_arguments(), apparently in the hope of saving a few
nanoseconds by not processing defaulted arguments. It got that quite wrong
though, leading to crashes with named arguments, as well as failure to enforce
writability of the argument for a defaulted INOUT parameter. Fix and simplify
the logic by using expand_function_arguments() before examining the list.
Also, move the argument-examination to just after producing the CALL command's
plan, before invoking the called procedure. This ensures that we'll track
possible changes in the procedure's argument list correctly, and avoids a
hazard of the plan cache being flushed while the procedure executes. Also fix
assorted falsehoods and omissions in associated documentation. Per bug #15477
from Alexey Stepanov. Patch by me, with some help from Pavel Stehule.
Back-patch to v11. Discussion:
https://postgr.es/m/15477-86075b1d1d319e0a@postgresql.org Discussion:

- Fix ExecuteCallStmt to not scribble on the passed-in parse tree. Modifying
the parse tree at execution time is, or at least ought to be, verboten. It
seems quite difficult to actually cause a crash this way in v11 (although you
can exhibit it pretty easily in HEAD by messing with plan_cache_mode).
Nonetheless, it's risky, so fix and back-patch. Discussion:

- Release notes for 11.1, 10.6, 9.6.11, 9.5.15, 9.4.20, 9.3.25.

Andres Freund pushed:

- Fix typo in xlog.c. Author: Daniel Gustafsson Discussion:

- Disallow starting server with insufficient wal_level for existing slot.
Previously it was possible to create a slot, change wal_level, and restart,
even if the new wal_level was insufficient for the slot. That's a problem for
both logical and physical slots, because the necessary WAL records are not
generated. This removes a few tests in newer versions that, somewhat
inexplicably, whether restarting with a too low wal_level worked (a buggy
behaviour!). Reported-By: Joshua D. Drake Author: Andres Freund Discussion:
Backpatch: 9.4-, where replication slots where introduced

- Fix error message typo introduced 691d79a07933. Reported-By: Michael Paquier
Discussion: https://postgr.es/m/20181101003405.GB1727@paquier.xyz Backpatch:
9.4-, like the previous commit

- Fix STRICT check for strict aggregates with NULL ORDER BY columns. I (Andres)
broke this unintentionally in 69c3936a14, by checking strictness for all input
expressions computed for an aggregate, rather than just the input for the
aggregate transition function. Reported-By: Ondřej Bouda Bisected-By: Tom
Lane Diagnosed-By: Andrew Gierth Discussion:
https://postgr.es/m/2a505161-2727-2473-7c46-591ed108ac52@email.cz Backpatch:
11-, like 69c3936a14

- Prevent generating EEOP_AGG_STRICT_INPUT_CHECK operations when nargs == 0.
This only became a problem with 4c640f4f38, which didn't synchronize the value
agg_strict_input_check.nargs is set to, with the guard condition for emitting
the operation. Besides such instructions being unnecessary overhead,
currently the LLVM JIT provider doesn't support them. It seems more sensible
to avoid generating such instruction than supporting them. Add assertions to
make it easier to debug a potential further occurance. Discussion:
https://postgr.es/m/2a505161-2727-2473-7c46-591ed108ac52@email.cz Backpatch:
11-, like 4c640f4f38.

Peter Geoghegan pushed:

- Adjust trace_sort log messages. The project message style guide dictates:
"When citing the name of an object, state what kind of object it is". The
parallel CREATE INDEX patch added a worker number to most of the trace_sort
messages within tuplesort.c without specifying the object type. Bring these
messages into compliance with the style guide. We're still treating a leader
or serial Tuplesortstate as having worker number -1. trace_sort is a
developer option, and these two cases are highly comparable, so this seems
appropriate. Per complaint from Tom Lane. Discussion:
https://postgr.es/m/8330.1540831863@sss.pgh.pa.us Backpatch: 11-, where
parallel CREATE INDEX was introduced.

Bruce Momjian pushed:

- GUC: adjust effective_cache_size docs and SQL description. Clarify that
effective_cache_size is both kernel buffers and shared buffers. Reported-by:
nat(at)makarevitch(dot)org Discussion:
Backpatch-through: 9.3

- doc: use simpler language for NULL return from ANY/ALL. Previously the
combination of "does not return" and "any row" caused ambiguity. Reported-by:
KES <kes-kes(at)yandex(dot)ru> Discussion:
Reviewed-by: David G. Johnston Backpatch-through: 9.3

- docs: adjust simpler language for NULL return from ANY/ALL. Adjustment to
commit 8610c973ddf1cbf0befc1369d2cf0d56c0efcd0a. Reported-by: Tom Lane
Discussion: https://postgr.es/m/17406.1541168421@sss.pgh.pa.us
Backpatch-through: 9.3

Stephen Frost pushed:

- Remove extra word from create sub docs. Improve the documentation in the
CREATE SUBSCRIPTION command a bit by removing an extraneous word and spelling
out 'information'.

Álvaro Herrera pushed:

- Fix tablespace handling for partitioned indexes. When creating partitioned
indexes, the tablespace was not being saved for the parent index. This meant
that subsequently created partitions would not use the right tablespace for
raised errors when tried; fix them too. This requires bespoke code for
ATExecCmd() that applies to the special case when the tablespace move is just
a catalog change. Discussion:

== Pending Patches ==

Masahiko Sawada sent in two more revisions of a patch to support atomic commit
among multiple foreign servers.

Pavan Deolasee sent in another revision of a patch to implement MERGE.

Pavel Stěhule sent in two revisions of a patch to fix some XMLTABLE-related

Vik Fearing sent in a patch to implement an optional date_trunc() parameter to
indicate the time zone within which the truncation should occur.

Amit Langote sent in another revision of a patch to implement
pg_partition_tree() to display information about partitions.

David Rowley sent in three more revisions of a patch to allow Append to be used
in place of MergeAppend for some cases.

Masahiko Sawada sent in another revision of a patch to add a FREEZE_ONLY option

Haribabu Kommi sent in a patch to handle some scenarios where a tuple is
concurrently updated/deleted and locking a tuple returns HeapTupleInvisible as a

Amit Langote sent in two more revisions of a patch to store the inheritance root
parent index in otherrel's RelOptInfo and use permissions granted on parent to
read stats on otherrel children.

Andres Freund sent in a patch to add "super" barriers for checksums.

Andrey Borodin sent in another revision of a patch to add ICU as a default
collation provider.

Aya Iwata sent in another revision of a patch to allow logging traces in libpq.

Daniel Gustafsson sent in another revision of a patch to enable auto_explain to
log NOTICEs.

Michael Banck sent in another revision of a patch to make it possible to verify
checksums online.

Sergei Kornilov sent in another revision of a patch to refactor the
recovery.conf API.

David Rowley sent in two revisions of a patch to document PostgreSQL's

David Rowley sent in two revisions of a patch to allow Pathkeys to derive their
order from a parent key.

Peter Eisentraut sent in two more revisions of a patch to implement generated

Tatsuo Ishii sent in a patch to fix the -M docs for pgbench.

Patrick Francelle sent in two revisions of a patch to document the fact that
PostgreSQL does not prevent CHECK constraints from looking at other tables, even
though it has no mechanism to ensure that such constraints do what they appear
to do.

Surafel Temesgen sent in two more revisions of a patch to add a WHEN option to

Pavel Stěhule sent in two more revisions of a patch to add a \dP command to psql
for natively partitioned tables.

Andreas Scherbaum sent in two revisions of a patch to document using the more
modern pg_ctl over places where it advised using the postgres binary directly.

Daniel Gustafsson sent in another revision of a patch to add infrastructure for
signalling backends with a payload, then use same to make it possible for
pg_{cancel|terminate}_backend to pass a message.

Andrew Dunstan sent in another revision of a patch to add an --exclude-database
option to pg_dumpall.

Masahiko Sawada sent in another revision of a patch to add a parallel option to
lazy vacuum.

Kevin Van sent in a patch to add a function that returns binary JSONB as a

John Naylor sent in another revision of a patch to Avoid creation of the free
space map for small tables and skip transfer of FSMs during upgrade if they
wouldn't have been created on the new cluster.

Amit Langote sent in a patch to move PartitionBoundInfo creation code to

Michaël Paquier sent in a patch to avoid duplicate XIDs at recovery when
building the initial snapshot.

Alexander Kuzmenkov sent in another revision of a patch to add a new GUC,

Surafel Temesgen sent in another revision of a patch to implement FETCH FIRST

Ildus Kurbangaliev sent in another revision of a patch to implement custom
compression methods.

Thomas Munro sent in a patch to implement UNDO logs.

Euler Taveira de Oliveira sent in another revision of a patch to implement row
filtering for logical replication.

Paul Ramsey sent in three revisions of a patch to add a code path to do a
partial decompression of the TOAST entry when the requested slice is at the
start of the object.

Andreas Scherbaum sent in three revisions of a patch to update the README for
modern tools and techniques.

David Rowley sent in another revision of a patch to fix an incorrect setting of
heap insert options for partitioned tables.

David Rowley and Amit Langote traded patches to speed up INSERT and UPDATE on
partitioned tables.

Haribabu Kommi sent in a patch to document pluggable storage and start laying
down infrastructure for same.

Amit Langote sent in two more revisions of a patch to ignore partitioned tables
during ON COMMIT DELETE ROWS and check whether a relation still exists before
applying any ON COMMIT action.

Nikolay Shaplov sent in another revision of a patch to add an enum relation
option type.

Daniel Vérité sent in another revision of a patch to add a csv format for psql.

LAM JUN RONG sent in two revisions of a patch to improve the "getting started"

Álvaro Herrera sent in a WIP patch to allow partitioned tables to be referenced
by foreign keys.

Thomas Munro sent in another revision of a patch to use pread()/pwrite() instead
of lseek() + read()/write() where available, and supply pread()/pwrite()
implementations for Windows.

Tatsuo Ishii sent in a patch to fix a bug wherein -M could be supplied more than
once to pgbench, even though -M options are mutually exclusive.

Edmund Horner sent in another revision of a patch to improve TID scans.

Browse pgsql-announce by date

  From Date Subject
Next Message David Christensen 2018-11-06 17:09:07 Bucardo version 5.5.0 released
Previous Message Andreas 'ads' Scherbaum 2018-11-04 00:37:24 FOSDEM PGDay and PostgreSQL Devroom 2019 - Call for Papers