== PostgreSQL Weekly News - April 29 2018 ==

From: David Fetter <david(at)fetter(dot)org>
To: PostgreSQL Announce <pgsql-announce(at)postgresql(dot)org>
Subject: == PostgreSQL Weekly News - April 29 2018 ==
Date: 2018-04-29 22:21:04
Message-ID: 20180429222104.GA25267@fetter.org
Views: Raw Message | Whole Thread | Download mbox
Lists: pgsql-announce

== PostgreSQL Weekly News - April 29 2018 ==

The Portland PostgreSQL Users Group will be holding a PGDay on September 10,
2018 in Portland, OR. The CfP is open at https://goo.gl/forms/E0CiUQGSZGMYwh922

== PostgreSQL Product News ==

pspg 1.1.0, a pager designed for PostgreSQL, released.

== PostgreSQL Jobs for April ==


== PostgreSQL Local ==

PGConfNepal 2018 will be held May 4-5, 2018 at Kathmandu University, Dhulikhel,

PGCon 2018 will take place in Ottawa on May 29 - June 1, 2018.

Swiss PGDay 2018 will take place in Rapperswil (near Zurich) on June 29, 2018.
The CfP is open through April 14, 2018, and registration is open through June
28, 2018.

PGConf.Brazil 2018 will take place in São Paulo, Brazil on August 3-4 2018.

== PostgreSQL in the News ==

Planet PostgreSQL: http://planet.postgresql.org/

PostgreSQL Weekly News is brought to you this week by David Fetter

Submit news and announcements by Sunday at 3:00pm EST5EDT. Please send English
language ones to david(at)fetter(dot)org, German language to pwn(at)pgug(dot)de, Italian
language to pwn(at)itpug(dot)org(dot)

== Applied Patches ==

Magnus Hagander pushed:

- Add comment explaining BGWORKER_BYPASS_ALLOWCONN. Suggested by Michael

- Fix typo. Author: Michael Paquier

Heikki Linnakangas pushed:

- Make PGJIT_* macros safer. Author: David Rowley Discussion:

- Remove outdated comment on how to set logtape's read buffer size. Commit
b75f467b6e removed the LogicalTapeAssignReadBufferSize() function, but forgot
to update this comment. The read buffer size is an argument to
LogicalTapeRewindForRead() now. Doesn't seem worth going into the details in
the file header comment, so remove the outdated sentence altogether.

Teodor Sigaev pushed:

- Fix wrong validation of top-parent pointer during page deletion in Btree.
After introducing usage of t_tid of inner or page high key for storing number
of attributes of tuple, validation of tuple's ItemPointer with
ItemPointerIsValid becomes incorrect, it's need to validate only blocknumber
of ItemPointer. Missing this causes a incorrect page deletion, fix that. Test
is added. BTW, current contrib/amcheck doesn't fail on index corrupted by
this way. Also introduce BTreeTupleGetTopParent/BTreeTupleSetTopParent
macroses to improve code readability and to avoid possible confusion with page
high key: high key is used to store top-parent link for branch to remove. Bug
found by Michael Paquier, but bug doesn't exist in previous versions because
t_tid was set to P_HIKEY. Author: Teodor Sigaev Reviewer: Peter Geoghegan

- Update trigram example in docs to correct state. Author: Liudmila Mantrova

- Make bms_prev_member work correctly with a 64 bit bitmapword. 5c067521
erroneously had coded bms_prev_member assuming that a bitmapword would always
hold 32 bits and started it's search on what it thought was the highest 8-bits
of the word. This was not the case if bitmapwords were 64 bits. In passing
add a test to exercise this function a little. Previously there was no
coverage at all. David Rowly

- Add missing and dangling downlink checks to amcheck. When
bt_index_parent_check() is called with the heapallindexed option, allocate a
second Bloom filter to fingerprint block numbers that appear in the downlinks
of internal pages. Use Bloom filter probes when walking the B-Tree to detect
missing downlinks. This can detect subtle problems with page deletion/VACUUM,
such as corruption caused by the bug just fixed in commit 6db4b499. The
downlink Bloom filter is bound in size by work_mem. Its optimal size is
typically far smaller than that of the regular heapallindexed Bloom filter,
especially when the index has high fan-out. Author: Peter Geoghegan Reviewer:
Teodor Sigaev Discussion:

- Add amcheck missing downlink tests. Also use palloc0() for main amcheck
state, and adjust a few comments. Somehow I pushed old version of patch in
commit 4eaf7eaccb291f5d32d9f05284bfca9c5744de9a, so commit the difference.
Peter Geoghegan

Álvaro Herrera pushed:

- Add missing pstrdup. Lifetime of the input string is not right, so create a
separate copy. Author: Amit Langote Discussion:

- Add GUC enable_partition_pruning. This controls both plan-time and
execution-time new-style partition pruning. While finer-grain control is
possible (maybe using an enum GUC instead of boolean), there doesn't seem to
be much need for that. This new parameter controls partition pruning for all
queries: trivially, SELECT queries that affect partitioned tables are
naturally under its control since they are using the new technology. However,
while UPDATE/DELETE queries do not use the new code, we make the new GUC
control their behavior also (stealing control from constraint_exclusion),
because it is more natural, and it leads to a more natural transition to the
future in which those queries will also use the new pruning code. Constraint
exclusion still controls pruning for regular inheritance situations (those not
involving partitioned tables). Author: David Rowley Review: Amit Langote,
Ashutosh Bapat, Justin Pryzby, David G. Johnston Discussion:

- Remove useless default clause in switch. The switch covers all values of the
enum driver variable, so having a default: clause is useless, even if it's
only to do Assert(false).

- Initialize ExprStates once in run-time partition pruning. Instead of doing
ExecInitExpr every time a Param needs to be evaluated in run-time partition
pruning, do it once during run-time pruning set-up and cache the exprstate in
PartitionPruneContext, saving a lot of work. Author: David Rowley
Reviewed-by: Amit Langote, Álvaro Herrera Discussion:

- Update ExecInitPartitionInfo comment. Remove the words "if not already done."
This obsolete wording corresponds to an early development version of what
became edd44738bc8. Author: Etsuro Fujita Reviewed-by: Amit Langote
Discussion: https://postgr.es/m/5ADF117B.5030606@lab.ntt.co.jp

Peter Eisentraut pushed:

- Make Emacs settings match perltidy configuration. Set Emacs's
perl-continued-statement-offset to match perltidy's
--continuation-indentation, which is 2 (not overridden in PostgreSQL's
profile) rather than the 4 that Emacs uses by default.

- Fix typo in logical truncate replication. This could result in some
misbehavior in a cascading replication setup.

- perltidy: Add option --nooutdent-long-comments.

- perltidy: Don't write backup files. Newer perltidy versions can just avoid
writing backup files, so we don't need the old dance of deleting them
afterwards. Supported since 20120619.

- perltidy: Add option --nooutdent-long-quotes.

Tom Lane pushed:

- Fix handling of partition bounds for boolean partitioning columns.
Previously, you could partition by a boolean column as long as you spelled the
bound values as string literals, for instance FOR VALUES IN ('t'). The
trouble with this is that ruleutils.c printed that as FOR VALUES IN (TRUE),
which is reasonable syntax but wasn't accepted by the grammar. That results
in dump-and-reload failures for such cases. Apply a minimal fix that just
causes TRUE and FALSE to be converted to strings 'true' and 'false'. This is
pretty grotty, but it's too late for a more principled fix in v11 (to say
nothing of v10). We should revisit the whole issue of how partition bound
values are parsed for v12. Amit Langote Discussion:

- Change pgindent/README to specify that we use perltidy version 20170521. Per
discussion, this is now the project's standard version. Update the obsolete
URL, and explain how to install a specific version. Discussion:

- Reindent Perl files with perltidy version 20170521. Discussion:

- Make Catalog.pm's representation of toast and index decls more abstract.
Instead of immediately constructing the string we need to emit into the .BKI
file, preserve the items we extracted from the header file in a hash. This
eases using the info for other purposes. John Naylor (with cosmetic
adjustments by me) Discussion:

- Convert unused_oids and duplicate_oids to use Catalog.pm infrastructure.
unused_oids was previously a shell script, which of course didn't work at all
on Windows. Also, commit 372728b0d introduced some other portability
problems, as complained of by Stas Kelvich. We can improve matters by
converting it to Perl. While we're at it, let's future-proof both this script
and duplicate_oids to use Catalog.pm rather than having a bunch of ad-hoc
logic for parsing catalog headers and .dat files. These scripts are thereby a
bit slower, which doesn't seem like a problem for typical manual use. It is a
little annoying for buildfarm purposes, but we should be able to fix that case
by having genbki.pl make the check instead of parsing the headers twice.
(That's not done in this commit, though.) Stas Kelvich, adjusted a bit by me

- Fix duplicate_oids and unused_oids so user needn't cd to catalog dir.
Previously, you had to cd into src/include/catalog before running either of
these scripts. That's a bit tedious, so let's make the scripts do it for you.
In passing, improve the initial comments in both scripts. Also remove
unused_oids' code to complain about duplicate oids. That was added in
yesterday's commit 5602265f7, but on second thought we shouldn't be randomly
redefining the script's behavior that way. John Naylor and Tom Lane

- Avoid parsing catalog data twice during BKI file construction. In the wake of
commit 5602265f7, we were doing duplicate-OID detection quite inefficiently,
by invoking duplicate_oids which does all the same parsing of catalog headers
and .dat files as genbki.pl does. That adds under half a second on modern
machines, but quite a bit more on slow buildfarm critters, so it seems worth
avoiding. Let's just extend genbki.pl a little so it can also detect
duplicate OIDs, and remove the duplicate_oids call from the build process.
(This also means that duplicate OID detection will happen during Windows
builds, which AFAICS it didn't before.) This makes the use-case for
duplicate_oids a bit dubious, but it's possible that people will still want to
run that check without doing a whole build run, so let's keep that script. In
passing, move down genbki.pl's creation of its temp output files so that it
doesn't happen until after we've done parsing and validation of the input.
This avoids leaving a lot of clutter around after a failure. John Naylor and
Tom Lane Discussion:

- Post-feature-freeze pgindent run. Discussion:

- Preliminary work for pgindent run. Update typedefs.list from current
buildfarm results. Adjust pgindent's typedef blacklist to block some more
unfortunate typedef names that have snuck in since last time. Manually tweak
a few places where I didn't like the initial results of pgindent'ing.

- Adjust hints and docs to suggest CREATE EXTENSION not CREATE LANGUAGE. The
core PLs have been extension-ified for seven years now, and we can reasonably
hope that all out-of-core PLs have been too. So adjust a few places that were
still recommending CREATE LANGUAGE as the user-level way to install a PL.

- Try to get some info about Windows can't-reattach-to-shared-memory errors.
Add some debug printouts focused on the idea that MapViewOfFileEx might be
rounding its virtual memory allocation up more than we expect (and, in
particular, more than VirtualAllocEx does). Once we've seen what this reports
in one of the failures on buildfarm members dory or jacana, we might revert
this ... or perhaps just decrease the log level. Discussion:

- Assorted minor doc/comment fixes. Identify pg_replication_origin as a shared
catalog in catalogs.sgml, using the same boilerplate wording used for most
other shared catalogs (and tweak another place where someone had randomly
deviated from that boilerplate). Make an example in mmgr/README more
consistent with surrounding text. Update an obsolete cross-reference in a
comment in storage/block.h. Zhuo Ql Discussion:

- Minor cleanups for install_llvm_module/uninstall_llvm_module Make macros.
Don't put comments inside the macros, per complaint from Michael Paquier.
Quote target directory path with single quotes, not double; that seems to be
our project standard. Not quoting it at all definitely isn't per standard.
Remove excess slash in one instance of path. Remove useless semicolon.
Discussion: https://postgr.es/m/20180428073935.GB1736@paquier.xyz

- Un-break contrib install with llvm. Apparently $(foreach ... $(call
install_llvm_module,...)) doesn't work too well without a blank line ending
the install_llvm_module macro. The previous coding hackishly dodged this
problem with some parens, but that's not really a good solution because make
misunderstands where the command boundaries are that way. Discussion:

- Clarify handling of special-case values in bootstrap catalog data. I (tgl)
originally coded the special case for pg_proc.pronargs as though it were a
kind of default value. It seems better though to treat computable columns as
an independent concern: this makes the code clearer, and probably a bit faster
too since we needn't do work inside the per-column loop. Improve related
comments, as well, in the expectation that there might be more cases like this
in future. John Naylor, some additional comment-hacking by me Discussion:

- Tweak reformat_dat_file.pl to make it more easily hand-invokable. Use the
same code we already applied in duplicate_oids and unused_oids to let this
script find Catalog.pm without help. This removes the need to supply a -I
switch in most cases. Also, mark the script executable, again to follow the
precedent of duplicate_oids and unused_oids. Now you can just do
"./reformat_dat_file.pl pg_proc.dat" if you want to reformat only one or a few
.dat files rather than all. It'd be possible to remove the -I switches in the
Makefile's convenience targets, but I chose to leave them: they don't hurt
anything, and it's possible that in weird VPATH situations they might be of

- Fix incorrect field type for PlannedStmt.jitFlags in outfuncs/readfuncs. This
field was a bool at one point, but now it's an int. Spotted by Hari Babu;
trivial patch is by Ashutosh Bapat. Discussion:

- In AtEOXact_Files, complain if any files remain unclosed at commit. This
change makes this module act more like most of our other low-level resource
management modules. It's a caller error if something is not explicitly closed
by the end of a successful transaction, so issue a WARNING about it. This
would not actually have caught the file leak bug fixed in commit 231bcd080,
because that was in a transaction-abort path; but it still seems like a good,
and pretty cheap, cross-check. Discussion:

- Cosmetic improvement: use BKI_DEFAULT and BKI_LOOKUP in pg_language. The
point of this is not really to remove redundancy in pg_language.dat; with only
three entries, it's hardly worth it. Rather, it is to get to a point where
there are exactly zero hard-coded numeric pg_proc OID references in the
catalog .dat files. The lanvalidator column was the only remaining location
of such references, and it seems like a good thing for future-proofing reasons
to make it not be a special case. There are still a few places in the .dat
files with numeric OID references to other catalogs, but after review I don't
see any that seem worth changing at present. In each case there are just too
few entries to make it worth the trouble to create lookup infrastructure.
This doesn't change the emitted postgres.bki file, so no catversion bump.

- Avoid wrong results for power() with NaN input on some platforms. Per spec,
the result of power() should be NaN if either input is NaN. It appears that
on some versions of Windows, the libc function does return NaN, but it also
sets errno = EDOM, confusing our code that attempts to work around
shortcomings of other platforms. Hence, add guard tests to avoid substituting
a wrong result for the right one. It's been like this for a long time (and
the odd behavior only appears in older MSVC releases, too) so back-patch to
all supported branches. Dang Minh Huong, reviewed by David Rowley Discussion:

- Update time zone data files to tzdata release 2018d. DST law changes in
Palestine and Antarctica (Casey Station). Historical corrections for Portugal
and its colonies, as well as Enderbury, Jamaica, Turks & Caicos Islands, and

- Update time zone data files to tzdata release 2018d. DST law changes in
Palestine and Antarctica (Casey Station). Historical corrections for Portugal
and its colonies, as well as Enderbury, Jamaica, Turks & Caicos Islands, and

Robert Haas pushed:

- Prevent generation of bogus subquery scan paths. Commit
0927d2f46ddd4cf7d6bf2cc84b3be923e0aedc52 didn't check that consider_parallel
was set for the target relation or account for the possibility that
required_outer might be non-empty. To prevent future bugs of this ilk, add
some assertions to add_partial_path and do a bit of future-proofing of the
code recently added to recurse_set_operations. Report by Andreas Seltenreich.
Patch by Jeevan Chalke. Review by Amit Kapila and by me. Discussion:

Noah Misch pushed:

- Correct pg_recvlogical server version test. The predecessor test boiled down
to "PQserverVersion(NULL) >= 100000", which is always false. No release
includes that, so it could not have reintroduced CVE-2018-1058. Back-patch to
9.4, like the addition of the predecessor in commit
8d2814f274def85f39fbe997d454b01628cb5667. Discussion:

Bruce Momjian pushed:

- docs: remove "III" version text from pgAdmin link. Reported-by:
vodevsh(at)gmail(dot)com Discussion:
Backpatch-through: 9.3

- C comment: add description of root_tuple_slot. Reported-by: Amit Langote

== Pending Patches ==

Heikki Linnakangas sent in a patch to tighten the movedpartitions magic value.

Michaël Paquier sent in a patch to fix gaps in modules with handling of
partitioned indexes.

Michaël Paquier sent in a patch to format waldump flags %X.

Kyotaro HORIGUCHI and Alexander Kuzmenkov traded patches to make pg_ctl work
better with logrotate.

Robert Haas, Amit Langote, and Etsuro Fujita traded patches to fix tuple routing
for foreign partitions.

Thomas Munro sent in patches to use signals for postmaster death on Linux and

Ildus Kurbangaliev sent in a patch to add PRIx64 macros.

Ashutosh Bapat sent in two more revisions of a patch to fix some expression
errors with "FOR UPDATE" and postgres_fdw with partition wise join enabled.

Mark Dilger sent in a patch to remove some redundancies from bootstrap data.

David Rowley sent in a patch to further document partition pruning.

Dmitry Dolgov sent in another revision of a patch to implement generic type
subscripting and use same for arrays and JSONB.

Álvaro Herrera sent in a patch to accept TEXT and CDATA nodes in XMLTABLE's

Kyotaro HORIGUCHI sent in a patch to fix an issue which caused subscriber
corruption after the publisher is restarted.

David Rowley and Amit Langote traded patches to remove from the docs the
no-longer-true assertion that foreign keys on partitioned tables are not

Peter Eisentraut sent in a patch to prevent infinity and NaN in jsonb/plperl

Tom Dunstan sent in a patch to enable auto_explain to log to NOTICE.

Ashutosh Bapat sent in another revision of a patch to add tests to show problem
when foreign table points to a partitioned table or inheritance table on the
foreign server, error out if one iteration of non-direct DML affects more than
one row on the foreign server, and and properly handle the fact that DML on a
foreign table pointing to an inherited or a partitioned table may affect
multiple rows on the foreign server.

Aleksandr Parfenov sent in a patch to add a range phrase operator to tsquery.

Fabien COELHO sent in another revision of a patch to pgbench to enable storing
SELECT results into variables.

Stas Kelvich sent in a patch to add a HOLD_INTERRUPTS section to

Peter Eisentraut sent in a patch to fix some new warnings from GCC 8.

Teodor Sigaev sent in two more revisions of a patch to fix predicate locking on
GIN indexes.

Pavel Stěhule sent in another revision of a patch to implement schema variables.

Joe Wildish sent in another revision of a patch to implement ASSERTIONs.

Browse pgsql-announce by date

  From Date Subject
Next Message D'Arcy Cain 2018-05-01 10:08:44 PyGreSQL 5.0.5
Previous Message David Fetter 2018-04-22 18:24:33 == PostgreSQL Weekly News - April 22 2018 ==