== PostgreSQL Weekly News - December 22, 2019 ==

== PostgreSQL Weekly News - December 22, 2019 ==

Postgres(at)CERN will be on January 17, 2020.
https://indico.cern.ch/e/PGCERN2020

== PostgreSQL Jobs for December ==

http://archives.postgresql.org/pgsql-jobs/2019-12/

== PostgreSQL Local ==

PGDay SF will take place on January 21, 2020 at the Swedish American Hall in San
Francisco.
https://2020.pgdaysf.org/

pgDay Israel 2020 will take place on March 19, 2020 in Tel Aviv.
The CfP is open through January 15, 2020.
http://pgday.org.il/

pgDay Paris 2020 will be held in Paris, France on March 26, 2020 at Espace
Saint-Martin. The CfP is open through December 31, 2019 at midnight, Paris time
at https://2020.pgday.paris/callforpapers/
https://2020.pgday.paris/

Nordic PGDay 2020 will be held in Helsinki, Finland at the Hilton Helsinki
Strand Hotel on March 24, 2020. The CfP is open through December 31, 2019 at
https://2020.nordicpgday.org/cfp/

PGConf India 2020 will be on February 26-28, 2020 in Bengaluru, Karnataka.
http://pgconf.in/

PostgreSQL(at)SCaLE is a two day, two track event which takes place on
March 5-6, 2020, at Pasadena Convention Center, as part of SCaLE 18X.
https://www.socallinuxexpo.org/scale/18x/postgresscale

The German-speaking PostgreSQL Conference 2020 will take place on May 15, 2019
in Stuttgart.

== PostgreSQL in the News ==

Planet PostgreSQL: http://planet.postgresql.org/

PostgreSQL Weekly News is brought to you this week by David Fetter

Submit news and announcements by Sunday at 3:00pm PST8PDT to david(at)fetter(dot)org(dot)

== Applied Patches ==

Etsuro Fujita pushed:

- Clean up some misplaced comments in partition_join.sql regression test. Also,
add a comment explaining a test case. Back-patch to 11 where the regression
test was added. Discussion:
https://postgr.es/m/CAPmGK15adZPh2B%2BmGUjSOMH%2BH39ogDRWfCfm4G6jncZCAs9V_Q%40mail.gmail.com
https://git.postgresql.org/pg/commitdiff/956ef5875341c22a602fb825e6c98eaabb1ecce7

Peter Eisentraut pushed:

- Sort out getpeereid() and peer auth handling on Windows. The getpeereid() uses
have so far been protected by HAVE_UNIX_SOCKETS, so they didn't ever care
about Windows support. But in anticipation of Unix-domain socket support on
Windows, that needs to be handled differently. Windows doesn't support
getpeereid() at this time, so we use the existing not-supported code path. We
let configure do its usual thing of picking up the replacement from libpgport,
instead of the custom overrides that it was doing before. But then Windows
doesn't have struct passwd, so this patch sprinkles some additional #ifdef
WIN32 around to make it work. This is similar to existing code that deals
with this issue. Reviewed-by: Andrew Dunstan <andrew(dot)dunstan(at)2ndquadrant(dot)com>
Discussion:
https://www.postgresql.org/message-id/5974caea-1267-7708-40f2-6009a9d653b0@2ndquadrant.com
https://git.postgresql.org/pg/commitdiff/f14413b684d57211068ee56ee04695efcc87a23a

- Fix build of Perl-using modules of Windows. Commit
f14413b684d57211068ee56ee04695efcc87a23a broke the build of Perl-using modules
on Windows. Perl might have its own definitions of uid_t and gid_t, so we
hide ours, but then we can't use ours in our header files such as port.h which
don't see the Perl definition. Hide our definition of getpeereid() on Windows
in Perl-using modules, using PLPERL_HAVE_UID_GID define. That means we can't
portably use getpeeruid() is such modules right now, but there is no need
anyway.
https://git.postgresql.org/pg/commitdiff/502423180a8cc9214861bffcb8405a42f146f160

- Add .editorconfig. The main use right now is getting properly spaced diff
views on GitHub, but perhaps this will also help developers with editors that
we currently don't have setup recipes for. The settings mirror mostly what's
currently in .dir-locals.el. Discussion:
https://www.postgresql.org/message-id/flat/273cb788-bbb2-ff34-ad6f-5192b44e5049%402ndquadrant.com
https://git.postgresql.org/pg/commitdiff/ecb09cd5de5279ab9cfa20a58fd3da44f7df5779

- Add support for MSYS2. It's basically a variant of Cygwin, so use that
template. Discussion:
https://www.postgresql.org/message-id/flat/6b467edc-4018-521f-ab18-171f098557ca%402ndquadrant.com
https://git.postgresql.org/pg/commitdiff/e975c1a6026adb9e248a408fe3ca2629bc8c0084

- Fix compiler warnings on MSYS2. The PS_USE_NONE case in ps_status.c left a
couple of unused variables exposed. Discussion:
https://www.postgresql.org/message-id/flat/6b467edc-4018-521f-ab18-171f098557ca%402ndquadrant.com
https://git.postgresql.org/pg/commitdiff/8c6d30f211390df911072d33f0114a31f066a4cd

- Disallow dropping rules on system tables by default. This was previously not
covered by allow_system_table_mods, but now it is. The impact in practice is
probably low, but this makes it consistent with most other DDL commands.
Reviewed-by: Robert Haas <robertmhaas(at)gmail(dot)com> Discussion:
https://www.postgresql.org/message-id/flat/ee9df1af-c0d8-7c82-5be7-39ce4e3b0a9d%402ndquadrant.com
https://git.postgresql.org/pg/commitdiff/df7fe9e2d707da69a4437fb6f9e695c070882160

- Generate pg_config.h from pg_config.h.in on Windows. Previously, the Windows
MSVC build generated pg_config.h from a hard-coded pg_config.h.win32 with some
ad hoc postprocessing. The pg_config.h.win32 file required manual maintenance
and was as a result frequently out of date. Instead, have the MSVC build
scripts emulate what configure and config.status do: collect a list of defines
and then create pg_config.h from pg_config.h.in by changing the appropriate
lines. The previous setup was made to support old Windows build systems that
didn't have any text processing capabilities, but the current system has Perl,
so it's not a problem. pg_config.h.win32 is removed. In order to try to keep
the Windows side of things more up to date in the future, we now also require
that all symbols found in pg_config.h.in are defined in the MSVC build system.
So if there is a change in configure that results in a new symbol, an update
in Solution.pm will be required. The other headers managed by
AC_CONFIG_HEADERS in configure, namely src/include/pg_config_ext.h and
src/interfaces/ecpg/include/ecpg_config.h, get the same treatment, so this
removes even more ad hoc code in the MSVC build scripts. Reviewed-by: Michael
Paquier <michael(at)paquier(dot)xyz> Discussion:
https://www.postgresql.org/message-id/flat/1441b834-f434-e0bf-46ed-9c4d5c29c2d4%402ndquadrant.com
https://git.postgresql.org/pg/commitdiff/8f4fb4c648ee8df00f78bee3f8099a6ae510071a

- Clean up inconsistent backslash use in paths. Most of the MSVC Perl code uses
forward slashes for file paths. Make the few places that use backslashes the
same. This also helps running that code on non-Windows.
https://git.postgresql.org/pg/commitdiff/77f416af6e821b40649c971ded43089e0450791a

- Fix compiler warning for ppoll() on Cygwin. _GNU_SOURCE is required to get the
prototype, so just define that globally, as was already done in the linux
template. Discussion:
https://www.postgresql.org/message-id/flat/6b467edc-4018-521f-ab18-171f098557ca%402ndquadrant.com
https://git.postgresql.org/pg/commitdiff/127ccb37251744c0fad2df0f3f67dd2c38fe8389

Heikki Linnakangas pushed:

- Fix yet another crash in page split during GiST index creation. Commit
a7ee7c8513 fixed a bug in GiST page split during index creation, where we
failed to re-find the position of a downlink after the page containing it was
split. However, that fix was incomplete; the other call to gistinserttuples()
in the same function needs to also clear 'downlinkoffnum'. Fixes bug #16134
reported by Alexander Lakhin, for real this time. The previous fix was enough
to fix the crash with the reproducer script for bug #16162, but the original
script for #16134 was still crashing. Backpatch to v12, like the previous
incomplete fix. Discussion:
https://www.postgresql.org/message-id/d869f537-abe4-d2ea-0510-38cd053f5152%40gmail.com
https://git.postgresql.org/pg/commitdiff/741b884353e4803abc15d4392ad287b0d5953fc4

Álvaro Herrera pushed:

- Demote variable from global to local. recoveryDelayUntilTime was introduced by
commit 36da3cfb457b as a global because its method of operation was devilishly
intrincate. Commit c945af80cfda removed all that complexity and could have
turned it into a local variable, but didn't. Do so now. Discussion:
https://postgr.es/m/20191213200751.GA10731@alvherre.pgsql Reviewed-by: Michaël
Paquier, Daniel Gustafsson
https://git.postgresql.org/pg/commitdiff/91fca4bb60e8c00e8b0e2755555b39f4b1c1659c

- makeArrayTypeName: Remove pointless relation open/close. Discussion:
https://postgr.es/m/20191218221326.GA25537@alvherre.pgsql
https://git.postgresql.org/pg/commitdiff/2b93e3d96b941740877b2ae196511564e5cc989b

Tom Lane pushed:

- On Windows, wait a little to see if ERROR_ACCESS_DENIED goes away. Attempting
to open a file fails with ERROR_ACCESS_DENIED if the file is flagged for
deletion but not yet actually gone (another in a long list of reasons why
Windows is broken, if you ask me). This seems likely to explain a lot of
irreproducible failures we see in the buildfarm. This state generally
persists for only a millisecond or so, so just wait a bit and retry. If it's
a real permissions problem, we'll eventually give up and report it as such.
If it's the pending deletion case, we'll see file-not-found and report that
after the deletion completes, and the caller will treat that in an appropriate
way. In passing, rejigger the existing retry logic for some other error cases
so that we don't uselessly wait an extra time when we're not going to retry
anymore. Alexander Lakhin (with cosmetic tweaks by me). Back-patch to all
supported branches, since this seems like a pretty safe change and the problem
is definitely real. Discussion:
https://postgr.es/m/16161-7a985d2f1bbe8f71@postgresql.org
https://git.postgresql.org/pg/commitdiff/6d7547c219adf2436323cdbd4bebc5e872d53546

- Fix "force_parallel_mode = regress" to work with ANALYZE + VERBOSE.
force_parallel_mode = regress is supposed to force use of a Gather node
without having any impact on EXPLAIN output. But it failed to accomplish that
if both ANALYZE and VERBOSE are given, because that enables per-worker output
data that you wouldn't see if the Gather hadn't been inserted. Improve the
logic so that we suppress the per-worker data too. This allows putting the
new test case added by commit 5935917ce back into the originally intended form
(cf. 776a2c887, 22864f6e0). We can also get rid of a kluge in subselect.sql,
which previously had to clean up after force_parallel_mode's failure to do
what it said on the tin. Discussion:
https://postgr.es/m/18445.1576177309@sss.pgh.pa.us
https://git.postgresql.org/pg/commitdiff/b925a00f4ef65db9359e1c60fbf0e56d05afb25a

- Fix error reporting for index expressions of prohibited types. If
CheckAttributeType() threw an error about the datatype of an index expression
column, it would report an empty column name, which is pretty unhelpful and
certainly not the intended behavior. I (tgl) evidently broke this in commit
cfc5008a5, by not noticing that the column's attname was used above where I'd
placed the assignment of it. In HEAD and v12, this is trivially fixable by
moving up the assignment of attname. Before v12 the code is a bit more messy;
to avoid doing substantial refactoring, I took the lazy way out and just put
in two copies of the assignment code. Report and patch by Amit Langote.
Back-patch to all supported branches. Discussion:
https://postgr.es/m/CA+HiwqFA+BGyBFimjiYXXMa2Hc3fcL0+OJOyzUNjhU4NCa_XXw@mail.gmail.com
https://git.postgresql.org/pg/commitdiff/2acab054b3ff8e46707727980ce3fa1a1897381f

- Minimal portability fix for commit e1551f96e. Older gcc versions are not happy
with having multiple declarations for the same typedef name (not struct name).
I'm a bit dubious as to how well-thought-out that patch was at all, but for
the moment just fix it enough so I can get some work done today. Discussion:
https://postgr.es/m/20191218101338.GB325369@paquier.xyz
https://git.postgresql.org/pg/commitdiff/bf7427bdd389aa6c266768f2a07214a2a02b85c9

- Doc: add a short summary of available authentication methods. The
"auth-methods" <sect1> used to include descriptions of all our authentication
methods. Commit 56811e573 promoted its child <sect2>'s to <sect1>'s, which
has advantages but also created some issues: * The auth-methods page itself is
essentially empty/useless. * Links that pointed to "auth-methods" as a
placeholder for all auth methods were rendered a bit nonsensical. * DocBook no
longer provides a subsection table-of-contents here, which formerly was a
useful if terse summary of available auth methods. To improve matters, add a
handwritten list of all the auth methods. Per gripe from Dave Cramer.
Back-patch to v11 where the previous commit came in. Discussion:
https://postgr.es/m/CADK3HH+xQLhcPgg=kWqfogtXGGZr-JdSo=x=WQC0PkAVyxUWyQ@mail.gmail.com
https://git.postgresql.org/pg/commitdiff/54fbd155cc6fdbf875185035b3d9823f739b617d

- libpq should expose GSS-related parameters even when not implemented. We
realized years ago that it's better for libpq to accept all connection
parameters syntactically, even if some are ignored or restricted due to lack
of the feature in a particular build. However, that lesson from the SSL
support was for some reason never applied to the GSSAPI support. This is
causing various buildfarm members to have problems with a test case added by
commit 6136e94dc, and it's just a bad idea from a user-experience standpoint
anyway, so fix it. While at it, fix some places where parameter-related
infrastructure was added with the aid of a dartboard, or perhaps with the aid
of the anti-pattern "add new stuff at the end". It should be safe to
rearrange the contents of struct pg_conn even in released branches, since
that's private to libpq (and we'd have to move some fields in some builds to
fix this, anyway). Back-patch to all supported branches. Discussion:
https://postgr.es/m/11297.1576868677@sss.pgh.pa.us
https://git.postgresql.org/pg/commitdiff/e60b480d39ee3401727a994988dd9117a3b48466

- Adjust test case added by commit 6136e94dc. Per project policy, transient
roles created by regression test cases should be named "regress_something", to
reduce the risks of running such cases against installed servers. And no such
role should ever be left behind after running a test. Discussion:
https://postgr.es/m/11297.1576868677@sss.pgh.pa.us
https://git.postgresql.org/pg/commitdiff/0af0504da91e5e15f0b203309a1e49a4829dac64

- In pgwin32_open, loop after ERROR_ACCESS_DENIED only if we can't stat. This
fixes a performance problem introduced by commit 6d7547c21.
ERROR_ACCESS_DENIED is returned in some other cases besides the delete-pending
case considered by that commit; notably, if the given path names a directory
instead of a plain file. In that case we'll uselessly loop for 1 second
before returning the failure condition. That slows down some usage scenarios
enough to cause test timeout failures on our Windows buildfarm critters. To
fix, try to stat() the file, and sleep/loop only if that fails. It will fail
in the delete-pending case, and also in the case where the deletion completed
before we could stat(), so we have the cases where we want to loop covered.
In the directory case, the stat() should succeed, letting us exit without a
wait. One case where we'll still wait uselessly is if the access-denied
problem pertains to a directory in the given pathname. But we don't expect
that to happen in any performance-critical code path. There might be room to
refine this further, but I'll push it now in hopes of making the buildfarm
green again. Back-patch, like the preceding commit. Alexander Lakhin and Tom
Lane Discussion: https://postgr.es/m/23073.1576626626@sss.pgh.pa.us
https://git.postgresql.org/pg/commitdiff/5406513e997f5ee9de79d4076ae91c04af0c52f6

- Avoid low-probability regression test failures in timestamp[tz] tests. If the
first transaction block in these tests were entered exactly at midnight
(California time), they'd report a bogus failure due to 'now' and 'midnight'
having the same values. Commit 8c2ac75c5 had dismissed this as being of
negligible probability, but we've now seen it happen in the buildfarm, so
let's prevent it. We can get pretty much the same test coverage without an
it's-not-midnight assumption by moving the does-'now'-work cases into their
own test step. While here, apply commit 47169c255's s/DELETE/TRUNCATE/ change
to timestamptz as well as timestamp (not sure why that didn't occur to me at
the time; the risk of failure is the same). Back-patch to all supported
branches, since the main point is to get rid of potential buildfarm failures.
Discussion: https://postgr.es/m/14821.1577031117@sss.pgh.pa.us
https://git.postgresql.org/pg/commitdiff/b265aa1f39b672d263e37bdb715516d32128d0c4

Peter Geoghegan pushed:

- Update nbtree README's "Scans during Recovery". get_actual_variable_range()
hasn't used a dirty snapshot since commit 3ca930fc3, which invented a new
snapshot type specifically to meet selfuncs.c's requirements
(HeapTupleSatisfiesNonVacuumable() type snapshots were added). Discussion:
https://postgr.es/m/CAH2-Wzn2pSqEOcBDAA40CnO82oEy-EOpE2bNh_XL_cfFoA86jw@mail.gmail.com
https://git.postgresql.org/pg/commitdiff/9067b83955da5fde49a2605510900e6d9fa273af

- Rename nbtree tuple macros. Rename two function-style macros, removing the
word "inner". This makes things more consistent.
https://git.postgresql.org/pg/commitdiff/fcf3b6917bd8f6f9f463e3e42e53d6ff9612e327

- Remove unneeded "pin scan" nbtree VACUUM code. The REDO routine for nbtree's
xl_btree_vacuum record type hasn't performed a "pin scan" since commit
3e4b7d87 went in, so clearly there isn't any point in VACUUM WAL-logging
information that won't actually be used. Finish off the work of commit
3e4b7d87 (and the closely related preceding commit 687f2cd7) by removing the
code that generates this unused information. Also remove the REDO routine
code disabled by commit 3e4b7d87. Replace the unneeded lastBlockVacuumed
field in xl_btree_vacuum with a new "ndeleted" field. The new field isn't
actually needed right now, since we could continue to infer the array length
from the overall record length. However, an upcoming patch to add
deduplication to nbtree needs to add an "items updated" field to
xl_btree_vacuum, so we might as well start being explicit about the number of
items now. (Besides, it doesn't seem like a good idea to leave the
xl_btree_vacuum struct without any fields; the C standard says that that's
undefined.) nbtree VACUUM no longer forces writing a WAL record for the last
block in the index. Writing out a WAL record with no items for the final
block was supposed to force processing of a lastBlockVacuumed field by a pin
scan. Bump XLOG_PAGE_MAGIC because xl_btree_vacuum changed. Discussion:
https://postgr.es/m/CAH2-WzmY_mT7UnTzFB5LBQDBkKpdV5UxP3B5bLb7uP%3D%3D6UQJRQ%40mail.gmail.com
https://git.postgresql.org/pg/commitdiff/9f83468b3536caf6fb7fe8f9dcdbb108a98d1257

Michaël Paquier pushed:

- Fix query cancellation handling in psql. The refactoring done in a4fd3aa for
query cancellation has messed up with the logic in psql by mixing
CancelRequested and cancel_pressed, breaking for example \watch. The former
would be switched to true if a cancellation request has been attempted and
that it actually succeeded, and the latter tracks if a cancellation attempt
has been done. This commit brings back the code of psql to a state consistent
to what it was before a4fd3aa, without giving up on the refactoring pieces
introduced. It should be actually possible to merge more both flags as their
concepts are close enough, however note that psql's --single-step mode relies
on cancel_pressed to be always set, so this requires more careful analysis
left for later. While on it, fix the declarations of CancelRequested (in
cancel.c) and cancel_pressed (in psql) to be volatile sig_atomic_t.
Previously, both were declared as booleans, which should be fine on modern
platforms, but the C standard recommends the use of sig_atomic_t for variables
used in signal handlers. Note that since its introduction in a1792320,
CancelRequested declaration was not volatile. Reported-by: Jeff Janes Author:
Michael Paquier Discussion:
https://postgr.es/m/CAMkU=1zpoUDGKqWKuMWkj7t-bOCaJDx0r=5te_-d0B2HVLABXg@mail.gmail.com
https://git.postgresql.org/pg/commitdiff/5d43c3c54d77f39135fe463539f5f438f460ae7e

- Remove shadow variables linked to RedoRecPtr in xlog.c. This changes the
routines in charge of recycling WAL segments past the last redo LSN to not use
anymore "RedoRecPtr" as a local variable, which is also available in the
context of the session as a static declaration, replacing it with
"lastredoptr". This confusion has been introduced by d9fadbf, so backpatch
down to v11 like the other commit. Thanks to Tom Lane, Robert Haas, Alvaro
Herrera, Mark Dilger and Kyotaro Horiguchi for the input provided. Author:
Ranier Vilela Discussion:
https://postgr.es/m/MN2PR18MB2927F7B5F690065E1194B258E35D0@MN2PR18MB2927.namprd18.prod.outlook.com
Backpatch-through: 11
https://git.postgresql.org/pg/commitdiff/70116493a8e07713f7e1270646ca1147898b4f6d

- Fix some OBJS lists in two Makefiles to be ordered alphabetically. These have
been missed in 01368e5, and count for plpython and the backend's tsearch code.
Author: Mahendra Singh Discussion:
https://postgr.es/m/CAKYtNAo4mxRRyDB0YqE6QLh17XD7pPQotpGm3GnHS+gQKz4zQQ@mail.gmail.com
https://git.postgresql.org/pg/commitdiff/aa3ef7ff505305d8ee5f733090b076c301a32cc8

- Doc: Improve readability of options for REINDEX. That's more consistent with
the style we have been using with for example EXPLAIN, VACUUM or ANALYZE (this
one had only one option in v11). Based on a suggestion from Pavel Stehule.
Author: Josef Šimánek Discussion:
https://postgr.es/m/CAFj8pRCrUS+eMFvssVPGZN-VDEMP3XN+1Dop0=CmeBq2D+dqOg@mail.gmail.com
Discussion:
https://postgr.es/m/CAFp7QwpeMPEtAR5AYpsG623ooMWX03wMjq5cpZn=X+6OCkfwJw@mail.gmail.com
https://git.postgresql.org/pg/commitdiff/52dcfda48778d16683c64ca4372299a099a15b96

- Refactor attribute mappings used in logical tuple conversion. Tuple conversion
support in tupconvert.c is able to convert rowtypes between two relations,
inner and outer, which are logically equivalent but have a different ordering
or even dropped columns (used mainly for inheritance tree and partitions).
This makes use of attribute mappings, which are simple arrays made of
AttrNumber elements with a length matching the number of attributes of the
outer relation. The length of the attribute mapping has been treated as
completely independent of the mapping itself until now, making it easy to pass
down an incorrect mapping length. This commit refactors the code related to
attribute mappings and moves it into an independent facility called attmap.c,
extracted from tupconvert.c. This merges the attribute mapping with its
length, avoiding to try to guess what is the length of a mapping to use as
this is computed once, when the map is built. This will avoid mistakes like
what has been fixed in dc816e58, which has used an incorrect mapping length by
matching it with the number of attributes of an inner relation (a child
partition) instead of an outer relation (a partitioned table). Author:
Michael Paquier Reviewed-by: Amit Langote Discussion:
https://postgr.es/m/20191121042556.GD153437@paquier.xyz
https://git.postgresql.org/pg/commitdiff/e1551f96e643a52a035c3b35777d968bc073f7fc

- Fix compiler warning in non-assert builds. Oversight in commit e1551f9.
Reported-by: Erik Rijkers Discussion:
https://postgr.es/m/b7ad911d3eaa29af9fcdb9ccb26c363c@xs4all.nl
https://git.postgresql.org/pg/commitdiff/2032645b195a53519b43dad57f55bc163b99f0ef

Amit Kapila pushed:

- Change overly strict Assert in TransactionGroupUpdateXidStatus. This Assert
thought that an overflowed transaction can never get registered for the group
update.  But that is not true, because even when the number of children for a
transaction got reduced, the overflow flag is not changed. And, for group
update, we only care about the current number of children for a transaction
that is being committed. Based on comments by Andres Freund, remove a
redundant Assert in TransactionIdSetPageStatus as we already had a static
Assert for the same condition a few lines earlier. Reported-by: Vignesh C
Author: Dilip Kumar Reviewed-by: Amit Kapila Backpatch-through: 11 Discussion:
https://postgr.es/m/CAFiTN-s5=uJw-Z6JC9gcqtBSjXsrHnU63PXBrA=pnBjqnkm5UA@mail.gmail.com
https://git.postgresql.org/pg/commitdiff/af3290f5e790dcd1be3ac209be1805626f4ebac8

- Fix subscriber invalid memory access on DDL. This patch allows building the
local relmap cache for a subscribed relation after processing pending
invalidation messages and potential relcache updates. Without this, the
attributes in the local cache don't tally with the updated relcache entry
leading to invalid memory access. Reported-by Jehan-Guillaume de Rorthais
Author: Jehan-Guillaume de Rorthais and Vignesh C Reviewed-by: Amit Kapila
Backpatch-through: 10 Discussion:
https://postgr.es/m/20191025175929.7e90dbf5@firost
https://git.postgresql.org/pg/commitdiff/04c8a69c0cccbc271e0feeb22a74c69fbd87c37e

Robert Haas pushed:

- Move interrupt-handling code into subroutines. Some auxiliary processes, as
well as the autovacuum launcher, have interrupt handling code directly in
their main loops. Try to abstract things a little better by moving it into
separate functions. This doesn't make any functional difference, and leaves
in place relatively large differences among processes in how interrupts are
handled, but hopefully it at least makes it easier to see the commonalities
and differences across process types. Patch by me, reviewed by Andres Freund
and Daniel Gustafsson. Discussion:
http://postgr.es/m/CA+TgmoZwDk=BguVDVa+qdA6SBKef=PKbaKDQALTC_9qoz1mJqg@mail.gmail.com
https://git.postgresql.org/pg/commitdiff/5910d6c7e311f0b14e3d3cb9ce3597c01d3a3cde

- Use PostgresSigHupHandler in more places. There seems to be no reason for
every background process to have its own flag indicating that a config-file
reload is needed. Instead, let's just use ConfigFilePending for that purpose
everywhere. Patch by me, reviewed by Andres Freund and Daniel Gustafsson.
Discussion:
http://postgr.es/m/CA+TgmoZwDk=BguVDVa+qdA6SBKef=PKbaKDQALTC_9qoz1mJqg@mail.gmail.com
https://git.postgresql.org/pg/commitdiff/1e53fe0e70f610c34f4c9e770d108cd94151342c

- Partially deduplicate interrupt handling for background processes. Where
possible, share signal handler code and main loop interrupt checking. This
saves quite a bit of code and should simplify maintenance, too. This commit
intends not to change the way anything works, even though that might allow
more code to be unified. It does unify a bunch of individual variables into a
ShutdownRequestPending flag that has is now used by a bunch of different
process types, though. Patch by me, reviewed by Andres Freund and Daniel
Gustafsson. Discussion:
http://postgr.es/m/CA+TgmoZwDk=BguVDVa+qdA6SBKef=PKbaKDQALTC_9qoz1mJqg@mail.gmail.com
https://git.postgresql.org/pg/commitdiff/7dbfea3c455e83a77213a92b9dfdc1c0577441ea

- Add missing "void" to prototypes. Commit
5910d6c7e311f0b14e3d3cb9ce3597c01d3a3cde got this wrong. Report and patch by
Andrew Gierth. Discussion:
http://postgr.es/m/8736diaj98.fsf@news-spur.riddles.org.uk
https://git.postgresql.org/pg/commitdiff/b1cc572f1274c946da42ed13ae4065e08b13262a

- simplehash: Allow use of simplehash without MemoryContext. If the
SH_RAW_ALLOCATOR is defined, it will be used to allocate bytes for the hash
table, and no dependencies on MemoryContext will exist. This means, in
particular, that the SH_CREATE function will not take a MemoryContext
argument. Patch by me, reviewed by Andres Freund. Discussion:
http://postgr.es/m/CA+Tgmob8oyh02NrZW=xCScB+5GyJ-jVowE3+TWTUmPF=FsGWTA@mail.gmail.com
https://git.postgresql.org/pg/commitdiff/48995040d5e7b1e9bac35d72aff326cae002219d

- simplehash: Allow for use in frontend code. Commit
48995040d5e7b1e9bac35d72aff326cae002219d removed the largest barrier to use of
simplehash in frontend code, but there's one more problem: it uses elog(ERROR,
...) or elog(LOG, ...) in a couple of places. Work around that by changing
those to pg_log_error() and pg_log_info() when FRONTEND is defined. Patch by
me, reviewed by Andres Freund. Discussion:
http://postgr.es/m/CA+Tgmob8oyh02NrZW=xCScB+5GyJ-jVowE3+TWTUmPF=FsGWTA@mail.gmail.com
https://git.postgresql.org/pg/commitdiff/da41d71070d14ecd9e2f4bbe275c98a136826d4b

- Code cleanup for toast_fetch_datum and toast_fetch_datum_slice. Rework some of
the checks for bad TOAST chunks to be a bit simpler and easier to understand.
These checks verify that (1) we get all and only the chunk numbers we expect
to see and (2) each chunk has the expected size. However, the existing code
was a bit hard to understand, at least for me; try to make it clearer. As
part of that, have toast_fetch_datum_slice check the relationship between
endchunk and totalchunks only with an Assert() rather than checking every
chunk number against both values. There's no need to check that relationship
in production builds because it's not a function of whether on-disk corruption
is present; it's just a question of whether the code does the right math.
Also, have toast_fetch_datum_slice() use ereport(ERROR) rather than
elog(ERROR). Commit fd6ec93bf890314ac694dc8a7f3c45702ecc1bbd made the two
functions inconsistent with each other. Rename assorted variables for better
clarity and consistency, and move assorted variables from function scope to
the function's main loop. Remove a few variables that are used only once
entirely. Patch by me, reviewed by Peter Eisentraut. Discussion:
http://postgr.es/m/CA+TgmobBzxwFojJ0zV0Own3dr09y43hp+OzU2VW+nos4PMXWEg@mail.gmail.com
https://git.postgresql.org/pg/commitdiff/d5406dea25b600408e7acf17d5a06e82d3ce6d0d

- Fix bad formula in previous commit. Commit
d5406dea25b600408e7acf17d5a06e82d3ce6d0d used a slightly novel, and wrong,
approach to compute the length of the last toast chunk. It worked fine unless
the last chunk happened to have the largest possible size.
https://git.postgresql.org/pg/commitdiff/5184f110aa4130ec87b0b3e0834292cd8cb1fd8a

- Move heap-specific detoasting logic into a separate function. The new
function, heap_fetch_toast_slice, is shared between toast_fetch_datum_slice
and toast_fetch_datum, and does all the work of scanning the TOAST table,
fetching chunks, and storing them into the space allocated for the result
varlena. As an incidental side effect, this allows toast_fetch_datum_slice to
perform the scan with only a single scankey if all chunks are being fetched,
which might have some tiny performance benefit. Discussion:
http://postgr.es/m/CA+TgmobBzxwFojJ0zV0Own3dr09y43hp+OzU2VW+nos4PMXWEg@mail.gmail.com
https://git.postgresql.org/pg/commitdiff/e9fd0415e6e27c8ce5f40152aa98347ca6fe4385

- Re-#include <time.h> in checkpointer.c. Commit
7dbfea3c455e83a77213a92b9dfdc1c0577441ea thought it could get away with
removing this, but Thomas Munro reports, on behalf of the buildfarm, that it's
still needed at least on Windows to avoid compiler warnings.
https://git.postgresql.org/pg/commitdiff/9aafc4529f50e027f05037d993c6dd60a6cde54d

- Fix minor problems with non-exclusive backup cleanup. The previous coding
imagined that it could call before_shmem_exit() when a non-exclusive backup
began and then remove the previously-added handler by calling
cancel_before_shmem_exit() when that backup ended. However, this only works
provided that nothing else in the system has registered a before_shmem_exit()
hook in the interim, because cancel_before_shmem_exit() is documented to
remove a callback only if it is the latest callback registered. It also only
works if nothing can ERROR out between the time that sessionBackupState is
reset and the time that cancel_before_shmem_exit(), which doesn't seem to be
strictly true. To fix, leave the handler installed for the lifetime of the
session, arrange to install it just once, and teach it to quietly do nothing
if there isn't a non-exclusive backup in process. This is a bug, but for now
I'm not going to back-patch, because the consequences are minor. It's possible
to cause a spurious warning to be generated, but that doesn't really matter.
It's also possible to trigger an assertion failure, but production builds
shouldn't have assertions enabled. Patch by me, reviewed by Kyotaro
Horiguchi, Michael Paquier (who preferred a different approach, but got
outvoted), Fujii Masao, and Tom Lane, and with comments by various others.
Discussion:
http://postgr.es/m/CA+TgmobMjnyBfNhGTKQEDbqXYE3_rXWpc4CM63fhyerNCes3mA@mail.gmail.com
https://git.postgresql.org/pg/commitdiff/303640199d0436c5e7acdf50b837a027b5726594

- Update neglected comment. Commit d986d4e87f61c68f52c68ebc274960dc664b7b4e
renamed a variable but neglected to update the corresponding comment. Amit
Langote
https://git.postgresql.org/pg/commitdiff/7cdcc747a9fe588f9e9b3a5d3feb650340093fb2

- Extend the ProcSignal mechanism to support barriers. A new function
EmitProcSignalBarrier() can be used to emit a global barrier which all
backends that participate in the ProcSignal mechanism must absorb, and a new
function WaitForProcSignalBarrier() can be used to wait until all relevant
backends have in fact absorbed the barrier. This can be used to coordinate
global state changes, such as turning checksums on while the system is
running. There's no real client of this mechanism yet, although two are
proposed, but an enum has to have at least one element, so this includes a
placeholder type (PROCSIGNAL_BARRIER_PLACEHOLDER) which should be replaced by
the first real client of this mechanism to get committed. Andres Freund and
Robert Haas, reviewed by Daniel Gustafsson and, in earlier versions, by Magnus
Hagander. Discussion:
http://postgr.es/m/CA+TgmoZwDk=BguVDVa+qdA6SBKef=PKbaKDQALTC_9qoz1mJqg@mail.gmail.com
https://git.postgresql.org/pg/commitdiff/16a4e4aecd47da7a6c4e1ebc20f6dd1a13f9133b

Bruce Momjian pushed:

- Remove redundant not-null test. Reported-by: Ranier Vilela Discussion:
https://postgr.es/m/MN2PR18MB2927E73FADCA8967B2302469E3490@MN2PR18MB2927.namprd18.prod.outlook.com
Author: Ranier Vilela Backpatch-through: master
https://git.postgresql.org/pg/commitdiff/181932a03212751102fb2c105fe556a26aee6ed7

- Remove meaningless assignments in nbtree code. Reported-by: Ranier Vilela
Discussion:
https://postgr.es/m/MN2PR18MB2927BB876D12A70FDBE8F35AE3450@MN2PR18MB2927.namprd18.prod.outlook.com
Backpatch-through: master
https://git.postgresql.org/pg/commitdiff/05684c8255af57258386c00354c61155ec519707

- revert: Remove meaningless assignments in nbtree code. Reverts commit
05684c8255. Reported-by: Tom Lane Discussion:
https://postgr.es/m/404.1576770942@sss.pgh.pa.us Backpatch-through: master
https://git.postgresql.org/pg/commitdiff/b93e9a5c94b4c89932a637798bd560971fe790d7

- docs: clarify handling of column lists in COPY TO/FROM. Previously it was
unclear how COPY FROM handled cases where not all columns were specified, or
if the order didn't match. Reported-by: pavlo(dot)golub(at)gmail(dot)com Discussion:
https://postgr.es/m/157487729344.7213.14245726713444755296@wrigleys.postgresql.org
Backpatch-through: 9.4
https://git.postgresql.org/pg/commitdiff/4cab43ec806a6298a545a399415ee8c4fe9307a8

- C comment: clarify why psql's help/exit/quit must alone. Document why no
indentation and why no non-whitespace postfix is supported.
Backpatch-through: master
https://git.postgresql.org/pg/commitdiff/4376fdbae11de9333012a31ba8d3a4f4b5d7a692

Andrew Dunstan pushed:

- Superuser can permit passwordless connections on postgres_fdw. Currently
postgres_fdw doesn't permit a non-superuser to connect to a foreign server
without specifying a password, or to use an authentication mechanism that
doesn't use the password. This is to avoid using the settings and identity of
the user running Postgres. However, this doesn't make sense for all
authentication methods. We therefore allow a superuser to set
"password_required 'false'" for user mappings for the postgres_fdw. The
superuser must ensure that the foreign server won't try to rely solely on the
server identity (e.g. trust, peer, ident) or use an authentication mechanism
that relies on the password settings (e.g. md5, scram-sha-256). This feature
is a prelude to better support for sslcert and sslkey settings in user
mappings. Author: Craig Ringer. Discussion:
https://postgr.es/m/075135da-545c-f958-fed0-5dcb462d6dae@2ndQuadrant.com
https://git.postgresql.org/pg/commitdiff/6136e94dcb88c50b6156aa646746565400e373d4

== Pending Patches ==

Tomas Zubiri sent in a patch to psql to try connecting on a TCP socket when a
Unix socket is not available.

Amit Langote sent in another revision of a patch to make it possible to add
partitioned tables to publications.

Surafel Temesgen sent in another revision of a patch to implement conflict
handling in COPY ... FROM.

Peter Eisentraut sent in another revision of a patch to have logical replication
subscriber fire column triggers.

John Naylor sent in a patch to optimize Bison by removing single character
literal tokens.

Daniel Vérité sent in a patch to make psql error out on output failures.

Ranier Vilela sent in three revisions of a patch to fix a memory leak in
src/common/exec.c.

Fujii Masao sent in another revision of a patch to ignore invalid pages.

Ranier Vilela sent in a patch to fix some crypto issues on Windows.

Ranier Vilela sent in two revisions of a patch to add support for
BCryptGenRandom on Windows.

Amit Kapila and Vigneshwaran C traded patches to fix subscriber invalid memory
access on DDL.

Peter Eisentraut sent in a patch to implement polymorphic table functions.

Artur Zakirov sent in another revision of a patch to fix a bug that manifested
as pg_upgrade fails with non-standard ACL.

Julien Rouhaud sent in another revision of a patch to fix collation versioning.

Robert Haas sent in another revision of a patch to tableam which allows a choice
of toast AM, and adds a new callback, relation_fetch_toast_slice.

Kyotaro HORIGUCHI and Robert Haas traded patches to fix a minor breakage in
non-exclusive backup.

Suraj Kharage sent in three more revisions of a patch to implement backup
manifests.

Masahiko Sawada and Amit Kapila traded patches to implement block-level parallel
vacuum.

Amit Langote sent in another revision of a patch to remove the dependency on
estate->es_result_relation_info from FDW APIs, remove es_result_relation_info,
rearrange the partition update row movement code a bit, and refactor the
transition tuple capture code a bit.

Peter Eisentraut sent in a patch to allow building without default socket
directory.

Ranier Vilela sent in a patch to add setenv support to Windows.

Ranier Vilela sent in a patch to remove an expression that's always false.

Robert Haas sent in two more revisions of a patch to add a pg_shmem_allocations
view.

Amit Khandekar sent in two more revisions of a patch to use vfd for logrep.

Simon Riggs and Mark Dilger traded patches to add a READ UNCOMMITTED isolation
level.

Peter Eisentraut sent in a patch to disallow dropping rules on system tables by
default.

Simon Riggs sent in a patch to optimize for the case when an xid has not yet
been assigned, i.e. for read-only transactions.

Imai Yoshikazu sent in another revision of a patch to add a pg_stat_waitaccum
view, and change the measuring method of wait event timed from INSTR_TIME (which
uses gettimeofday or clock_gettime) to rdtsc.

Tatsuro Yamada sent in another revision of a patch to implement a progress
report for ANALYZE.

Dmitry Dolgov sent in another revision of a patch to implement generic type
subscripting.

Michaël Paquier sent in another revision of a patch to clean up some old cruft
related to Windows.

Thomas Munro sent in another revision of a patch to add kqueue(2) support for
WaitEventSet.

Peter Geoghegan sent in another revision of a patch to add deduplication to
nbtree.

Álvaro Herrera sent in a patch to remove some duplicated code in pg_publication.

Álvaro Herrera sent in another revision of a patch to implement multiranges.

Tom Lane sent in a patch to add extra info to the ParseNamespaceItem structs,
which requires some refactoring to allow the ParseNamespaceItems, not just the
referenced RTEs, to be passed down through Var lookup/construction.

Mark Lorenz sent in two more revisions of a patch to fix WW in to_date.

Thomas Munro sent in a patch not to call ResetLatch() in
ConditionVariablePrepareToSleep().

Andrey Borodin sent in a patch to disallow cancellation of waiting for
synchronous replication.

Yugo Nagata sent in another revision of a patch to implement incremental view
maintenance.

Peter Smith sent in another revision of a patch to add more compile-time asserts
to expose inconsistencies.

Peter Eisentraut sent in a patch to optimize updates of tables with generated
only updating columns by those generated columns whose base columns have changed
in this update and keep the rest unchanged.

Tom Lane sent in three revisions of a patch to fix the fact that composite types
can't be partition keys.

Peter Eisentraut sent in another revision of a patch to simplify passing of
configure arguments to pg_config.

Maxence Ahlouche sent in another revision of a patch to Fix %w length in PROMPT2
when PROMPT1 contains a newline, in psql.

Noah Misch sent in two revisions of a patch to ensure that mdclose() copes with
FileClose() failure.

Jeff Davis sent in another revision of a patch to implement memory-bounded hash
aggregation.

Justin Pryzby sent in a patch to implement planner support functions for GROUP
BY f(), implemented for date_trunc().