== PostgreSQL Weekly News - November 10, 2019 ==

== PostgreSQL Weekly News - November 10, 2019 ==

PGDay SF will take place on January 21, 2020 at the Swedish American Hall in San
Francisco. The CfP is open through November 22, 2019 at
https://2020.pgdaysf.org/callforpapers/

Prague PostgreSQL Developer Day 2020 (P2D2 2020) is a two-day
conference that will be held on February 5-6, 2020 in Prague, Czech Republic.
The CfP is open until December 6, 2019 at https://p2d2.cz/callforpapers
http://www.p2d2.cz/

== PostgreSQL Product News ==

Postgresql Anonymizer, an extension that hides or replaces personally
identifiable information (PII) or commercially sensitive data from a PostgreSQL
database, released.
https://postgresql-anonymizer.readthedocs.io/en/latest/

Cybertec Migrator 1.0, a migration tool from Oracle to PostgreSQL, released.
https://www.cybertec-postgresql.com/en/products/cybertec-migrator/

pg_probackup 2.2.5, a utility to manage backup and recovery of PostgreSQL
database clusters, released.
https://github.com/postgrespro/pg_probackup

== PostgreSQL Jobs for November ==

http://archives.postgresql.org/pgsql-jobs/2019-11/

== PostgreSQL Local ==

2Q PGConf 2019 will be held December 4 & 5 in Chicago.
https://www.2qpgconf.com/

PGDay Down Under 2019 will be held on November 15, 2019 in Sydney,
Australia.
https://pgdu.org/

The CfP for FOSDEM PGDay is open through November 18th, 2019 at 24:00 CET.
https://2020.fosdempgday.org/

pgDay Israel 2020 will take place on March 19, 2020 in Tel Aviv.
The CfP is open through January 15, 2020.
http://pgday.org.il/

pgDay Paris 2020 will be held in Paris, France on March 26, 2020
at Espace Saint-Martin.
http://2020.pgday.paris/

Nordic PGDay 2020 will be held in Helsinki, Finland at the Hilton Helsinki
Strand Hotel on March 24, 2020. The CfP is open through December 31, 2019 at
https://2020.nordicpgday.org/cfp/

PGConf India 2020 will be on February 26-28, 2020 in Bengaluru, Karnataka. The
CfP is open until November 15, 2019.
http://pgconf.in/

PostgreSQL(at)SCaLE is a two day, two track event which takes place on
March 5-6, 2020, at Pasadena Convention Center, as part of SCaLE 18X. The CfP is
open through November 30, 2019.
https://www.socallinuxexpo.org/scale/18x/postgresscale

The German-speaking PostgreSQL Conference 2020 will take place on May 15, 2019
in Stuttgart.

== PostgreSQL in the News ==

Planet PostgreSQL: http://planet.postgresql.org/

PostgreSQL Weekly News is brought to you this week by David Fetter

Submit news and announcements by Sunday at 3:00pm PST8PDT to david(at)fetter(dot)org(dot)

== Applied Patches ==

Tom Lane pushed:

- Suppress warning from older compilers. Commit 8af1624e3 introduced a warning
about possibly returning without a value, on compilers that don't realize that
ereport(ERROR) doesn't return. Tweak the code to avoid that. Per buildfarm.
Back-patch to 9.6, like the aforesaid commit.
https://git.postgresql.org/pg/commitdiff/db27b60f07a039e236738fe90bd4215cafd35cf2

- Fix ginEntryInsert's counting of GIN leaf tuples. As the code stands, nEntries
counts the number of ginEntryInsert() calls, so that's what you end up with at
the end of a GIN index build. However, ginvacuumcleanup() recomputes nEntries
as the number of surviving leaf tuples, and that's generally consistent with
the way that gincostestimate() uses the value. So let's clearly define
nEntries as the number of leaf tuples, and therefore adjust ginEntryInsert()
to increment it only when we make a new one, not when we add TIDs into an
existing tuple or posting tree. In practice this inconsistency probably has
little impact, so I don't feel a need to back-patch. Insung Moon and Keisuke
Kuroda Discussion:
https://postgr.es/m/CAEMmqBuH_O-oXL+3_ArQ6F5cJ7kXVow2SGQB3HRacku_T+xkmA@mail.gmail.com
https://git.postgresql.org/pg/commitdiff/ec28808ba85853fa14b090199236ca555273607e

- Stabilize pg_dump output order for similarly-named triggers and policies. The
code only compared two triggers' names and namespaces (the latter being the
owning table's schema). This could result in falling back to an OID-based
sort of similarly-named triggers on different tables. We prefer to avoid that,
so add a comparison of the table names too. (The sort order is thus table
namespace, trigger name, table name, which is a bit odd, but it doesn't seem
worth contorting the code to work around that.) Likewise for policy objects,
in 9.5 and up. Complaint and fix by Benjie Gillam. Back-patch to all
supported branches. Discussion:
https://postgr.es/m/CAMThMzEEt2mvBbPgCaZ1Ap1N-moGn=Edxmadddjq89WG4NpPtQ@mail.gmail.com
https://git.postgresql.org/pg/commitdiff/5102f39440f758ea53c2e1cdea7d8411df1805d2

- Generate EquivalenceClass members for partitionwise child join rels. Commit
d25ea0127 got rid of what I thought were entirely unnecessary derived child
expressions in EquivalenceClasses for EC members that mention multiple
baserels. But it turns out that some of the child expressions that code
created are necessary for partitionwise joins, else we fail to find matching
pathkeys for Sort nodes. (This happens only for certain shapes of the
resulting plan; it may be that partitionwise aggregation is also necessary to
show the failure, though I'm not sure of that.) Reverting that commit
entirely would be quite painful performance-wise for large partition sets. So
instead, add code that explicitly generates child expressions that match only
partitionwise child join rels we have actually generated. Per report from
Justin Pryzby. (Amit Langote noticed the problem earlier, though it's not
clear if he recognized then that it could result in a planner error, not
merely failure to exploit partitionwise join, in the code as-committed.)
Back-patch to v12 where commit d25ea0127 came in. Amit Langote, with lots of
kibitzing from me Discussion:
https://postgr.es/m/CA+HiwqG2WVUGmLJqtR0tPFhniO=H=9qQ+Z3L_ZC+Y3-EVQHFGg@mail.gmail.com
Discussion: https://postgr.es/m/20191011143703.GN10470@telsasoft.com
https://git.postgresql.org/pg/commitdiff/529ebb20aaa5eb68e4fb7a656271bbb83efe9529

- Fix "unexpected relkind" error when denying permissions on toast tables.
get_relkind_objtype, and hence get_object_type, failed when applied to a toast
table. This is not a good thing, because it prevents reporting of perfectly
legitimate permissions errors. (At present, these functions are in fact
*only* used to determine the ObjectType argument for acl_error() calls.) It
seems best to have them fall back to returning OBJECT_TABLE in every case
where they can't determine an object type for a pg_class entry, so do that.
In passing, make some edits to alter.c to make it more obvious that those
calls of get_object_type() are used only for error reporting. This might save
a few cycles in the non-error code path, too. Back-patch to v11 where this
issue originated. John Hsu, Michael Paquier, Tom Lane Discussion:
https://postgr.es/m/C652D3DF-2B0C-4128-9420-FB5379F6B1E4@amazon.com
https://git.postgresql.org/pg/commitdiff/a30531c5c8a384363d410d4027e1c1eeed76e550

- Avoid logging complaints about abandoned connections when using PAM. For a
long time (since commit aed378e8d) we have had a policy to log nothing about a
connection if the client disconnects when challenged for a password. This is
because libpq-using clients will typically do that, and then come back for a
new connection attempt once they've collected a password from their user, so
that logging the abandoned connection attempt will just result in log spam.
However, this did not work well for PAM authentication: the bottom-level
function pam_passwd_conv_proc() was on board with it, but we logged messages
at higher levels anyway, for lack of any reporting mechanism. Add a flag and
tweak the logic so that the case is silent, as it is for other password-using
auth mechanisms. Per complaint from Yoann La Cancellera. It's been like this
for awhile, so back-patch to all supported branches. Discussion:
https://postgr.es/m/CACP=ajbrFFYUrLyJBLV8=q+eNCapa1xDEyvXhMoYrNphs-xqPw@mail.gmail.com
https://git.postgresql.org/pg/commitdiff/3affe76ef8227dad85b61cec769235f682132651

- Tweak some authentication debug messages to follow project style. Avoid
initial capital, since that's not how we do it. Discussion:
https://postgr.es/m/CACP=ajbrFFYUrLyJBLV8=q+eNCapa1xDEyvXhMoYrNphs-xqPw@mail.gmail.com
https://git.postgresql.org/pg/commitdiff/66c61c81b90c68db84d422092fbbf8a1a82ee09a

- Sync our DTrace infrastructure with c.h's definition of type bool. Since
commit d26a810eb, we've defined bool as being either _Bool from <stdbool.h>,
or "unsigned char"; but that commit overlooked the fact that probes.d has
"#define bool char". For consistency, make it say "unsigned char" instead.
This should be strictly a cosmetic change, but it seems best to be in sync.
Formally, in the now-normal case where we're using <stdbool.h>, it'd be better
to write "#define bool _Bool". However, then we'd need some build
infrastructure to inject that configuration choice into probes.d, and it
doesn't seem worth the trouble. We only use <stdbool.h> if sizeof(_Bool) is
1, so having DTrace think that bool parameters are "unsigned char" should be
close enough. Back-patch to v12 where d26a810eb came in. Discussion:
https://postgr.es/m/CAA4eK1LmaKO7Du9M9Lo=kxGU8sB6aL8fa3sF6z6d5yYYVe3BuQ@mail.gmail.com
https://git.postgresql.org/pg/commitdiff/ff43b3e88eceb94b3a2b2579e8510e4f3aedbe09

- Minor code review for tuple slot rewrite. Avoid creating
transiently-inconsistent slot states where possible, by not setting
TTS_FLAG_SHOULDFREE until after the slot actually has a free'able tuple
pointer, and by making sure that we reset tts_nvalid and related derived state
before we replace the tuple contents. This would only matter if something
were to examine the slot after we'd suffered some kind of error (e.g. out of
memory) while manipulating the slot. We typically don't do that, so these
changes might just be cosmetic --- but even if so, it seems like good
future-proofing. Also remove some redundant Asserts, and add a couple for
consistency. Back-patch to v12 where all this code was rewritten.
Discussion: https://postgr.es/m/16095-c3ff2e5283b8dba5@postgresql.org
https://git.postgresql.org/pg/commitdiff/22e44e8dbcfe4b9f3c4189f07a2361c951d72514

- Fix integer-overflow edge case detection in interval_mul and pgbench. This
patch adopts the overflow check logic introduced by commit cbdb8b4c0 into two
more places. interval_mul() failed to notice if it computed a new
microseconds value that was one more than INT64_MAX, and pgbench's
double-to-int64 logic had the same sorts of edge-case problems that cbdb8b4c0
fixed in the core code. To make this easier to get right in future, put the
guts of the checks into new macros in c.h, and add commentary about how to use
the macros correctly. Back-patch to all supported branches, as we did with
the previous fix. Yuya Watari Discussion:
https://postgr.es/m/CAJ2pMkbkkFw2hb9Qb1Zj8d06EhWAQXFLy73St4qWv6aX=vqnjw@mail.gmail.com
https://git.postgresql.org/pg/commitdiff/a7145f6bc8b7774a29f3841552a463fb20ea98d7

- Move declaration of ecpg_gettext() to a saner place. Declaring this in the
client-visible header ecpglib.h was a pretty poor decision. It's not meant to
be application-callable (and if it was, putting it outside the extern "C" {
... } wrapper means that C++ clients would fail to call it). And the
declaration would not even compile for a client, anyway, since it would not
have the macro pg_attribute_format_arg(). Fortunately, it seems that no
clients have tried to include this header with ENABLE_NLS defined, or we'd
have gotten complaints about that. But we have no business putting such a
restriction on client code. Move the declaration to ecpglib_extern.h, since
in fact nothing outside src/interfaces/ecpg/ecpglib/ needs to call it. The
practical effect of this is just that clients can now safely #include
ecpglib.h while having ENABLE_NLS defined, but that seems like enough of a
reason to back-patch it. Discussion:
https://postgr.es/m/20590.1573069709@sss.pgh.pa.us
https://git.postgresql.org/pg/commitdiff/c8cb98ec41f0c41ac3b4c3e8be01f12c4c53d3aa

Peter Eisentraut pushed:

- Catch invalid typlens in a couple of places. Rearrange the logic in
record_image_cmp() and datum_image_eq() to error out on unexpected typlens
(either not supported there or completely invalid due to corruption). Barring
corruption, this is not possible today but it seems more future-proof and
robust to fix this. Reported-by: Peter Geoghegan <pg(at)bowt(dot)ie>
https://git.postgresql.org/pg/commitdiff/8557a6f10ca6f01f4b2f2f25e197292f3f46bb5c

- Fix some compiler warnings on older compilers. Some older compilers appear to
not understand the recently introduced PG_FINALLY code structure that well in
some circumstances and complain about possibly uninitialized variables. So to
fix, initialize the variables explicitly in the cases complained about.
Discussion:
https://www.postgresql.org/message-id/flat/95a822c3-728b-af0e-d7e5-71890507ae0c%402ndquadrant.com
https://git.postgresql.org/pg/commitdiff/a63c84e59acf9f5e2b54aad4974a80e5075af646

- Remove unused function argument. The cache_plan argument to ri_PlanCheck has
not been used since e8c9fd5fdf768323911f7088e8287f63b513c3c6. Reviewed-by:
vignesh C <vignesh21(at)gmail(dot)com> Discussion:
https://www.postgresql.org/message-id/flat/ec8a8b45-a30b-9193-cd4b-985d60d1497e%402ndquadrant.com
https://git.postgresql.org/pg/commitdiff/5b7ba75f7ff854003231e8099e3038c7e2eba875

- Fix memory allocation mistake. The previous code was allocating more memory
than necessary because the formula used the wrong data type. Reported-by:
Jehan-Guillaume de Rorthais <jgdr(at)dalibo(dot)com> Discussion:
https://www.postgresql.org/message-id/20191105172918.3e32a446@firost
https://git.postgresql.org/pg/commitdiff/d40abd5fcfb25d764419f8e9bffa5cdbdb247c1b

- Fix nested error handling in PG_FINALLY. We need to pop the error stack before
running the user-supplied PG_FINALLY code. Otherwise an error in the cleanup
code would end up at the same sigsetjmp() invocation and result in an infinite
error handling loop. Reviewed-by: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> Discussion:
https://www.postgresql.org/message-id/flat/95a822c3-728b-af0e-d7e5-71890507ae0c%402ndquadrant.com
https://git.postgresql.org/pg/commitdiff/581a55889ba7f76dd87a270e37e5137f684bfdb7

- Remove HAVE_LONG_LONG_INT. The presence of long long int is now implied in the
requirement for C99 and the configure check for the same. We keep the define
hard-coded in ecpg_config.h for backward compatibility with ecpg-using user
code. Discussion:
https://www.postgresql.org/message-id/flat/5cdd6a2b-b2c7-c6f6-344c-a406d5c1a254%402ndquadrant.com
https://git.postgresql.org/pg/commitdiff/effa40281bb1f6c71e81c980f86852ab3be603c3

- More precise errors from initial pg_control check. Use a separate error
message for invalid checkpoint location and invalid state instead of just
"invalid data" for both. Reviewed-by: Michael Paquier <michael(at)paquier(dot)xyz>
Discussion:
https://www.postgresql.org/message-id/20191107041630.GK1768@paquier.xyz
https://git.postgresql.org/pg/commitdiff/b85e43feb3e837699239aba6b8e0f280a59417be

- Fix gratuitous error message variation.
https://git.postgresql.org/pg/commitdiff/3dcffb381c81c9c8f8254100feacac256b9e75a6

- Fix negative bitmapset member not allowed error in logical replication. This
happens when we add a replica identity column on a subscriber that does not
yet exist on the publisher, according to the mapping maintained by the
subscriber. Code that checks whether the target relation on the subscriber is
updatable would check the replica identity attribute bitmap with a column
number -1, which would result in an error. To fix, skip such columns in the
bitmap lookup and consider the relation not updatable. The result is
consistent with the rule that the replica identity columns on the subscriber
must be a subset of those on the publisher, since if the column doesn't exist
on the publisher, the column set on the subscriber can't be a subset.
Reported-by: Tim Clarke <tim(dot)clarke(at)minerva(dot)info> Analyzed-by: Jehan-Guillaume
de Rorthais <jgdr(at)dalibo(dot)com> Discussion:
https://www.postgresql.org/message-id/flat/a9139c29-7ddd-973b-aa7f-71fed9c38d75%40minerva.info
https://git.postgresql.org/pg/commitdiff/1c60e40ad54b18685436443da3c56a7018a35475

- doc: Further tweak recovery parameters documentation. Remove one sentence that
was deemed misleading. Discussion:
https://www.postgresql.org/message-id/flat/E1iEgSp-0004R5-2E%40gemulon.postgresql.org
https://git.postgresql.org/pg/commitdiff/27b59d619ddfb512e3f158a87f557a46f2d5794f

- doc: Clarify documentation about SSL passphrases. The previous statement that
using a passphrase disables the ability to change the server's SSL
configuration without a server restart was no longer completely true since the
introduction of ssl_passphrase_command_supports_reload.
https://git.postgresql.org/pg/commitdiff/d2d4c35080026ba3da49583e30c1fd53c9fb22dd

- Fix subscription test. After altering a subscription, we should wait until the
updated table sync data has been fetched by the subscriber.
https://git.postgresql.org/pg/commitdiff/ef8fcbff56e8904ba83b2659e9d0290e6106928e

Michaël Paquier pushed:

- Refactor code building relation options. Historically, the code to build
relation options has been shaped the same way in multiple code paths by using
a set of datums in input with the options parsed with a static table which is
then filled with the option values. This introduces a new common routine in
reloptions.c to do most of the legwork for the in-core code paths. Author:
Amit Langote Reviewed-by: Michael Paquier Discussion:
https://postgr.es/m/CA+HiwqGsoSn_uTPPYT19WrtR7oYpYtv4CdS0xuedTKiHHWuk_g@mail.gmail.com
https://git.postgresql.org/pg/commitdiff/3534fa2233285c1fab1e668871aabf05e5541213

- Doc: Improve description around ALTER TABLE ATTACH PARTITION. This clarifies
more how to use and how to take advantage of constraints when attaching a new
partition. Author: Justin Pryzby Reviewed-by: Amit Langote, Álvaro Herrera,
Michael Paquier Discussion:
https://postgr.es/m/20191028001207.GB23808@telsasoft.com Backpatch-through: 10
https://git.postgresql.org/pg/commitdiff/ea881338014fd5c7d864d78d2eb441d784d1cc39

- Doc: Clarify locks taken when using ALTER TABLE ATTACH PARTITION. Since
898e5e32, this command uses partially ShareUpdateExclusiveLock, but the docs
did not get the call. Author: Justin Pryzby Reviewed-by: Amit Langote, Álvaro
Herrera, Michael Paquier Discussion:
https://postgr.es/m/20191028001207.GB23808@telsasoft.com Backpatch-through: 12
https://git.postgresql.org/pg/commitdiff/2a4d96ebbd65be9aa421a8a4550a51ff12bc6d2d

- Fix timestamp of sent message for write context in logical decoding. When
sending data for logical decoding using the streaming replication protocol via
a WAL sender, the timestamp of the sent write message is allocated at the
beginning of the message when preparing for the write, and actually computed
when the write message is ready to be sent. The timestamp was getting
computed after sending the message. This impacts anything using logical
decoding, causing for example logical replication to report mostly NULL for
last_msg_send_time in pg_stat_subscription. This commit makes sure that the
timestamp is computed before sending the message. This is wrong since
5a991ef, so backpatch down to 9.4. Author: Jeff Janes Discussion:
https://postgr.es/m/CAMkU=1z=WMn8jt7iEdC5sYNaPgAgOASb_OW5JYv-vMdYaJSL-w@mail.gmail.com
Backpatch-through: 9.4
https://git.postgresql.org/pg/commitdiff/5f6b1eb0cf4b0fb101478da51c294a63fd4726ef

- Check after errors of SPI_execute() in xml.c. SPI gets used to build a list of
relation OIDs for XML object generation, and one code path building a list
uses SPI_execute() without looking at errors it produces. So fix that.
Author: Mark Dilger Reviewed-by: Michael Paquier, Pavel Stehule Discussion:
https://postgr.es/m/17d30445-4862-7917-170f-84328dcd292d@gmail.com
https://git.postgresql.org/pg/commitdiff/3feb6ace7cfe8edbf6db702de06dc9295f307a8e

- Add tests for COPY in PL/pgSQL. This stresses the error handling of COPY
inside SPI which does not support the operation using stdin or stdout, and
these scenarios were not tested up to now. Author: Mark Dilger Discussion:
https://postgr.es/m/a6e9b130-7fd5-387b-4ec5-89bda24373ab@gmail.com
https://git.postgresql.org/pg/commitdiff/1858b105b05fcded43e9f2b767dec7268431043b

- Fix new COPY test of PL/pgSQL with VPATH builds. The buildfarm has turned red
after 1858b10 because VPATH builds need to use "@abs_srcdir@" and not
"@abs_builddir@" for paths coming directly from the source tree. The input
file of the new test got that right, but not the output file. Per complaints
from several buildfarm animals, including desmoxytes and culicidae. I have
also reproduced the error by myself.
https://git.postgresql.org/pg/commitdiff/943b447d303e6d0a7635d39ac552550b77b96cef

Andres Freund pushed:

- Split all OBJS style lines in makefiles into one-line-per-entry style. When
maintaining or merging patches, one of the most common sources for conflicts
are the list of objects in makefiles. Especially when the split across lines
has been changed on both sides, which is somewhat common due to attempting to
stay below 80 columns, those conflicts are unnecessarily laborious to resolve.
By splitting, and alphabetically sorting, OBJS style lines into one object per
line, conflicts should be less frequent, and easier to resolve when they still
occur. Author: Andres Freund Discussion:
https://postgr.es/m/20191029200901.vww4idgcxv74cwes@alap3.anarazel.de
https://git.postgresql.org/pg/commitdiff/01368e5d9da77099b38aac527b01b85cc7869b25

- Make StringInfo available to frontend code. There's plenty places in frontend
code that could benefit from a string buffer implementation. Some because it
yields simpler and faster code, and some others because of the desire to share
code between backend and frontend. While there is a string buffer
implementation available to frontend code, libpq's PQExpBuffer, it is clunkier
than stringinfo, it introduces a libpq dependency, doesn't allow for sharing
between frontend and backend code, and has a higher API/ABI stability
requirement due to being exposed via libpq. Therefore it seems best to just
making StringInfo being usable by frontend code. There's not much to do for
that, except for rewriting two subsequent elog/ereport calls into others types
of error reporting, and deciding on a maximum string length. For the maximum
string size I decided to privately define MaxAllocSize to the same value as
used in the backend. It seems likely that we'll want to reconsider this for
both backend and frontend code in the not too far away future. For now I've
left stringinfo.h in lib/, rather than common/, to reduce the likelihood of
unnecessary breakage. We could alternatively decide to provide a redirecting
stringinfo.h in lib/, or just not provide compatibility. Author: Andres
Freund Reviewed-By: Kyotaro Horiguchi, Daniel Gustafsson Discussion:
https://postgr.es/m/20190920051857.2fhnvhvx4qdddviz@alap3.anarazel.de
https://git.postgresql.org/pg/commitdiff/26aaf97b683d6258c098859e6b1268e1f5da242f

- Pass ItemPointer not HeapTuple to IndexBuildCallback. Not all AMs use
HeapTuples internally, making it inconvenient to pass a HeapTuple. As the
index callbacks really only need the TID, not the full tuple, modify callback
to only take ItemPointer. Author: Ashwin Agrawal Reviewed-By: Andres Freund
Discussion:
https://postgr.es/m/CALfoeis6=8ehuR=VNtHvj3z16cYfCwPdTcpaxU+sfSUJ5QgR3g@mail.gmail.com
https://git.postgresql.org/pg/commitdiff/aae50236e4ce95c05a3962be0814c74c5a22206d

Bruce Momjian pushed:

- doc: fix plurality typo on bgwriter doc sentence. Reported-by:
matthew(dot)alton(at)gmail(dot)com Discussion:
https://postgr.es/m/157204060717.1042.8194076510523669244@wrigleys.postgresql.org
Backpatch-through: 9.4
https://git.postgresql.org/pg/commitdiff/4b5e58b86e3b09daa7384dbbd0bb4cacbd9bd7c6

- docs: clarify that only INSERT and UPDATE triggers can mod. NEW. The point is
that DELETE triggers cannot modify any values. Reported-by: Eugen Konkov
Discussion: https://postgr.es/m/919823407.20191029175436@yandex.ru
Backpatch-through: 9.4
https://git.postgresql.org/pg/commitdiff/c10fae21428fe926568e67b7662ed8577faf9235

Fujii Masao pushed:

- Add "G" (server-side data generation) as an initialization step in pgbench.
This commit allows --init-steps option in pgbench to accept "G" character
meaning server-side data generation as an initialization step. With "G", only
limited queries are sent from pgbench client and then data is actually
generated in the server. This might make the initialization phase faster if
the bandwidth between pgbench client and the server is low. Author: Fabien
Coelho Reviewed-by: Anna Endo, Ibrar Ahmed, Fujii Masao Discussion:
https://postgr.es/m/alpine.DEB.2.21.1904061826420.3678@lancre
https://git.postgresql.org/pg/commitdiff/a386942bd29b0ef0c9df061392659880d22cdf43

- Correct the command tags for ALTER ... RENAME COLUMN. Previously ALTER
MATERIALIZED VIEW / FOREIGN TABLE ... RENAME COLUMN ... returned "ALTER TABLE"
as a command tag. This commit fixes them so that they return "ALTER
MATERIALIZED VIEW" and "ALTER FOREIGN TABLE" as command tags, respectively.
This issue exists in all supported versions, but we don't back-patch this
because it's not enough of a bug to justify taking any compatibility risks
for. Otherwise, the back-patch would cause minor version update to break, for
example, the existing event trigger functions using TG_TAG. Author: Fujii
Masao Reviewed-by: Ibrar Ahmed Discussion:
https://postgr.es/m/CAHGQGwGUaC03FFdTFoHsCuDrrNvFvNVQ6xyd40==P25WvuBJjg@mail.gmail.com
https://git.postgresql.org/pg/commitdiff/979766c0afceb95c4b96323b57a0f6edf8472612

- Fix assertion failure when running pgbench -s. If there is the WAL page that
the continuation WAL record just fits within (i.e., the continuation record
ends just at the end of the page) and the LSN in such page is specified with
-s option, previously pg_waldump caused an assertion failure. The cause of
this assertion failure was that XLogFindNextRecord() that pg_waldump -s calls
mistakenly handled such special WAL page. This commit changes
XLogFindNextRecord() so that it can handle such WAL page correctly.
Back-patch to all supported versions. Author: Andrey Lepikhov Reviewed-by:
Fujii Masao, Michael Paquier Discussion:
https://postgr.es/m/99303554-5dd5-06e6-f943-b3005ccd6edd@postgrespro.ru
https://git.postgresql.org/pg/commitdiff/a0c96856e8b38a102daac76e3d385d9f8876744e

Andrew Gierth pushed:

- Request small targetlist for input to WindowAgg. WindowAgg will potentially
store large numbers of input rows into tuplestores to allow access to other
rows in the frame. If the input is coming via an explicit Sort node, then
unneeded columns will already have been discarded (since Sort requests a small
tlist); but there are idioms like COUNT(*) OVER () that result in the input
not being sorted at all, and cases where the input is being sorted by some
means other than a Sort; if we don't request a small tlist, then WindowAgg's
storage requirement is inflated by the unneeded columns. Backpatch back to
9.6, where the current tlist handling was added. (Prior to that, WindowAgg
would always use a small tlist.) Discussion:
https://postgr.es/m/87a7ator8n.fsf@news-spur.riddles.org.uk
https://git.postgresql.org/pg/commitdiff/a9056cc637f2d183eefa165ebbd0308d9a89abb9

Tomáš Vondra pushed:

- Document log_transaction_sample_rate as superuser-only. The docs do say which
GUCs can be changed only by superusers, but we forgot to mention this for the
new log_transaction_sample_rate. This GUC was introduced in PostgreSQL 12, so
backpatch accordingly. Author: Adrien Nayrat Backpatch-through: 12
https://git.postgresql.org/pg/commitdiff/11d9ac28e5e0077d8f23761c2962a30423f44cee

- Allow sampling of statements depending on duration. This allows logging a
sample of statements, without incurring excessive log traffic (which may
impact performance). This can be useful when analyzing workloads with lots of
short queries. The sampling is configured using two new GUC parameters: *
log_min_duration_sample - minimum required statement duration *
log_statement_sample_rate - sample rate (0.0 - 1.0) Only statements with
duration exceeding log_min_duration_sample are considered for sampling. To
enable sampling, both those GUCs have to be set correctly. The existing
log_min_duration_statement GUC has a higher priority, i.e. statements with
duration exceeding log_min_duration_statement will be always logged,
irrespectedly of how the sampling is configured. This means only
configurations log_min_duration_sample < log_min_duration_statement do
actually sample the statements, instead of logging everything. Author: Adrien
Nayrat Reviewed-by: David Rowley, Vik Fearing, Tomas Vondra Discussion:
https://postgr.es/m/bbe0a1a8-a8f7-3be2-155a-888e661cc06c@anayrat.info
https://git.postgresql.org/pg/commitdiff/6e3e6cc0e884a6091e1094dff29db430af08fb93

Thomas Munro pushed:

- Add reusable routine for making arrays unique. Introduce qunique() and
qunique_arg(), which can be used after qsort() and qsort_arg() respectively to
remove duplicate values. Use it where appropriate. Author: Thomas Munro
Reviewed-by: Tom Lane (in an earlier version) Discussion:
https://postgr.es/m/CAEepm%3D2vmFTNpAmwbGGD2WaryM6T3hSDVKQPfUwjdD_5XY6vAA%40mail.gmail.com
https://git.postgresql.org/pg/commitdiff/7815e7efdb4ce9575b5d8460beb0dd2569d7ca3a

Álvaro Herrera pushed:

- Fix SET CONSTRAINTS .. DEFERRED on partitioned tables. SET CONSTRAINTS ...
DEFERRED failed on partitioned tables, because of a sanity check that ensures
that the affected constraints have triggers. On partitioned tables, the
triggers are in the leaf partitions, not in the partitioned relations
themselves, so the sanity check fails. Removing the sanity check solves the
problem, because the code needed to support the case is already there.
Backpatch to 11. Note: deferred unique constraints are not affected by this
bug, because they do have triggers in the parent partitioned table. I did not
add a test for this scenario. Discussion:
https://postgr.es/m/20191105212915.GA11324@alvherre.pgsql
https://git.postgresql.org/pg/commitdiff/b4bcc6bfdfa2b564b0171d437f9988a3159fd58d

- Add backtrace support for error reporting. Add some support for automatically
showing backtraces in certain error situations in the server. Backtraces are
shown on assertion failure; also, a new setting backtrace_functions can be set
to a list of C function names, and all ereport()s and elog()s from the
mentioned functions will have backtraces generated. Finally, the function
errbacktrace() can be manually added to an ereport() call to generate a
backtrace for that call. Authors: Peter Eisentraut, Álvaro Herrera
Discussion:
https://postgr.es/m//5f48cb47-bf1e-05b6-7aae-3bf2cd01586d(at)2ndquadrant(dot)com
Discussion:
https://postgr.es/m/CAMsr+YGL+yfWE=JvbUbnpWtrRZNey7hJ07+zT4bYJdVp4Szdrg@mail.gmail.com
https://git.postgresql.org/pg/commitdiff/71a8a4f6e36547bb060dbcc961ea9b57420f7190

Peter Geoghegan pushed:

- Use "low key" terminology in nbtsort.c. nbtree index builds once stashed the
"minimum key" for a page, which was used as the basis of the pivot tuple that
gets placed in the next level up (i.e. the tuple that stores the downlink to
the page in question). It doesn't quite work that way anymore, so the "minimum
key" terminology now seems misleading (these days the minimum key is actually
a straight copy of the high key from the left sibling, which is a distinct
thing in subtle but important ways). Rename this concept to "low key". This
name is a lot clearer given that there is now a sharp distinction between
pivot and non-pivot tuples. Also remove comments that describe obsolete
details about how the minimum key concept used to work. Rather than
generating the minus infinity item for the leftmost page on a level by copying
the new item and truncating that copy, simply allocate a small buffer. The
old approach confusingly created the impression that the new item had some
kind of significance. This was another artifact of how things used to work
before commits 8224de4f and dd299df8.
https://git.postgresql.org/pg/commitdiff/e86c8ef243aad4570f66a406c81211f75774ced1

Etsuro Fujita pushed:

- postgres_fdw: Fix error message for PREPARE TRANSACTION. Currently,
postgres_fdw does not support preparing a remote transaction for two-phase
commit even in the case where the remote transaction is read-only, but the old
error message appeared to imply that that was not supported only if the remote
transaction modified remote tables. Change the message so as to include the
case where the remote transaction is read-only. Also fix a comment above the
message. Also add a note about the lack of supporting PREPARE TRANSACTION to
the postgres_fdw documentation. Reported-by: Gilles Darold Author: Gilles
Darold and Etsuro Fujita Reviewed-by: Michael Paquier and Kyotaro Horiguchi
Backpatch-through: 9.4 Discussion:
https://postgr.es/m/08600ed3-3084-be70-65ba-279ab19618a5%40darold.net
https://git.postgresql.org/pg/commitdiff/879c1176157175e0a83742b810f137aebccef4a4

== Pending Patches ==

Tom Lane sent in a patch to fix some infelicities between tab completion and
COPY in psql.

Thomas Munro sent in another revision of a patch to add an SQL type xid8 to
expose FullTransactionId to users, and introduce xid8 variants of the txid_XXX()
fmgr functions.

Asif Rehman sent in a patch to add the grammar for parallel backups.

Tom Lane sent in a patch to suppress PAM logging for EOF.

Daniel Vérité sent in a patch to cover update the unaccent.rules for Arabic
letters.

Euler Taveira de Oliveira sent in a patch to show only rows that open
connections in some system views.

Tom Lane sent in another revision of a patch to include RELKIND_TOASTVALUE in
get_relkind_objtype.

Álvaro Herrera sent in a patch to make pg_restore -f- dump to stdout instead of to ./-

Thomas Munro sent in two more revisions of a patch to show parallel leader stats
in EXPLAIN output, and improve EXPLAIN of Sort in parallel queries.

Amit Khandekar sent in another revision of a patch to implement minimal logical
decoding on standbys.

Grigory Smolkin sent in four revisions of a patch to add a "latest" recovery
target.

Peter Eisentraut sent in a patch to refactor parse analysis of the EXECUTE
command.

Peter Geoghegan sent in two more revisions of a patch to teach datum_image_eq()
about cstring datums, add pageinspect instrumentation for same, and use it to
add deduplication to nbtree.

Thomas Munro and Julien Rouhaud traded patches to implement collation
versioning.

Quan Zongliang sent in another revision of a patch to restore replication
settings when modifying a field type.

Andrey V. Lepikhov sent in another revision of a patch to remove unneeded
self-joins.

Alexander Korotkov sent in a patch to introduce RRRR and RR, revise YYY, YY and
Y datetime format patterns.

Amit Langote sent in another revision of a patch to fix a bug that manifested as
ERROR: could not find pathkey item to sort.

Tatsuro Yamada sent in two more revisions of a patch to add a progress report
for ANALYZE.

Amul Sul sent in two more revisions of a patch to improve the partition-matching
algorithm for partition-wise JOINs.

Rafa Torre sent in another revision of a patch to allow using arbitrary queries
in the postgres_fdw.

David Cramer sent in another revision of a patch to use pg_strncasecmp for
windows compatibility.

Kirk Jamison sent in another revision of a patch to optimize dropping of
relation buffers using dlist.

Peter Eisentraut sent in a patch to fix nested error handling in PG_FINALLY.

Fujii Masao, Yu Kimura, and Fabien COELHO traded patches to extend the
initialization phase control for pgbench.

Álvaro Herrera sent in two revisions of a patch to fix deferred constraints on
partitioned rels.

Alexey Bashtanov sent in another revision of a patch to log bind parameter
values on error.

Yuya Watari sent in three more revisions of a patch to silence some compiler
warnings about implicit conversion from 'long' to 'double'.

Moon Insung sent in a patch to make the ssl_passphrase_command superuser-only.

Pavel Stěhule sent in another revision of a patch to reduce the overhead of SPI.

David Cramer sent in another revision of a patch to add binary support for the
pgoutput plugin.

Etsuro Fujita and Gilles Darold traded patches to fix the PREPARE TRANSACTION
doc and postgres_fdw message.

Kyotaro HORIGUCHI sent in another revision of a patch to fix the WAL-skipping
feature.

Amit Langote sent in a patch to make some cosmetic improvements to partition-
wise join code, move some code from joinrel.c to relnode.c, and fix
partition-wise join to handle FULL JOINs correctly.

Fabien COELHO sent in another revision of a patch to factor out the query
cancellation code.

btfujiitkp sent in another revision of a patch to enable CREATE OR REPLACE VIEW
to rename the columns.

Fabien COELHO sent in another revision of a patch to pgbench to make variable
management more efficient.

Pierre Ducroquet sent in a patch to simplify AND clauses that have NOT NULL
clauses on similar variables.

Tom Lane sent in a patch to rewrite windows locale name matching so they're all
lower case.

Vigneshwaran C and Amit Kapila traded patches to fix an infelicity between
logical_work_mem and logical streaming of large in-progress transactions.

Juan José Santamaría Flecha sent in a patch to add the FILE_SHARE options to
TestLib::slurp_file using Win32API::File

Mark Dilger sent in a patch to check the return value of SPI_execute.

Mahendra Singh sent in two revisions of a patch to Force all vacuum to use
parallel vacuum in tests.

Zeng Wenjing and Konstantin Knizhnik traded patches to implement global
temporary tables.

Tomáš Vondra and Mark Dilger traded patches to use multiple extended statistics
for estimates.

Paul A Jungwirth sent in two more revisions of a patch to implement multiranges.

Paul A Jungwirth sent in another revision of a patch to add temporal PKs and FKs,
and adds FOR PORTION OF to gram.y.

Dmitry Dolgov sent in another revision of a patch to implement index skip scans.

Mark Dilger sent in another revision of a patch to add safeguards against
incorrect fd flags for fsync().

Michaël Paquier sent in a patch to cover SKIP_LOCKED better in the tests.

Sergei Fedorov sent in a patch to make it possible to bind a portal to a
previously prepared statement with parameters in libpq.

Mark Dilger sent in a patch to test COPY inside PL/pgSQL.

Andrew Dunstan sent in another revision of a patch to add an SSL passphrase
callback.

Amit Langote sent in another revision of a patch to do some refactoring of
publication and subscription code, and support adding partitioned tables to a
publication.

Amit Kapila sent in another revision of a patch to implement dropdb --force.

Mark Dilger sent in a patch to refactor SPI by deprecating unused SPI error
codes, changing SPI_connect and SPI_connect_ext to return void, deprecating
SPI_ERROR_REL_DUPLICATE and SPI_ERROR_REL_NOT_FOUND error codes in favor of
throwing an error, deprecating SPI_ERROR_PARAM in favor of throwing an error,
and deprecating SPI_ERROR_OPUNKNOWN error code in favor of throwing an
exception instead.

Yuli Khodorkovskiy sent in another revision of a patch to add a MAC check for
TRUNCATE.

Thomas Munro sent in a patch to make TransactionIdIsCurrentTransactionId() more
efficient by exiting earlier if the xid is in the current top transaction, make
PredicateLockTuple() exit earlier if tuple was written by current transaction,
and remove the HeapTuple dependency for predicate locking functions.

Ranier VF and sent in two revisions of a patch to avoid calling strlen
repeatedly in a loop.

Thomas Munro and Juan José Santamaría Flecha traded patches to add collation
versions for Windows with major and minor versions included.

Christoph Berg sent in a patch to add pg_tablespace_statfs() functions.

Robert Haas sent in another revision of a patch to do some code cleanup for
toast_fetch_datum and toast_fetch_datum_slice, and move heap-specific detoasting
logic into a separate function.

John Dent sent in another revision of a patch to add pipelined function scans.

Julien Rouhaud sent in another revision of a patch to pass the query string to
the planner, and add planning counters to pg_stat_statements.

Tom Lane sent in a patch to add a "trusted" attribute to extensions that denotes
that they can be installed by the database owner, which need not be a database
superuser.

Daniel Gustafsson sent in another revision of a patch to enable pg_dump to dump
foreign data.

Peter Eisentraut sent in a patch to make walsender use a temporary replication
slot by default.

Euler Taveira de Oliveira sent in a patch to skip empty transactions for logical
replication.

Amit Kapila sent in another revision of a patch to to rearrange dropdb() to
avoid errors after allowing other sessions to exit.

Shawn Wang sent in a patch to implement TDE.

Vigneshwaran C sent in another revision of a patch to make the order of the
header file includes consistent in backend modules.

Tom Lane sent in a patch to fix an issue where pg_upgrade fails to detect
unsupported arrays and ranges.