== PostgreSQL Weekly News - March 15, 2020 ==

== PostgreSQL Weekly News - March 15, 2020 ==

Several conferences have been cancelled due to COVID-19. These include
PGConfNepal, Nordic PGDay, and pgDay Paris.

pgDay Israel 2020 will take place on September 10, 2020 in Tel Aviv.
The CfP is open through January 15, 2020.
http://pgday.org.il/

== PostgreSQL Product News ==

pg_featureserv a RESTful web service that provides access to spatial data
stored in PostGIS tables, released.
https://github.com/crunchydata/pg_featureserv

Postgresql Anonymizer 0.6, an extension that hides or replaces personally
identifiable information (PII) or commercially sensitive data from a PostgreSQL
database, released.
https://postgresql-anonymizer.readthedocs.io/en/latest/

PostgreSQL Automatic Failover (PAF) v2.3.0 released.
https://github.com/ClusterLabs/PAF/releases/latest

pg_tileserv, a PostGIS-only tile server written in Go, released.
https://github.com/crunchydata/pg_tileserv

== PostgreSQL Local ==

PGDay Ukraine will take place April 25th, 2020 in Lviv at the Bank Hotel.
https://pgday.org.ua/

PgConf Belgium will take place on May 8, 2020 at the UCLL Campus in Haasrode
(near Leuven). The CfS is open through March 31, 2020 at https://pgconf.be/#cfs
http://pgconf.be

The German-speaking PostgreSQL Conference 2020 will take place on May 15, 2019
in Stuttgart.

PGCon 2020 will take place in Ottawa on May 26-29, 2020.
https://www.pgcon.org/2020/

PGDay.IT 2020 will take place June 11-12 in Bergamo, Italy.
https://2020.pgday.it/en/

Swiss PGDay 2020 will take place in Rapperswil (near Zurich) on June 18-19, 2020.
The Call for Speakers is open through March 17, 2020.
https://www.pgday.ch/2020/

Postgres Ibiza will be held in Ibiza, Spain on June 25-26, 2020.
https://pgibz.io/

PostgresLondon 2020 will be July 7-8, 2020 with an optional training day on
July 6. The CfP is open at https://forms.gle/5m8ybUt9YDZG4gVU7 through March
27, 2020.
http://postgreslondon.org

PG Day Russia will be in Saint Petersburg on July 10, 2020. The CfP is open at
https://pgday.ru/en/2020/for-speakers through April 6, 2020.
https://pgday.ru/en/2020/

FOSS4G 2020, will take place in Calgary, Alberta, Canada August 24-29 2020.
the Call for Papers is currently open at https://2020.foss4g.org/speakers/
https://2020.foss4g.org/

pgDay Israel 2020 will take place on September 10, 2020 in Tel Aviv.
http://pgday.org.il/

Austrian pgDay will take place September 18, 2020 at Schloss Schoenbrunn
(Apothekertrakt) in Vienna. The CfP is open until April 19, 2020 at
https://pgday.at/en/talk-commitee/
https://pgday.at/en/

== PostgreSQL in the News ==

Planet PostgreSQL: http://planet.postgresql.org/

PostgreSQL Weekly News is brought to you this week by David Fetter

Submit news and announcements by Sunday at 3:00pm PST8PDT to david(at)fetter(dot)org(dot)

== Applied Patches ==

Tom Lane pushed:

- Add an explicit test to catch changes in checksumming calculations. Seems like
a good idea in view of 006517432 and addd034ae. Michael Paquier, Tom Lane
Discussion: https://postgr.es/m/20200306075230.GA118430@paquier.xyz
https://git.postgresql.org/pg/commitdiff/38ce06c37e9cc8fec9125262b129602c2274f344

- Add an "absval" parameter to allow contrib/dict_int to ignore signs. Jeff
Janes Discussion:
https://postgr.es/m/CAMkU=1xRcs_BUPzR0+V3WndaCAv0E_m3h6aUEJ8NF-sY1nnHsw@mail.gmail.com
https://git.postgresql.org/pg/commitdiff/806eb92c018102da9514443ad19160c7b1841062

- Fix pg_dump/pg_restore to restore event triggers later. Previously, event
triggers were restored just after regular triggers (and FK constraints, which
are basically triggers). This is risky since an event trigger, once
installed, could interfere with subsequent restore commands. Worse, because
event triggers don't have any particular dependencies on any post-data
objects, a parallel restore would consider them eligible to be restored the
moment the post-data phase starts, allowing them to also interfere with
restoration of a whole bunch of objects that would have been restored before
them in a serial restore. There's no way to completely remove the risk of a
misguided event trigger breaking the restore, since if nothing else it could
break other event triggers. But we can certainly push them to later in the
process to minimize the hazard. To fix, tweak the RestorePass mechanism
introduced by commit 3eb9a5e7c so that event triggers are handled as part of
the post-ACL processing pass (renaming the "REFRESH" pass to "POST_ACL" to
reflect its more general use). This will cause them to restore after
everything except matview refreshes, which seems OK since matview refreshes
really ought to run in the post-restore state of the database. In a parallel
restore, event triggers and matview refreshes might be intermixed, but that
seems all right as well. Also update the code and comments in pg_dump_sort.c
so that its idea of how things are sorted agrees with what actually happens
due to the RestorePass mechanism. This is mostly cosmetic: it'll affect the
order of objects in a dump's TOC, but not the actual restore order. But not
changing that would be quite confusing to somebody reading the code.
Back-patch to all supported branches. Fabrízio de Royes Mello, tweaked a bit
by me Discussion:
https://postgr.es/m/CAFcNs+ow1hmFox8P--3GSdtwz-S3Binb6ZmoP6Vk+Xg=K6eZNA@mail.gmail.com
https://git.postgresql.org/pg/commitdiff/8728b2c703573c6cbb431e01e228ae489117f974

- Preserve integer and float values accurately in (de)serialize_deflist.
Previously, this code just smashed all types of DefElem values to strings,
cavalierly reasoning that nobody would care. But in point of fact, most of
the defGetFoo functions do distinguish among different input syntaxes; for
instance defGetBoolean will accept 1 as an integer but not "1" as a string.
This led to CREATE/ALTER TEXT SEARCH DICTIONARY accepting 0 and 1 as values
for boolean dictionary properties, only to have the dictionary fail at
runtime. We can upgrade this behavior by teaching serialize_deflist that it
does not need to quote T_Integer or T_Float nodes' values on output, and then
teaching deserialize_deflist to restore unquoted integer or float values as
the appropriate node type. This should not break anything using
pg_ts_dict.dictinitoption, since that field is just defined as being something
valid to include in CREATE TEXT SEARCH DICTIONARY. deserialize_deflist is
also used to parse the options arguments for the ts_headline family of
functions, but so far as I can see this won't cause any problems there either:
the only consumer of that output is prsd_headline which always uses
defGetString. (Really that's a bad idea, but I won't risk changing it here.)
This is surely a bug fix, but given the lack of field complaints I don't think
it's necessary to back-patch. Discussion:
https://postgr.es/m/CAMkU=1xRcs_BUPzR0+V3WndaCAv0E_m3h6aUEJ8NF-sY1nnHsw@mail.gmail.com
https://git.postgresql.org/pg/commitdiff/d01f03a495294f726001a90ec27a675428d9fc45

- Avoid portability problem introduced in 0a42a2e9c. Apparently not all shells
handle nested quotes quite the same. But we don't need the inner double quotes
in this case, so just drop them. Discussion:
https://postgr.es/m/50ecb345-4dc5-1f7d-64ca-7018195fcc8d@2ndquadrant.com
https://git.postgresql.org/pg/commitdiff/dbf05a14399430751e54cd0b4bd8fb0e4f1fb309

- Ensure that CREATE TABLE LIKE copies any NO INHERIT constraint property. Since
the documentation about LIKE doesn't say that a copied constraint has
properties different from the original, it seems that ignoring a NO INHERIT
property doesn't meet the principle of least surprise. So make it copy that.
(Note, however, that we still don't copy a NOT VALID property; CREATE TABLE
offers no way to do that, plus it seems pointless.) Arguably this is a bug
fix; but no back-patch, as it seems barely possible somebody is depending on
the current behavior. Ildar Musin and Chris Travers; reviewed by Amit Langote
and myself Discussion:
https://postgr.es/m/CAONYFtMC6C+3AWCVp7Yd8H87Zn0GxG1_iQG6_bQKbaqYZY0=-g@mail.gmail.com
https://git.postgresql.org/pg/commitdiff/cacef172237fd3426b578f81b7414c0de56cbaaf

- Marginal comments and docs cleanup. Fix up some imprecise comments and poor
markup from ba79cb5dc. Also try to convert the documentation of
log_min_duration_sample and friends into passable English.
https://git.postgresql.org/pg/commitdiff/c8e8b2f9dfee21fa4ef1ec7da5c10e6ef706df36

- Avoid holding a directory FD open across pg_ls_dir_files() calls. This coding
technique is undesirable because (a) it leaks the FD for the rest of the
transaction if the SRF is not run to completion, and (b) allocated FDs are a
scarce resource, but multiple interleaved uses of the relevant functions could
eat many such FDs. In v11 and later, a query such as "SELECT pg_ls_waldir()
LIMIT 1" yields a warning about the leaked FD, and the only reason there's no
warning in earlier branches is that fd.c didn't whine about such leaks before
commit 9cb7db3f0. Even disregarding the warning, it wouldn't be too hard to
run a backend out of FDs with careless use of these SQL functions. Hence,
rewrite the function so that it reads the directory within a single call,
returning the results as a tuplestore rather than via value-per-call mode.
There are half a dozen other built-in SRFs with similar problems, but let's
fix this one to start with, just to see if the buildfarm finds anything wrong
with the code. In passing, fix bogus error report for stat() failure: it was
whining about the directory when it should be fingering the individual file.
Doubtless a copy-and-paste error. Back-patch to v10 where this function was
added. Justin Pryzby, with cosmetic tweaks and test cases by me Discussion:
https://postgr.es/m/20200308173103.GC1357@telsasoft.com
https://git.postgresql.org/pg/commitdiff/085b6b6679e73b9b386f209b4d625c7bc60597c0

- Fix test case instability introduced in 085b6b667. I forgot that the WAL
directory might hold other files besides WAL segments, notably including new
segments still being filled. That means a blind test for the first file's size
being 16MB can fail. Restrict based on file name length to make it more
robust. Per buildfarm.
https://git.postgresql.org/pg/commitdiff/a029a0641cce002acbb5e8ac6ec5533e599a9aea

- Doc: fix mistaken reference to "PG_ARGNULL_xxx()" macro. This should of course
be just "PG_ARGISNULL()". Also reorder a couple of paras to make the
discussion of PG_ARGISNULL less disjointed. Back-patch to v10 where the error
was introduced. Laurenz Albe and Tom Lane, per an anonymous docs comment
Discussion:
https://postgr.es/m/158399487096.5708.10696365251766477013@wrigleys.postgresql.org
https://git.postgresql.org/pg/commitdiff/dbf95c843a3d66cf9a692f5937a1bec4f2261035

- Restructure polymorphic-type resolution in funcapi.c.
resolve_polymorphic_tupdesc() and resolve_polymorphic_argtypes() failed to
cover the case of having to resolve anyarray given only an anyrange input. The
bug was masked if anyelement was also used (as either input or output), which
probably helps account for our not having noticed. While looking at this I
noticed that resolve_generic_type() would produce the wrong answer if asked to
make that same resolution. ISTM that resolve_generic_type() is confusingly
defined and overly complex, so rather than fix it, let's just make funcapi.c
do the specific lookups it requires for itself. With this change,
resolve_generic_type() is not used anywhere, so remove it in HEAD. In the
back branches, leave it alone (complete with bug) just in case any external
code is using it. While we're here, make some other refactoring adjustments
in funcapi.c with an eye to upcoming future expansion of the set of
polymorphic types: * Simplify quick-exit tests by adding an overall
have_polymorphic_result flag. This is about a wash now but will be a win when
there are more flags. * Reduce duplication of code between
resolve_polymorphic_tupdesc() and resolve_polymorphic_argtypes(). * Don't
bother to validate correct matching of anynonarray or anyenum; the parser
should have done that, and even if it didn't, just doing "return false" here
would lead to a very confusing, off-point error message. (Really, "return
false" in these two functions should only occur if the call_expr isn't
supplied or we can't obtain data type info from it.) * For the same reason,
throw an elog rather than "return false" if we fail to resolve a polymorphic
type. The bug's been there since we added anyrange, so back-patch to all
supported branches. Discussion:
https://postgr.es/m/6093.1584202130@sss.pgh.pa.us
https://git.postgresql.org/pg/commitdiff/4dbcb3f844eca4a401ce06aa2781bd9a9be433e9

- Rearrange pseudotypes.c to get rid of duplicative code. Commit a5954de10
replaced a lot of manually-coded stub I/O routines with code generated by
macros. That was a good idea but it didn't go far enough, because there were
still manually-coded stub input routines for types that had live output
routines. Refactor the macro so that we can generate just a stub input
routine at need. Also create similar macros to generate stub binary I/O
routines, since we have some of those now. The only stub functions that
remain hand-coded are shell_in() and shell_out(), which need to be separate
because they use different error messages. While here, rearrange the
commentary to discuss each type not each function. This provides a better way
to explain the *why* of which types need which support, rather than just
duplicatively annotating the functions. Discussion:
https://postgr.es/m/24137.1584139352@sss.pgh.pa.us
https://git.postgresql.org/pg/commitdiff/87c9c2571c8146f1594830072253ba39e62ccbc8

Michaël Paquier pushed:

- Doc: fix some description of environment variables with frontend tools. This
addresses a couple of issues in the documentation: - Description of PG_COLOR
was missing for some tools (pg_archivecleanup and pg_test_fsync), while the
other descriptions had grammar mistakes. - pgbench supports more environment
variables: PGUSER, PGHOST and PGPORT. - vacuumlo, oid2name and pgbench support
coloring (HEAD only) Author: Michael Paquier Reviewed-by: Fabien Coelho,
Daniel Gustafsson, Juan José Santamaría Flecha Discussion:
https://postgr.es/m/20200304075418.GJ2593@paquier.xyz Backpatch-through: 12
https://git.postgresql.org/pg/commitdiff/5aaa584f819e2d8d2fb23cdf2174ce6e5628e1f5

- Prevent reindex of invalid indexes on TOAST tables. Such indexes can only be
duplicated leftovers of a previously failed REINDEX CONCURRENTLY command, and
a valid equivalent is guaranteed to exist. As toast indexes can only be
dropped if invalid, reindexing these would lead to useless duplicated indexes
that can't be dropped anymore, except if the parent relation is dropped.
Thanks to Justin Pryzby for reminding that this problem was reported long ago
during the review of the original patch of REINDEX CONCURRENTLY, but the issue
was never addressed. Reported-by: Sergei Kornilov, Justin Pryzby Author:
Julien Rouhaud Reviewed-by: Michael Paquier Discussion:
https://postgr.es/m/36712441546604286%40sas1-890ba5c2334a.qloud-c.yandex.net
Discussion: https://postgr.es/m/20200216190835.GA21832@telsasoft.com
Backpatch-through: 12
https://git.postgresql.org/pg/commitdiff/61d7c7bce3686ec02bd64abac742dd35ed9b9b01

- Refactor compile-time assertion checks in c.h. This commit refactors and
simplifies the definitions of StaticAssertStmt, StaticAssertExpr and
StaticAssertDecl. By unifying the C and C++ fallback implementations, this
reduces the number of different implementations from four to three. Author:
Michael Paquier Reviewed-by: Georgios Kokolatos, Tom Lane Discussion:
https://postgr.es/m/20200204081503.GF2287@paquier.xyz
https://git.postgresql.org/pg/commitdiff/b7f64c64d3262c309e2c5e8c11fd111b1dbd1a4a

Fujii Masao pushed:

- Mark ssl_passphrase_command as GUC_SUPERUSER_ONLY. This commit changes the GUC
ssl_passphrase_command so that it's examinable by only superuser and a member
of pg_read_all_settings. Per discussion, we determined to do this because the
parameter may contain a sensitive informtaion like a passphrase itself.
Author: Insung Moon Reviewed-by: Keisuke Kuroda Discussion:
https://postgr.es/m/CAEMmqBuHVGayc+QkYKgx3gWSdqwTAQGw+0DYn3WhcX-eNa2ntA@mail.gmail.com
https://git.postgresql.org/pg/commitdiff/d9249441efc5b4adc5b1a71f3d16538c68e198fc

- Avoid assertion failure with targeted recovery in standby mode. At the end of
recovery, standby mode is turned off to re-fetch the last valid record from
archive or pg_wal. Previously, if recovery target was reached and standby mode
was turned off while the current WAL source was stream, recovery could try to
retrieve WAL file containing the last valid record unexpectedly from stream
even though not in standby mode. This caused an assertion failure. That is,
the assertion test confirms that WAL file should not be retrieved from stream
if standby mode is not true. This commit moves back the current WAL source to
archive if it's stream even though not in standby mode, to avoid that
assertion failure. This issue doesn't cause the server to crash when built
with assertion disabled. In this case, the attempt to retrieve WAL file from
stream not in standby mode just fails. And then recovery tries to retrieve WAL
file from archive or pg_wal. Back-patch to all supported branches. Author:
Kyotaro Horiguchi Reviewed-by: Fujii Masao Discussion:
https://postgr.es/m/20200227.124830.2197604521555566121.horikyota.ntt@gmail.com
https://git.postgresql.org/pg/commitdiff/ef34ab42a85968f3ab4c8e490b77dfb3565dd5e7

- Fix bug that causes to report waiting in PS display twice, in hot standby.
Previously "waiting" could appear twice via PS in case of lock conflict in hot
standby mode. Specifically this issue happend when the delay in WAL
application determined by max_standby_archive_delay and
max_standby_streaming_delay had passed but it took more than 500 msec to
cancel all the conflicting transactions. Especially we can observe this easily
by setting those delay parameters to -1. The cause of this issue was that
WaitOnLock() and ResolveRecoveryConflictWithVirtualXIDs() added "waiting" to
the process title in that case. This commit prevents
ResolveRecoveryConflictWithVirtualXIDs() from reporting waiting in case of
lock conflict, to fix the bug. Back-patch to all back branches. Author:
Masahiko Sawada Reviewed-by: Fujii Masao Discussion:
https://postgr.es/m/CA+fd4k4mXWTwfQLS3RPwGr4xnfAEs1ysFfgYHvmmoUgv6Zxvmg@mail.gmail.com
https://git.postgresql.org/pg/commitdiff/17d3fcdc39577b1e43624392c25e5384da42e522

- Tidy up XLogSource code in xlog.c. This commit replaces 0 used as an initial
value of XLogSource variable, with XLOG_FROM_ANY. Also this commit changes
those variable so that XLogSource instead of int is used as the type for them.
These changes are for code readability and debugger-friendliness. Author:
Kyotaro Horiguchi Reviewed-by: Fujii Masao Discussion:
https://postgr.es/m/20200227.124830.2197604521555566121.horikyota.ntt@gmail.com
https://git.postgresql.org/pg/commitdiff/71e0d0a73773b3985db658d3c5366ce5ceef76ae

Peter Eisentraut pushed:

- Add tg_updatedcols to TriggerData. This allows a trigger function to determine
for an UPDATE trigger which columns were actually updated. This allows some
optimizations in generic trigger functions such as lo_manage and
tsvector_update_trigger. Reviewed-by: Daniel Gustafsson <daniel(at)yesql(dot)se>
Discussion:
https://www.postgresql.org/message-id/flat/11c5f156-67a9-0fb5-8200-2a8018eb2e0c(at)2ndquadrant(dot)com
https://git.postgresql.org/pg/commitdiff/71d60e2aa05157efec28393b15c0b0b9fc1b210c

- Code simplification. Initialize TriggerData to 0 for the whole struct
together, instead of each field separately. Reviewed-by: Daniel Gustafsson
<daniel(at)yesql(dot)se> Discussion:
https://www.postgresql.org/message-id/flat/11c5f156-67a9-0fb5-8200-2a8018eb2e0c(at)2ndquadrant(dot)com
https://git.postgresql.org/pg/commitdiff/8f152b6c50c764d4c300e73a535da88cd1b18fa5

- Support adding partitioned tables to publication. When a partitioned table is
added to a publication, changes of all of its partitions (current or future)
are published via that publication. This change only affects which tables a
publication considers as its members. The receiving side still sees the data
coming from the individual leaf partitions. So existing restrictions that
partition hierarchies can only be replicated one-to-one are not changed by
this. Author: Amit Langote <amitlangote09(at)gmail(dot)com> Reviewed-by: Rafia Sabih
<rafia(dot)pghackers(at)gmail(dot)com> Reviewed-by: Peter Eisentraut
<peter(dot)eisentraut(at)2ndquadrant(dot)com> Discussion:
https://www.postgresql.org/message-id/flat/CA+HiwqH=Y85vRK3mOdjEkqFK+E=ST=eQiHdpj43L=_eJMOOznQ(at)mail(dot)gmail(dot)com
https://git.postgresql.org/pg/commitdiff/17b9e7f9fe238eeb5f6b40061b444ebf28d9e06f

- Remove utils/acl.h from catalog/objectaddress.h. The need for this was removed
by 8b9e9644dc6a9bd4b7a97950e6212f63880cf18b. A number of files now need to
include utils/acl.h or parser/parse_node.h explicitly where they previously
got it indirectly somehow. Since parser/parse_node.h already includes
nodes/parsenodes.h, the latter is then removed where the former was added.
Also, remove nodes/pg_list.h from objectaddress.h, since that's included via
nodes/parsenodes.h. Reviewed-by: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> Reviewed-by:
Alvaro Herrera <alvherre(at)2ndquadrant(dot)com> Discussion:
https://www.postgresql.org/message-id/flat/7601e258-26b2-8481-36d0-dc9dca6f28f1%402ndquadrant.com
https://git.postgresql.org/pg/commitdiff/3c173a53a825075f3efe32b9917eff5063e81f4d

- Remove win32ver.rc from version_stamp.pl. This removes another relic from the
old nmake-based Windows build. version_stamp.pl put version number information
into win32ver.rc. But win32ver.rc already gets other version number
information from the preprocessor at build time, so it would make more sense
if all version number information would be handled in the same way and we
don't have two places that do it. What we need for this is having the major
version number and the minor version number as separate integer symbols. Both
configure and Solution.pm already have that logic, because they compute
PG_VERSION_NUM. So we just keep all the logic there now. Put the minor
version number into a new symbol PG_MINORVERSION_NUM. Also, add a symbol
PG_MAJORVERSION_NUM, which is a number, alongside the existing
PG_MAJORVERSION, which is a string. Reviewed-by: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
Discussion:
https://www.postgresql.org/message-id/flat/1ee46ac4-a9b2-4531-bf54-5ec2e374634d(at)2ndquadrant(dot)com
https://git.postgresql.org/pg/commitdiff/0a42a2e9ce8481a024d085f2cc526a366db8df59

- Remove HAVE_WORKING_LINK. Previously, hard links were not used on Windows and
Cygwin, but they support them just fine in currently supported OS versions, so
we can use them there as well. Since all supported platforms now support hard
links, we can remove the alternative code paths. Rename
durable_link_or_rename() to durable_rename_excl() to make the purpose more
clear without referencing the implementation details. Discussion:
https://www.postgresql.org/message-id/flat/72fff73f-dc9c-4ef4-83e8-d2e60c98df48%402ndquadrant.com
https://git.postgresql.org/pg/commitdiff/aaa3aeddee51dd0058d38469907865052706a590

- Clean up order in miscinit.c a bit. The code around InitPostmasterChild() from
commit 31c453165b5 somehow ended up in the middle of a block of code related
to "User ID state". Move it into its own block instead.
https://git.postgresql.org/pg/commitdiff/1c918381814ca6ff976aab2bf130b0c582ca30e4

- Refactor ps_status.c API. The init_ps_display() arguments were mostly lies by
now, so to match typical usage, just use one argument and let the caller
assemble it from multiple sources if necessary. The only user of the
additional arguments is BackendInitialize(), which was already doing string
assembly on the caller side anyway. Remove the second argument of
set_ps_display() ("force") and just handle that in init_ps_display()
internally. BackendInitialize() also used to set the initial status as
"authentication", but that was very far from where authentication actually
happened. So now it's set to "initializing" and then "authentication" just
before the actual call to ClientAuthentication(). Reviewed-by: Julien Rouhaud
<rjuju123(at)gmail(dot)com> Reviewed-by: Kuntal Ghosh <kuntalghosh(dot)2007(at)gmail(dot)com>
Reviewed-by: Alvaro Herrera <alvherre(at)2ndquadrant(dot)com> Discussion:
https://www.postgresql.org/message-id/flat/c65e5196-4f04-4ead-9353-6088c19615a3(at)2ndquadrant(dot)com
https://git.postgresql.org/pg/commitdiff/bf68b79e50e3359accc85c94fa23cc03abb9350a

- Preserve replica identity index across ALTER TABLE rewrite. If an index was
explicitly set as replica identity index, this setting was lost when a table
was rewritten by ALTER TABLE. Because this setting is part of pg_index but
actually controlled by ALTER TABLE (not part of CREATE INDEX, say), we have to
do some extra work to restore it. Based-on-patch-by: Quan Zongliang
<quanzongliang(at)gmail(dot)com> Reviewed-by: Euler Taveira
<euler(dot)taveira(at)2ndquadrant(dot)com> Discussion:
https://www.postgresql.org/message-id/flat/c70fcab2-4866-0d9f-1d01-e75e189db342(at)gmail(dot)com
https://git.postgresql.org/pg/commitdiff/1cc9c2412cc9a2fbe6a381170097d315fd40ccca

- Unify several ways to tracking backend type. Add a new global variable
MyBackendType that uses the same BackendType enum that was previously only
used by the stats collector. That way several duplicate ways of checking what
type a particular process is can be simplified. Since it's no longer just for
stats, move to miscinit.c and rename existing functions to match the expanded
purpose. Reviewed-by: Julien Rouhaud <rjuju123(at)gmail(dot)com> Reviewed-by: Kuntal
Ghosh <kuntalghosh(dot)2007(at)gmail(dot)com> Reviewed-by: Alvaro Herrera
<alvherre(at)2ndquadrant(dot)com> Discussion:
https://www.postgresql.org/message-id/flat/c65e5196-4f04-4ead-9353-6088c19615a3(at)2ndquadrant(dot)com
https://git.postgresql.org/pg/commitdiff/8e8a0becb335529c66a9f82f88e1419e49b458ae

- Remove am_syslogger global variable. Use the new MyBackendType instead. More
similar changes for other "am something" variables are possible. This one was
just particularly simple. Reviewed-by: Julien Rouhaud <rjuju123(at)gmail(dot)com>
Reviewed-by: Kuntal Ghosh <kuntalghosh(dot)2007(at)gmail(dot)com> Reviewed-by: Alvaro
Herrera <alvherre(at)2ndquadrant(dot)com> Discussion:
https://www.postgresql.org/message-id/flat/c65e5196-4f04-4ead-9353-6088c19615a3(at)2ndquadrant(dot)com
https://git.postgresql.org/pg/commitdiff/d90bd24391fdde2b73906e16052821c9e3c1ce82

- doc: Remove unused title ids. FOP issues warnings about them. These aren't
even used, so just remove them. For the ones that are actually used, we'll
come up with a different solution. Discussion:
https://www.postgresql.org/message-id/flat/e29b580e-79ab-a371-5ea4-6946e4d3af0b%402ndQuadrant.com
https://git.postgresql.org/pg/commitdiff/340de72780e4eb769d5cf052b03084808bac476a

- Add backend type to csvlog and optionally log_line_prefix. The backend type,
which corresponds to what pg_stat_activity.backend_type shows, is added as a
column to the csvlog and can optionally be added to log_line_prefix using the
new %b placeholder. Reviewed-by: Julien Rouhaud <rjuju123(at)gmail(dot)com>
Reviewed-by: Kuntal Ghosh <kuntalghosh(dot)2007(at)gmail(dot)com> Reviewed-by: Alvaro
Herrera <alvherre(at)2ndquadrant(dot)com> Reviewed-by: Justin Pryzby
<pryzby(at)telsasoft(dot)com> Discussion:
https://www.postgresql.org/message-id/flat/c65e5196-4f04-4ead-9353-6088c19615a3(at)2ndquadrant(dot)com
https://git.postgresql.org/pg/commitdiff/70a7b4776be4e776221e578d8ca53b2b9f8118c0

Jeff Davis pushed:

- Introduce LogicalTapeSetExtend(). Increases the number of tapes in a logical
tape set. This will be important for disk-based hash aggregation, because the
maximum number of tapes is not known ahead of time. While discussing this
change, it was observed to regress the performance of Sort for at least one
test case. The performance regression was because some versions of GCC switch
to an inlined version of memcpy() in LogicalTapeWrite() after this change. No
performance regression for clang was observed. Because the regression is due
to an arbitrary decision by the compiler, I decided it shouldn't hold up this
change. If it needs to be fixed, we can find a workaround. Author: Adam Lee,
Jeff Davis Discussion:
https://postgr.es/m/e54bfec11c59689890f277722aaaabd05f78e22c.camel%40j-davis.com
https://git.postgresql.org/pg/commitdiff/24d85952a57b16090ca8ad9cf800fbdd9ddd104f

Álvaro Herrera pushed:

- Split out CreateCast into src/backend/catalog/pg_cast.c. This catalog-handling
code was previously together with the rest of CastCreate() in
src/backend/commands/functioncmds.c. A future patch will need a way to add
casts internally, so this will be useful to have separate. Also, move the
nearby get_cast_oid() function from functioncmds.c to lsyscache.c, which seems
a more natural place for it. Author: Paul Jungwirth, minor edits by Álvaro
Discussion: https://postgr.es/m/20200309210003.GA19992@alvherre.pgsql
https://git.postgresql.org/pg/commitdiff/40b3e2c201af10c27a2c4c5bfcd029026b4cdff3

- Avoid duplicates in ALTER ... DEPENDS ON EXTENSION. If the command is
attempted for an extension that the object already depends on, silently do
nothing. In particular, this means that if a database containing multiple
such entries is dumped, the restore will silently do the right thing and
record just the first one. (At least, in a world where pg_dump does dump such
entries -- which it doesn't currently, but it will.) Backpatch to 9.6, where
this kind of dependency was introduced. Reviewed-by: Ibrar Ahmed, Tom Lane
(offlist) Discussion:
https://postgr.es/m/20200217225333.GA30974@alvherre.pgsql
https://git.postgresql.org/pg/commitdiff/899a04f5ed61c3db7a2bd84957ecf530c09fd05a

- Add pg_dump support for ALTER obj DEPENDS ON EXTENSION. pg_dump is oblivious
to this kind of dependency, so they're lost on dump/restores (and pg_upgrade).
Have pg_dump emit ALTER lines so that they're preserved. Add some pg_dump
tests for the whole thing, also. Reviewed-by: Tom Lane (offlist) Reviewed-by:
Ibrar Ahmed Reviewed-by: Ahsan Hadi (who also reviewed commit 899a04f5ed61)
Discussion: https://postgr.es/m/20200217225333.GA30974@alvherre.pgsql
https://git.postgresql.org/pg/commitdiff/b08dee24a5574ba6657d238b91c10f1f8e2608f8

Peter Geoghegan pushed:

- nbtree: Demote minus infinity "can't happen" error. Only a very basic logic
bug in a _bt_insertonpg() caller could lead to a violation of this invariant.
Besides, any newitemoff used for an internal page is sanitized using other
"can't happen" errors in _bt_getstackbuf() or its callers, before
_bt_insertonpg() even gets called. Also, move the error/assertion from the
insert-without-split path of _bt_insertonpg() to the top of the same function.
There is no reason why this invariant only applies to insertions that happen
to not result in a page split; cover every insertion. The assertion naturally
belongs next to the existing generic assertions that document relatively
high-level invariants for the item being inserted.
https://git.postgresql.org/pg/commitdiff/d1e241c22602a5369b54e29b7430553445f6a52b

- nbtree: Move fastpath NULL descent stack assertion. Commit 074251db added an
assertion that verified the fastpath/rightmost page insert optimization's
assumption about free space: There should always be enough free space on the
page to insert the new item without splitting the page. Otherwise, we end up
using the "concurrent root page split" phony/fake stack path in
_bt_insert_parent(). This does not lead to incorrect behavior, but it is
likely to be far slower than simply using the regular _bt_search() path. The
assertion catches serious performance bugs that would probably take a long
time to detect any other way. It seems much more natural to make this
assertion just before the point that we generate a fake/phony descent stack.
Move the assert there. This also makes _bt_insertonpg() a bit more readable.
https://git.postgresql.org/pg/commitdiff/39eabec90451d8badbba9b3e938d6d05432a0517

- Remove stray parenthesis in nbtree.h. Oversight in commit 0d861bbb702.
https://git.postgresql.org/pg/commitdiff/a88a285c7e343502b55204a211818d12b1b7d6ff

Alexander Korotkov pushed:

- Improve checking of child pages in contrib/amcheck. This commit eliminates
lossiness in check for missing parent downlinks in B-tree. Instead of
collecting lossy bitmap, we check for missing downlinks while visiting child
pages referenced by downlinks of target level. We traverse from previous
child page to the subsequent child page by right links. Intermediate pages are
candidates to have lost parent downlinks. Also this commit introduces
matching of child high key to the pivot key of it's parent. Discussion:
https://postgr.es/m/CAPpHfduoF-c4RhOyOm%3D4-Y367%2B8txq9Q6iM_ty0OYc8si1Abww%40mail.gmail.com
Author: Alexander Korotkov Reviewed-by: Peter Geoghegan
https://git.postgresql.org/pg/commitdiff/d114cc538715e14d29d6de8b6ea1a1d5d3e0edb4

Tomáš Vondra pushed:

- Use functional dependencies to estimate ScalarArrayOpExpr. Until now
functional dependencies supported only simple equality clauses and clauses
that can be trivially translated to equalities. This commit allows estimation
of some ScalarArrayOpExpr (IN/ANY) clauses. For IN clauses we can do this
thanks to using operator with equality semantics, which means an IN clause
WHERE c IN (1, 2, ..., N) can be translated to WHERE (c = 1 OR c = 2 OR
... OR c = N) IN clauses are now considered compatible with functional
dependencies, and rely on the same assumption of consistency of queries with
data (which is an assumption we already used for simple equality clauses).
This applies also to ALL clauses with an equality operator, which can be
considered equivalent to IN clause. ALL clauses are still considered
incompatible, although there's some discussion about maybe relaxing this in
the future. Author: Pierre Ducroquet Reviewed-by: Tomas Vondra, Dean Rasheed
Discussion:
https://www.postgresql.org/message-id/flat/13902317.Eha0YfKkKy%40pierred-pdoc
https://git.postgresql.org/pg/commitdiff/8f321bd16cdb11307f17007eb3c55b65d7d323ef

- Use multi-variate MCV lists to estimate ScalarArrayOpExpr. Commit 8f321bd16c
added support for estimating ScalarArrayOpExpr clauses (IN/ANY) clauses using
functional dependencies. There's no good reason not to support estimation of
these clauses using multi-variate MCV lists too, so this commits implements
that. That makes the behavior consistent and MCV lists can estimate all
variants (ANY/ALL, inequalities, ...). Author: Tomas Vondra Review: Dean
Rasheed Discussion:
https://www.postgresql.org/message-id/flat/13902317.Eha0YfKkKy%40pierred-pdoc
https://git.postgresql.org/pg/commitdiff/e83daa7e331e0718bb254ab685f2ed95df423554

- Improve test coverage for functional dependencies. The regression tests for
functional dependencies were only using clauses of the form (Var op Const),
i.e. with Var on the left side. This adds a couple of queries with Var on the
right, to test other code paths. It also prints one of the functional
dependencies, to test the data type output function. The functional
dependencies are "perfect" with degree of 1.0 so this should be stable.
Discussion:
https://www.postgresql.org/message-id/flat/20200113230008(dot)g67iyk4cs3xbnjju(at)development
Author: Tomas Vondra
https://git.postgresql.org/pg/commitdiff/f9696782c70168e575c0f2d17a1cfe71847fef3d

- Improve test coverage for multi-column MCV lists. The regression tests for
extended statistics were not testing a couple of important cases for the MCV
lists: * IS NOT NULL clauses - We did have queries with IS NULL clauses,
but not the negative case. * clauses with variable on the right - All
the clauses had the Var on the left, i.e. (Var op Const), so this adds
(Const op Var) too. * columns with fixed-length types passed by reference -
All columns were using either by-value or varlena types, so add a test
with UUID columns too. This matters for (de)serialization. * NULL-only
dimension - When one of the columns contains only NULL values, we treat it
a a special case during (de)serialization. * arrays containing NULL - When
the constant parameter contains NULL value, we need to handle it correctly
_during estimation, for all IN, ANY and ALL clauses. Discussion:
https://www.postgresql.org/message-id/flat/20200113230008(dot)g67iyk4cs3xbnjju(at)development
Author: Tomas Vondra
https://git.postgresql.org/pg/commitdiff/d8cfa82d51f812f886d06bdc7bb73470e55d74b7

== Pending Patches ==

David Rowley sent in another revision of a patch to implement run-time pruning
for ModifyTable.

James Coleman sent in another revision of a patch to improve the pg_rewind
explanation and warnings.

Rémi Lapeyre sent in another revision of a patch to allow COPY "text" to output
a header, and add header matching mode to COPY FROM so round trips work.

Tom Lane sent in another revision of a patch to support external parameters in
PL/pgsql's EXECUTE command.

Kyoaro HORIGUCHI sent in another revision of a patch to tidy up XLogSource
usage.

Alexander Korotkov sent in two more revisions of a patch to improve checking of
child pages in contrib/amcheck.

Fujii Masao and Masahiko Sawada traded patches to fix double updating of ps
title when recovery conflict is occurring.

Kyotaro HORIGUCHI sent in a PoC patch to Make pg_event_trigger_ddl_command work
for DROP.

Justin Pryzby sent in another revision of a patch to allow CLUSTER, VACUUM FULL
and REINDEX to change tablespace on the fly.

Michaël Paquier sent in five more revisions of a patch to move routine
generating restore_command to src/common/, and add -c/--restore-target-wal to
pg_rewind.

Surafel Temesgen sent in another revision of a patch to add conflict handling to
COPY FROM.

Zeng Wenjing sent in three more revisions of a patch to implement global
temporary tables.

Tom Lane sent in another revision of a patch to implement a shared-memory-based
stats collector.

Julien Rouhaud sent in three more revisions of a patch to pass the query string
to the planner, and add planning counters to pg_stat_statements.

Tomáš Vondra and Dean Rasheed traded patches to improve estimation of OR clauses
with extended statistics, and support clauses of the form Var op Var.

Dilip Kumar and Amit Kapila traded patches to assert that we don't acquire a
heavyweight lock on another object after relation extension lock, add an assert
to ensure that page locks don't participate in a deadlock cycle, allow relation
extension lock to conflict among parallel group members, and allow page lock to
conflict among parallel group members.

Thomas Munro sent in a patch to simplify the effective_io_concurrency setting,
and introduce a maintenance_io_concurrency setting.

James Coleman sent in a patch to improve the standby connection denied error
message.

Kyotaro HORIGUCHI sent in another revision of a patch to add TAP test for a
crash bug.

Surafel Temesgen sent in another revision of a patch to implement
system-versioned temporal tables.

Fujii Masao sent in three revisions of a patch to add a --no-estimate-size
option to pg_basebackup.

Corey Huinker and Jürgen Purtz traded patches to add a glossary.

Euler Taveira de Oliveira sent in another revision of a patch to preserve
replica identity index across ALTER TABLE rewrite.

Hugh McMaster and Tom Lane traded patches to use PKG_CHECK_MODULES to detect the
libxml2 library.

Tom Lane sent in another revision of a patch to control the maximum length of
parameter values logged.

Laurenz Albe sent in another revision of a patch to autovacuum tables that have
received only inserts.

Justin Pryzby sent in two more revisions of a patch to make pg_ls_tmpdir and
friends show directories and shared filesets.

Julien Rouhaud sent in another revision of a patch to implement online checksum
verification in the backend.

Peter Eisentraut sent in another revision of a patch to improve handling of
parameter differences in physical replication.

Thomas Munro and Julien Rouhaud traded patches to implement collation
versioning.

Justin Pryzby sent in a patch to pg_ls_dir_files to avoid leaking DIR if not run
to completion.

Justin Pryzby sent in a patch to SRF to avoid leaking resources if not run to
completion.

Thomas Munro sent in a patch to fix nextXid tracking bug on standbys (9.5-11
only).

Michaël Paquier sent in three more revisions of a patch to refactor compile-time
assertion checks for C/C++.

Robert Haas sent in two more revisions of a patch to implement backup manifests.

Craig Ringer sent in a patch to skip llvm bytecode generation if LLVM is
missing.

Tomáš Vondra, Álvaro Herrera, and James Coleman traded patches to implement
incremental sort.

Paul A Jungwirth sent in another revision of a patch to extend primary and
foreign key updates and deletes to the temporal versions of primary and foreign
keys.

Michael Banck sent in a patch to fix checksum verification in base backups for
random or zero page headers.

Vigneshwaran C sent in a patch to add instrumenting code to COPY.

Nikita Glukhov sent in another revision of a patch to implement SQL/JSON
functions.

Jeff Davis sent in another revision of a patch to implement memory-bounded hash
aggregation.

Movead Li sent in two revisions of a patch to fix a bug when using the get_bit()
function for a long bytea string.

Tom Lane sent in another revision of a patch to pipeline functionscans.

Pengzhou Tang sent in a patch to set numtrans correctly when building hash
aggregate tables.

David Rowley and Andy Fan traded patches to Bypass distinctClause &
groupbyClause if the exprs is unique already.

Kyotaro HORIGUCHI sent in another revision of a patch to reduce WaitEventSet
syscall churn.

Justin Pryzby sent in another revision of a patch to make more clear the
computation of min/max IO, and use the correlation statistic in costing bitmap
scans.

Asif Rehman sent in another revision of a patch to implement parallel backup.

Álvaro Herrera sent in another revision of a patch to enable BEFORE row-level
triggers for partitioned tables.

Pavel Stěhule sent in another revision of a patch to implement schema variables.

Paul A Jungwirth and Álvaro Herrera traded patches to implement multi-ranges and
range aggregation.

John Naylor sent in another revision of a patch to make it possible to truncate
timestamps on arbitrary intervals.

Tom Lane sent in two more revisions of a patch to add new polymorphic types:
commontype and commontypearray.

Pengzhou Tang sent in another revision of a patch to implement parallel grouping
sets.

Julien Rouhaud sent in another revision of a patch to expose queryid in
pg_stat_activity and log_line_prefix.

Tomáš Vondra sent in another revision of a patch to add support for IN and @> in
functional-dependency statistics use.

Justin Pryzby sent in another revision of a patch to add backend type in
log_line_prefix.

Kirill Bychik sent in another revision of a patch to add stats for WAL usage.