== PostgreSQL Weekly News - November 17, 2019 ==

== PostgreSQL Weekly News - November 17, 2019 ==

PostgreSQL bug fix releases 12.1, 11.6, 10.11, 9.6.16, 9.5.20, and 9.4.25 are out.
Upgrade as soon as possible. The 9.4 series will get its last update on
February 13, 2020.
https://www.postgresql.org/about/news/1994/

pgDay Paris 2020 will be held in Paris, France on March 26, 2020 at Espace
Saint-Martin. The CfP is open through December 31, 2019 at midnight, Paris time
at https://2020.pgday.paris/callforpapers/
http://2020.pgday.paris/

== PostgreSQL Product News ==

PGroonga 2.2.2 a full text search platform for all languages, released.
https://pgroonga.github.io/

check_pgbackrest 1.6, a Nagios-compatible monitor for pgBackRest, released.
https://github.com/dalibo/check_pgbackrest/releases

pgAdmin4 4.15, a web- and native GUI control center for PostgreSQL, released.
https://www.pgadmin.org/docs/pgadmin4/dev/release_notes_4_15.html

postgres-checkup 1.3.0 "Double Eagle", a tool that automates detailed
health checks of PostgreSQL clusters, released.
https://gitlab.com/postgres-ai-team/postgres-checkup/-/tags/1.3.0

== PostgreSQL Jobs for November ==

http://archives.postgresql.org/pgsql-jobs/2019-11/

== PostgreSQL Local ==

PGDay Down Under 2019 will be held on November 15, 2019 in Sydney,
Australia.
https://pgdu.org/

2Q PGConf 2019 will be held December 4 & 5 in Chicago.
https://www.2qpgconf.com/

PGDay SF will take place on January 21, 2020 at the Swedish American Hall in San
Francisco. The CfP is open through November 22, 2019 at
https://2020.pgdaysf.org/callforpapers/

The CfP for FOSDEM PGDay is open through November 18th, 2019 at 24:00 CET.
https://2020.fosdempgday.org/

pgDay Israel 2020 will take place on March 19, 2020 in Tel Aviv.
The CfP is open through January 15, 2020.
http://pgday.org.il/

pgDay Paris 2020 will be held in Paris, France on March 26, 2020
at Espace Saint-Martin.
http://2020.pgday.paris/

Nordic PGDay 2020 will be held in Helsinki, Finland at the Hilton Helsinki
Strand Hotel on March 24, 2020. The CfP is open through December 31, 2019 at
https://2020.nordicpgday.org/cfp/

PGConf India 2020 will be on February 26-28, 2020 in Bengaluru, Karnataka.
http://pgconf.in/

PostgreSQL(at)SCaLE is a two day, two track event which takes place on
March 5-6, 2020, at Pasadena Convention Center, as part of SCaLE 18X. The CfP is
open through November 30, 2019.
https://www.socallinuxexpo.org/scale/18x/postgresscale

The German-speaking PostgreSQL Conference 2020 will take place on May 15, 2019
in Stuttgart.

== PostgreSQL in the News ==

Planet PostgreSQL: http://planet.postgresql.org/

PostgreSQL Weekly News is brought to you this week by David Fetter

Submit news and announcements by Sunday at 3:00pm PST8PDT to david(at)fetter(dot)org(dot)

== Applied Patches ==

Amit Kapila pushed:

- Rearrange dropdb() to avoid errors after allowing other sessions to exit.
During Drop Database, it is better to error out before allowing other sessions
to exit and forcefully terminating autovacuum workers. All the other errors
except for checking subscriptions are already done before. Author: Amit
Kapila Discussion:
https://postgr.es/m/CAA4eK1+qhLkCYG2oy9xug9ur_j=G2wQNRYAyd+-kZfZ1z42pLw@mail.gmail.com
https://git.postgresql.org/pg/commitdiff/9fab25c6cd1f943284b8e0014007cd5750d54308

- Make the order of the header file includes consistent in backend modules.
Similar to commits 7e735035f2 and dddf4cdc33, this commit makes the order of
header file inclusion consistent for backend modules. In the passing, removed
a couple of duplicate inclusions. Author: Vignesh C Reviewed-by: Kuntal Ghosh
and Amit Kapila Discussion:
https://postgr.es/m/CALDaNm2Sznv8RR6Ex-iJO6xAdsxgWhCoETkaYX=+9DW3q0QCfA@mail.gmail.com
https://git.postgresql.org/pg/commitdiff/14aec03502302eff6c67981d8fd121175c436ce9

- Introduce the 'force' option for the Drop Database command. This new option
terminates the other sessions connected to the target database and then drop
it. To terminate other sessions, the current user must have desired
permissions (same as pg_terminate_backend()). We don't allow to terminate the
sessions if prepared transactions, active logical replication slots or
subscriptions are present in the target database. Author: Pavel Stehule with
changes by me Reviewed-by: Dilip Kumar, Vignesh C, Ibrar Ahmed, Anthony
Nowocien, Ryan Lambert and Amit Kapila Discussion:
https://postgr.es/m/CAP_rwwmLJJbn70vLOZFpxGw3XD7nLB_7+NKz46H5EOO2k5H7OQ@mail.gmail.com
https://git.postgresql.org/pg/commitdiff/1379fd537f9fc7941c8acff8c879ce3636dbdb77

Thomas Munro pushed:

- Optimize TransactionIdIsCurrentTransactionId(). If the passed in xid is the
current top transaction, we can do a fast check and exit early. This should
work well for the current heap but also works very well for proposed AMs that
don't use a separate xid for subtransactions. Author: Ashwin Agrawal, based
on a suggestion from Andres Freund Reviewed-by: Thomas Munro Discussion:
https://postgr.es/m/CALfoeiv0k3hkEb3Oqk%3DziWqtyk2Jys1UOK5hwRBNeANT_yX%2Bng%40mail.gmail.com
https://git.postgresql.org/pg/commitdiff/695c5977c8bc115029a85dcc1821d7b0136b4e4c

- Optimize PredicateLockTuple(). PredicateLockTuple() has a fast exit if tuple
was written by the current transaction, as in that case it already has a lock.
This check can be performed using TransactionIdIsCurrentTransactionId()
instead of SubTransGetTopmostTransaction(), to avoid any chance of having to
hit the disk. Author: Ashwin Agrawal, based on a suggestion from Andres
Freund Reviewed-by: Thomas Munro Discussion:
https://postgr.es/m/CALfoeiv0k3hkEb3Oqk%3DziWqtyk2Jys1UOK5hwRBNeANT_yX%2Bng%40mail.gmail.com
https://git.postgresql.org/pg/commitdiff/db2687d1f3787aa8113b3dbb358153feee30c64c

- Always call ExecShutdownNode() if appropriate. Call ExecShutdownNode() after
ExecutePlan()'s loop, rather than at each break. We had forgotten to do that
in one case. The omission caused intermittent "temporary file leak" warnings
from multi-batch parallel hash joins with a LIMIT clause. Back-patch to 11.
Though the problem exists in theory in earlier parallel query releases,
nothing really depended on it. Author: Kyotaro Horiguchi Reviewed-by: Thomas
Munro, Amit Kapila Discussion:
https://postgr.es/m/20191111.212418.2222262873417235945.horikyota.ntt%40gmail.com
https://git.postgresql.org/pg/commitdiff/76cbfcdf3a0dff3f029ca079701418b861ce86c8

Peter Eisentraut pushed:

- Rerun autoheader. This puts pg_config.h.in content back into the "correct"
order.
https://git.postgresql.org/pg/commitdiff/bbaa8232729f2aa1b1061503a3d9f7db503d2788

- Fix whitespace.
https://git.postgresql.org/pg/commitdiff/d0c92527cc77a8f6b235c2a5abba95e01367f825

- gitattributes: Remove entries for no longer existing files.
https://git.postgresql.org/pg/commitdiff/2cd75e4e795bea110c1d32e8cbfdf1e33a109e80

- gitattributes: Add new file.
https://git.postgresql.org/pg/commitdiff/de7c2d30b6dc2544b181cf074e942fff54d7999d

Tom Lane pushed:

- Further improve stability of partition_prune regression test. Commits
4ea03f3f4 et al arranged to filter out row counts in parallel plans, because
those are dependent on the number of workers actually obtained. Somehow I
missed that the 'Rows Removed by Filter' counts can also vary, so fix that
too. Per buildfarm. This seems worth a last-minute patch because unreliable
regression tests are a serious pain in the rear for packagers. Like the
previous patch, back-patch to v11 where this test was introduced.
https://git.postgresql.org/pg/commitdiff/13e8b2ee896d76dfcd02dddee40919fd6f2cd937

- Doc: fix ancient mistake, or at least obsolete info, in rules example. The
example of expansion of multiple views claimed that the resulting subquery
nest would not get fully flattened because of an aggregate function. There's
no aggregate in the example, though, only a user defined function confusingly
named MIN(). In a modern server, the reason for the non-flattening is that
MIN() is volatile, but I'm unsure whether that was true back when this text
was written. Let's reduce the confusion level by using LEAST() instead (which
we didn't have at the time this example was created). And then we can just
say that the planner will flatten the sub-queries, so the rewrite system
doesn't have to. Noted by Paul Jungwirth. This text is old enough to vote,
so back-patch to all supported branches. Discussion:
https://postgr.es/m/CA+renyXZFnmp9PcvX1EVR2dR=XG5e6E-AELr8AHCNZ8RYrpnPw@mail.gmail.com
https://git.postgresql.org/pg/commitdiff/b6423e92abfadaa1ed9642319872aa1654403cd6

- Fix ecpglib.h to declare bool consistently with c.h. This completes the task
begun in commit 1408d5d86, to synchronize ECPG's exported definitions with the
definition of bool used by c.h (and, therefore, the one actually in use in the
ECPG library). On practically all modern platforms, ecpglib.h will now just
include <stdbool.h>, which should surprise nobody anymore. That removes a
header-inclusion-order hazard for ECPG clients, who previously might get build
failures or unexpected behavior depending on whether they'd included
<stdbool.h> themselves, and if so, whether before or after ecpglib.h. On
platforms where sizeof(_Bool) is not 1 (only old PPC-based Mac systems, as far
as I know), things are still messy, as inclusion of <stdbool.h> could still
break ECPG client code. There doesn't seem to be any clean fix for that, and
given the probably-negligible population of users who would care anymore, it's
not clear we should go far out of our way to cope with it. This change at
least fixes some header-inclusion-order hazards for our own code, since c.h
and ecpglib.h previously disagreed on whether bool should be char or unsigned
char. To implement this with minimal invasion of ECPG client namespace, move
the choice of whether to rely on <stdbool.h> into configure, and have it
export a configuration symbol PG_USE_STDBOOL. ecpglib.h no longer exports
definitions for TRUE and FALSE, only their lowercase brethren. We could undo
that if we get push-back about it. Ideally we'd back-patch this as far as
v11, which is where c.h started to rely on <stdbool.h>. But the odds of
creating problems for formerly-working ECPG client code seem about as large as
the odds of fixing any non-working cases, so we'll just do this in HEAD.
Discussion:
https://postgr.es/m/CAA4eK1LmaKO7Du9M9Lo=kxGU8sB6aL8fa3sF6z6d5yYYVe3BuQ@mail.gmail.com
https://git.postgresql.org/pg/commitdiff/7a0574b50ee9c2b96ce94c29e031c103285c0b1d

- Finish reverting commit 0a52d378b. Apply the solution adopted in commit
dcb7d3caf (ie, explicitly don't call memcmp for a zero-length comparison) to
func_get_detail() as well, removing one other place where we were passing an
uninitialized array to a parse_func.c entry point. Discussion:
https://postgr.es/m/MN2PR18MB2927F24692485D754794F01BE3740@MN2PR18MB2927.namprd18.prod.outlook.com
Discussion:
https://postgr.es/m/MN2PR18MB2927F6873DF2774A505AC298E3740@MN2PR18MB2927.namprd18.prod.outlook.com
https://git.postgresql.org/pg/commitdiff/112caf9039f4c8fb286bb610461ced8253313e9f

- Handle arrays and ranges in pg_upgrade's test for non-upgradable types.
pg_upgrade needs to check whether certain non-upgradable data types appear
anywhere on-disk in the source cluster. It knew that it has to check for
these types being contained inside domains and composite types; but it somehow
overlooked that they could be contained in arrays and ranges, too. Extend the
existing recursive-containment query to handle those cases. We probably
should have noticed this oversight while working on commit 0ccfc2822 and
follow-ups, but we failed to :-(. The whole thing's possibly a bit
overdesigned, since we don't really expect that any of these types will appear
on disk; but if we're going to the effort of doing a recursive search then
it's silly not to cover all the possibilities. While at it, refactor so that
we have only one copy of the search logic, not three-and-counting. Also, to
keep the branches looking more alike, back-patch the output wording change of
commit 1634d3615. Back-patch to all supported branches. Discussion:
https://postgr.es/m/31473.1573412838@sss.pgh.pa.us
https://git.postgresql.org/pg/commitdiff/29aeda6e4e609b66b6f1c3c73b4def6fb34d90ed

- Include TableFunc references when computing expression dependencies. The
TableFunc node (i.e., XMLTABLE) includes type and collation OIDs that might
not be referenced anywhere else in the expression tree, so they need to be
accounted for when extracting dependencies. Fortunately, the practical
effects of this are limited, since (a) it's somewhat unlikely that people
would be extracting columns of non-builtin types from an XML document, and (b)
in many scenarios, the query would contain other references to such types, or
functions depending on them. However, it's not hard to construct examples
wherein the existing code lets one drop a type used in XMLTABLE and thereby
break a view. This is evidently an original oversight in the XMLTABLE patch,
so back-patch to v10 where that came in. Discussion:
https://postgr.es/m/18427.1573508501@sss.pgh.pa.us
https://git.postgresql.org/pg/commitdiff/2c7b5dad6eb1602839e5961b4d4eb6494fe4a1ee

- Avoid downcasing/truncation of RADIUS authentication parameters. Commit
6b76f1bb5 changed all the RADIUS auth parameters to be lists rather than
single values. But its use of SplitIdentifierString to parse the list format
was not very carefully thought through, because that function thinks it's
parsing SQL identifiers, which means it will (a) downcase the strings and (b)
truncate them to be shorter than NAMEDATALEN. While downcasing should be
harmless for the server names and ports, it's just wrong for the shared
secrets, and probably for the NAS Identifier strings as well. The truncation
aspect is at least potentially a problem too, though typical values for these
parameters would fit in 63 bytes. Fortunately, we now have a function
SplitGUCList that is exactly the same except for not doing the two unwanted
things, so fixing this is a trivial matter of calling that function instead.
While here, improve the documentation to show how to double-quote the
parameter values. I failed to resist the temptation to do some copy-editing
as well. Report and patch from Marcos David (bug #16106); doc changes by me.
Back-patch to v10 where the aforesaid commit came in, since this is arguably a
regression from our previous behavior with RADIUS auth. Discussion:
https://postgr.es/m/16106-7d319e4295d08e70@postgresql.org
https://git.postgresql.org/pg/commitdiff/7618eaf5f315f53619b718e571cd2a2020fb0226

- Avoid using SplitIdentifierString to parse ListenAddresses, too. This gets rid
of our former behavior of forcibly downcasing the postmaster's hostname list
and truncating the elements to NAMEDATALEN. In principle, DNS hostnames are
case-insensitive so the first behavior should be harmless, and server
hostnames are seldom long enough for the second behavior to be an issue. But
it's still dubious, and an easy fix is available: just use SplitGUCList
instead. AFAICT, all other SplitIdentifierString calls in the backend are OK:
either the items actually are SQL identifiers, or they are keywords that are
short and case-insensitive. Per thinking about bug #16106. While this has
been wrong for a very long time, the lack of field complaints means there's
little reason to back-patch. Discussion:
https://postgr.es/m/16106-7d319e4295d08e70@postgresql.org
https://git.postgresql.org/pg/commitdiff/7bf40ea0d028e6dc49c152e5820a65d69b74b409

- Fix silly initializations (cosmetic only). Initializing a pointer to "false"
isn't per project style, and reportedly some compilers warn about it (though
I've not seen any such warnings in the buildfarm). Seems to have come in with
commit ff11e7f4b, so back-patch to v12 where that was added. Didier Gautheron
Discussion:
https://postgr.es/m/CAJRYxu+XQuM0qnSqt1Ujztu6fBPzMMAT3VEn6W32rgKG6A2Fsw@mail.gmail.com
https://git.postgresql.org/pg/commitdiff/0cafdd03a850265006c0ada1b0bf4f83e087a409

- Add missing check_collation_set call to bpcharne(). We should throw an error
for indeterminate collation, but bpcharne() was missing that logic, resulting
in a much less user-friendly error (either an assertion failure or "cache
lookup failed for collation 0"). Per report from Manuel Rigger. Back-patch
to v12 where the mistake came in, evidently in commit 5e1963fb7. (Before
non-deterministic collations, this function wasn't collation sensitive.)
Discussion:
https://postgr.es/m/CA+u7OA4HOjtymxAbuGNh4-X_2R0Lw5n01tzvP8E5-i-2gQXYWA@mail.gmail.com
https://git.postgresql.org/pg/commitdiff/d57d61533a2b5b27b60cc9024c54688390871bf6

- Further fix dumping of views that contain just VALUES(...). It turns out that
commit e9f1c01b7 missed a case: we must print a VALUES clause in long format
if get_query_def is given a resultDesc that would require the query's output
column name(s) to be different from what the bare VALUES clause would produce.
This applies in case an ALTER ... RENAME COLUMN has been done to a view that
formerly could be printed in simple format, as shown in the added regression
test case. It also explains bug #16119 from Dmitry Telpt, because it turns
out that (unlike CREATE VIEW) CREATE MATERIALIZED VIEW fails to apply any
column aliases it's given to the stored ON SELECT rule. So to get them to be
printed, we have to account for the resultDesc renaming. It might be worth
changing the matview code so that it creates the ON SELECT rule with the
correct aliases; but we'd still need these messy checks in
get_simple_values_rte to handle the case of a subsequent column rename, so any
such change would be just neatnik-ism not a bug fix. Like the previous patch,
back-patch to all supported branches. Discussion:
https://postgr.es/m/16119-e64823f30a45a754@postgresql.org
https://git.postgresql.org/pg/commitdiff/bf2efc55da9a1a33da32fa383db9db2f2c49b2cb

Peter Geoghegan pushed:

- Teach datum_image_eq() about cstring datums. Bring datum_image_eq() in line
with datumIsEqual() by adding support for comparing cstring datums. An
upcoming patch that adds deduplication to the nbtree AM will use
datum_image_eq(). datum_image_eq() will need to work with all datatypes that
can be used as the storage type of a B-Tree index column, including cstring.
(cstring is used as the storage type for columns of type "name" as a
space-saving optimization.) Discussion:
https://postgr.es/m/CAH2-Wzn3Ee49Gmxb7V1VJ3-AC8fWn-Fr8pfWQebHe8rYRxt5OQ@mail.gmail.com
https://git.postgresql.org/pg/commitdiff/8c951687f58ad604be13e6addfd56446afb36e13

- Make _bt_keep_natts_fast() use datum_image_eq(). An upcoming patch that adds
deduplication to the nbtree AM will rely on _bt_keep_natts_fast()
understanding that differences in TOAST input state can never affect its
answer. In particular, two opclass-equal datums (with opclasses deemed safe
for deduplication) should never be treated as unequal by _bt_keep_natts_fast()
due to TOAST input differences. This also seems like a good idea on general
principle. nbtsplitloc.c will now occasionally make better decisions about
where to split a leaf page. The behavior of _bt_keep_natts_fast() is now
somewhat closer to the behavior of _bt_keep_natts(). Discussion:
https://postgr.es/m/CAH2-Wzn3Ee49Gmxb7V1VJ3-AC8fWn-Fr8pfWQebHe8rYRxt5OQ@mail.gmail.com
https://git.postgresql.org/pg/commitdiff/1f55ebae27225111d0a5c4d89d7e77d917357ff4

- Add tuplesort test to serial_schedule. Oversight in commit 4a252996.
https://git.postgresql.org/pg/commitdiff/815bd578076672a2325d026a2c48b6075454be9b

Álvaro Herrera pushed:

- Have LookupFuncName accept NULL argtypes for 0 args. Prior to this change, it
requires to be passed a valid pointer just to be able to pass it to a
zero-byte memcmp, per 0a52d378b03b. Given the strange resulting code in
callsites, it seems better to test for the case specifically and remove the
requirement. Reported-by: Ranier Vilela Discussion:
https://postgr.es/m/MN2PR18MB2927F24692485D754794F01BE3740@MN2PR18MB2927.namprd18.prod.outlook.com
Discussion:
https://postgr.es/m/MN2PR18MB2927F6873DF2774A505AC298E3740@MN2PR18MB2927.namprd18.prod.outlook.com
https://git.postgresql.org/pg/commitdiff/dcb7d3cafa3197c5425c129ba0dc5eddd23c0532

- pg_stat_activity: document client_port being null. As suggested by Stephen
Frost. Discussion:
https://postgr.es/m/20191104160605.GC6962@tamriel.snowman.net
https://git.postgresql.org/pg/commitdiff/c5e8ea978d88ea4aa731516836e14d54c50cc957

- pg_stat_{ssl,gssapi}: Show only processes with connections. It is pointless to
show in those views auxiliary processes that don't open network connections.
A small incompatibility is that anybody joining pg_stat_activity and
pg_stat_ssl/pg_stat_gssapi will have to use a left join if they want to see
such auxiliary processes. Author: Euler Taveira Discussion:
https://postgr.es/m/20190904151535.GA29108@alvherre.pgsql
https://git.postgresql.org/pg/commitdiff/5c46e7d82e88859395c2688c37bb643ae7dbbfa1

- Remove the word "virgins" for documentation. Apparently, it's no longer
welcome. Therefore replace it with "pristine", and add some explanatory text
while at it. Reported by Brian Williams Discussion:
https://postgr.es/m/157313712259.14261.16141263269989647311@wrigleys.postgresql.org
https://git.postgresql.org/pg/commitdiff/6ae4d271879b62c4325df3ddf75bff3f2b911086

Michaël Paquier pushed:

- Add regression test for two-phase transaction in postgres_fdw. postgres_fdw
does not support two-phase transactions, so let's add a small negative test
case to check after it. Note that this is checked using an end-of-xact
callback to ensure a proper connection cleanup with the foreign server, which
is called before checking if a server is able to handle 2PC with
max_prepared_xacts, so this test does not need an alternate output file.
Author: Gilles Darold Discussion:
https://postgr.es/m/20191108090507.GC1768@paquier.xyz
https://git.postgresql.org/pg/commitdiff/94fec48516a77f219ab94890219d724b973e4674

- Split handling of reloptions for partitioned tables. Partitioned tables do not
have relation options yet, but, similarly to what's done for views which have
their own parsing table, it could make sense to introduce new parameters for
some of the existing default ones like fillfactor, autovacuum, etc. Splitting
things has the advantage to make the information stored in rd_options include
only the necessary information, reducing the amount of memory used for a
relcache entry with partitioned tables if new reloptions are introduced at
this level. Author: Nikolay Shaplov Reviewed-by: Amit Langote, Michael
Paquier Discussion: https://postgr.es/m/1627387.Qykg9O6zpu@x200m
https://git.postgresql.org/pg/commitdiff/1bbd608fdae7af314d8e2229e369a45a3da83cd8

- Cleanup code in reloptions.h regarding reloption handling. reloptions.h
includes since ba748f7 a set of macros to handle reloption types in a way
similar to how parseRelOptions() works. They have never been used in the core
code, and we have more simple methods now to parse and fill in rd_options for
a given relation depending on its relkind, so remove this interface to
simplify things. Per discussion between Amit Langote, Álvaro Herrera and me.
Discussion:
https://postgr.es/m/CA+HiwqE6zbNO92az6pp5GiTw4tr-9rfCE0t84whQSP+YwSKjMQ@mail.gmail.com
https://git.postgresql.org/pg/commitdiff/50d22de9325f41a32faeb0d1055f50b43d0507b3

- Improve stability of tests for VACUUM (SKIP_LOCKED). Concurrent autovacuums
running with the main regression test suite could cause the tests with VACUUM
(SKIP_LOCKED) to generate randomly WARNING messages. For these tests, set
client_min_messages to ERROR to get rid of those random failures, as disabling
autovacuum for the relations operated would not completely close the failure
window. For isolation tests, disable autovacuum for the relations vacuumed
with SKIP_LOCKED. The tests are designed so as LOCK commands are taken in a
first session before running a concurrent VACUUM (SKIP_LOCKED) in a second to
generate WARNING messages, but a concurrent autovacuum could cause the tests
to be slower. Reported-by: Tom Lane Author: Michael Paquier Reviewed-by:
Andres Freund, Tom Lane Discussion:
https://postgr.es/m/25294.1573077278@sss.pgh.pa.us Backpatch-through: 12
https://git.postgresql.org/pg/commitdiff/3db0598d908f22435ab8138d1c76b971f17a5633

Fujii Masao pushed:

- Make pg_waldump report more detail information about PREPARE TRANSACTION
record. This commit changes xact_desc() so that it reports the detail
information about PREPARE TRANSACTION record, like GID (global transaction
identifier), timestamp at prepare transaction, delete-on-abort/commit
relations, XID of subtransactions, and invalidation messages. These are
helpful when diagnosing 2PC-related troubles. Author: Fujii Masao
Reviewed-by: Michael Paquier, Andrey Lepikhov, Kyotaro Horiguchi, Julien
Rouhaud, Alvaro Herrera Discussion:
https://postgr.es/m/CAHGQGwEvhASad4JJnCv=0dW2TJypZgW_Vpb-oZik2a3utCqcrA@mail.gmail.com
https://git.postgresql.org/pg/commitdiff/7b8a899bdeb638f46e102d1714c079a0874e9fa0

Andres Freund pushed:

- Add tests for tuplesort.c. Previously significant parts of tuplesort.c were
untested. This commit, while not testing every path, significantly increases
coverage. In particular, this adds tests for abbreviated key logic,
forward/backward scans & scrolling and mark/restore. I tried to keep the
table sizes reasonable, and stress the on-disk paths by setting work_mem to
low values for specific tests. The buildfarm will tell whether more attention
to test time is needed. Author: Andres Freund Discussion:
https://postgr.es/m/20191013144153.ooxrfglvnaocsrx2@alap3.anarazel.de
https://git.postgresql.org/pg/commitdiff/4a252996d5fda7662b2afdf329a5c95be0fe3b01

- Remove unused code from tuplesort. copytup_index() is unused, as
tuplesort_putindextuplevalues() doesn't use COPYTUP(). Replace function body
with an elog(ERROR), as already done e.g. for copytup_datum(). Author: Andres
Freund Discussion:
https://postgr.es/m/20191013144153.ooxrfglvnaocsrx2@alap3.anarazel.de
https://git.postgresql.org/pg/commitdiff/7d962eaf50c093b4bb1d35a7b495d416a67b40e2

- Fix plan instability in the new tuplesort test. At least buildfarm member
florican doesn't use a material node above a sort in the mark/restore case. As
material is not intended to be tested with that query, disallow.
https://git.postgresql.org/pg/commitdiff/80ef34fc7075b37fc23f4ab714a5ce60f82400de

Tomáš Vondra pushed:

- Skip system attributes when applying mvdistinct stats. When estimating number
of distinct groups, we failed to ignore system attributes when matching the
group expressions to mvdistinct stats, causing failures like ERROR:
negative bitmapset member not allowed Fix that by simply skipping anything
that is not a regular attribute. Backpatch to PostgreSQL 10, where the
extended stats were introduced. Bug: #16111 Reported-by: Tuomas Leikola
Author: Tomas Vondra Backpatch-through: 10 Discussion:
https://postgr.es/m/16111-687799584c3a7e73@postgresql.org
https://git.postgresql.org/pg/commitdiff/d482f7f867b58bbd29f65a4471eca8c5b57a7da0

- Properly determine length for on-disk TOAST values. In detoast_attr_slice,
VARSIZE_ANY was used to compute compressed length of on-disk TOAST values.
That's incorrect, because the varlena value may be just a TOAST pointer,
producing either bogus value or crashing. This is likely why the code was
crashing on big-endian machines before 540f31680913 replaced the VARSIZE with
VARSIZE_ANY, which however only masked the issue. Reported-by: Rushabh Lathia
Discussion:
https://postgr.es/m/CAL-OGkthU9Gs7TZchf5OWaL-Gsi=hXqufTxKv9qpNG73d5na_g@mail.gmail.com
https://git.postgresql.org/pg/commitdiff/2dc08bd6179d8cf480c93701010c19ad7a9891d8

== Pending Patches ==

Justin Pryzby sent in a patch for psql to enable filtering the columns shown by \d.

Tomáš Vondra and Jesper Pedersen traded patches to implement index skip scans.

Thomas Munro sent in a patch to add collation versions for Windows.

Pengzhou Tang sent in a patch to extend the Table AM routine to get total blocks
that can be analyzed.

Nino Floris sent in a patch to implements ltree, lquery, and ltxtquery binary
protocol support.

Pavel Stěhule sent in a patch to implement minscale and trimscale for NUMERIC.

Konstantin Knizhnik sent in another revision of a patch to implement global
temporary tables.

David Cramer sent in two more revisions of a patch to add binary support for the
pgoutput plugin.

Amit Langote sent in another revision of a patch to do some refactoring of the
publication and subscription code, and use same to support adding partitioned
tables to a publication.

Guram Duka sent in two revisions of a patch to fix the build and install
problems under MSYS2, including llvmjit.

Ranier Vilela sent in two revisions of a patch to fix a missing initialization
of fargtypes in event_trigger.c.

Gilles Darold sent in another revision of a patch to fix the PREPARE TRANSACTION
doc and postgres_fdw message.

Thomas Munro sent in another revision of a patch to remove the dependency on
HeapTuple from predicate locking functions.

Álvaro Herrera sent in another revision of a patch to implement FETCH FIRST ...
WITH TIES.

Fujii Masao sent in two more revisions of a patch to fix an infelicity between
pg_waldump and PREPARE.

Nikolay Shaplov and Michaël Paquier traded patches to use an empty structure for
partitioned options.

Michaël Paquier and Álvaro Herrera traded patches to fix some of the coding in
WalSndWaitForWal.

Michaël Paquier and Antonin Houska fix the hintbit LSN compare in
src/backend/storage/buffer/bufmgr.c.

Mark Dilger sent in a patch to change instances of "else if (IsA(node, XXX))" to
"switch (nodeTag(node)) { case XXX:...", which would make node handling simpler.

Dilip Kumar sent in five more revisions of a patch to add cost to parallel
vacuum.

Andrew Dunstan sent in another revision of a patch to add an ssl passphrase
callback.

Surafel Temesgen sent in another revision of a patch to handle conflicts in COPY
... FROM.

Antonin Houska and Álvaro Herrera traded patches to consolidate the reading of
XLOG pages to a single code path.

Ranier Vilela sent in a patch to fix some uninitialized variables.

Kirk Jamison sent in another revision of a patch to make dropping of relation
buffers more efficient by using dlist.

Ranier Vilela sent in a patch to fix a potential null pointer dereference by
ensuring that ExecClearTuple checks TupIsNull.

Asif Rehman sent in two more revisions of a patch to implement parallel backup.

Konstantin Knizhnik sent in two more revisions of a patch to implement a
built-in connection pooler.

Luis Carril sent in two more revisions of a patch to make it possible to dump
foreign data with pg_dump.

Andres Freund sent in a patch to change pg_attribute_noreturn() to pg_noreturn,
move to before return type.

Daniel Gustafsson sent in two more revisions of a patch to use heap_multi_insert
for catalog relations.

Nikita Glukhov sent in two more revisions of a patch to implement JSON_TABLE.

Peter Smith sent in another revision of a patch to add a new SQL counter
statistics view: pg_stat_sql.

Iwata Aya sent in another revision of a patch to make it possible to log libpq
PQtrace output on one line.

Kyotaro HORIGUCHI sent in another revision of a patch to add a "latest" recovery
target.

Julien Rouhaud sent in another revision of a patch to show planning buffers.

Lætitia Avrot sent in a patch to ensure that the pg_restore documentation
includes how to use connection string.

Ranier Vilela sent in a patch to tests in AtSubCommit_childXids from
(s->nChildXids > 0) to (s->childXids != NULL), the latter covering more cases.

Robert Haas sent in another revision of a patch to move AtEOXact_Snapshot() back
to AbortTransaction().

Zheng Li sent in another revision of a patch to execute some cases of NOT IN as
anti-JOINs.

Thomas Munro sent in a patch to make it possible to have an all-whitespace
PROMPT2 in psql.

Tomáš Vondra and Mark Dilger traded patches to use multiple extended statistics
for estimates.

Michaël Paquier and Nikolay Shaplov traded patches to remove calls to
StdRdOptions from Access Methods.

Etsuro Fujita and Amul Sul traded patches to use a better partition-matching
algorithm for partition-wise JOINs.

Álvaro Herrera sent in two more revisions of a patch to speed up foreign keys to
partitioned tables by adding a partdesc memcxt, then using same to invoke
RelationBuildPartitionDesc lazily.

Peter Geoghegan sent in another revision of a patch to add deduplication to
nbtree.

Fujii Masao sent in another revision of a patch to enable ALTER MATERIALIZED VIEW
to rename columns.

Nikita Glukhov sent in another revision of a patch to implement the SQL/JSON
functions.

Bruce Momjian sent in a patch to fix a pg_upgrade loadable_libraries warning.

Tomáš Vondra sent in another revision of a patch to make partial TOAST
compression more efficient.

Robert Haas sent in another revision of a patch to add some CHECK_FOR_INTERRUPTS
calls to src/backend/postmaster/autovacuum.c.

Juan José Santamaría Flecha sent in another revision of a patch to fix logging
colorization on Windows.

Anastasia Lubennikova sent in another revision of a patch to add a bitwise
equality parameter to OPCLASSes.

Ranier Vilela sent in a patch to fix some unsafe access pointers in BRIN.

Konstantin Knizhnik sent in another revision of a patch to add a
wal_receiver_start_condition GUC.

Laurenz Albe sent in a patch to document the fact that the right to ALTER or
DROP can be inherited.

Robert Haas sent in a patch to add a pg_shmem_allocations view.

Tom Lane sent in another revision of a patch to invent trusted extensions.

Thomas Munro sent in a patch to make more accurate partial path row estimates.

Amit Kapila sent in another revision of a patch to add logical_decoding_work_mem
to limit ReorderBuffer memory usage, and track statistics for spilling of
changes from ReorderBuffer.

Vigneshwaran C sent in a patch to make the copyright information consistent in
the source files.

Alexander Korotkov sent in another revision of a patch to fix a deadlock between
ginDeletePage() and ginStepRight(), fix traversing to the deleted GIN page via
downlink, and revise GIN README to better document its concurrency, and adds an
explanation of the differences in page layout from nbtree.

David Fetter sent in a patch to make it possible to have reverse collations.
These would simplify keyset pagination.