PostgreSQL Weekly News - November 1, 2020

## PostgreSQL Weekly News - November 1, 2020

The next Commitfest for PostgreSQL 14 has begun. If you are the author of a
patch, please make sure to follow up on the reviews so your contribution can get
ready to be committed.

## PostgreSQL Product News

pgbitmap 0.93, a space-optimised, non-sparse, bitmap type, released.
[https://pgxn.org/dist/pgbitmap/](https://pgxn.org/dist/pgbitmap/)

Pgpool-II 4.2 beta1
[https://www.pgpool.net/docs/42/en/html/release-4-2-0.html](https://www.pgpool.net/docs/42/en/html/release-4-2-0.html)

pglogical 2.3.3, a logical-WAL-based replication system for PostgreSQL, released.
[https://www.2ndquadrant.com/en/resources/pglogical/release-notes/](https://www.2ndquadrant.com/en/resources/pglogical/release-notes/)

pg\_activity 1.6.2, a top-like application for PostgreSQL server activity
monitoring, released.
[https://github.com/dalibo/pg\_activity/releases/tag/v1.6.2](https://github.com/dalibo/pg\_activity/releases/tag/v1.6.2)

pg\_dumpbinary 2.3, a program used to dump a PostgreSQL database in binary
format, released.
[https://github.com/lzlabs/pg\_dumpbinary/releases/](https://github.com/lzlabs/pg\_dumpbinary/releases/)

pgCenter 0.6.6, a command-line admin tool for observing and troubleshooting
PostgreSQL, released.
[https://github.com/lesovsky/pgcenter/releases/tag/v0.6.6](https://github.com/lesovsky/pgcenter/releases/tag/v0.6.6)

## PostgreSQL in the News

Planet PostgreSQL: [http://planet.postgresql.org/](http://planet.postgresql.org/)

PostgreSQL Weekly News is brought to you this week by David Fetter

Submit news and announcements by Sunday at 3:00pm PST8PDT to [david(at)fetter(dot)org](david(at)fetter(dot)org).

## Applied Patches

David Rowley pushed:

- Prevent overly large and NaN row estimates in relations. Given a query with
enough joins, it was possible that the query planner, after multiplying the
row estimates with the join selectivity that the estimated number of rows
would exceed the limits of the double data type and become infinite. To give
an indication on how extreme a case is required to hit this, the particular
example case reported required 379 joins to a table without any statistics,
which resulted in the 1.0/DEFAULT\_NUM\_DISTINCT being used for the join
selectivity. This eventually caused the row estimates to go infinite and
resulted in an assert failure in initial\_cost\_mergejoin() where the infinite
row estimated was multiplied by an outerstartsel of 0.0 resulting in NaN. The
failing assert verified that NaN <= Inf, which is false. To get around this
we use clamp\_row\_est() to cap row estimates at a maximum of 1e100. This value
is thought to be low enough that costs derived from it would remain within the
bounds of what the double type can represent. Aside from fixing the failing
Assert, this also has the added benefit of making it so add\_path() will still
receive proper numerical values as costs which will allow it to make more sane
choices when determining the cheaper path in extreme cases such as the one
described above. Additionally, we also get rid of the isnan() checks in the
join costing functions. The actual case which originally triggered those
checks to be added in the first place never made it to the mailing lists. It
seems likely that the new code being added to clamp\_row\_est() will result in
those becoming checks redundant, so just remove them. The fairly harmless
assert failure problem does also exist in the backbranches, however, a more
minimalistic fix will be applied there. Reported-by: Onder Kalaci
Reviewed-by: Tom Lane Discussion:
[https://postgr.es/m/DM6PR21MB1211FF360183BCA901B27F04D80B0@DM6PR21MB1211.namprd21.prod.outlook.com](https://postgr.es/m/DM6PR21MB1211FF360183BCA901B27F04D80B0@DM6PR21MB1211.namprd21.prod.outlook.com)
[https://git.postgresql.org/pg/commitdiff/a90c950fc7fd8796daa8c7948e7046bceb272894](https://git.postgresql.org/pg/commitdiff/a90c950fc7fd8796daa8c7948e7046bceb272894)

- Optimize a few list\_delete\_ptr calls. There is a handful of places where we
called list\_delete\_ptr() to remove some element from a List. In many of these
places we know, or with very little additional effort know the index of the
ListCell that we need to remove. Here we change all of those places to
instead either use one of; list\_delete\_nth\_cell(), foreach\_delete\_current() or
list\_delete\_last(). Each of these saves from having to iterate over the list
to search for the element to remove by its pointer value. There are some
small performance gains to be had by doing this, but in the general case, none
of these lists are likely to be very large, so the lookup was probably never
that expensive anyway. However, some of the calls are in fairly hot code
paths, e.g process\_equivalence(). So any small gains there are useful.
Author: Zhijie Hou and David Rowley Discussion:
[https://postgr.es/m/b3517353ec7c4f87aa560678fbb1034b@G08CNEXMBPEKD05.g08.fujitsu.local](https://postgr.es/m/b3517353ec7c4f87aa560678fbb1034b@G08CNEXMBPEKD05.g08.fujitsu.local)
[https://git.postgresql.org/pg/commitdiff/e7c2b95d37a2b9c01367f7ccc55703555b39c81c](https://git.postgresql.org/pg/commitdiff/e7c2b95d37a2b9c01367f7ccc55703555b39c81c)

- Fix incorrect parameter name in a function header comment. Author: Zhijie Hou
Discussion:
[https://postgr.es/m/14cd74ea00204cc8a7ea5d738ac82cd1@G08CNEXMBPEKD05.g08.fujitsu.local](https://postgr.es/m/14cd74ea00204cc8a7ea5d738ac82cd1@G08CNEXMBPEKD05.g08.fujitsu.local)
Backpatch-through: 12, where the mistake was introduced
[https://git.postgresql.org/pg/commitdiff/e83c9f913c6197586af8ac53c1d3652db15a3c91](https://git.postgresql.org/pg/commitdiff/e83c9f913c6197586af8ac53c1d3652db15a3c91)

Michaël Paquier pushed:

- Fix potential memory leak in pgcrypto. When allocating a EVP context, it would
have been possible to leak some memory allocated directly by OpenSSL, that
PostgreSQL lost track of if the initialization of the context allocated
failed. The cleanup can be done with EVP\_MD\_CTX\_destroy(). Note that EVP
APIs exist since OpenSSL 0.9.7 and we have in the tree equivalent
implementations for older versions since ce9b75d (code removed with 9b7cd59a
as of 10~). However, in 9.5 and 9.6, the existing code makes use of
EVP\_MD\_CTX\_destroy() and EVP\_MD\_CTX\_create() without an equivalent
implementation when building the tree with OpenSSL 0.9.6 or older, meaning
that this code is in reality broken with such versions since it got introduced
in e2838c5. As we have heard no complains about that, it does not seem worth
bothering with in 9.5 and 9.6, so I have left that out for simplicity.
Author: Michael Paquier Discussion:
[https://postgr.es/m/20201015072212.GC2305@paquier.xyz](https://postgr.es/m/20201015072212.GC2305@paquier.xyz) Backpatch-through: 9.5
[https://git.postgresql.org/pg/commitdiff/ca2a12c935f75fb56c3b14527d6f2ff6f549ea85](https://git.postgresql.org/pg/commitdiff/ca2a12c935f75fb56c3b14527d6f2ff6f549ea85)

- Review format of code generated by PerfectHash.pm. 80f8eb7 has added to the
normalization quick check headers some code generated by PerfectHash.pm that
is incompatible with the settings of gitattributes for this repository, as
whitespaces followed a set of tabs for the first element of a line in the
table. Instead of adding a new exception to gitattributes, rework the format
generated so as a right padding with spaces is used instead of a left padding.
This keeps the table generated in a readable shape with its set of columns,
making unnecessary an update of gitattributes. Reported-by: Peter Eisentraut
Author: John Naylor Discussion:
[https://postgr.es/m/d601b3b5-a3c7-5457-2f84-3d6513d690fc@2ndquadrant.com](https://postgr.es/m/d601b3b5-a3c7-5457-2f84-3d6513d690fc@2ndquadrant.com)
[https://git.postgresql.org/pg/commitdiff/19ae53c92d5f5bdfb971d560a562e84c5f65c8b0](https://git.postgresql.org/pg/commitdiff/19ae53c92d5f5bdfb971d560a562e84c5f65c8b0)

- Improve performance of Unicode {de,re}composition in the backend. This
replaces the existing binary search with two perfect hash functions for the
composition and the decomposition in the backend code, at the cost of
slightly-larger binaries there (35kB in libpgcommon\_srv.a). Per the
measurements done, this improves the speed of the recomposition and
decomposition by up to 30~40 times for the NFC and NFKC conversions, while all
other operations get at least 40% faster. This is not as "good" as what
libicu has, but it closes the gap a lot as per the feedback from Daniel
Verite. The decomposition table remains the same, getting used for the binary
search in the frontend code, where we care more about the size of the
libraries like libpq over performance as this gets involved only in code paths
related to the SCRAM authentication. In consequence, note that the perfect
hash function for the recomposition needs to use a new inverse lookup array
back to to the existing decomposition table. The size of all frontend
deliverables remains unchanged, even with --enable-debug, including libpq.
Author: John Naylor Reviewed-by: Michael Paquier, Tom Lane Discussion:
[https://postgr.es/m/CAFBsxsHUuMFCt6-pU+oG-F1==CmEp8wR+O+bRouXWu6i8kXuqA@mail.gmail.com](https://postgr.es/m/CAFBsxsHUuMFCt6-pU+oG-F1==CmEp8wR+O+bRouXWu6i8kXuqA@mail.gmail.com)
[https://git.postgresql.org/pg/commitdiff/783f0cc64dcc05e3d112a06b1cd181e5a1ca9099](https://git.postgresql.org/pg/commitdiff/783f0cc64dcc05e3d112a06b1cd181e5a1ca9099)

- Add tab completion for ALTER TABLE .. FORCE ROW LEVEL SECURITY in psql. This
completes both the FORCE and NO FORCE options, NO INHERIT needing a small
adjustment. Author: Li Japin Discussion:
[https://postgr.es/m/15B10F9F-5847-4F5E-BD66-8E25AA473C95@hotmail.com](https://postgr.es/m/15B10F9F-5847-4F5E-BD66-8E25AA473C95@hotmail.com)
[https://git.postgresql.org/pg/commitdiff/0b46e82c06b0c4b0dc6a94a890d23945ebf720fd](https://git.postgresql.org/pg/commitdiff/0b46e82c06b0c4b0dc6a94a890d23945ebf720fd)

- Fix issue with --enable-coverage and the new unicode {de,re}composition code.
genhtml has been generating the following warning with this new code: WARNING:
function data mismatch at /path/src/common/unicode\_norm.c:102 HTML coverage
reports care about the uniqueness of functions defined in source files,
ignoring any assumptions around CFLAGS. 783f0cc introduced a duplicated
definition of get\_code\_entry(), leading to a warning and potentially some
incorrect data generated in the reports. This refactors the code so as the
code has only one function declaration, fixing the warning. Oversight in
783f0cc. Reported-by: Tom Lane Author: Michael Paquier Reviewed-by: Tom Lane
Discussion: [https://postgr.es/m/207789.1603469272@sss.pgh.pa.us](https://postgr.es/m/207789.1603469272@sss.pgh.pa.us)
[https://git.postgresql.org/pg/commitdiff/2771fcee18be0b86d2e008add20f73d175e06e90](https://git.postgresql.org/pg/commitdiff/2771fcee18be0b86d2e008add20f73d175e06e90)

- Extend PageIsVerified() to handle more custom options. This is useful for
checks of relation pages without having to load the pages into the shared
buffers, and two cases can make use of that: page verification in base backups
and the online, lock-safe, flavor. Compatibility is kept with past versions
using a macro that calls the new extended routine with the set of options
compatible with the original version. Extracted from a larger patch by the
same author. Author: Anastasia Lubennikova Reviewed-by: Michael Paquier,
Julien Rouhaud Discussion:
[https://postgr.es/m/608f3476-0598-2514-2c03-e05c7d2b0cbd@postgrespro.ru](https://postgr.es/m/608f3476-0598-2514-2c03-e05c7d2b0cbd@postgrespro.ru)
[https://git.postgresql.org/pg/commitdiff/d401c5769ef6aeef0a28c147f3fb5afedcd59984](https://git.postgresql.org/pg/commitdiff/d401c5769ef6aeef0a28c147f3fb5afedcd59984)

- Add CheckBuffer() to check on-disk pages without shared buffer loading.
CheckBuffer() is designed to be a concurrent-safe function able to run sanity
checks on a relation page without loading it into the shared buffers. The
operation is done using a lock on the partition involved in the shared buffer
mapping hashtable and an I/O lock for the buffer itself, preventing the risk
of false positives due to any concurrent activity. The primary use of this
function is the detection of on-disk corruptions for relation pages. If a
page is found in shared buffers, the on-disk page is checked if not dirty (a
follow-up checkpoint would flush a valid version of the page if dirty anyway),
as it could be possible that a page was present for a long time in shared
buffers with its on-disk version corrupted. Such a scenario could lead to a
corrupted cluster if a host is plugged off for example. If the page is not
found in shared buffers, its on-disk state is checked.
PageIsVerifiedExtended() is used to apply the same sanity checks as when a
page gets loaded into shared buffers. This function will be used by an
upcoming patch able to check the state of on-disk relation pages using a SQL
function. Author: Julien Rouhaud, Michael Paquier Reviewed-by: Masahiko
Sawada Discussion:
[https://postgr.es/m/CAOBaU\_aVvMjQn=ge5qPiJOPMmOj5=ii3st5Q0Y+WuLML5sR17w@mail.gmail.com](https://postgr.es/m/CAOBaU\_aVvMjQn=ge5qPiJOPMmOj5=ii3st5Q0Y+WuLML5sR17w@mail.gmail.com)
[https://git.postgresql.org/pg/commitdiff/c780a7a90a8e7b074405ea2007e34f94e227e695](https://git.postgresql.org/pg/commitdiff/c780a7a90a8e7b074405ea2007e34f94e227e695)

- Add pg\_relation\_check\_pages() to check on-disk pages of a relation. This makes
use of CheckBuffer() introduced in c780a7a, adding a SQL wrapper able to do
checks for all the pages of a relation. By default, all the fork types of a
relation are checked, and it is possible to check only a given relation fork.
Note that if the relation given in input has no physical storage or is
temporary, then no errors are generated, allowing full-database checks when
coupled with a simple scan of pg\_class for example. This is not limited to
clusters with data checksums enabled, as clusters without data checksums can
still apply checks on pages using the page headers or for the case of a page
full of zeros. This function returns a set of tuples consisting of: - The
physical file where a broken page has been detected (without the segment
number as that can be AM-dependent, which can be guessed from the block number
for heap). A relative path from PGPATH is used. - The block number of the
broken page. By default, only superusers have an access to this function but
execution rights can be granted to other users. The feature introduced here
is still minimal, and more improvements could be done, like: - Addition of a
start and end block number to run checks on a range of blocks, which would
apply only if one fork type is checked. - Addition of some progress reporting.
- Throttling, with configuration parameters in function input or potentially
some cost-based GUCs. Regression tests are added for positive cases in the
main regression test suite, and TAP tests are added for cases involving the
emulation of page corruptions. Bump catalog version. Author: Julien Rouhaud,
Michael Paquier Reviewed-by: Masahiko Sawada, Justin Pryzby Discussion:
[https://postgr.es/m/CAOBaU\_aVvMjQn=ge5qPiJOPMmOj5=ii3st5Q0Y+WuLML5sR17w@mail.gmail.com](https://postgr.es/m/CAOBaU\_aVvMjQn=ge5qPiJOPMmOj5=ii3st5Q0Y+WuLML5sR17w@mail.gmail.com)
[https://git.postgresql.org/pg/commitdiff/f2b883969557f4572cdfa87e1a40083d2b1272e7](https://git.postgresql.org/pg/commitdiff/f2b883969557f4572cdfa87e1a40083d2b1272e7)

- Use correct GetDatum() in pg\_relation\_check\_pages(). UInt32GetDatum() was
getting used, while the result needs Int64GetDatum(). Oversight in f2b8839.
Per buildfarm member florican. Discussion:
[https://postgr.es/m/1226629.1603859189@sss.pgh.pa.us](https://postgr.es/m/1226629.1603859189@sss.pgh.pa.us)
[https://git.postgresql.org/pg/commitdiff/ce7f772c5e6066e0bbafea5759e652c9757c8e6b](https://git.postgresql.org/pg/commitdiff/ce7f772c5e6066e0bbafea5759e652c9757c8e6b)

- Fix incorrect placement of pfree() in pg\_relation\_check\_pages(). This would
cause the function to crash when more than one page is considered as broken
and reported in the SRF. Reported-by: Noriyoshi Shinoda Discussion:
[https://postgr.es/m/TU4PR8401MB11523D42C315AAF822E74275EE170@TU4PR8401MB1152.NAMPRD84.PROD.OUTLOOK.COM](https://postgr.es/m/TU4PR8401MB11523D42C315AAF822E74275EE170@TU4PR8401MB1152.NAMPRD84.PROD.OUTLOOK.COM)
[https://git.postgresql.org/pg/commitdiff/60a51c6b32960822d3987ea7d2816c65bdbcb314](https://git.postgresql.org/pg/commitdiff/60a51c6b32960822d3987ea7d2816c65bdbcb314)

- Add error code for encryption failure in pgcrypto. PXE\_DECRYPT\_FAILED exists
already for decryption errors, and an equivalent for encryption did not exist.
There is one code path that deals with such failures for OpenSSL but it used
PXE\_ERR\_GENERIC, which was inconsistent. This switches this code path to use
the new error PXE\_ENCRYPT\_FAILED instead of PXE\_ERR\_GENERIC, making the code
used for encryption more consistent with the decryption. Author: Daniel
Gustafsson Discussion:
[https://postgr.es/m/03049139-CB7A-436E-B71B-42696D3E2EF7@yesql.se](https://postgr.es/m/03049139-CB7A-436E-B71B-42696D3E2EF7@yesql.se)
[https://git.postgresql.org/pg/commitdiff/aecaa04418f39c32adb3dbf91c4aa7f6e175f01c](https://git.postgresql.org/pg/commitdiff/aecaa04418f39c32adb3dbf91c4aa7f6e175f01c)

- Preserve index data in pg\_statistic across REINDEX CONCURRENTLY. Statistics
associated to an index got lost after running REINDEX CONCURRENTLY, while the
non-concurrent case preserves these correctly. The concurrent and
non-concurrent operations need to be consistent for the end-user, and missing
statistics would force to wait for a new analyze to happen, which could take
some time depending on the activity of the existing autovacuum workers. This
issue is fixed by copying any existing entries in pg\_statistic associated to
the old index to the new one. Note that this copy is already done with the
data of the index in the stats collector. Reported-by: Fabrízio de Royes
Mello Author: Michael Paquier, Fabrízio de Royes Mello Reviewed-by: Justin
Pryzby Discussion:
[https://postgr.es/m/CAFcNs+qpFPmiHd1oTXvcPdvAHicJDA9qBUSujgAhUMJyUMb+SA@mail.gmail.com](https://postgr.es/m/CAFcNs+qpFPmiHd1oTXvcPdvAHicJDA9qBUSujgAhUMJyUMb+SA@mail.gmail.com)
Backpatch-through: 12
[https://git.postgresql.org/pg/commitdiff/b17ff07aa3eb142d2cde2ea00e4a4e8f63686f96](https://git.postgresql.org/pg/commitdiff/b17ff07aa3eb142d2cde2ea00e4a4e8f63686f96)

Amit Kapila pushed:

- Change the docs for PARALLEL option of Vacuum. The rules to choose the number
of parallel workers to perform parallel vacuum operation were not clearly
specified. Reported-by: Peter Eisentraut Author: Amit Kapila
Backpatch-through: 13, where it was introduced Discussion:
[https://postgr.es/m/36aa8aea-61b7-eb3c-263b-648e0cb117b7@2ndquadrant.com](https://postgr.es/m/36aa8aea-61b7-eb3c-263b-648e0cb117b7@2ndquadrant.com)
[https://git.postgresql.org/pg/commitdiff/560d260d7852dc54a8c587c1b388843e8c433bc8](https://git.postgresql.org/pg/commitdiff/560d260d7852dc54a8c587c1b388843e8c433bc8)

- Change the attribute name in pg\_stat\_replication\_slots view. Change the
attribute 'name' to 'slot\_name' in pg\_stat\_replication\_slots view to make it
clear and that way we will be consistent with the other places like
pg\_stat\_wal\_receiver view where we display the same attribute. In the
passing, fix the typo in one of the macros in the related code. Bump the
catversion as we have modified the name in the catalog as well. Reported-by:
Noriyoshi Shinoda Author: Noriyoshi Shinoda Reviewed-by: Sawada Masahiko and
Amit Kapila Discussion:
[https://postgr.es/m/CA+fd4k5\_pPAYRTDrO2PbtTOe0eHQpBvuqmCr8ic39uTNmR49Eg@mail.gmail.com](https://postgr.es/m/CA+fd4k5\_pPAYRTDrO2PbtTOe0eHQpBvuqmCr8ic39uTNmR49Eg@mail.gmail.com)
[https://git.postgresql.org/pg/commitdiff/03d51b776d88badbeb4452c1ab452feb78dce36a](https://git.postgresql.org/pg/commitdiff/03d51b776d88badbeb4452c1ab452feb78dce36a)

- Update description of spilled counters in pg\_stat\_replication\_slots view. This
is to make the description of spilled counters clear. Author: Amit Kapila
Reviewed-by: Sawada Masahiko Discussion:
[https://postgr.es/m/CA+fd4k5\_pPAYRTDrO2PbtTOe0eHQpBvuqmCr8ic39uTNmR49Eg@mail.gmail.com](https://postgr.es/m/CA+fd4k5\_pPAYRTDrO2PbtTOe0eHQpBvuqmCr8ic39uTNmR49Eg@mail.gmail.com)
[https://git.postgresql.org/pg/commitdiff/2f0760c9ff827bb3d23ee327e6b46038000c7ef9](https://git.postgresql.org/pg/commitdiff/2f0760c9ff827bb3d23ee327e6b46038000c7ef9)

- Minor improvements in description of spilled counters in
pg\_stat\_replication\_slots view. Per a suggestion by Justin Pryzby.
Discussion:
[https://postgr.es/m/CA+fd4k5\_pPAYRTDrO2PbtTOe0eHQpBvuqmCr8ic39uTNmR49Eg@mail.gmail.com](https://postgr.es/m/CA+fd4k5\_pPAYRTDrO2PbtTOe0eHQpBvuqmCr8ic39uTNmR49Eg@mail.gmail.com)
[https://git.postgresql.org/pg/commitdiff/9e0f87a4955ede0769d2f9a93171145b6ddab901](https://git.postgresql.org/pg/commitdiff/9e0f87a4955ede0769d2f9a93171145b6ddab901)

- Track statistics for streaming of changes from ReorderBuffer. This adds the
statistics about transactions streamed to the decoding output plugin from
ReorderBuffer. Users can query the pg\_stat\_replication\_slots view to check
these stats and call pg\_stat\_reset\_replication\_slot to reset the stats of a
particular slot. Users can pass NULL in pg\_stat\_reset\_replication\_slot to
reset stats of all the slots. Commit 9868167500 has added the basic
infrastructure to capture the stats of slot and this commit extends the
statistics collector to track additional information about slots. Bump the
catversion as we have added new columns in the catalog entry. Author: Ajin
Cherian and Amit Kapila Reviewed-by: Sawada Masahiko and Dilip Kumar
Discussion:
[https://postgr.es/m/CAA4eK1+chpEomLzgSoky-D31qev19AmECNiEAietPQUGEFhtVA@mail.gmail.com](https://postgr.es/m/CAA4eK1+chpEomLzgSoky-D31qev19AmECNiEAietPQUGEFhtVA@mail.gmail.com)
[https://git.postgresql.org/pg/commitdiff/8e90ec5580d5345fef31005d7cc2215ba2125070](https://git.postgresql.org/pg/commitdiff/8e90ec5580d5345fef31005d7cc2215ba2125070)

Peter Eisentraut pushed:

- Improve whitespace. The SQL file was using a mix of tabs and spaces
inconsistently. Convert all to spaces and adjust indentation accordingly.
[https://git.postgresql.org/pg/commitdiff/afa0d53d4de6d2f43fbc0a76c88dceb22ba65e86](https://git.postgresql.org/pg/commitdiff/afa0d53d4de6d2f43fbc0a76c88dceb22ba65e86)

- Avoid invalid alloc size error in shm\_mq. In shm\_mq\_receive(), a huge payload
could trigger an unjustified "invalid memory alloc request size" error due to
the way the buffer size is increased. Add error checks (documenting the upper
limit) and avoid the error by limiting the allocation size to MaxAllocSize.
Author: Markus Wanner [markus(dot)wanner(at)2ndquadrant(dot)com](markus(dot)wanner(at)2ndquadrant(dot)com) Discussion:
[https://www.postgresql.org/message-id/flat/3bb363e7-ac04-0ac4-9fe8-db1148755bfa%402ndquadrant.com](https://www.postgresql.org/message-id/flat/3bb363e7-ac04-0ac4-9fe8-db1148755bfa%402ndquadrant.com)
[https://git.postgresql.org/pg/commitdiff/26ec6b5948a73d0e07ed9435ee4554594acdf34f](https://git.postgresql.org/pg/commitdiff/26ec6b5948a73d0e07ed9435ee4554594acdf34f)

- Fix -Wcast-function-type warnings on Windows/MinGW. After
de8feb1f3a23465b5737e8a8c160e8ca62f61339, some warnings remained that were
only visible when using GCC on Windows. Fix those as well. Note that the
ecpg test source files don't use the full pg\_config.h, so we can't use
pg\_funcptr\_t there but have to do it the long way.
[https://git.postgresql.org/pg/commitdiff/8a58347a3c10126ce9eb38872a46c55e91faa174](https://git.postgresql.org/pg/commitdiff/8a58347a3c10126ce9eb38872a46c55e91faa174)

- Remove obsolete ifdefs. Commit 8dace66e0735ca39b779922d02c24ea2686e6521 added
\#ifdefs for a number of errno symbols because they were not present on
Windows. Later, commit 125ad539a275db5ab8f4647828b80a16d02eabd2 added
replacement \#defines for some of those symbols. So some of the changes from
the first commit are made dead code by the second commit and can now be
removed. Discussion:
[https://www.postgresql.org/message-id/flat/6dee8574-b0ad-fc49-9c8c-2edc796f0033(at)2ndquadrant(dot)com](https://www.postgresql.org/message-id/flat/6dee8574-b0ad-fc49-9c8c-2edc796f0033(at)2ndquadrant(dot)com)
[https://git.postgresql.org/pg/commitdiff/555eb1a4f0114e1c4017093fe840045dde1ed9ea](https://git.postgresql.org/pg/commitdiff/555eb1a4f0114e1c4017093fe840045dde1ed9ea)

- Use croak instead of die in Perl code when appropriate.
[https://git.postgresql.org/pg/commitdiff/f8721bd752790859df747905bc44fb5ad8dbf07d](https://git.postgresql.org/pg/commitdiff/f8721bd752790859df747905bc44fb5ad8dbf07d)

- doc: Fix order of protocol messages in listing. The order of
AuthenticationGSSContinue and AuthenticationSSPI was swapped, based on the
other Authentication\* protocol messages being listed in subcode order.
[https://git.postgresql.org/pg/commitdiff/83d727e5b23c43f3fb7221963ddec24277c1126f](https://git.postgresql.org/pg/commitdiff/83d727e5b23c43f3fb7221963ddec24277c1126f)

- doc: Fix order of protocol messages in listing. Move GSSENCRequest to the
correct alphabetical position.
[https://git.postgresql.org/pg/commitdiff/8fed2eadb8558185d162cc7fd09192c2e10b915f](https://git.postgresql.org/pg/commitdiff/8fed2eadb8558185d162cc7fd09192c2e10b915f)

- Make procedure OUT parameters work with JDBC. The JDBC driver sends OUT
parameters with type void. This makes sense when calling a function, so that
the parameters are ignored in ParseFuncOrColumn(). For a procedure call we
want to treat them as unknown. Reviewed-by: Andrew Dunstan
andrew [AT] dunslane.net Discussion:
[https://www.postgresql.org/message-id/flat/d7e49540-ea92-b4e2-5fff-42036102f968%402ndquadrant.com](https://www.postgresql.org/message-id/flat/d7e49540-ea92-b4e2-5fff-42036102f968%402ndquadrant.com)
[https://git.postgresql.org/pg/commitdiff/9213462c539e6412fe0498a7f8e20b662e15c4ec](https://git.postgresql.org/pg/commitdiff/9213462c539e6412fe0498a7f8e20b662e15c4ec)

- Fix enum errdetail to mention bytes, not chars. The enum label length is in
terms of bytes, not charactes. Author: Ian Lawrence Barwick
barwick [AT] gmail.com Reviewed-by: Julien Rouhaud rjuju123 [AT] gmail.com
Discussion:
[https://www.postgresql.org/message-id/flat/CAB8KJ=itZEJ7C9BacTHSYgeUysH4xx8wDiOnyppnSLyn6-g+Bw(at)mail(dot)gmail(dot)com](https://www.postgresql.org/message-id/flat/CAB8KJ=itZEJ7C9BacTHSYgeUysH4xx8wDiOnyppnSLyn6-g+Bw(at)mail(dot)gmail(dot)com)
[https://git.postgresql.org/pg/commitdiff/0525572860335d050a1bea194a5278c8833304d1](https://git.postgresql.org/pg/commitdiff/0525572860335d050a1bea194a5278c8833304d1)

- Add select\_common\_typmod(). This accompanies select\_common\_type() and
select\_common\_collation(). Typmods were previously combined using hand-coded
logic in several places. The logic in select\_common\_typmod() isn't very
exciting, but it makes the code more compact and readable in a few locations,
and in the future we can perhaps do more complicated things if desired. As a
small enhancement, the type unification of the direct and aggregate arguments
of hypothetical-set aggregates now unifies the typmod as well using this new
function, instead of just dropping it. Reviewed-by: Heikki Linnakangas
hlinnaka [AT] iki.fi Discussion:
[https://www.postgresql.org/message-id/flat/97df3af9-8b5e-fb7f-a029-3eb7e80d7af9(at)2ndquadrant(dot)com](https://www.postgresql.org/message-id/flat/97df3af9-8b5e-fb7f-a029-3eb7e80d7af9(at)2ndquadrant(dot)com)
[https://git.postgresql.org/pg/commitdiff/f893e68d761adbee7f888109b1adf76151e3e17a](https://git.postgresql.org/pg/commitdiff/f893e68d761adbee7f888109b1adf76151e3e17a)

Heikki Linnakangas pushed:

- Clean up code to resolve the "root target relation" in nodeModifyTable.c. When
executing DDL on a partitioned table or on a table with inheritance children,
statement-level triggers must be fired against the table given in the original
statement. The code to look that up was a bit messy and duplicative. Commit
501ed02cf6 added a helper function, getASTriggerResultRelInfo() (later renamed
to getTargetResultRelInfo()) for it, but for some reason it was only used when
firing AFTER STATEMENT triggers and the code to fire BEFORE STATEMENT triggers
duplicated the logic. Determine the target relation in ExecInitModifyTable(),
and set it always in ModifyTableState. Code that used to call
getTargetResultRelInfo() can now use ModifyTableState->rootResultRelInfo
directly. Discussion:
[https://www.postgresql.org/message-id/CA%2BHiwqFViT47Zbr\_ASBejiK7iDG8%3DQ1swQ-tjM6caRPQ67pT%3Dw%40mail.gmail.com](https://www.postgresql.org/message-id/CA%2BHiwqFViT47Zbr\_ASBejiK7iDG8%3DQ1swQ-tjM6caRPQ67pT%3Dw%40mail.gmail.com)
[https://git.postgresql.org/pg/commitdiff/f49b85d783f6781138f33bbe5f6e98da86907d84](https://git.postgresql.org/pg/commitdiff/f49b85d783f6781138f33bbe5f6e98da86907d84)

- Revise child-to-root tuple conversion map management. Store the tuple
conversion map to convert a tuple from a child table's format to the root
format in a new ri\_ChildToRootMap field in ResultRelInfo. It is initialized if
transition tuple capture for FOR STATEMENT triggers or INSERT tuple routing on
a partitioned table is needed. Previously, ModifyTable kept the maps in the
per-subplan ModifyTableState->mt\_per\_subplan\_tupconv\_maps array, or when tuple
routing was used, in ResultRelInfo->ri\_Partitioninfo->pi\_PartitionToRootMap.
The new field replaces both of those. Now that the child-to-root tuple
conversion map is always available in ResultRelInfo (when needed), remove the
TransitionCaptureState.tcs\_map field. The callers of Exec\*Trigger() functions
no longer need to set or save it, which is much less confusing and bug-prone.
Also, as a future optimization, this will allow us to delay creating the map
for a given result relation until the relation is actually processed during
execution. Author: Amit Langote Discussion:
[https://www.postgresql.org/message-id/CA%2BHiwqHtCWLdK-LO%3DNEsvOdHx%2B7yv4mE\_zYK0i3BH7dXb-wxog%40mail.gmail.com](https://www.postgresql.org/message-id/CA%2BHiwqHtCWLdK-LO%3DNEsvOdHx%2B7yv4mE\_zYK0i3BH7dXb-wxog%40mail.gmail.com)
[https://git.postgresql.org/pg/commitdiff/6973533650c04653d9018f61c1ac99ecb11094bd](https://git.postgresql.org/pg/commitdiff/6973533650c04653d9018f61c1ac99ecb11094bd)

- Remove PartitionRoutingInfo struct. The extra indirection neeeded to access
its members via its enclosing ResultRelInfo seems pointless. Move all the
fields from PartitionRoutingInfo to ResultRelInfo. Author: Amit Langote
Reviewed-by: Alvaro Herrera Discussion:
[https://www.postgresql.org/message-id/CA%2BHiwqFViT47Zbr\_ASBejiK7iDG8%3DQ1swQ-tjM6caRPQ67pT%3Dw%40mail.gmail.com](https://www.postgresql.org/message-id/CA%2BHiwqFViT47Zbr\_ASBejiK7iDG8%3DQ1swQ-tjM6caRPQ67pT%3Dw%40mail.gmail.com)
[https://git.postgresql.org/pg/commitdiff/fb5883da86154c3126264bfd97b0cd6f293bcebd](https://git.postgresql.org/pg/commitdiff/fb5883da86154c3126264bfd97b0cd6f293bcebd)

- Fix doc for full text search distance operator. Commit 028350f619 changed its
behavior from "at most" to "exactly", but forgot to update the documentation.
Backpatch to 9.6. Patch by Justin Pryzby, per Yaroslav Schekin's report.
Discussion:
[https://www.postgresql.org/message-id/20201005191922.GE17626%40telsasoft.com](https://www.postgresql.org/message-id/20201005191922.GE17626%40telsasoft.com)
[https://git.postgresql.org/pg/commitdiff/1a64c7636f24096155f7aa15d7164eba3cdef075](https://git.postgresql.org/pg/commitdiff/1a64c7636f24096155f7aa15d7164eba3cdef075)

- Fix output of tsquery example in docs. The output for this query changed in
commit 4e2477b7b8. Backport to 9.6 like that commit. Patch by Justin Pryzby,
per Yaroslav Schekin's report. Discussion:
[https://www.postgresql.org/message-id/20201005191922.GE17626%40telsasoft.com](https://www.postgresql.org/message-id/20201005191922.GE17626%40telsasoft.com)
[https://git.postgresql.org/pg/commitdiff/c0bc4c682ee2a84dee2b562db3d17d173a6abc06](https://git.postgresql.org/pg/commitdiff/c0bc4c682ee2a84dee2b562db3d17d173a6abc06)

- Fix TRUNCATE doc: ALTER SEQUENCE RESTART is now transactional. ALTER SEQUENCE
RESTART was made transactional in commit 3d79013b97. Backpatch to v10, where
that was introduced. Patch by Justin Pryzby, per Yaroslav Schekin's report.
Discussion:
[https://www.postgresql.org/message-id/20201005191922.GE17626%40telsasoft.com](https://www.postgresql.org/message-id/20201005191922.GE17626%40telsasoft.com)
[https://git.postgresql.org/pg/commitdiff/2a972e0165c9b34da4cff3001a2dad8e5a9a5a2b](https://git.postgresql.org/pg/commitdiff/2a972e0165c9b34da4cff3001a2dad8e5a9a5a2b)

- Misc documentation fixes. - Misc grammar and punctuation fixes. - Stylistic
cleanup: use spaces between function arguments and JSON fields in examples.
For example "foo(a,b)" -> "foo(a, b)". Add semicolon after last END in a few
PL/pgSQL examples that were missing them. - Make sentence that talked about
"..." and ".." operators more clear, by avoiding to end the sentence with
"..". That makes it look the same as "..." - Fix syntax description for
HAVING: HAVING conditions cannot be repeated Patch by Justin Pryzby, per
Yaroslav Schekin's report. Backpatch to all supported versions, to the extent
that the patch applies easily. Discussion:
[https://www.postgresql.org/message-id/20201005191922.GE17626%40telsasoft.com](https://www.postgresql.org/message-id/20201005191922.GE17626%40telsasoft.com)
[https://git.postgresql.org/pg/commitdiff/c5f42daa6077a4c309c5280a47d0e114c12dc572](https://git.postgresql.org/pg/commitdiff/c5f42daa6077a4c309c5280a47d0e114c12dc572)

- Fix initialization of es\_result\_relations in EvalPlanQualStart(). Thinko in
commit 1375422c782. EvalPlanQualStart() was mistakenly resetting the parent
EState's es\_result\_relations, when it should initialize the field in the child
EPQ EState it just created. That was clearly wrong, but it didn't cause any
ill effects, because es\_result\_relations is currently not used after the
ExecInit\* phase. Author: Amit Langote Discussion:
[https://www.postgresql.org/message-id/CA%2BHiwqFEuq8AAAmxXsTDVZ1r38cHbfYuiPQx\_%3DYyKe2DC-6q4A%40mail.gmail.com](https://www.postgresql.org/message-id/CA%2BHiwqFEuq8AAAmxXsTDVZ1r38cHbfYuiPQx\_%3DYyKe2DC-6q4A%40mail.gmail.com)
[https://git.postgresql.org/pg/commitdiff/22b73d3cb0b5bb4c141421f98dd67f091dda3c20](https://git.postgresql.org/pg/commitdiff/22b73d3cb0b5bb4c141421f98dd67f091dda3c20)

- doc: Remove reference to pre-8.2 pg\_dump behaviour. The behavioural change in
the -t/--table option happened around 15 years ago and there seems little
point in keeping it around. Author: Ian Barwick Discussion:
[https://www.postgresql.org/message-id/CAB8KJ%3Dh-XALik4M7gv-pX48%3D%2BSPWexfaYwa%2ByTnPwD3DxceXrg%40mail.gmail.com](https://www.postgresql.org/message-id/CAB8KJ%3Dh-XALik4M7gv-pX48%3D%2BSPWexfaYwa%2ByTnPwD3DxceXrg%40mail.gmail.com)
[https://git.postgresql.org/pg/commitdiff/902b57c9bf27b3ea2963ef832cddb38b1203130e](https://git.postgresql.org/pg/commitdiff/902b57c9bf27b3ea2963ef832cddb38b1203130e)

- docs: Remove notes about incompatibilies with very old versions. These are old
enough that they'll cause more confusion and distraction to new readers, than
they could help anyone upgrade from very old servers. Discussion:
[https://www.postgresql.org/message-id/fd93f1c5-7818-a02c-01e5-1075ac0d4def%40iki.fi](https://www.postgresql.org/message-id/fd93f1c5-7818-a02c-01e5-1075ac0d4def%40iki.fi)
[https://git.postgresql.org/pg/commitdiff/fa42c2ecb0f6e89f74bc1cc37b56a1d43e45d513](https://git.postgresql.org/pg/commitdiff/fa42c2ecb0f6e89f74bc1cc37b56a1d43e45d513)

- Fix missing validation for the new GiST sortsupport functions. Because of
this, if you tried to create an operator family with the new sortsupport
function, you got an error: ERROR: support function number 11 is invalid for
access method gist We missed this in commit 16fa9b2b30 that added the
sortsupport function, because it only added sortsupport to a built-in operator
family. Author: Andrey Borodin Discussion:
[https://www.postgresql.org/message-id/3520A18A-5C38-4697-A2E3-F3BDE3496CD5%40yandex-team.ru](https://www.postgresql.org/message-id/3520A18A-5C38-4697-A2E3-F3BDE3496CD5%40yandex-team.ru)
[https://git.postgresql.org/pg/commitdiff/6f0bc5e1daf09686c526aa161da5336f7c94f4eb](https://git.postgresql.org/pg/commitdiff/6f0bc5e1daf09686c526aa161da5336f7c94f4eb)

Magnus Hagander pushed:

- Update link for pllua. Author: Daniel Gustafsson daniel [AT] yesql.se Discussion:
[https://postgr.es/m/A05874AE-8771-4C61-A24E-0B6249B8F3C2@yesql.se](https://postgr.es/m/A05874AE-8771-4C61-A24E-0B6249B8F3C2@yesql.se)
[https://git.postgresql.org/pg/commitdiff/b4d5b458e6fb4c861f92888753deea6a5005a68d](https://git.postgresql.org/pg/commitdiff/b4d5b458e6fb4c861f92888753deea6a5005a68d)

Tom Lane pushed:

- Fix list-munging bug that broke SQL function result coercions. Since commit
913bbd88d, check\_sql\_fn\_retval() can either insert type coercion steps in-line
in the Query that produces the SQL function's results, or generate a new
top-level Query to perform the coercions, if modifying the Query's output
in-place wouldn't be safe. However, it appears that the latter case has never
actually worked, because the code tried to inject the new Query back into the
query list it was passed ... which is not the list that will be used for later
processing when we execute the SQL function "normally" (without inlining it).
So we ended up with no coercion happening at run-time, leading to wrong
results or crashes depending on the datatypes involved. While the regression
tests look like they cover this area well enough, through a huge bit of bad
luck all the test cases that exercise the separate-Query path were checking
either inline-able cases (which accidentally didn't have the bug) or cases
that are no-ops at runtime (e.g., varchar to text), so that the failure to
perform the coercion wasn't obvious. The fact that the cases that don't work
weren't allowed at all before v13 probably contributed to not noticing the
problem sooner, too. To fix, get rid of the separate "flat" list of Query
nodes and instead pass the real two-level list that is going to be used later.
I chose to make the same change in check\_sql\_fn\_statements(), although that
has no actual bug, just so that we don't need that data structure at all.
This is an API change, as evidenced by the adjustments needed to callers
outside functions.c. That's a bit scary to be doing in a released branch, but
so far as I can tell from a quick search, there are no outside callers of
these functions (and they are sufficiently specific to our semantics for
SQL-language functions that it's not apparent why any extension would need to
call them). In any case, v13 already changed the API of check\_sql\_fn\_retval()
compared to prior branches. Per report from pinker. Back-patch to v13 where
this code came in. Discussion:
[https://postgr.es/m/1603050466566-0.post@n3.nabble.com](https://postgr.es/m/1603050466566-0.post@n3.nabble.com)
[https://git.postgresql.org/pg/commitdiff/c8ab9701791e22f7a8e1badf362654db179c9703](https://git.postgresql.org/pg/commitdiff/c8ab9701791e22f7a8e1badf362654db179c9703)

- Fix connection string handling in src/bin/scripts/ programs. When told to
process all databases, clusterdb, reindexdb, and vacuumdb would reconnect by
replacing their --maintenance-db parameter with the name of the target
database. If that parameter is a connstring (which has been allowed for a
long time, though we failed to document that before this patch), we'd lose any
other options it might specify, for example SSL or GSS parameters, possibly
resulting in failure to connect. Thus, this is the same bug as commit
a45bc8a4f fixed in pg\_dump and pg\_restore. We can fix it in the same way, by
using libpq's rules for handling multiple "dbname" parameters to add the
target database name separately. I chose to apply the same refactoring
approach as in that patch, with a struct to handle the command line parameters
that need to be passed through to connectDatabase. (Maybe someday we can
unify the very similar functions here and in pg\_dump/pg\_restore.) Per Peter
Eisentraut's comments on bug \#16604. Back-patch to all supported branches.
Discussion: [https://postgr.es/m/16604-933f4b8791227b15@postgresql.org](https://postgr.es/m/16604-933f4b8791227b15@postgresql.org)
[https://git.postgresql.org/pg/commitdiff/8e5793ab60bba65ffaa0f2237b39c9580d8972c7](https://git.postgresql.org/pg/commitdiff/8e5793ab60bba65ffaa0f2237b39c9580d8972c7)

- Remove the option to build thread\_test.c outside configure. Theoretically one
could go into src/test/thread and build/run this program there. In practice,
that hasn't worked since 96bf88d52, and probably much longer on some platforms
(likely including just the sort of hoary leftovers where this test might be of
interest). While it wouldn't be too hard to repair the breakage, the fact that
nobody has noticed for two years shows that there is zero usefulness in
maintaining this build pathway. Let's get rid of it and decree that
thread\_test.c is \*only\* meant to be built/used in configure. Given that
decision, it makes sense to put thread\_test.c under config/ and get rid of
src/test/thread altogether, so that's what I did. In passing, update
src/test/README, which had been ignored by some not-so-recent additions of
subdirectories. Discussion:
[https://postgr.es/m/227659.1603041612@sss.pgh.pa.us](https://postgr.es/m/227659.1603041612@sss.pgh.pa.us)
[https://git.postgresql.org/pg/commitdiff/8a2121185b26d7eb439860c7a96a08c0aa3c8508](https://git.postgresql.org/pg/commitdiff/8a2121185b26d7eb439860c7a96a08c0aa3c8508)

- Fix connection string handling in psql's \connect command. psql's \connect
claims to be able to re-use previous connection parameters, but in fact it
only re-uses the database name, user name, host name (and possibly hostaddr,
depending on version), and port. This is problematic for assorted use cases.
Notably, pg\_dump[all] emits "\connect databasename" commands which we would
like to have re-use all other parameters. If such a script is loaded in a
psql run that initially had "-d connstring" with some non-default parameters,
those other parameters would be lost, potentially causing connection failure.
(Thus, this is the same kind of bug addressed in commits a45bc8a4f and
8e5793ab6, although the details are much different.) To fix, redesign
do\_connect() so that it pulls out all properties of the old PGconn using
PQconninfo(), and then replaces individual properties in that array. In the
case where we don't wish to re-use anything, get libpq's default settings
using PQconndefaults() and replace entries in that, so that we don't need
different code paths for the two cases. This does result in an additional
behavioral change for cases where the original connection parameters allowed
multiple hosts, say "psql -h host1,host2", and the \connect request allows
re-use of the host setting. Because the previous coding relied on PQhost(),
it would only permit reconnection to the same host originally selected.
Although one can think of scenarios where that's a good thing, there are
others where it is not. Moreover, that behavior doesn't seem to meet the
principle of least surprise, nor was it documented; nor is it even clear it
was intended, since that coding long pre-dates the addition of multi-host
support to libpq. Hence, this patch is content to drop it and re-use the host
list as given. Per Peter Eisentraut's comments on bug \#16604. Back-patch to
all supported branches. Discussion:
[https://postgr.es/m/16604-933f4b8791227b15@postgresql.org](https://postgr.es/m/16604-933f4b8791227b15@postgresql.org)
[https://git.postgresql.org/pg/commitdiff/85c54287af56fe351b53913ea2b81e9d6145f964](https://git.postgresql.org/pg/commitdiff/85c54287af56fe351b53913ea2b81e9d6145f964)

- Clean up some unpleasant behaviors in psql's \connect command. The check for
whether to complain about not having an old connection to get parameters from
was seriously out of date: it had not been rethought when we invented
connstrings, nor when we invented the -reuse-previous option. Replace it with
a check that throws an error if reuse-previous is active and we lack an old
connection to reuse. While that doesn't move the goalposts very far in terms
of easing reconnection after a server crash, at least it's consistent. If the
user specifies a connstring plus additional parameters (which is invalid per
the documentation), the extra parameters were silently ignored. That seems
like it could be really confusing, so let's throw a syntax error instead.
Teach the connstring code path to re-use the old connection's password in the
same cases as the old-style-syntax code path would, ie if we are reusing
parameters and the values of username, host/hostaddr, and port are not being
changed. Document this behavior, too, since it was unmentioned before. Also
simplify the implementation a bit, giving rise to two new and useful
properties: if there's a "password=xxx" in the connstring, we'll use it not
ignore it, and by default (i.e., except with --no-password) we will prompt for
a password if the re-used password or connstring password doesn't work. The
previous code just failed if the re-used password didn't work. Given the
paucity of field complaints about these issues, I don't think that they rise
to the level of back-patchable bug fixes, and in any case they might represent
undesirable behavior changes in minor releases. So no back-patch.
Discussion: [https://postgr.es/m/235210.1603321144@sss.pgh.pa.us](https://postgr.es/m/235210.1603321144@sss.pgh.pa.us)
[https://git.postgresql.org/pg/commitdiff/94929f1cf6cb3ea070d0919d1303379b525a72a3](https://git.postgresql.org/pg/commitdiff/94929f1cf6cb3ea070d0919d1303379b525a72a3)

- Add documentation and tests for quote marks in ECPG literal queries. ECPG's
PREPARE ... FROM and EXECUTE IMMEDIATE can optionally take the target query as
a simple literal, rather than the more usual string-variable reference. This
was previously documented as being a C string literal, but that's a lie in one
critical respect: you can't write a data double quote as \" in such literals.
That's because the lexer is in SQL mode at this point, so it'll parse
double-quoted strings as SQL identifiers, within which backslash is not
special, so \" ends the literal. I looked into making this work as
documented, but getting the lexer to switch behaviors at just the right point
is somewhere between very difficult and impossible. It's not really worth the
trouble, because these cases are next to useless: if you have a fixed SQL
statement to execute or prepare, you might as well write it as a direct EXEC
SQL, saving the messiness of converting it into a string literal and gaining
the opportunity for compile-time SQL syntax checking. Instead, let's just
document (and test) the workaround of writing a double quote as an octal
escape (\042) in such cases. There's no code behavioral change here, so in
principle this could be back-patched, but it's such a niche case I doubt it's
worth the trouble. Per report from 1250kv. Discussion:
[https://postgr.es/m/673825.1603223178@sss.pgh.pa.us](https://postgr.es/m/673825.1603223178@sss.pgh.pa.us)
[https://git.postgresql.org/pg/commitdiff/c16a1bbcf498f0aa053a3e55008f57d7f67357dd](https://git.postgresql.org/pg/commitdiff/c16a1bbcf498f0aa053a3e55008f57d7f67357dd)

- Avoid premature de-doubling of quote marks in ECPG strings. If you write the
literal 'abc''def' in an EXEC SQL command, that will come out the other end as
'abc'def', triggering a syntax error in the backend. Likewise, "abc""def" is
reduced to "abc"def" which is wrong syntax for a quoted identifier. The cause
is that the lexer thinks it should emit just one quote mark, whereas what it
really should do is keep the string as-is. Add some docs and test cases, too.
Although this seems clearly a bug, I fear users wouldn't appreciate changing
it in minor releases. Some may well be working around it by applying an extra
doubling of affected quotes, as for example sql/dyntest.pgc has been doing.
Per investigation of a report from 1250kv, although this isn't exactly what
he/she was on about. Discussion:
[https://postgr.es/m/673825.1603223178@sss.pgh.pa.us](https://postgr.es/m/673825.1603223178@sss.pgh.pa.us)
[https://git.postgresql.org/pg/commitdiff/3dfb1942d9b8748b93094a430289e2f7f3b3ae0d](https://git.postgresql.org/pg/commitdiff/3dfb1942d9b8748b93094a430289e2f7f3b3ae0d)

- Sync our copy of the timezone library with IANA release tzcode2020d. There's
no functional change at all here, but I'm curious to see whether this change
successfully shuts up Coverity's warning about a useless strcmp(), which
appeared with the previous update. Discussion:
[http://mm.icann.org/pipermail/tz/2020-October/029370.html](http://mm.icann.org/pipermail/tz/2020-October/029370.html)
[https://git.postgresql.org/pg/commitdiff/c5054da0d7ee59a93ca6d7f7fc669ef0210dbad3](https://git.postgresql.org/pg/commitdiff/c5054da0d7ee59a93ca6d7f7fc669ef0210dbad3)

- Update time zone data files to tzdata release 2020d. DST law changes in
Palestine, with a whopping 120 hours' notice. Also some historical corrections
for Palestine.
[https://git.postgresql.org/pg/commitdiff/7d6d6bce43c60bb7b77237e2cc6ab845646b911f](https://git.postgresql.org/pg/commitdiff/7d6d6bce43c60bb7b77237e2cc6ab845646b911f)

- Fix broken XML formatting in EXPLAIN output for incremental sorts. The
ExplainCloseGroup arguments for incremental sort usage data didn't match the
corresponding ExplainOpenGroup. This only matters for XML-format output,
which is probably why we'd not noticed. Daniel Gustafsson, per bug \#16683
from Frits Jalvingh Discussion:
[https://postgr.es/m/16683-8005033324ad34e9@postgresql.org](https://postgr.es/m/16683-8005033324ad34e9@postgresql.org)
[https://git.postgresql.org/pg/commitdiff/87a174c0e77eed0bec7d53ef6d470d60335f6444](https://git.postgresql.org/pg/commitdiff/87a174c0e77eed0bec7d53ef6d470d60335f6444)

- Fix portability issues in new amcheck test. The tests added by commit
866e24d47 failed on big-endian machines due to lack of attention to endianness
considerations. Fix that. While here, improve a few small cosmetic things,
such as running it through perltidy. Mark Dilger Discussion:
[https://postgr.es/m/30B8E99A-2D9C-48D4-A55C-741C9D5F1563@enterprisedb.com](https://postgr.es/m/30B8E99A-2D9C-48D4-A55C-741C9D5F1563@enterprisedb.com)
[https://git.postgresql.org/pg/commitdiff/860593ec3bd15e8969effdfcb5cbd98c561dd722](https://git.postgresql.org/pg/commitdiff/860593ec3bd15e8969effdfcb5cbd98c561dd722)

- Allow psql to re-use connection parameters after a connection loss. Instead of
immediately PQfinish'ing a dead connection, save it aside so that we can still
extract its parameters for \connect attempts. (This works because PQconninfo
doesn't care whether the PGconn is in CONNECTION\_BAD state.) This allows
developers to reconnect with just \c after a database crash and restart. It's
tempting to use the same approach instead of closing the old connection after
a failed non-interactive \connect command. However, that would not be very
safe: consider a script containing \c db1 user1 live\_server \c db2 user2
dead\_server \c db3 The script would be expecting to connect to db3 at
dead\_server, but if we re-use parameters from the first connection then it
might successfully connect to db3 at live\_server. This'd defeat the goal of
not letting a script accidentally execute commands against the wrong database.
Discussion: [https://postgr.es/m/38464.1603394584@sss.pgh.pa.us](https://postgr.es/m/38464.1603394584@sss.pgh.pa.us)
[https://git.postgresql.org/pg/commitdiff/1b62d0fb3e50ede570d0d4e4a2be69d5645b48a7](https://git.postgresql.org/pg/commitdiff/1b62d0fb3e50ede570d0d4e4a2be69d5645b48a7)

- Fix more portability issues in new amcheck code. verify\_heapam() wasn't being
careful to sanity-check tuple line pointers before using them, resulting in
SIGBUS on alignment-picky architectures. Fix that, add some more test
coverage. Mark Dilger, some tweaking by me Discussion:
[https://postgr.es/m/30B8E99A-2D9C-48D4-A55C-741C9D5F1563@enterprisedb.com](https://postgr.es/m/30B8E99A-2D9C-48D4-A55C-741C9D5F1563@enterprisedb.com)
[https://git.postgresql.org/pg/commitdiff/321633e17b07968e68ca5341429e2c8bbf15c331](https://git.postgresql.org/pg/commitdiff/321633e17b07968e68ca5341429e2c8bbf15c331)

- Fix ancient bug in ecpg's pthread\_once() emulation for Windows. We must not
set the "done" flag until after we've executed the initialization function.
Otherwise, other threads can fall through the initial unlocked test before
initialization is really complete. This has been seen to cause rare failures
of ecpg's thread/descriptor test, and it could presumably cause other sorts of
misbehavior in threaded ECPG-using applications, since ecpglib relies on
pthread\_once() in several places. Diagnosis and patch by me, based on
investigation by Alexander Lakhin. Back-patch to all supported branches (the
bug dates to 2007). Discussion:
[https://postgr.es/m/16685-d6cd241872c101d3@postgresql.org](https://postgr.es/m/16685-d6cd241872c101d3@postgresql.org)
[https://git.postgresql.org/pg/commitdiff/21d36747d4fafe16539a0c55ebb91a01e4053e3c](https://git.postgresql.org/pg/commitdiff/21d36747d4fafe16539a0c55ebb91a01e4053e3c)

- Fix corner case for a BEFORE ROW UPDATE trigger returning OLD. If the old row
has any "missing" attributes that are supposed to be retrieved from an
associated tuple descriptor, the wrong things happened because the trigger
result is shoved directly into an executor slot that lacks the
missing-attribute data. Notably, CHECK-constraint verification would
incorrectly see those columns as NULL, and so would RETURNING-list evaluation.
Band-aid around this by forcibly expanding the tuple before passing it to the
trigger function. (IMO it was a fundamental misdesign to put the
missing-attribute data into tuple constraints, which so much of the system
considers to be optional. But we're probably stuck with that now, and will
have to continue to apply band-aids as we find other places with similar
issues.) Back-patch to v12. v11 would also have the issue, except that
commit 920311ab1 already applied a similar band-aid. That forced expansion in
more cases than seem really necessary, though, so this isn't a directly
equivalent fix. Amit Langote, with some cosmetic changes by me Discussion:
[https://postgr.es/m/16644-5da7ef98a7ac4545@postgresql.org](https://postgr.es/m/16644-5da7ef98a7ac4545@postgresql.org)
[https://git.postgresql.org/pg/commitdiff/ba9f18abd3650e385e9a35df7145a7c38af17e92](https://git.postgresql.org/pg/commitdiff/ba9f18abd3650e385e9a35df7145a7c38af17e92)

- In INSERT/UPDATE, use the table's real tuple descriptor as target. Previously,
ExecInitModifyTable relied on ExecInitJunkFilter, and thence
ExecCleanTypeFromTL, to build the target descriptor from the query tlist.
While we just checked (in ExecCheckPlanOutput) that the tlist produces
compatible output, this is not a great substitute for the relation's actual
tuple descriptor that's available from the relcache. For one thing, dropped
columns will not be correctly marked attisdropped; it's a bit surprising that
we've gotten away with that this long. But the real reason for being
concerned with this is that using the table's descriptor means that the slot
will have correct attrmissing data, allowing us to revert the klugy fix of
commit ba9f18abd. (This commit undoes that one's changes in trigger.c, but
keeps the new test case.) Thus we can solve the bogus-trigger-tuple problem
with fewer cycles rather than more. No back-patch, since this doesn't fix any
additional bug, and it seems somewhat more likely to have unforeseen side
effects than ba9f18abd's narrow fix. Discussion:
[https://postgr.es/m/16644-5da7ef98a7ac4545@postgresql.org](https://postgr.es/m/16644-5da7ef98a7ac4545@postgresql.org)
[https://git.postgresql.org/pg/commitdiff/20d3fe9009ddbbbb3da3a2da298f922054b43f8c](https://git.postgresql.org/pg/commitdiff/20d3fe9009ddbbbb3da3a2da298f922054b43f8c)

- Doc: improve explanation of how to use our code coverage infrastructure. The
reference to running "make coverage" in a subdirectory was a bit obscure, so
clarify what happens when you do that. Do a little desultory copy-editing,
too. Per a question from Peter Smith. Discussion:
[https://postgr.es/m/CAHut+Pu0r3AjRSyu5E0v2-zRj8r24OSrkWs3fEBxOuaw1i8DKA@mail.gmail.com](https://postgr.es/m/CAHut+Pu0r3AjRSyu5E0v2-zRj8r24OSrkWs3fEBxOuaw1i8DKA@mail.gmail.com)
[https://git.postgresql.org/pg/commitdiff/8d132b2850d4c95ed5666a09b295dea645bbc884](https://git.postgresql.org/pg/commitdiff/8d132b2850d4c95ed5666a09b295dea645bbc884)

- Fix foreign-key selectivity estimation in the presence of constants.
get\_foreign\_key\_join\_selectivity() looks for join clauses that equate the two
sides of the FK constraint. However, if we have a query like "WHERE fktab.a =
pktab.a and fktab.a = 1", it won't find any such join clause, because
equivclass.c replaces the given clauses with "fktab.a = 1 and pktab.a = 1",
which can be enforced at the scan level, leaving nothing to be done for column
"a" at the join level. We can fix that expectation without much trouble, but
then a new problem arises: applying the foreign-key-based selectivity rule
produces a rowcount underestimate, because we're effectively double-counting
the selectivity of the "fktab.a = 1" clause. So we have to cancel that
selectivity out of the estimate. To fix, refactor process\_implied\_equality()
so that it can pass back the new RestrictInfo to its callers in equivclass.c,
allowing the generated "fktab.a = 1" clause to be saved in the
EquivalenceClass's ec\_derives list. Then it's not much trouble to dig out the
relevant RestrictInfo when we need to adjust an FK selectivity estimate.
(While at it, we can also remove the expensive use of
initialize\_mergeclause\_eclasses() to set up the new RestrictInfo's left\_ec and
right\_ec pointers. The equivclass.c code can set those basically for free.)
This seems like clearly a bug fix, but I'm hesitant to back-patch it, first
because there's some API/ABI risk for extensions and second because we're
usually loath to destabilize plan choices in stable branches. Per report from
Sigrid Ehrenreich. Discussion:
[https://postgr.es/m/1019549.1603770457@sss.pgh.pa.us](https://postgr.es/m/1019549.1603770457@sss.pgh.pa.us) Discussion:
[https://postgr.es/m/AM6PR02MB5287A0ADD936C1FA80973E72AB190@AM6PR02MB5287.eurprd02.prod.outlook.com](https://postgr.es/m/AM6PR02MB5287A0ADD936C1FA80973E72AB190@AM6PR02MB5287.eurprd02.prod.outlook.com)
[https://git.postgresql.org/pg/commitdiff/ad1c36b0709e47cdb3cc4abd6c939fe64279b63f](https://git.postgresql.org/pg/commitdiff/ad1c36b0709e47cdb3cc4abd6c939fe64279b63f)

- Don't use custom OID symbols in pg\_proc.dat. We have a perfectly good
convention for OID macros for built-in functions already, so making custom
symbols is just introducing unnecessary deviation from the convention. Remove
the one case that had snuck in, and add an error check in genbki.pl to
discourage future instances. Although this touches pg\_proc.dat, there's no
need for a catversion bump since the actual catalog data isn't changed. John
Naylor Discussion:
[https://postgr.es/m/CAFBsxsHpCbjfoddNGpnnnY5pHwckWfiYkMYSF74PmP1su0+ZOw@mail.gmail.com](https://postgr.es/m/CAFBsxsHpCbjfoddNGpnnnY5pHwckWfiYkMYSF74PmP1su0+ZOw@mail.gmail.com)
[https://git.postgresql.org/pg/commitdiff/36b93121436cbbf357974144068c23bac75154fa](https://git.postgresql.org/pg/commitdiff/36b93121436cbbf357974144068c23bac75154fa)

- Calculate extraUpdatedCols in query rewriter, not parser. It's unsafe to do
this at parse time because addition of generated columns to a table would not
invalidate stored rules containing UPDATEs on the table ... but there might
now be dependent generated columns that were not there when the rule was made.
This also fixes an oversight that rewriteTargetView failed to update
extraUpdatedCols when transforming an UPDATE on an updatable view. (Since the
new calculation is downstream of that, rewriteTargetView doesn't actually need
to do anything; but before, there was a demonstrable bug there.) In v13 and
HEAD, this leads to easily-visible bugs because (since commit c6679e4fc) we
won't recalculate generated columns that aren't listed in extraUpdatedCols.
In v12 this bitmap is mostly just used for trigger-firing decisions, so you'd
only notice a problem if a trigger cared whether a generated column had been
updated. I'd complained about this back in May, but then forgot about it
until bug \#16671 from Michael Paul Killian revived the issue. Back-patch to
v12 where this field was introduced. If existing stored rules contain any
extraUpdatedCols values, they'll be ignored because the rewriter will
overwrite them, so the bug will be fixed even for existing rules. (But note
that if someone were to update to 13.1 or 12.5, store some rules with UPDATEs
on tables having generated columns, and then downgrade to a prior minor
version, they might observe issues similar to what this patch fixes. That
seems unlikely enough to not be worth going to a lot of effort to fix.)
Discussion: [https://postgr.es/m/10206.1588964727@sss.pgh.pa.us](https://postgr.es/m/10206.1588964727@sss.pgh.pa.us) Discussion:
[https://postgr.es/m/16671-2fa55851859fb166@postgresql.org](https://postgr.es/m/16671-2fa55851859fb166@postgresql.org)
[https://git.postgresql.org/pg/commitdiff/ad77039fad0f4128b0e4a05ddbf5dbc3ab5f3fa4](https://git.postgresql.org/pg/commitdiff/ad77039fad0f4128b0e4a05ddbf5dbc3ab5f3fa4)

- Use mode "r" for popen() in psql's evaluate\_backtick(). In almost all other
places, we use plain "r" or "w" mode in popen() calls (the exceptions being
for COPY data). This one has been overlooked (possibly because it's buried in
a ".l" flex file?), but it's using PG\_BINARY\_R. Kensuke Okamura complained in
bug \#16688 that we fail to strip \r when stripping the trailing newline from a
backtick result string. That's true enough, but we'd also fail to convert
embedded \r\n cleanly, which also seems undesirable. Fixing the popen() mode
seems like the best way to deal with this. It's been like this for a long
time, so back-patch to all supported branches. Discussion:
[https://postgr.es/m/16688-c649c7b69cd7e6f8@postgresql.org](https://postgr.es/m/16688-c649c7b69cd7e6f8@postgresql.org)
[https://git.postgresql.org/pg/commitdiff/66f8687a8ff867f656de81e367314604d29dbd59](https://git.postgresql.org/pg/commitdiff/66f8687a8ff867f656de81e367314604d29dbd59)

- Doc: clean up verify\_heapam() documentation. I started with the intention of
just suppressing a PDF build warning by removing the example output, but ended
up doing more: correcting factual errors in the function's signature, moving a
bunch of generalized handwaving into the "Using amcheck Effectively" section
which seemed a better place for it, and improving wording and markup a little
bit. Discussion: [https://postgr.es/m/732904.1603728748@sss.pgh.pa.us](https://postgr.es/m/732904.1603728748@sss.pgh.pa.us)
[https://git.postgresql.org/pg/commitdiff/4c49d8fc15eeb1dc69b0ddb2d986a1884a5d7f5f](https://git.postgresql.org/pg/commitdiff/4c49d8fc15eeb1dc69b0ddb2d986a1884a5d7f5f)

- Doc: clean up pg\_relation\_check\_pages() documentation. Commit f2b883969 did
not get the memo about the new formatting style for tables documenting
built-in functions. I noticed because of a PDF build warning about an
overwidth table.
[https://git.postgresql.org/pg/commitdiff/b787d4ce6d910080065025bcd5f968544997271f](https://git.postgresql.org/pg/commitdiff/b787d4ce6d910080065025bcd5f968544997271f)

- Don't use custom OID symbols in pg\_type.dat, either. On the same reasoning as
in commit 36b931214, forbid using custom oid\_symbol macros in pg\_type as well
as pg\_proc, so that we always rely on the predictable macro names generated by
genbki.pl. We do continue to grant grandfather status to the names CASHOID
and LSNOID, although those are now considered deprecated aliases for the
preferred names MONEYOID and PG\_LSNOID. This is because there's likely to be
client-side code using the old names, and this bout of neatnik-ism doesn't
quite seem worth breaking client code. There might be a case for
grandfathering EVTTRIGGEROID, too, since externally-maintained PLs may
reference that symbol. But renaming such references to EVENT\_TRIGGEROID
doesn't seem like a particularly heavy lift --- we make far more significant
backend API changes in every major release. For now I didn't add that, but we
could reconsider if there's pushback. The other names changed here seem
pretty unlikely to have any outside uses. Again, we could add alias macros if
there are complaints, but for now I didn't. As before, no need for a
catversion bump. John Naylor Discussion:
[https://postgr.es/m/CAFBsxsHpCbjfoddNGpnnnY5pHwckWfiYkMYSF74PmP1su0+ZOw@mail.gmail.com](https://postgr.es/m/CAFBsxsHpCbjfoddNGpnnnY5pHwckWfiYkMYSF74PmP1su0+ZOw@mail.gmail.com)
[https://git.postgresql.org/pg/commitdiff/f90149e6285aaae6b48559afce1bd638ee26c33e](https://git.postgresql.org/pg/commitdiff/f90149e6285aaae6b48559afce1bd638ee26c33e)

- Stabilize timetz test across DST transitions. The timetz test cases I added in
commit a9632830b were unintentionally sensitive to whether or not DST is
active in the PST8PDT time zone. Thus, they'll start failing this coming
weekend, as reported by Bernhard M. Wiedemann in bug \#16689. Fortunately,
DST-awareness is not significant to the purpose of these test cases, so we can
just force them all to PDT (DST hours) to preserve stability of the results.
Back-patch to v10, as the prior patch was. Discussion:
[https://postgr.es/m/16689-57701daa23b377bf@postgresql.org](https://postgr.es/m/16689-57701daa23b377bf@postgresql.org)
[https://git.postgresql.org/pg/commitdiff/4a071afbd056282746a5bc9362e87f579a56402d](https://git.postgresql.org/pg/commitdiff/4a071afbd056282746a5bc9362e87f579a56402d)

- Doc: clarify description for pg\_constraint.convalidated. Jimmy Angelakos
Discussion:
[https://postgr.es/m/CABgVKCW\_zPnvFXn24FTF0299\_yU6+1p6JRUc0xpiZFWEXH1\_jg@mail.gmail.com](https://postgr.es/m/CABgVKCW\_zPnvFXn24FTF0299\_yU6+1p6JRUc0xpiZFWEXH1\_jg@mail.gmail.com)
[https://git.postgresql.org/pg/commitdiff/b401fa206d446f224ec4760f21e0a351816c97b3](https://git.postgresql.org/pg/commitdiff/b401fa206d446f224ec4760f21e0a351816c97b3)

- Fix assertion failure in check\_new\_partition\_bound(). Commit 6b2c4e59d was
overly confident about not being able to see a negative cmpval result from
partition\_range\_bsearch(). Adjust the code to cope with that. Report and
patch by Amul Sul; some additional cosmetic changes by me Discussion:
[https://postgr.es/m/CAAJ\_b97WCO=EyVA7fKzc86kKfojHXLU04\_zs7-7+yVzm=-1QkQ@mail.gmail.com](https://postgr.es/m/CAAJ\_b97WCO=EyVA7fKzc86kKfojHXLU04\_zs7-7+yVzm=-1QkQ@mail.gmail.com)
[https://git.postgresql.org/pg/commitdiff/970c05057593c2f5919a69b43fd917c4fa86f51c](https://git.postgresql.org/pg/commitdiff/970c05057593c2f5919a69b43fd917c4fa86f51c)

- Avoid null pointer dereference if error result lacks SQLSTATE. Although error
results received from the backend should always have a SQLSTATE field, ones
generated by libpq won't, making this code vulnerable to a crash after, say,
untimely loss of connection. Noted by Coverity. Oversight in commit
403a3d91c. Back-patch to 9.5, as that was.
[https://git.postgresql.org/pg/commitdiff/7f4235032f0d75ea1ad29b192d57fee3d8fe533e](https://git.postgresql.org/pg/commitdiff/7f4235032f0d75ea1ad29b192d57fee3d8fe533e)

Álvaro Herrera pushed:

- Fix ALTER TABLE .. ENABLE/DISABLE TRIGGER recursion. More precisely, correctly
handle the ONLY flag indicating not to recurse. This was implemented in
86f575948c77 by recursing in trigger.c, but that's the wrong place; use
ATSimpleRecursion instead, which behaves properly. However, because legacy
inheritance has never recursed in that situation, make sure to do that only
for new-style partitioning. I noticed this problem while testing a fix for
another bug in the vicinity. This has been wrong all along, so backpatch to
11. Discussion: [https://postgr.es/m/20201016235925.GA29829@alvherre.pgsql](https://postgr.es/m/20201016235925.GA29829@alvherre.pgsql)
[https://git.postgresql.org/pg/commitdiff/bbb927b4db9b3b449ccd0f76c1296de382a2f0c1](https://git.postgresql.org/pg/commitdiff/bbb927b4db9b3b449ccd0f76c1296de382a2f0c1)

- Use fast checkpoint in PostgresNode::backup(). Should cause tests to be a bit
faster
[https://git.postgresql.org/pg/commitdiff/831611b11c34ed0066633864f42ff67a10286aee](https://git.postgresql.org/pg/commitdiff/831611b11c34ed0066633864f42ff67a10286aee)

- Accept relations of any kind in LOCK TABLE. The restriction that only tables
and views can be locked by LOCK TABLE is quite arbitrary, since the underlying
mechanism can lock any relation type. Drop the restriction so that programs
such as pg\_dump can lock all relations they're interested in, preventing
schema changes that could cause a dump to fail after expending much effort.
Backpatch to 9.5. Author: Álvaro Herrera alvherre [AT] alvh.no-ip.org
Reviewed-by: Tom Lane tgl [AT] sss.pgh.pa.us Reported-by: Wells Oliver
wells.oliver [AT] gmail.com Discussion:
[https://postgr.es/m/20201021200659.GA32358@alvherre.pgsql](https://postgr.es/m/20201021200659.GA32358@alvherre.pgsql)
[https://git.postgresql.org/pg/commitdiff/59ab4ac32460a6a93b665f4e487d7ff64979ba4d](https://git.postgresql.org/pg/commitdiff/59ab4ac32460a6a93b665f4e487d7ff64979ba4d)

- pg\_dump: Lock all relations, not just plain tables. Now that LOCK TABLE can
take any relation type, acquire lock on all relations that are to be dumped.
This prevents schema changes or deadlock errors that could cause a dump to
fail after expending much effort. The server is tested to have the capability
and the feature disabled if it doesn't, so that a patched pg\_dump doesn't fail
when connecting to an unpatched server. Backpatch to 9.5. Author: Álvaro
Herrera alvherre [AT] alvh.no-ip.org Reviewed-by: Tom Lane tgl [AT] sss.pgh.pa.us
Reported-by: Wells Oliver wells.oliver [AT] gmail.com Discussion:
[https://postgr.es/m/20201021200659.GA32358@alvherre.pgsql](https://postgr.es/m/20201021200659.GA32358@alvherre.pgsql)
[https://git.postgresql.org/pg/commitdiff/403a3d91c841beabf3efd7bffddb47a2bce4481f](https://git.postgresql.org/pg/commitdiff/403a3d91c841beabf3efd7bffddb47a2bce4481f)

Robert Haas pushed:

- Extend amcheck to check heap pages. Mark Dilger, reviewed by Peter Geoghegan,
Andres Freund, Álvaro Herrera, Michael Paquier, Amul Sul, and by me. Some
last-minute cosmetic revisions by me. Discussion:
[http://postgr.es/m/12ED3DA8-25F0-4B68-937D-D907CFBF08E7@enterprisedb.com](http://postgr.es/m/12ED3DA8-25F0-4B68-937D-D907CFBF08E7@enterprisedb.com)
[https://git.postgresql.org/pg/commitdiff/866e24d47db1743dfcff5bd595b57e3a143f2cb1](https://git.postgresql.org/pg/commitdiff/866e24d47db1743dfcff5bd595b57e3a143f2cb1)

- Try to avoid a compiler warning about using fxid uninitialized. Mark Dilger,
with a couple of stray semicolons removed by me. Discussion:
[http://postgr.es/m/2A7DA1A8-C4AA-43DF-A985-3CA52F4DC775@enterprisedb.com](http://postgr.es/m/2A7DA1A8-C4AA-43DF-A985-3CA52F4DC775@enterprisedb.com)
[https://git.postgresql.org/pg/commitdiff/8bb0c9770e80fa79f15b27af8f3c3f2833225aa3](https://git.postgresql.org/pg/commitdiff/8bb0c9770e80fa79f15b27af8f3c3f2833225aa3)

Bruce Momjian pushed:

- doc: make blooms docs match reality. Parallel execution changed the way bloom
queries are executed, so update the EXPLAIN output, and restructure the docs
to be clearer and more accurate. Reported-by: Daniel Westermann Discussion:
[https://postgr.es/m/ZR0P278MB0122119FAE78721A694C30C8D2340@ZR0P278MB0122.CHEP278.PROD.OUTLOOK.COM](https://postgr.es/m/ZR0P278MB0122119FAE78721A694C30C8D2340@ZR0P278MB0122.CHEP278.PROD.OUTLOOK.COM)
Author: Daniel Westermann and me Backpatch-through: 9.6
[https://git.postgresql.org/pg/commitdiff/e9661f2b0f3310d8bacc3f0802b309f599300cfb](https://git.postgresql.org/pg/commitdiff/e9661f2b0f3310d8bacc3f0802b309f599300cfb)

- doc: simplify wording of function error affects. Reported-by:
bob(dot)henkel(at)gmail(dot)com Discussion:
[https://postgr.es/m/160324449781.693.8298142858847611071@wrigleys.postgresql.org](https://postgr.es/m/160324449781.693.8298142858847611071@wrigleys.postgresql.org)
Backpatch-through: 9.5
[https://git.postgresql.org/pg/commitdiff/2f17fe431860fc434084d99d8b789e6ec7e6977b](https://git.postgresql.org/pg/commitdiff/2f17fe431860fc434084d99d8b789e6ec7e6977b)

- docs: remove reference to src/tools/thread. This directory and the ability to
build the thread test independently were removed in commit 8a2121185b.
Reported-by: e(dot)indrupskaya(at)postgrespro(dot)ru Discussion:
[https://postgr.es/m/160379609706.24746.7506163279454026608@wrigleys.postgresql.org](https://postgr.es/m/160379609706.24746.7506163279454026608@wrigleys.postgresql.org)
Backpatch-through: 9.5
[https://git.postgresql.org/pg/commitdiff/4066b642909176aede03b6c64d29734ce2a34716](https://git.postgresql.org/pg/commitdiff/4066b642909176aede03b6c64d29734ce2a34716)

- Makefile comment: remove reference to tools/thread/thread\_test. You can't
compile thread\_test alone anymore, and the location moved too. Reported-by:
Tom Lane Discussion: [https://postgr.es/m/1062278.1603819969@sss.pgh.pa.us](https://postgr.es/m/1062278.1603819969@sss.pgh.pa.us)
Backpatch-through: 9.5
[https://git.postgresql.org/pg/commitdiff/8e5d1aef8eed26977739b662cddcea2de5efe834](https://git.postgresql.org/pg/commitdiff/8e5d1aef8eed26977739b662cddcea2de5efe834)

Andres Freund pushed:

- Centralize horizon determination for temp tables, fixing bug due to skew. This
fixes a bug in the edge case where, for a temp table, heap\_page\_prune() can
end up with a different horizon than heap\_vacuum\_rel(). Which can trigger
errors like "ERROR: cannot freeze committed xmax ...". The bug was introduced
due to interaction of a7212be8b9e "Set cutoff xmin more aggressively when
vacuuming a temporary table." with dc7420c2c92 "snapshot scalability: Don't
compute global horizons while building snapshots.". The problem is caused by
lazy\_scan\_heap() assuming that the only reason its HeapTupleSatisfiesVacuum()
call would return HEAPTUPLE\_DEAD is if the tuple is a HOT tuple, or if the
tuple's inserting transaction has aborted since the heap\_page\_prune() call.
But after a7212be8b9e that was also possible in other cases for temp tables,
because heap\_page\_prune() uses a different visibility test after dc7420c2c92.
The fix is fairly simple: Move the special case logic for temp tables from
vacuum\_set\_xid\_limits() to the infrastructure introduced in dc7420c2c92. That
ensures that the horizon used for pruning is at least as aggressive as the one
used by lazy\_scan\_heap(). The concrete horizon used for temp tables is
slightly different than the logic in dc7420c2c92, but should always be as
aggressive as before (see comments). A significant benefit to centralizing
the logic procarray.c is that now the more aggressive horizons for temp tables
does not just apply to VACUUM but also to e.g. HOT pruning and the nbtree
killtuples logic. Because isTopLevel is not needed by vacuum\_set\_xid\_limits()
anymore, I undid the the related changes from a7212be8b9e. This commit also
adds an isolation test ensuring that the more aggressive vacuuming and pruning
of temp tables keeps working. Debugged-By: Amit Kapila
amit.kapila16 [AT] gmail.com Debugged-By: Tom Lane tgl [AT] sss.pgh.pa.us
Debugged-By: Ashutosh Sharma ashu.coek88 [AT] gmail.com Author: Andres Freund
andres [AT] anarazel.de Discussion:
[https://postgr.es/m/20201014203103.72oke6hqywcyhx7s@alap3.anarazel.de](https://postgr.es/m/20201014203103.72oke6hqywcyhx7s@alap3.anarazel.de)
Discussion:
[https://postgr.es/m/20201015083735.derdzysdtqdvxshp@alap3.anarazel.de](https://postgr.es/m/20201015083735.derdzysdtqdvxshp@alap3.anarazel.de)
[https://git.postgresql.org/pg/commitdiff/94bc27b57680b4e757577e3f5b65dc32f96d33c1](https://git.postgresql.org/pg/commitdiff/94bc27b57680b4e757577e3f5b65dc32f96d33c1)

- Fix wrong data table horizon computation during backend startup. When
ComputeXidHorizons() was called before MyDatabaseOid is set, e.g. because a
dead row in a shared relation is encountered during InitPostgres(), the
horizon for normal tables was computed too aggressively, ignoring all backends
connected to a database. During subsequent pruning in a data table the too
aggressive horizon could end up still being used, possibly leading to still
needed tuples being removed. Not good. This is a bug in dc7420c2c92, which
the test added in 94bc27b5768 made visible, if run with force\_parallel\_mode
set to regress. In that case the bug is reliably triggered, because
"pruning\_query" is run in a parallel worker and the start of that parallel
worker is likely to encounter a dead row in pg\_database. The fix is trivial:
Compute a more pessimistic data table horizon if MyDatabaseId is not yet
known. Author: Andres Freund Discussion:
[https://postgr.es/m/20201029040030.p4osrmaywhqaesd4@alap3.anarazel.de](https://postgr.es/m/20201029040030.p4osrmaywhqaesd4@alap3.anarazel.de)
[https://git.postgresql.org/pg/commitdiff/1c7675a7a4265064a2c8e1ed02b2c042c2521664](https://git.postgresql.org/pg/commitdiff/1c7675a7a4265064a2c8e1ed02b2c042c2521664)

Noah Misch pushed:

- Reproduce debug\_query\_string\=\=NULL on parallel workers. Certain background
workers initiate parallel queries while debug\_query\_string\=\=NULL, at which
point they attempted strlen(NULL) and died to SIGSEGV. Older
debug\_query\_string observers allow NULL, so do likewise in these newer ones.
Back-patch to v11, where commit 7de4a1bcc56f494acbd0d6e70781df877dc8ecb5
introduced the first of these. Discussion:
[https://postgr.es/m/20201014022636.GA1962668@rfd.leadboat.com](https://postgr.es/m/20201014022636.GA1962668@rfd.leadboat.com)
[https://git.postgresql.org/pg/commitdiff/f90e80b9138355a51d2d5b5b63e1f89c4ba53325](https://git.postgresql.org/pg/commitdiff/f90e80b9138355a51d2d5b5b63e1f89c4ba53325)

- Set debug\_query\_string in worker\_spi. This makes elog.c emit the string, which
is good practice for a background worker that executes SQL strings. Reviewed
by Tom Lane. Discussion:
[https://postgr.es/m/20201014022636.GA1962668@rfd.leadboat.com](https://postgr.es/m/20201014022636.GA1962668@rfd.leadboat.com)
[https://git.postgresql.org/pg/commitdiff/d2246cde825e4e1a85408390c66367b85b51a233](https://git.postgresql.org/pg/commitdiff/d2246cde825e4e1a85408390c66367b85b51a233)

## Pending Patches

Peter Eisentraut sent in a patch to add a default\_result\_formats GUC, which
specifies the default result formats by data type for rows returned in the
extended query protocol when no result formats are specified in the Bind
message.

Álvaro Herrera sent in another revision of a patch to add tracing to libpq.

Julien Rouhaud sent in two revisions of a patch to handle = operator in pg\_trgm.

Álvaro Herrera sent in another revision of a patch to add batch and pipelining
support to libpq.

Craig Ringer sent in a patch to report libpq version and configuration.

Jakub Wartak sent in two revisions of a patch to add "IO read time" log message
to autoanalyze.

Alexander Korotkov and Andrey Borodin traded patches to use a shared lock, as
opposed to an exclusive control lock, in GetMultiXactIdMembers for offsets and
members, make the MultiXact local cache size configurable, add a conditional
variable to wait for next MultXact offset in corner case, and add GUCs to tune
MultiXact SLRUs.

Pavel Borisov sent in another revision of a patch to make GROUP BY more
efficient by making it possible to reorder the columns.

Peter Geoghegan sent in another revision of a patch to add sort\_template.h for
making fast sort functions, and add delete deduplication to nbtree.

Michaël Paquier sent in another revision of a patch to fix checksum verification
in base backups for zero page headers.

Ajin Cherian and Peter Smith traded patches to implement logical decoding for
two-phase transactions.

Zhenghua Lyu sent in a patch to consider the case when stanullfrac is 1.0 in
get\_variable\_numdistinct.

Konstantin Knizhnik sent in five more revisions of a patch to implement
compression in libpq.

Justin Pryzby sent in a patch to remove deprecated v8.2 containment operators.

Peter Eisentraut sent in another revision of a patch to support for writing
CREATE FUNCTION and CREATE PROCEDURE statements for language SQL with a function
body that conforms to the SQL standard and is portable to other implementations.

Ibrar Ahmed sent in another revision of a patch to implement temporal PK/FK
deletes and updates.

Justin Pryzby sent in another revision of a patch to pg\_dump to output
DISABLE/ENABLE for child triggers if their state does not match their parent.

Arseny Sher sent in a patch to fix a use-after-free.

Amit Kapila sent in a patch to make some minor improvements to the description
of spilled counters in pg\_stat\_replication\_slots view.

Dilip Kumar sent in three more revisions of a patch to implement custom
compression methods for tables and ways to change same.

John Naylor sent in two revisions of a patch to use the standard symbol for the
builtin function heap\_tableam\_handler().

Michaël Paquier sent in another revision of a patch to enable online checksum
verification in the backend.

Greg Nancarrow sent in two more revisions of a patch to enable parallel SELECT
for "INSERT INTO ... SELECT ...", where it is safe to do so, and enable parallel
INSERT and/or SELECT for "INSERT INTO ... SELECT ...", where it is safe to do
so.

Masahiko Sawada and Bertrand Drouvot traded patches to log the standby recovery
conflict waits.

Vigneshwaran C sent in another revision of a patch to parallelize parts of COPY.

Daniel Gustafsson sent in four more revisions of a patch to support NSS as a
libpq TLS backend.

Michaël Paquier, Fabrízio de Royes Mello, and Nikolay Samokhvalov traded patches
to add important information about ANALYZE after creating an expressional index.

Justin Pryzby sent in a patch to implement CLUSTER for partitioned tables,
reserve indisclustered on children of clustered, partitioned indexes, and
invalidate indisclustered when attaching unclustered indexes.

Justin Pryzby sent in a WIP patch to implement DROP INDEX CONCURRENTLY for
partitioned indexes.

Hayato Kuroda sent in a patch to fix an issue in pgbench where queries were
firing before connections were established by adding a call to
pthread\_barrier\_wait().

Hou Zhijie sent in a patch to enhance libpq to support multiple host for non hot
standby.

Pavel Borisov sent in a patch to fix for bug when running Valgrind on OSX.

Amul Sul sent in another revision of a patch to implement ALTER SYSTEM READ
{ONLY|WRITE}.

Justin Pryzby sent in another revision of a patch to ensure that all the pg\_ls\_\*
functions show directories and shared filesets.

Tsutomu Yamada sent in four revisions of a patch to add list extended stats to
psql's \dX and \dX+.

Kirk Jamison sent in another revision of a patch to make dropping relation
buffers more efficient using dlist.

Bruce Momjian sent in two more revisions of a patch to implement a key
management system.

Xiao Bai Yan sent in another revision of a patch to extend the shared tuple
store and add batch store module, and in passing, use atomic operations instead
of LWLock for shared tuple stores when getting the next read page, and use this
to implement parallel distinct union aggregates and grouping sets support using
batch hash aggregate.

Amit Kapila sent in another revision of a patch to track statistics for
streaming of changes from ReorderBuffer.

John Naylor sent in a patch to forbid custom pg type symbols.

Heikki Linnakangas and Andres Freund traded patches to deduplicate aggregates
and transition functions in the planner.

Amit Langote sent in another revision of a patch to rearrange partition routing
layering in nodeModifyTable.c.

Takamichi Osumi make it possible to disable WAL logging to speed up data
loading.

John Naylor sent in four revisions of a patch to document the fact that
pg\_settings view doesn't display custom options.

Bharath Rupireddy sent in two more revisions of a patch to adds two new
functions: pg\_terminate\_backend(pid, wait, timeout), which terminates and waits
or times out for a given backend, and pg\_wait\_backend(pid, timeout), which
checks for the existence of the backend with a given PID and waits or times out
until it goes away.

Euler Taveira de Oliveira sent in a patch to control temporary file removal
after a crash.

Vigneshwaran C sent in four revisions of a patch to add a log message to include
GSS authentication, encryption & principal information.

Noah Misch sent in another revision of a patch to prevent excess
SimpleLruTruncate() deletion, and unlink less in SimpleLruTruncate(), as
insurance against bugs.

Takamichi Osumi sent in another revision of a patch to implement CREATE OR
REPLACE TRIGGER.

Justin Pryzby sent in another revision of a patch to pg\_dump: Allow child
partitions to be independently restored, even if the parent doesn't exist, or
has missing/incompatible columns.

Justin Pryzby sent in a patch to retire pg\_standby and pg\_archivecleanup.

Masahiko Sawada sent in a PoC patch that adds some xx\_vacuum GUC parameters to
control the method of recording TIDs.

Michaël Paquier sent in another revision of a patch to fix page verifications in
base backups.

Masahiro Ikeda sent in another revision of a patch to add statistics to the
pg\_stat\_wal view.

Justin Pryzby sent in another revision of a patch to allow INSERT SELECT to use
a BulkInsertState, and use same to make INSERT SELECT use multi\_insert.

Daniel Gustafsson sent in another revision of a patch to clean up
contrib/sslinfo and OpenSSL error handling.

Amit Langote sent in another revision of a patch to make update/delete of
inheritance trees scale better.

Tomáš Vondra and James Coleman traded patches to fix
get\_useful\_pathkeys\_for\_relation for volatile exprs below joins, and add
comments explaining where projections aren't necessary.

Thomas Munro sent in two more revisions of a patch to implement collation
versioning.

Amit Langote sent in another revision of a patch to call BeginDirectModify from
ExecInitModifyTable, and initialize result relation information lazily.

Amit Langote sent in another revision of a patch to fix a bug with RETURNING
when UPDATE moves tuples.

Ashutosh Bapat sent in two more revisions of a patch to enumize logical
replication message actions.

Jimmy Angelakos sent in a patch to clarify the wording for convalidated in
pg\_constraint.

John Naylor sent in a patch to document deviation from the standard on REVOKE
ROLE.

Andrey Borodin sent in another revision of a patch to add Sortsupport for
sorting GiST build for gist\_btree types.

Tomáš Vondra sent in another revision of a patch to extend range type operators
to cope with elements.

Heikki Linnakangas sent in a WIP patch to find all line-endings in COPY in
chunks.

Jürgen Purtz, Erik Rijkers, and Justin Pryzby traded patches to add a new
chapter to the tutorial.

Tomáš Vondra sent in another revision of a patch to add extra statistics to
explain for Nested Loop.

Justin Pryzby sent in another revision of a patch to fix typos.

Justin Pryzby sent in another revision of a patch to allow CREATE INDEX
CONCURRENTLY on partitioned table, add a SKIPVALID flag for more integration,
and make eindexPartitions() set indisvalid.

Andrey V. Lepikhov sent in another revision of a patch to remove unneeded
self-joins.

Ranier Vilela sent in a patch to fix a dereference before NULL check in
src/backend/storage/ipc/latch.c.

Ranier Vilela sent in a patch to fix an explicit NULL dereference in
src/backend/utils/adt/ruleutils.c.

Justin Pryzby sent in another revision of a patch to allow CLUSTER, VACUUM FULL
and REINDEX to change tablespace on the fly.

Tom Lane sent in a patch to add a new CoercionForm variant, COERCE\_SQL\_SYNTAX.

Greg Sabino Mullane sent in another revision of a patch to help psql's \df
choose functions by their arguments.

Tom Lane sent in a patch to get Gen\_fmgrtab.pl to generate macros for all
pg\_proc entries.

Tom Lane sent in a patch to get rid of aggregate\_dummy().