PostgreSQL Weekly News - May 16, 2021

# PostgreSQL Weekly News - May 16, 2021

Security releases 13.3, 12.7, 11.12, 10.17, and 9.6.22 are out. Please
upgrade ASAP. The 9.6 series will stop getting fixes on November 11, 2021. Plan
major version upgrades now.
[https://www.postgresql.org/about/news/postgresql-133-127-1112-1017-and-9622-released-2210/](https://www.postgresql.org/about/news/postgresql-133-127-1112-1017-and-9622-released-2210/)

Person of the week: [https://postgresql.life/post/laurenz_albe/](https://postgresql.life/post/laurenz_albe/)

# PostgreSQL Jobs for May

[https://archives.postgresql.org/pgsql-jobs/2021-05/](https://archives.postgresql.org/pgsql-jobs/2021-05/)

# PostgreSQL in the News

Planet PostgreSQL: [https://planet.postgresql.org/](https://planet.postgresql.org/)

PostgreSQL Weekly News is brought to you this week by David Fetter

Submit news and announcements by Sunday at 3:00pm PST8PDT to david(at)fetter(dot)org(dot)

# Applied Patches

Tom Lane pushed:

- Improve comments about USE_VALGRIND in pg_config_manual.h. These comments left
the impression that USE_VALGRIND isn't really essential for valgrind testing.
But that's wrong, as I learned the hard way today. Discussion:
[https://postgr.es/m/512778.1620588546@sss.pgh.pa.us](https://postgr.es/m/512778.1620588546@sss.pgh.pa.us)
[https://git.postgresql.org/pg/commitdiff/8dc3d68cbe676deb5e74d1b1b565f57fffaf107e](https://git.postgresql.org/pg/commitdiff/8dc3d68cbe676deb5e74d1b1b565f57fffaf107e)

- Prevent integer overflows in array subscripting calculations. While we were
(mostly) careful about ensuring that the dimensions of arrays aren't large
enough to cause integer overflow, the lower bound values were generally not
checked. This allows situations where lower_bound + dimension overflows an
integer. It seems that that's harmless so far as array reading is concerned,
except that array elements with subscripts notionally exceeding INT_MAX are
inaccessible. However, it confuses various array-assignment logic, resulting
in a potential for memory stomps. Fix by adding checks that array lower
bounds aren't large enough to cause lower_bound + dimension to overflow.
(Note: this results in disallowing cases where the last subscript position
would be exactly INT_MAX. In principle we could probably allow that, but
there's a lot of code that computes lower_bound + dimension and would need
adjustment. It seems doubtful that it's worth the trouble/risk to allow it.)
Somewhat independently of that, array_set_element() was careless about
possible overflow when checking the subscript of a fixed-length array,
creating a different route to memory stomps. Fix that too. Security:
CVE-2021-32027
[https://git.postgresql.org/pg/commitdiff/f02b9085ad2f6fefd9c5cdf85579cb9f0ff0f0ea](https://git.postgresql.org/pg/commitdiff/f02b9085ad2f6fefd9c5cdf85579cb9f0ff0f0ea)

- Fix mishandling of resjunk columns in ON CONFLICT ... UPDATE tlists. It's
unusual to have any resjunk columns in an ON CONFLICT ... UPDATE list, but it
can happen when MULTIEXPR_SUBLINK SubPlans are present. If it happens, the ON
CONFLICT UPDATE code path would end up storing tuples that include the values
of the extra resjunk columns. That's fairly harmless in the short run, but if
new columns are added to the table then the values would become accessible,
possibly leading to malfunctions if they don't match the datatypes of the new
columns. This had escaped notice through a confluence of missing sanity
checks, including: `*` There's no cross-check that a tuple presented to
heap_insert or heap_update matches the table rowtype. While it's difficult to
check that fully at reasonable cost, we can easily add assertions that there
aren't too many columns. `*` The output-column-assignment cases in
execExprInterp.c lacked any sanity checks on the output column numbers, which
seems like an oversight considering there are plenty of assertion checks on
input column numbers. Add assertions there too. `*` We failed to apply
nodeModifyTable's ExecCheckPlanOutput() to the ON CONFLICT UPDATE tlist. That
wouldn't have caught this specific error, since that function is chartered to
ignore resjunk columns; but it sure seems like a bad omission now that we've
seen this bug. In HEAD, the right way to fix this is to make the processing
of ON CONFLICT UPDATE tlists work the same as regular UPDATE tlists now do,
that is don't add "SET x = x" entries, and use ExecBuildUpdateProjection to
evaluate the tlist and combine it with old values of the not-set columns.
This adds a little complication to ExecBuildUpdateProjection, but allows
removal of a comparable amount of now-dead code from the planner. In the back
branches, the most expedient solution seems to be to (a) use an output slot
for the ON CONFLICT UPDATE projection that actually matches the target table,
and then (b) invent a variant of ExecBuildProjectionInfo that can be told to
not store values resulting from resjunk columns, so it doesn't try to store
into nonexistent columns of the output slot. (We can't simply ignore the
resjunk columns altogether; they have to be evaluated for MULTIEXPR_SUBLINK to
work.) This works back to v10. In 9.6, projections work much differently and
we can't cheaply give them such an option. The 9.6 version of this patch
works by inserting a JunkFilter when it's necessary to get rid of resjunk
columns. In addition, v11 and up have the reverse problem when trying to
perform ON CONFLICT UPDATE on a partitioned table. Through a further
oversight, adjust_partition_tlist() discarded resjunk columns when re-ordering
the ON CONFLICT UPDATE tlist to match a partition. This accidentally prevented
the storing-bogus-tuples problem, but at the cost that MULTIEXPR_SUBLINK cases
didn't work, typically crashing if more than one row has to be updated. Fix
by preserving resjunk columns in that routine. (I failed to resist the
temptation to add more assertions there too, and to do some minor code
beautification.) Per report from Andres Freund. Back-patch to all supported
branches. Security: CVE-2021-32028
[https://git.postgresql.org/pg/commitdiff/049e1e2edb06854d7cd9460c22516efaa165fbf8](https://git.postgresql.org/pg/commitdiff/049e1e2edb06854d7cd9460c22516efaa165fbf8)

- Replace opr_sanity test's binary_coercible() function with C code.
opr_sanity's binary_coercible() function has always been meant to match the
parser's notion of binary coercibility, but it also has always been a rather
poor approximation of the parser's real rules (as embodied in
IsBinaryCoercible()). That hasn't bit us so far, but it's predictable that it
will eventually. It also now emerges that implementing this check in plpgsql
performs absolutely horribly in clobber-cache-always testing. (Perhaps we
could do something about that, but I suspect it just means that plpgsql is
exploiting catalog caching to the hilt.) Hence, let's replace
binary_coercible() with a C shim that directly invokes IsBinaryCoercible(),
eliminating both the semantic hazard and the performance issue. Most of
regress.c's C functions are declared in create_function_1, but we can't simply
move that to before opr_sanity/type_sanity since those tests would complain
about the resulting shell types. I chose to split it into create_function_0
and create_function_1. Since create_function_0 now runs as part of a parallel
group while create_function_1 doesn't, reduce the latter to create just those
functions that opr_sanity and type_sanity would whine about. To make room for
create_function_0 in the second parallel group of tests, move tstypes to the
third parallel group. In passing, clean up some ordering deviations between
parallel_schedule and serial_schedule. Discussion:
[https://postgr.es/m/292305.1620503097@sss.pgh.pa.us](https://postgr.es/m/292305.1620503097@sss.pgh.pa.us)
[https://git.postgresql.org/pg/commitdiff/6303a5730914dfe6ef2709b28b225553315c573c](https://git.postgresql.org/pg/commitdiff/6303a5730914dfe6ef2709b28b225553315c573c)

- Get rid of the separate serial_schedule list of tests. Having to maintain two
lists of regression test scripts has proven annoyingly error-prone. We can
achieve the effect of the serial_schedule by running the parallel_schedule
with "--max_connections=1"; so do that and remove serial_schedule. This
causes cosmetic differences in the progress output, but it doesn't seem worth
restructuring pg_regress to avoid that. Discussion:
[https://postgr.es/m/899209.1620759506@sss.pgh.pa.us](https://postgr.es/m/899209.1620759506@sss.pgh.pa.us)
[https://git.postgresql.org/pg/commitdiff/1df3555acc78dedc3ca25eb5e83649b3da1f298f](https://git.postgresql.org/pg/commitdiff/1df3555acc78dedc3ca25eb5e83649b3da1f298f)

- Fix vcregress.pl's ancient misspelling of --max-connections. I copied the
existing spelling of "--max_connections", but that's just wrong :-(.
Evidently setting $ENV{MAX_CONNECTIONS} has never actually worked in this
script. Given the lack of complaints, it's probably not worth back-patching a
fix. Per buildfarm. Discussion:
[https://postgr.es/m/899209.1620759506@sss.pgh.pa.us](https://postgr.es/m/899209.1620759506@sss.pgh.pa.us)
[https://git.postgresql.org/pg/commitdiff/0b85fa93e4575183aa5a71ebe3c6bae8d97704ed](https://git.postgresql.org/pg/commitdiff/0b85fa93e4575183aa5a71ebe3c6bae8d97704ed)

- Reduce runtime of privileges.sql test under CLOBBER_CACHE_ALWAYS. Several
queries in the privileges regression test cause the planner to apply the
plpgsql function "leak()" to every element of the histogram for atest12.b.
Since commit 0c882e52a increased the size of that histogram to 10000 entries,
the test invokes that function over 100000 times, which takes an absolutely
unreasonable amount of time in clobber-cache-always mode. However, there's no
real reason why that has to be a plpgsql function: for the purposes of this
test, all that matters is that it not be marked leakproof. So we can replace
the plpgsql implementation with a direct call of int4lt, which has the same
behavior and is orders of magnitude faster. This is expected to cut several
hours off the buildfarm cycle time for CCA animals. It has some positive
impact in normal builds too, though that's probably lost in the noise.
Back-patch to v13 where 0c882e52a came in. Discussion:
[https://postgr.es/m/575884.1620626638@sss.pgh.pa.us](https://postgr.es/m/575884.1620626638@sss.pgh.pa.us)
[https://git.postgresql.org/pg/commitdiff/e135743ef07ea59088d09c459f41ee2eaabe95c3](https://git.postgresql.org/pg/commitdiff/e135743ef07ea59088d09c459f41ee2eaabe95c3)

- Initial pgindent and pgperltidy run for v14. Also "make reformat-dat-files".
The only change worthy of note is that pgindent messed up the formatting of
launcher.c's struct LogicalRepWorkerId, which led me to notice that that
struct wasn't used at all anymore, so I just took it out.
[https://git.postgresql.org/pg/commitdiff/def5b065ff22a16a80084587613599fe15627213](https://git.postgresql.org/pg/commitdiff/def5b065ff22a16a80084587613599fe15627213)

- Do pre-release housekeeping on catalog data. Run renumber_oids.pl to move
high-numbered OIDs down, as per pre-beta tasks specified by RELEASE_CHANGES.
For reference, the command was ./renumber_oids.pl --first-mapped-oid 8000
--target-oid 6150
[https://git.postgresql.org/pg/commitdiff/14472442861ca95cc9158518acdedf740c4bff55](https://git.postgresql.org/pg/commitdiff/14472442861ca95cc9158518acdedf740c4bff55)

- Doc: update bki.sgml's statements about OID ranges. Commit ab596105b neglected
to make the docs match the code.
[https://git.postgresql.org/pg/commitdiff/1f9b0e6938054515b8c9df545437c3d347eed683](https://git.postgresql.org/pg/commitdiff/1f9b0e6938054515b8c9df545437c3d347eed683)

- Double-space commands in system_constraints.sql/system_functions.sql.
Previously, any error reported by the backend while reading
system_constraints.sql would report the entire file, not just the particular
command it was working on. (Ask me how I know.) Likewise, there were chunks
of system_functions.sql that would be read as one command, which would be
annoying if anything failed there. The issue for system_constraints.sql is an
oversight in commit dfb75e478. I didn't try to trace down where the poor
formatting in system_functions.sql started, but it's certainly contrary to the
advice at the head of that file.
[https://git.postgresql.org/pg/commitdiff/7dde98728a2ef6d48ef397ee783dd130fdb34e6b](https://git.postgresql.org/pg/commitdiff/7dde98728a2ef6d48ef397ee783dd130fdb34e6b)

- Refactor CHECK_FOR_INTERRUPTS() to add flexibility. Split up
CHECK_FOR_INTERRUPTS() to provide an additional macro
INTERRUPTS_PENDING_CONDITION(), which just tests whether an interrupt is
pending without attempting to service it. This is useful in situations where
the caller knows that interrupts are blocked, and would like to find out if
it's worth the trouble to unblock them. Also add
INTERRUPTS_CAN_BE_PROCESSED(), which indicates whether CHECK_FOR_INTERRUPTS()
can be relied on to clear the pending interrupt. This commit doesn't actually
add any uses of the new macros, but a follow-on bug fix will do so.
Back-patch to all supported branches to provide infrastructure for that fix.
Alvaro Herrera and Tom Lane Discussion:
[https://postgr.es/m/20210513155351.GA7848@alvherre.pgsql](https://postgr.es/m/20210513155351.GA7848@alvherre.pgsql)
[https://git.postgresql.org/pg/commitdiff/e47f93f981ccb70b4c4c5a0966e5fa0400e11a7e](https://git.postgresql.org/pg/commitdiff/e47f93f981ccb70b4c4c5a0966e5fa0400e11a7e)

- Fix query-cancel handling in spgdoinsert(). Knowing that a buggy opclass could
cause an infinite insertion loop, spgdoinsert() intended to allow its loop to
be interrupted by query cancel. However, that never actually worked, because
in iterations after the first, we'd be holding buffer lock(s) which would
cause InterruptHoldoffCount to be positive, preventing servicing of the
interrupt. To fix, check if an interrupt is pending, and if so fall out of
the insertion loop and service the interrupt after we've released the buffers.
If it was indeed a query cancel, that's the end of the matter. If it was a
non-canceling interrupt reason, make use of the existing provision to retry
the whole insertion. (This isn't as wasteful as it might seem, since any
upper-level index tuples we already created should be usable in the next
attempt.) While there's no known instance of such a bug in existing release
branches, it still seems like a good idea to back-patch this to all supported
branches, since the behavior is fairly nasty if a loop does happen --- not
only is it uncancelable, but it will quickly consume memory to the point of an
OOM failure. In any case, this code is certainly not working as intended.
Per report from Dilip Kumar. Discussion:
[https://postgr.es/m/CAFiTN-uxP_soPhVG840tRMQTBmtA_f_Y8N51G7DKYYqDh7XN-A@mail.gmail.com](https://postgr.es/m/CAFiTN-uxP_soPhVG840tRMQTBmtA_f_Y8N51G7DKYYqDh7XN-A@mail.gmail.com)
[https://git.postgresql.org/pg/commitdiff/eb7a6b9229432dcb791f4bf0c44fe97bab661134](https://git.postgresql.org/pg/commitdiff/eb7a6b9229432dcb791f4bf0c44fe97bab661134)

- Prevent infinite insertion loops in spgdoinsert(). Formerly we just relied on
operator classes that assert longValuesOK to eventually shorten the leaf value
enough to fit on an index page. That fails since the introduction of
INCLUDE-column support (commit 09c1c6ab4), because the INCLUDE columns might
alone take up more than a page, meaning no amount of leaf-datum compaction
will get the job done. At least with spgtextproc.c, that leads to an infinite
loop, since spgtextproc.c won't throw an error for not being able to shorten
the leaf datum anymore. To fix without breaking cases that would otherwise
work, add logic to spgdoinsert() to verify that the leaf tuple size is
decreasing after each "choose" step. Some opclasses might not decrease the
size on every single cycle, and in any case, alignment roundoff of the tuple
size could obscure small gains. Therefore, allow up to 10 cycles without
additional savings before throwing an error. (Perhaps this number will need
adjustment, but it seems quite generous right now.) As long as we've
developed this logic, let's back-patch it. The back branches don't have
INCLUDE columns to worry about, but this seems like a good defense against
possible bugs in operator classes. We already know that an infinite loop here
is pretty unpleasant, so having a defense seems to outweigh the risk of
breaking things. (Note that spgtextproc.c is actually the only known opclass
with longValuesOK support, so that this is all moot for known non-core
opclasses anyway.) Per report from Dilip Kumar. Discussion:
[https://postgr.es/m/CAFiTN-uxP_soPhVG840tRMQTBmtA_f_Y8N51G7DKYYqDh7XN-A@mail.gmail.com](https://postgr.es/m/CAFiTN-uxP_soPhVG840tRMQTBmtA_f_Y8N51G7DKYYqDh7XN-A@mail.gmail.com)
[https://git.postgresql.org/pg/commitdiff/c3c35a733c77b298d3cf7e7de2eeb4aea540a631](https://git.postgresql.org/pg/commitdiff/c3c35a733c77b298d3cf7e7de2eeb4aea540a631)

- Be more careful about barriers when releasing BackgroundWorkerSlots.
ForgetBackgroundWorker lacked any memory barrier at all, while
BackgroundWorkerStateChange had one but unaccountably did additional
manipulation of the slot after the barrier. AFAICS, the rule must be that the
barrier is immediately before setting or clearing slot->in_use. It looks like
back in 9.6 when ForgetBackgroundWorker was first written, there might have
been some case for not needing a barrier there, but I'm not very convinced of
that --- the fact that the load of bgw_notify_pid is in the caller doesn't
seem to guarantee no memory ordering problem. So patch 9.6 too. It's likely
that this doesn't fix any observable bug on Intel hardware, but machines with
weaker memory ordering rules could have problems here. Discussion:
[https://postgr.es/m/4046084.1620244003@sss.pgh.pa.us](https://postgr.es/m/4046084.1620244003@sss.pgh.pa.us)
[https://git.postgresql.org/pg/commitdiff/30d8bad494ad1f604295033e4f4de4b8f258fe74](https://git.postgresql.org/pg/commitdiff/30d8bad494ad1f604295033e4f4de4b8f258fe74)

Michaël Paquier pushed:

- doc: Fix some gaps with the documentation related to LZ4. The upstream project
is officially named "LZ4", and the documentation was confused with the option
value that can be used with DDLs supporting this option, and the project name.
Documentation related to the configure option --with-lz4 was missing, so add
something for that. Author: Dilip Kumar, Michael Paquier Reviewed-by: Justin
Pryzby Discussion:
[https://postgr.es/m/YJaOZQDXBVySq+Cc@paquier.xyz](https://postgr.es/m/YJaOZQDXBVySq+Cc@paquier.xyz)
[https://git.postgresql.org/pg/commitdiff/02a93e7ef9612788081ef07ea1bbd0a8cc99ae99](https://git.postgresql.org/pg/commitdiff/02a93e7ef9612788081ef07ea1bbd0a8cc99ae99)

- Add more TAP tests for pg_dump with attribute compression. Some relations with
LZ4 used as the toast compression methods have been left around in the main
regression test suite to stress pg_upgrade, but pg_dump, that includes tests
much more picky in terms of output generated, had no actual coverage with this
feature. Similarly to collations, tests only working with LZ4 are tracked
with an additional flag, and this uses TestLib::check_pg_config() to check if
the build supports LZ4 or not. This stresses more scenarios with tables,
materialized views and pg_dump --no-toast-compression. Author: Dilip Kumar
Reviewed-by: Michael Paquier Discussion:
[https://postgr.es/m/CAFiTN-twgPmohG7qj1HXhySq16h123y5OowsQR+5h1YeZE9fmQ@mail.gmail.com](https://postgr.es/m/CAFiTN-twgPmohG7qj1HXhySq16h123y5OowsQR+5h1YeZE9fmQ@mail.gmail.com)
[https://git.postgresql.org/pg/commitdiff/63db0ac3f9e6bae313da67f640c95c0045b7f0ee](https://git.postgresql.org/pg/commitdiff/63db0ac3f9e6bae313da67f640c95c0045b7f0ee)

- Fix generation of ./INSTALL for the distribution tarball. "make dist", in
charge of creating a distribution tarball, failed when attempting to generate
./INSTALL as a new reference added to guc-default-toast-compression on the
documentation for the installation details was not getting translated properly
to plain text. Like all the other link references on this page, this adds a
new entry to standalone-profile.xsl to allow the generation of ./INSTALL to
finish properly. Oversight in 02a93e7, per buildfarm member guaibasaurus.
[https://git.postgresql.org/pg/commitdiff/45aa88fe1d4028ea50ba7d26d390223b6ef78acc](https://git.postgresql.org/pg/commitdiff/45aa88fe1d4028ea50ba7d26d390223b6ef78acc)

- Fix typos in operatorcmds.c. Author: Kyotaro Horiguchi, Justin Pryzby
Discussion:
[https://postgr.es/m/20210428.173633.1525059946206239295.horikyota.ntt@gmail.com](https://postgr.es/m/20210428.173633.1525059946206239295.horikyota.ntt@gmail.com)
[https://git.postgresql.org/pg/commitdiff/829daab4bbe356a2f9ae0b2ee0fc98bc2279d754](https://git.postgresql.org/pg/commitdiff/829daab4bbe356a2f9ae0b2ee0fc98bc2279d754)

- Add support for LZ4 build in MSVC scripts. Since its introduction in bbe0a81,
compression of table data supports LZ4, but nothing had been done within the
MSVC scripts to allow users to build the code with this library. This commit
closes the gap by extending the MSVC scripts to be able to build optionally
with LZ4. Getting libraries that can be used for compilation and execution is
possible as LZ4 can be compiled down to MSVC 2010 using its source tarball.
MinGW may require extra efforts to be able to work, and I have been able to
test this only with MSVC, still this is better than nothing to give users a
way to test the feature on Windows. Author: Dilip Kumar Reviewed-by: Michael
Paquier Discussion:
[https://postgr.es/m/YJPdNeF68XpwDDki@paquier.xyz](https://postgr.es/m/YJPdNeF68XpwDDki@paquier.xyz)
[https://git.postgresql.org/pg/commitdiff/9ca40dcd4d0cad43d95a9a253fafaa9a9ba7de24](https://git.postgresql.org/pg/commitdiff/9ca40dcd4d0cad43d95a9a253fafaa9a9ba7de24)

- Simplify one use of ScanKey in pg_subscription.c. The section of the code in
charge of returning all the relations associated to a subscription only need
one ScanKey, but allocated two of them. This code was introduced as a
copy-paste from a different area on the same file by 7c4f524, making the
result confusing to follow. Author: Peter Smith Reviewed-by: Tom Lane, Julien
Rouhaud, Bharath Rupireddy Discussion:
[https://postgr.es/m/CAHut+PsLKe+rN3FjchoJsd76rx2aMsFTB7CTFxRgUP05p=kcpQ@mail.gmail.com](https://postgr.es/m/CAHut+PsLKe+rN3FjchoJsd76rx2aMsFTB7CTFxRgUP05p=kcpQ@mail.gmail.com)
[https://git.postgresql.org/pg/commitdiff/e6ccd1ce1644d1b40b7981f8bc172394de524f99](https://git.postgresql.org/pg/commitdiff/e6ccd1ce1644d1b40b7981f8bc172394de524f99)

- Make saner the tab completion of INSERT and DELETE in psql. When specified
directly as DML queries, INSERT was not getting always completed to "INSERT
INTO", same for DELETE with "DELETE FROM". This makes the completion behavior
more consistent for both commands, saving a few keystrokes. Commands on
policies, triggers, grant/revoke, etc. require only DELETE as completion
keyword. Author: Haiying Tang Reviewed-by: Dilip Kumar, Julien Rouhaud
Discussion:
[https://postgr.es/m/OS0PR01MB61135AE2B07CCD1AB8C6A0F6FB549@OS0PR01MB6113.jpnprd01.prod.outlook.com](https://postgr.es/m/OS0PR01MB61135AE2B07CCD1AB8C6A0F6FB549@OS0PR01MB6113.jpnprd01.prod.outlook.com)
[https://git.postgresql.org/pg/commitdiff/1906cc07d90a8e58fd381dba43c1085e9231f236](https://git.postgresql.org/pg/commitdiff/1906cc07d90a8e58fd381dba43c1085e9231f236)

Thomas Munro pushed:

- Revert recovery prefetching feature. This set of commits has some bugs with
known fixes, but at this late stage in the release cycle it seems best to
revert and resubmit next time, along with some new automated test coverage for
this whole area. Commits reverted: dc88460c: Doc: Review for "Optionally
prefetch referenced data in recovery." 1d257577: Optionally prefetch
referenced data in recovery. f003d9f8: Add circular WAL decoding buffer.
323cbe7c: Remove read_page callback from XLogReader. Remove the new GUC group
WAL_RECOVERY recently added by a55a9847, as the corresponding section of
config.sgml is now reverted. Discussion:
[https://postgr.es/m/CAOuzzgrn7iKnFRsB4MHp3UisEQAGgZMbk_ViTN4HV4-Ksq8zCg%40mail.gmail.com](https://postgr.es/m/CAOuzzgrn7iKnFRsB4MHp3UisEQAGgZMbk_ViTN4HV4-Ksq8zCg%40mail.gmail.com)
[https://git.postgresql.org/pg/commitdiff/c2dc19342e05e081dc13b296787baa38352681ef](https://git.postgresql.org/pg/commitdiff/c2dc19342e05e081dc13b296787baa38352681ef)

Bruce Momjian pushed:

- doc: first draft of the PG 14 release notes.
[https://git.postgresql.org/pg/commitdiff/dc0260861063b125d297c0f3caca359feb381c6a](https://git.postgresql.org/pg/commitdiff/dc0260861063b125d297c0f3caca359feb381c6a)

- doc: update PG 14 release notes based on feedback so far.
[https://git.postgresql.org/pg/commitdiff/ff51679220ce31091bfdbc96d2e90fc02ac6f329](https://git.postgresql.org/pg/commitdiff/ff51679220ce31091bfdbc96d2e90fc02ac6f329)

- doc: update PG 14 release notes based on feedback.
[https://git.postgresql.org/pg/commitdiff/5b2d09beaffa915edd6e74fcf030b13844d3326f](https://git.postgresql.org/pg/commitdiff/5b2d09beaffa915edd6e74fcf030b13844d3326f)

- doc: update PG 14 release notes based on current feedback.
[https://git.postgresql.org/pg/commitdiff/b35f827b68dc1e761e17f621fbf17c3ecd073cb0](https://git.postgresql.org/pg/commitdiff/b35f827b68dc1e761e17f621fbf17c3ecd073cb0)

- doc: PG 14 release notes, adjust updates/deletes on partitions.
[https://git.postgresql.org/pg/commitdiff/b2d0c7c96711843c6e47fce71335d43127f81647](https://git.postgresql.org/pg/commitdiff/b2d0c7c96711843c6e47fce71335d43127f81647)

- doc: PG 14 release notes, reorder items by significance.
[https://git.postgresql.org/pg/commitdiff/521d08a21a2b1ba7038ccc815b8bccc3c9be1351](https://git.postgresql.org/pg/commitdiff/521d08a21a2b1ba7038ccc815b8bccc3c9be1351)

- doc: update PG 14 release notes with recent feedback. Reported-by: Justin
Pryzby Discussion:
[https://postgr.es/m/20210514020141.GQ27406@telsasoft.com](https://postgr.es/m/20210514020141.GQ27406@telsasoft.com)
[https://git.postgresql.org/pg/commitdiff/5eb1b27d20670b378508391fab01a6871a86a8e9](https://git.postgresql.org/pg/commitdiff/5eb1b27d20670b378508391fab01a6871a86a8e9)

- doc: update PG 14 release notes for compute_query_id change. Also remove ALTER
TYPE ...SUBSCRIPT, and update for all current commits.
[https://git.postgresql.org/pg/commitdiff/6cb5346cb15d56e6ba8288b891c7098f0aecdadc](https://git.postgresql.org/pg/commitdiff/6cb5346cb15d56e6ba8288b891c7098f0aecdadc)

- doc: remove XML comments around compute_query_id PG14 rel text.
[https://git.postgresql.org/pg/commitdiff/f39b21e6a25c7269f50a709aa874e321e6f84b20](https://git.postgresql.org/pg/commitdiff/f39b21e6a25c7269f50a709aa874e321e6f84b20)

Peter Eisentraut pushed:

- Remove unused function arguments. Was present in original commit
198b3716dba68544b55cb97bd120738a86d5df2d but apparently never used.
[https://git.postgresql.org/pg/commitdiff/3c554100307f4e57c0881e205dbdbc173bb84d56](https://git.postgresql.org/pg/commitdiff/3c554100307f4e57c0881e205dbdbc173bb84d56)

- Emit dummy statements for probes.d probes when disabled. When building without
--enable-dtrace, emit dummy do {} while (0) statements for the
stubbed-out TRACE_POSTGRESQL_foo() macros instead of empty macros that totally
elide the original probe statement. This fixes the warning: suggest
braces around empty body in an ‘if’ statement [-Wempty-body] introduced by
b94409a02f. Author: Craig Ringer <craig(dot)ringer(at)2ndquadrant(dot)com> Discussion:
[https://www.postgresql.org/message-id/flat/20210504221531.cfvpmmdfsou6eitb%40alap3.anarazel.de](https://www.postgresql.org/message-id/flat/20210504221531.cfvpmmdfsou6eitb%40alap3.anarazel.de)
[https://git.postgresql.org/pg/commitdiff/fa8fbadb934b4727a7aeff074995e799f4685a75](https://git.postgresql.org/pg/commitdiff/fa8fbadb934b4727a7aeff074995e799f4685a75)

- Translation updates. Source-Git-URL:
git://git.postgresql.org/git/pgtranslation/messages.git Source-Git-Hash:
1c361d3ac016b61715d99f2055dee050397e3f13
[https://git.postgresql.org/pg/commitdiff/6206454bdac1ccd6f6ed9d811e1a1139e663a8b9](https://git.postgresql.org/pg/commitdiff/6206454bdac1ccd6f6ed9d811e1a1139e663a8b9)

- Fix typo.
[https://git.postgresql.org/pg/commitdiff/6d177e2813a2b4415539e2861b595583cc1a8f71](https://git.postgresql.org/pg/commitdiff/6d177e2813a2b4415539e2861b595583cc1a8f71)

- Refactor some error messages for easier translation.
[https://git.postgresql.org/pg/commitdiff/ec6e70c79fffe9292402ee602d3742a8c7d31bd2](https://git.postgresql.org/pg/commitdiff/ec6e70c79fffe9292402ee602d3742a8c7d31bd2)

- pg_amcheck: Message style and formatting improvements.
[https://git.postgresql.org/pg/commitdiff/5a73a9e3b5b24cf2dd90ab4a7ae3724b2c12a0cc](https://git.postgresql.org/pg/commitdiff/5a73a9e3b5b24cf2dd90ab4a7ae3724b2c12a0cc)

- Message style improvements.
[https://git.postgresql.org/pg/commitdiff/09ae329957b739dfbaf722eb5624d0a71fdff3b4](https://git.postgresql.org/pg/commitdiff/09ae329957b739dfbaf722eb5624d0a71fdff3b4)

David Rowley pushed:

- Doc: Remove outdated note about run-time partition pruning. The note is no
longer true as of 86dc90056, so remove it. Author: Amit Langote Discussion:
[https://postgr.es/m/CA+HiwqFxQn7Hz1wT+wYgnf_9SK0c4BwOOwFFT8jcSZwJrd8HEA@mail.gmail.com](https://postgr.es/m/CA+HiwqFxQn7Hz1wT+wYgnf_9SK0c4BwOOwFFT8jcSZwJrd8HEA@mail.gmail.com)
[https://git.postgresql.org/pg/commitdiff/1692d0c3a3fc7716d7d00e0d657248cb98bf4df8](https://git.postgresql.org/pg/commitdiff/1692d0c3a3fc7716d7d00e0d657248cb98bf4df8)

- Convert misleading while loop into an if condition. This seems to be leftover
from ea15e1867 and from when we used to evaluate SRFs at each node. Since
there is an unconditional "return" at the end of the loop body, only 1 loop is
ever possible, so we can just change this into an if condition. There is no
actual bug being fixed here so no back-patch. It seems fine to just fix this
anomaly in master only. Author: Greg Nancarrow Discussion:
[https://postgr.es/m/CAJcOf-d7T1q0az-D8evWXnsuBZjigT04WkV5hCAOEJQZRWy28w@mail.gmail.com](https://postgr.es/m/CAJcOf-d7T1q0az-D8evWXnsuBZjigT04WkV5hCAOEJQZRWy28w@mail.gmail.com)
[https://git.postgresql.org/pg/commitdiff/6cb93beddd33d00e0ce2ee55edfa32cd2a935394](https://git.postgresql.org/pg/commitdiff/6cb93beddd33d00e0ce2ee55edfa32cd2a935394)

Andrew Dunstan pushed:

- Tweak generation of Gen_dummy_probes.pl. Use a static prolog file instead of
generating the prolog from the existing perl script. Also, support generation
of the file in a vpath build. Discussion:
[https://postgr.es/m/700620.1620662868@sss.pgh.pa.us](https://postgr.es/m/700620.1620662868@sss.pgh.pa.us)
[https://git.postgresql.org/pg/commitdiff/0bf62931cae0db1294937eb9190b183494af4cf8](https://git.postgresql.org/pg/commitdiff/0bf62931cae0db1294937eb9190b183494af4cf8)

Fujii Masao pushed:

- Change data type of counters in BufferUsage and WalUsage from long to int64.
Previously long was used as the data type for some counters in BufferUsage and
WalUsage. But long is only four byte, e.g., on Windows, and it's entirely
possible to wrap a four byte counter. For example, emitting more than four
billion WAL records in one transaction isn't actually particularly rare. To
avoid the overflows of those counters, this commit changes the data type of
them from long to int64. Suggested-by: Andres Freund Author: Masahiro Ikeda
Reviewed-by: Fujii Masao Discussion:
[https://postgr.es/m/20201221211650.k7b53tcnadrciqjo@alap3.anarazel.de](https://postgr.es/m/20201221211650.k7b53tcnadrciqjo@alap3.anarazel.de)
Discussion:
[https://postgr.es/m/af0964ac-7080-1984-dc23-513754987716@oss.nttdata.com](https://postgr.es/m/af0964ac-7080-1984-dc23-513754987716@oss.nttdata.com)
[https://git.postgresql.org/pg/commitdiff/d780d7c0882fe9a385102b292907baaceb505ed0](https://git.postgresql.org/pg/commitdiff/d780d7c0882fe9a385102b292907baaceb505ed0)

Etsuro Fujita pushed:

- Fix EXPLAIN ANALYZE for async-capable nodes. EXPLAIN ANALYZE for an
async-capable ForeignScan node associated with postgres_fdw is done just by
using instrumentation for ExecProcNode() called from the node's callbacks,
causing the following problems: 1) If the remote table to scan is empty, the
node is incorrectly considered as "never executed" by the command even if
the node is executed, as ExecProcNode() isn't called from the node's
callbacks at all in that case. 2) The command fails to collect timings for
things other than ExecProcNode() done in the node, such as creating a
cursor for the node's remote query. To fix these problems, add
instrumentation for async-capable nodes, and modify postgres_fdw accordingly.
My oversight in commit 27e1f1456. While at it, update a comment for the
AsyncRequest struct in execnodes.h and the documentation for the
ForeignAsyncRequest API in fdwhandler.sgml to match the code in
ExecAsyncAppendResponse() in nodeAppend.c, and fix typos in comments in
nodeAppend.c. Per report from Andrey Lepikhov, though I didn't use his patch.
Reviewed-by: Andrey Lepikhov Discussion:
[https://postgr.es/m/2eb662bb-105d-fc20-7412-2f027cc3ca72%40postgrespro.ru](https://postgr.es/m/2eb662bb-105d-fc20-7412-2f027cc3ca72%40postgrespro.ru)
[https://git.postgresql.org/pg/commitdiff/a363bc6da96b14d27e1cae1bae97242eb6ade5e6](https://git.postgresql.org/pg/commitdiff/a363bc6da96b14d27e1cae1bae97242eb6ade5e6)

- Prevent asynchronous execution of direct foreign-table modifications. Commits
27e1f1456 and 86dc90056, which were independently discussed, cause a crash
when executing an inherited foreign UPDATE/DELETE query with asynchronous
execution enabled, where children of an Append node that is the
direct/indirect child of the ModifyTable node are rewritten so as to modify
foreign tables directly by postgresPlanDirectModify(); as in that case the
direct modifications are executed asynchronously, which is not currently
supported by asynchronous execution. Fix by disabling asynchronous execution
of the direct modifications in that function. Author: Etsuro Fujita
Reviewed-by: Amit Langote Discussion:
[https://postgr.es/m/CAPmGK158e9sJOfuWxfn%2B0ynrspXQU3JhNjSCbaoeSzMvnga%2Bbw%40mail.gmail.com](https://postgr.es/m/CAPmGK158e9sJOfuWxfn%2B0ynrspXQU3JhNjSCbaoeSzMvnga%2Bbw%40mail.gmail.com)
[https://git.postgresql.org/pg/commitdiff/a784859f4480ceaa05a00ca35311071ca33483d1](https://git.postgresql.org/pg/commitdiff/a784859f4480ceaa05a00ca35311071ca33483d1)

Álvaro Herrera pushed:

- Rename the logical replication global "wrconn". The worker.c global wrconn is
only meant to be used by logical apply/ tablesync workers, but there are other
variables with the same name. To reduce future confusion rename the global
from "wrconn" to "LogRepWorkerWalRcvConn". While this is just cosmetic, it
seems better to backpatch it all the way back to 10 where this code appeared,
to avoid future backpatching issues. Author: Peter Smith
<smithpb2250(at)gmail(dot)com> Discussion:
[https://postgr.es/m/CAHut+Pu7Jv9L2BOEx_Z0UtJxfDevQSAUW2mJqWU+CtmDrEZVAg@mail.gmail.com](https://postgr.es/m/CAHut+Pu7Jv9L2BOEx_Z0UtJxfDevQSAUW2mJqWU+CtmDrEZVAg@mail.gmail.com)
[https://git.postgresql.org/pg/commitdiff/db16c656478b815627a03bb0a31833391a733eb0](https://git.postgresql.org/pg/commitdiff/db16c656478b815627a03bb0a31833391a733eb0)

- Describe (auto-)analyze behavior for partitioned tables. This explains the new
behavior introduced by 0827e8af70f4 as well as preexisting. Author: Justin
Pryzby <pryzby(at)telsasoft(dot)com> Author: Álvaro Herrera <alvherre(at)alvh(dot)no-ip(dot)org>
Discussion:
[https://postgr.es/m/20210423180152.GA17270@telsasoft.com](https://postgr.es/m/20210423180152.GA17270@telsasoft.com)
[https://git.postgresql.org/pg/commitdiff/1b5617eb844cd2470a334c1d2eec66cf9b39c41a](https://git.postgresql.org/pg/commitdiff/1b5617eb844cd2470a334c1d2eec66cf9b39c41a)

- Allow compute_query_id to be set to 'auto' and make it default. Allowing only
on/off meant that all either all existing configuration guides would become
obsolete if we disabled it by default, or that we would have to accept a
performance loss in the default config if we enabled it by default. By
allowing 'auto' as a middle ground, the performance cost is only paid by those
who enable pg_stat_statements and similar modules. I only edited the release
notes to comment-out a paragraph that is now factually wrong; further edits
are probably needed to describe the related change in more detail. Author:
Julien Rouhaud <rjuju123(at)gmail(dot)com> Reviewed-by: Justin Pryzby
<pryzby(at)telsasoft(dot)com> Discussion:
[https://postgr.es/m/20210513002623.eugftm4nk2lvvks3@nol](https://postgr.es/m/20210513002623.eugftm4nk2lvvks3@nol)
[https://git.postgresql.org/pg/commitdiff/cafde58b337e007cb6a719f5ab4dd6459d932a39](https://git.postgresql.org/pg/commitdiff/cafde58b337e007cb6a719f5ab4dd6459d932a39)

- Unbreak EXEC_BACKEND build. Per buildfarm
[https://git.postgresql.org/pg/commitdiff/354f32d01dedc2c86a05be298a62cdae9710d203](https://git.postgresql.org/pg/commitdiff/354f32d01dedc2c86a05be298a62cdae9710d203)

Amit Kapila pushed:

- Fix tests for replication slots stats. Some of the tests were not considering
that the slot's spill stats could be received by the stats collector after we
have reset the stats. Remove those tests and don't check total bytes decoded
and sent to output plugin in the spilled stats test as we can send the spilled
stats to the stats collector before actually sending the changes to output
plugin. Reported-by: Tom Lane as per buildfarm Author: Vignesh C, Sawada
Masahiko Reviewed-by: Amit Kapila Discussion:
[https://postgr.es/m/20210319185247.ldebgpdaxsowiflw@alap3.anarazel.de](https://postgr.es/m/20210319185247.ldebgpdaxsowiflw@alap3.anarazel.de)
[https://git.postgresql.org/pg/commitdiff/fc69509131c33c298e39dd25d542374e86aa3295](https://git.postgresql.org/pg/commitdiff/fc69509131c33c298e39dd25d542374e86aa3295)

Alexander Korotkov pushed:

- Improve documentation example for jsonpath like_regex operator. Make sample
like_regex match string values of the root object instead of the whole
document. The corrected example seems to represent a more relevant use case.
Backpatch to 12, when jsonpath was introduced. Discussion:
[https://postgr.es/m/13440f8b-4c1f-5875-c8e3-f3f65606af2f%40xs4all.nl](https://postgr.es/m/13440f8b-4c1f-5875-c8e3-f3f65606af2f%40xs4all.nl)
Author: Erik Rijkers Reviewed-by: Michael Paquier, Alexander Korotkov
Backpatch-through: 12
[https://git.postgresql.org/pg/commitdiff/9b7286c2b394381c937559a98f35df64a92ffbac](https://git.postgresql.org/pg/commitdiff/9b7286c2b394381c937559a98f35df64a92ffbac)

Peter Geoghegan pushed:

- Fix autovacuum log output heap truncation issue. The percentage of blocks from
the table value reported by autovacuum log output (following commit
5100010ee4d) should never exceed 100% because it describes the state of the
table back when lazy_vacuum() was called. The value could nevertheless exceed
100% in the event of heap relation truncation. We failed to compensate for
how truncation affects rel_pages. Fix the faulty accounting by using the
original rel_pages value instead of the current/final rel_pages value.
Reported-By: Andres Freund <andres(at)anarazel(dot)de> Discussion:
[https://postgr.es/m/20210423204306.5osfpkt2ggaedyvy@alap3.anarazel.de](https://postgr.es/m/20210423204306.5osfpkt2ggaedyvy@alap3.anarazel.de)
[https://git.postgresql.org/pg/commitdiff/fbe9b80610fe17ed27ee318bdc5ba06ed86b1a71](https://git.postgresql.org/pg/commitdiff/fbe9b80610fe17ed27ee318bdc5ba06ed86b1a71)

- Harden nbtree deduplication posting split code. Add a defensive "can't happen"
error to code that handles nbtree posting list splits (promote an existing
assertion). This avoids a segfault in the event of an insertion of a newitem
that is somehow identical to an existing non-pivot tuple in the index. An
nbtree index should never have two index tuples with identical TIDs. This
scenario is not particular unlikely in the event of any kind of corruption
that leaves the index in an inconsistent state relative to the heap relation
that is indexed. There are two known reports of preventable hard crashes.
Doing nothing seems unacceptable given the general expectation that nbtree
will cope reasonably well with corrupt data. Discussion:
[https://postgr.es/m/CAH2-Wz=Jr_d-dOYEEmwz0-ifojVNWho01eAqewfQXgKfoe114w@mail.gmail.com](https://postgr.es/m/CAH2-Wz=Jr_d-dOYEEmwz0-ifojVNWho01eAqewfQXgKfoe114w@mail.gmail.com)
Backpatch: 13-, where nbtree deduplication was introduced.
[https://git.postgresql.org/pg/commitdiff/8f72bbac3e4b1d1be9598e8edb9353fa5dc48138](https://git.postgresql.org/pg/commitdiff/8f72bbac3e4b1d1be9598e8edb9353fa5dc48138)

# Pending Patches

Amit Langote sent in another revision of a patch to allow batching of inserts
during cross-partition updates.

Nitin Jadhav sent in a patch to match the file path with the actual file path in
comments in src/backend/utils/activity/backend_status.c.

Bharath Rupireddy sent in a patch to make the PostgreSQL FDW cached connection
functions tests meaningful.

Thomas Munro sent in a patch to clobber memory released by HASH_REMOVE, and
avoid accessing local locks after freeing them.

Andres Freund sent in a patch to zero partial pages in walreceiver.

Peter Smith sent in a patch to ensure that GetSubscriptionRelations declares
only 1 scan key.

Vigneshwaran C sent in three more revisions of a patch to include the actual
datatype used in logical replication message descriptions.

Tang sent in three more revisions of a patch to make tab completion for DELETE
FORM and INSERT INTO consistent in psql.

Vigneshwaran C sent in three more revisions of a patch to make error messages
more informative as to the actual error.

Kyotaro HORIGUCHI sent in two more revisions of a patch to set expectedTLEs
correctly based on recoveryTargetTLI.

Etsuro Fujita sent in another revision of a patch to fix EXPLAIN ANALYZE for
async capable nodes.

Hou Zhijie sent in another revision of a patch to check UDF parallel safety in
fmgr_info, fix UDF the parallel safety label in test case, check built-in
function parallel safety in fmgr_info, and fix the builtin parallel safety label
in test case.

Honza Horak sent in a patch to fix the subtransaction test for Python 3.10.

Masahiko Sawada sent in another revision of a patch to introduce a transaction
manager for foreign transactions and use same to handle two-phase commits that
involve them.

Masahiro Ikeda sent in two more revisions of a patch to improve the performance
of reporting wal stats.

Hou Zhijie sent in two more revisions of a patch to make it possible for users
to declare that a table allows parallel data modification, enable parallel
select for insert, implement a utility function,
pg_get_parallel_safety(regclass) that returns records of (objid, classid,
parallel_safety) that represent the parallel safety of objects that determine
the parallel safety of the specified table, and update the regression tests to
include same.

Pavel Stěhule sent in two more revisions of a patch to implement schema
variables.

Pavel Stěhule sent in another revision of a patch to make it possible for
pg_dump to read table names from a file.

Pavel Stěhule sent in another revision of a patch to make it possible to specify
a separate pager for \watch in psql.

Zeng Wenjing sent in another revision of a patch to implement global temporary
tables.

Tom Lane sent in another revision of a patch to replace pg_depend PIN entries
with a fixed range check.

Atsushi Torikoshi sent in a patch to add a pg_log_current_plan(pid) function
which allows people to get the plan of a currently executing query.

Mark Dilger sent in another revision of a patch to delegate setting of GUC
variables to some newly created ROLEs.

Michail Nikolaev sent in two more revisions of a patch to add full support for
index LP_DEAD hint bits on standby.

Amul Sul sent in two more revisions of a patch to implement ALTER DATABASE READ
{ONLY | WRITE}.

Vigneshwaran C and Bharath Rupireddy traded patches to clarify the messages and
documentation for ALTER SUBSCRIPTION ... DROP PUBLICATION.

Ranier Vilela sent in a patch to fix an explicity NULL pointer dereference in
src/backend/commands/tablecmds.c.

Michaël Paquier sent in a patch to add a vacuum_cleanup_index_scale_factor GUC.

Vigneshwaran C sent in two revisions of a patch intended to fix a bug that
manifested as a message text failure by by checking the active column in
pg_replication_slot instead of pg_stat_replication, the pg_replication_slot test
being more reliable.

Bharath Rupireddy sent in a patch to reword error messages and docs for parallel
VACUUM.

Nathan Bossart sent in a patch to allow specifying direct role membership in
pg_hba.conf.

Peter Smith and Ajin Cherian traded patches to support logical decoding of
two-phase transactions.

Peter Geoghegan sent in a patch to consider triggering failsafe during the first
scan in vacuumlazy.c.

Amit Langote sent in a patch to fix a pgoutput tupeconv map leak.

Kyotaro HORIGUCHI sent in a patch to match errors about braces with their name.

Vigneshwaran C sent in three revisions of a patch to add tab completion for
ALTER SUBSCRIPTION SET options streaming and binary to psql.

Ranier Vilela sent in two revisions of a patch to fix a possible memory
corruption in src/timezone/zic.

Michaël Paquier sent in another revision of a patch to rewrite the test of
pg_upgrade as a TAP test.

Nitin Jadhav and Justin Pryzby traded patches to remove extra memory allocations
from create_list_bounds, allocate the PartitionListValue and create_hash_bounds
PartitionHashBound as a single chunk, allocate datum arrays in bulk to avoid
palloc overhead, and pfree() intermediate results in create_range_bounds.

Tom Lane sent in two revisions of a patch intended to fix a bug that manifested
as prion failed with ERROR: missing chunk number 0 for toast value 14334 in
pg_toast_2619.

Tatsuo Ishii sent in two revisions of a patch to fix a typo in README.barrier.

Nitin Jadhav sent in a patch to support tzh_tzm patterns in to_char().