PostgreSQL Weekly News - April 4, 2021

# PostgreSQL Weekly News - April 4, 2021

Person of the week: https://postgresql.life/post/jan_karremans/

# PostgreSQL Product News

Ora2Pg 21.1, a tool for migrating Oracle databases to PostgreSQL, released.
[https://github.com/darold/ora2pg/blob/master/changelog](https://github.com/darold/ora2pg/blob/master/changelog)

pgtt 2.3, an extension to implement global temporary tables, released.
[https://github.com/darold/pgtt/releases/tag/v2.3](https://github.com/darold/pgtt/releases/tag/v2.3)

SB Data Generator, GUI tool for generating and populating databases with test
data, released.
[SB Data Generator](https://soft-builder.com/sb-data-generator/)

# PostgreSQL Jobs for April

[https://archives.postgresql.org/pgsql-jobs/2021-04/](https://archives.postgresql.org/pgsql-jobs/2021-04/)

# PostgreSQL in the News

Planet PostgreSQL: [https://planet.postgresql.org/](https://planet.postgresql.org/)

PostgreSQL Weekly News is brought to you this week by David Fetter

Submit news and announcements by Sunday at 3:00pm PST8PDT to david(at)fetter(dot)org(dot)

# Applied Patches

David Rowley pushed:

- Cache if PathTarget and RestrictInfos contain volatile functions. Here we aim
to reduce duplicate work done by contain_volatile_functions() by caching
whether PathTargets and RestrictInfos contain any volatile functions the first
time contain_volatile_functions() is called for them. Any future calls for
these nodes just use the cached value rather than going to the trouble of
recursively checking the sub-node all over again. Thanks to Tom Lane for the
idea. Any locations in the code which make changes to a PathTarget or
RestrictInfo which could change the outcome of the volatility check must
change the cached value back to VOLATILITY_UNKNOWN again.
contain_volatile_functions() is the only code in charge of setting the cache
value to either VOLATILITY_VOLATILE or VOLATILITY_NOVOLATILE. Some existing
code does benefit from this additional caching, however, this change is mainly
aimed at an upcoming patch that must check for volatility during the join
search. Repeated volatility checks in that case can become very expensive
when the join search contains more than a few relations. Author: David Rowley
Discussion:
[https://postgr.es/m/3795226.1614059027@sss.pgh.pa.us](https://postgr.es/m/3795226.1614059027@sss.pgh.pa.us)
[https://git.postgresql.org/pg/commitdiff/f58b230ed0dba2a3d396794a2ec84541e321d92d](https://git.postgresql.org/pg/commitdiff/f58b230ed0dba2a3d396794a2ec84541e321d92d)

- Adjust design of per-worker parallel seqscan data struct. The design of the
data structures which allow storage of the per-worker memory during parallel
seq scans were not ideal. The work done in 56788d215 required an additional
data structure to allow workers to remember the range of pages that had been
allocated to them for processing during a parallel seqscan. That commit added
a void pointer field to TableScanDescData to allow heapam to store the
per-worker allocation information. However putting the field there made very
little sense given that we have AM specific structs for that, e.g.
HeapScanDescData. Here we remove the void pointer field from
TableScanDescData and add a dedicated field for this purpose to
HeapScanDescData. Previously we also allocated memory for this parallel
per-worker data for all scans, regardless if it was a parallel scan or not.
This was just a wasted allocation for non-parallel scans, so here we make the
allocation conditional on the scan being parallel. Also, add previously
missing pfree() to free the per-worker data in heap_endscan(). Reported-by:
Andres Freund Reviewed-by: Andres Freund Discussion:
[https://postgr.es/m/20210317023101.anvejcfotwka6gaa@alap3.anarazel.de](https://postgr.es/m/20210317023101.anvejcfotwka6gaa@alap3.anarazel.de)
[https://git.postgresql.org/pg/commitdiff/af527705edc3fd0b335264d17e0521c05edc5cca](https://git.postgresql.org/pg/commitdiff/af527705edc3fd0b335264d17e0521c05edc5cca)

- Allow users of simplehash.h to perform direct deletions. Previously
simplehash.h only exposed a method to perform a hash table delete using the
hash table key. This meant that the delete function had to perform a hash
lookup in order to find the entry to delete. Here we add a new function so
that users of simplehash.h can perform a hash delete directly using the entry
pointer, thus saving the hash lookup. An upcoming patch that uses
simplehash.h already has performed the hash lookup so already has the entry
pointer. This change will allow the code in that patch to perform the hash
delete without the code in simplehash.h having to perform an additional hash
lookup. Author: David Rowley Reviewed-by: Andres Freund Discussion:
[https://postgr.es/m/CAApHDvqFLXXge153WmPsjke5VGOSt7Ez0yD0c7eBXLfmWxs3Kw@mail.gmail.com](https://postgr.es/m/CAApHDvqFLXXge153WmPsjke5VGOSt7Ez0yD0c7eBXLfmWxs3Kw@mail.gmail.com)
[https://git.postgresql.org/pg/commitdiff/ff53d7b159b93ce9fc884897f9d96b97744781e2](https://git.postgresql.org/pg/commitdiff/ff53d7b159b93ce9fc884897f9d96b97744781e2)

- Fix compiler warning in unistr function. Some compilers are not aware that
elog/ereport ERROR does not return.
[https://git.postgresql.org/pg/commitdiff/efd9d92bb39c74c2aded64fc08e2d601ce20c39d](https://git.postgresql.org/pg/commitdiff/efd9d92bb39c74c2aded64fc08e2d601ce20c39d)

- Allow estimate_num_groups() to pass back further details about the estimation.
Here we add a new output parameter to estimate_num_groups() to allow it to
inform the caller of additional, possibly useful information about the
estimation. The new output parameter is a struct that currently contains just
a single field with a set of flags. This was done rather than having the
flags as an output parameter to allow future fields to be added without having
to change the signature of the function at a later date when we want to pass
back further information that might not be suitable to store in the flags
field. It seems reasonable that one day in the future that the planner would
want to know more about the estimation. For example, how many individual sets
of statistics was the estimation generated from? The planner may want to take
that into account if we ever want to consider risks as well as costs when
generating plans. For now, there's only 1 flag we set in the flags field.
This is to indicate if the estimation fell back on using the hard-coded
constants in any part of the estimation. Callers may like to change their
behavior if this is set, and this gives them the ability to do so. Callers
may pass the flag pointer as NULL if they have no interest in obtaining any
additional information about the estimate. We're not adding any actual usages
of these flags here. Some follow-up commits will make use of this feature.
Additionally, we're also not making any changes to add support for
clauselist_selectivity() and clauselist_selectivity_ext(). However, if this
is required in the future then the same struct being added here should be fine
to use as a new output argument for those functions too. Author: David Rowley
Discussion:
[https://postgr.es/m/CAApHDvqQqpk=1W-G_ds7A9CsXX3BggWj_7okinzkLVhDubQzjA@mail.gmail.com](https://postgr.es/m/CAApHDvqQqpk=1W-G_ds7A9CsXX3BggWj_7okinzkLVhDubQzjA@mail.gmail.com)
[https://git.postgresql.org/pg/commitdiff/ed934d4fa30f0f94e6f7125ad2154e6a58d1c7f7](https://git.postgresql.org/pg/commitdiff/ed934d4fa30f0f94e6f7125ad2154e6a58d1c7f7)

- Add Result Cache executor node. Here we add a new executor node type named
"Result Cache". The planner can include this node type in the plan to have
the executor cache the results from the inner side of parameterized nested
loop joins. This allows caching of tuples for sets of parameters so that in
the event that the node sees the same parameter values again, it can just
return the cached tuples instead of rescanning the inner side of the join all
over again. Internally, result cache uses a hash table in order to quickly
find tuples that have been previously cached. For certain data sets, this can
significantly improve the performance of joins. The best cases for using this
new node type are for join problems where a large portion of the tuples from
the inner side of the join have no join partner on the outer side of the join.
In such cases, hash join would have to hash values that are never looked up,
thus bloating the hash table and possibly causing it to multi-batch. Merge
joins would have to skip over all of the unmatched rows. If we use a nested
loop join with a result cache, then we only cache tuples that have at least
one join partner on the outer side of the join. The benefits of using a
parameterized nested loop with a result cache increase when there are fewer
distinct values being looked up and the number of lookups of each value is
large. Also, hash probes to lookup the cache can be much faster than the hash
probe in a hash join as it's common that the result cache's hash table is much
smaller than the hash join's due to result cache only caching useful tuples
rather than all tuples from the inner side of the join. This variation in
hash probe performance is more significant when the hash join's hash table no
longer fits into the CPU's L3 cache, but the result cache's hash table does.
The apparent "random" access of hash buckets with each hash probe can cause a
poor L3 cache hit ratio for large hash tables. Smaller hash tables generally
perform better. The hash table used for the cache limits itself to not
exceeding work_mem * hash_mem_multiplier in size. We maintain a dlist of keys
for this cache and when we're adding new tuples and realize we've exceeded the
memory budget, we evict cache entries starting with the least recently used
ones until we have enough memory to add the new tuples to the cache. For
parameterized nested loop joins, we now consider using one of these result
cache nodes in between the nested loop node and its inner node. We determine
when this might be useful based on cost, which is primarily driven off of what
the expected cache hit ratio will be. Estimating the cache hit ratio relies
on having good distinct estimates on the nested loop's parameters. For now,
the planner will only consider using a result cache for parameterized nested
loop joins. This works for both normal joins and also for LATERAL type joins
to subqueries. It is possible to use this new node for other uses in the
future. For example, to cache results from correlated subqueries. However,
that's not done here due to some difficulties obtaining a distinct estimation
on the outer plan to calculate the estimated cache hit ratio. Currently we
plan the inner plan before planning the outer plan so there is no good way to
know if a result cache would be useful or not since we can't estimate the
number of times the subplan will be called until the outer plan is generated.
The functionality being added here is newly introducing a dependency on the
return value of estimate_num_groups() during the join search. Previously,
during the join search, we only ever needed to perform selectivity
estimations. With this commit, we need to use estimate_num_groups() in order
to estimate what the hit ratio on the result cache will be. In simple terms,
if we expect 10 distinct values and we expect 1000 outer rows, then we'll
estimate the hit ratio to be 99%. Since cache hits are very cheap compared to
scanning the underlying nodes on the inner side of the nested loop join, then
this will significantly reduce the planner's cost for the join. However,
it's fairly easy to see here that things will go bad when
estimate_num_groups() incorrectly returns a value that's significantly lower
than the actual number of distinct values. If this happens then that may
cause us to make use of a nested loop join with a result cache instead of some
other join type, such as a merge or hash join. Our distinct estimations have
been known to be a source of trouble in the past, so the extra reliance on
them here could cause the planner to choose slower plans than it did previous
to having this feature. Distinct estimations are also fairly hard to estimate
accurately when several tables have been joined already or when a WHERE clause
filters out a set of values that are correlated to the expressions we're
estimating the number of distinct value for. For now, the costing we perform
during query planning for result caches does put quite a bit of faith in the
distinct estimations being accurate. When these are accurate then we should
generally see faster execution times for plans containing a result cache.
However, in the real world, we may find that we need to either change the
costings to put less trust in the distinct estimations being accurate or
perhaps even disable this feature by default. There's always an element of
risk when we teach the query planner to do new tricks that it decides to use
that new trick at the wrong time and causes a regression. Users may opt to
get the old behavior by turning the feature off using the enable_resultcache
GUC. Currently, this is enabled by default. It remains to be seen if we'll
maintain that setting for the release. Additionally, the name "Result Cache"
is the best name I could think of for this new node at the time I started
writing the patch. Nobody seems to strongly dislike the name. A few people
did suggest other names but no other name seemed to dominate in the brief
discussion that there was about names. Let's allow the beta period to see if
the current name pleases enough people. If there's some consensus on a better
name, then we can change it before the release. Please see the 2nd discussion
link below for the discussion on the "Result Cache" name. Author: David
Rowley Reviewed-by: Andy Fan, Justin Pryzby, Zhihong Yu Tested-By: Konstantin
Knizhnik Discussion:
[https://postgr.es/m/CAApHDvrPcQyQdWERGYWx8J%2B2DLUNgXu%2BfOSbQ1UscxrunyXyrQ%40mail.gmail.com](https://postgr.es/m/CAApHDvrPcQyQdWERGYWx8J%2B2DLUNgXu%2BfOSbQ1UscxrunyXyrQ%40mail.gmail.com)
Discussion:
[https://postgr.es/m/CAApHDvq=yQXr5kqhRviT2RhNKwToaWr9JAN5t+5_PzhuRJ3wvg@mail.gmail.com](https://postgr.es/m/CAApHDvq=yQXr5kqhRviT2RhNKwToaWr9JAN5t+5_PzhuRJ3wvg@mail.gmail.com)
[https://git.postgresql.org/pg/commitdiff/b6002a796dc0bfe721db5eaa54ba9d24fd9fd416](https://git.postgresql.org/pg/commitdiff/b6002a796dc0bfe721db5eaa54ba9d24fd9fd416)

- Revert b6002a796. This removes "Add Result Cache executor node". It seems
that something weird is going on with the tracking of cache hits and misses as
highlighted by many buildfarm animals. It's not yet clear what the problem is
as other parts of the plan indicate that the cache did work correctly, it's
just the hits and misses that were being reported as 0. This is especially a
bad time to have the buildfarm so broken, so reverting before too many more
animals go red. Discussion:
[https://postgr.es/m/CAApHDvq_hydhfovm4=izgWs+C5HqEeRScjMbOgbpC-jRAeK3Yw@mail.gmail.com](https://postgr.es/m/CAApHDvq_hydhfovm4=izgWs+C5HqEeRScjMbOgbpC-jRAeK3Yw@mail.gmail.com)
[https://git.postgresql.org/pg/commitdiff/28b3e3905c982c42fb10ee800e6f881e9742c89d](https://git.postgresql.org/pg/commitdiff/28b3e3905c982c42fb10ee800e6f881e9742c89d)

- Add Result Cache executor node (take 2). Here we add a new executor node type
named "Result Cache". The planner can include this node type in the plan to
have the executor cache the results from the inner side of parameterized
nested loop joins. This allows caching of tuples for sets of parameters so
that in the event that the node sees the same parameter values again, it can
just return the cached tuples instead of rescanning the inner side of the join
all over again. Internally, result cache uses a hash table in order to
quickly find tuples that have been previously cached. For certain data sets,
this can significantly improve the performance of joins. The best cases for
using this new node type are for join problems where a large portion of the
tuples from the inner side of the join have no join partner on the outer side
of the join. In such cases, hash join would have to hash values that are
never looked up, thus bloating the hash table and possibly causing it to
multi-batch. Merge joins would have to skip over all of the unmatched rows.
If we use a nested loop join with a result cache, then we only cache tuples
that have at least one join partner on the outer side of the join. The
benefits of using a parameterized nested loop with a result cache increase
when there are fewer distinct values being looked up and the number of lookups
of each value is large. Also, hash probes to lookup the cache can be much
faster than the hash probe in a hash join as it's common that the result
cache's hash table is much smaller than the hash join's due to result cache
only caching useful tuples rather than all tuples from the inner side of the
join. This variation in hash probe performance is more significant when the
hash join's hash table no longer fits into the CPU's L3 cache, but the result
cache's hash table does. The apparent "random" access of hash buckets with
each hash probe can cause a poor L3 cache hit ratio for large hash tables.
Smaller hash tables generally perform better. The hash table used for the
cache limits itself to not exceeding work_mem * hash_mem_multiplier in size.
We maintain a dlist of keys for this cache and when we're adding new tuples
and realize we've exceeded the memory budget, we evict cache entries starting
with the least recently used ones until we have enough memory to add the new
tuples to the cache. For parameterized nested loop joins, we now consider
using one of these result cache nodes in between the nested loop node and its
inner node. We determine when this might be useful based on cost, which is
primarily driven off of what the expected cache hit ratio will be. Estimating
the cache hit ratio relies on having good distinct estimates on the nested
loop's parameters. For now, the planner will only consider using a result
cache for parameterized nested loop joins. This works for both normal joins
and also for LATERAL type joins to subqueries. It is possible to use this new
node for other uses in the future. For example, to cache results from
correlated subqueries. However, that's not done here due to some difficulties
obtaining a distinct estimation on the outer plan to calculate the estimated
cache hit ratio. Currently we plan the inner plan before planning the outer
plan so there is no good way to know if a result cache would be useful or not
since we can't estimate the number of times the subplan will be called until
the outer plan is generated. The functionality being added here is newly
introducing a dependency on the return value of estimate_num_groups() during
the join search. Previously, during the join search, we only ever needed to
perform selectivity estimations. With this commit, we need to use
estimate_num_groups() in order to estimate what the hit ratio on the result
cache will be. In simple terms, if we expect 10 distinct values and we
expect 1000 outer rows, then we'll estimate the hit ratio to be 99%. Since
cache hits are very cheap compared to scanning the underlying nodes on the
inner side of the nested loop join, then this will significantly reduce the
planner's cost for the join. However, it's fairly easy to see here that
things will go bad when estimate_num_groups() incorrectly returns a value
that's significantly lower than the actual number of distinct values. If this
happens then that may cause us to make use of a nested loop join with a result
cache instead of some other join type, such as a merge or hash join. Our
distinct estimations have been known to be a source of trouble in the past, so
the extra reliance on them here could cause the planner to choose slower plans
than it did previous to having this feature. Distinct estimations are also
fairly hard to estimate accurately when several tables have been joined
already or when a WHERE clause filters out a set of values that are correlated
to the expressions we're estimating the number of distinct value for. For
now, the costing we perform during query planning for result caches does put
quite a bit of faith in the distinct estimations being accurate. When these
are accurate then we should generally see faster execution times for plans
containing a result cache. However, in the real world, we may find that we
need to either change the costings to put less trust in the distinct
estimations being accurate or perhaps even disable this feature by default.
There's always an element of risk when we teach the query planner to do new
tricks that it decides to use that new trick at the wrong time and causes a
regression. Users may opt to get the old behavior by turning the feature off
using the enable_resultcache GUC. Currently, this is enabled by default. It
remains to be seen if we'll maintain that setting for the release.
Additionally, the name "Result Cache" is the best name I could think of for
this new node at the time I started writing the patch. Nobody seems to
strongly dislike the name. A few people did suggest other names but no other
name seemed to dominate in the brief discussion that there was about names.
Let's allow the beta period to see if the current name pleases enough people.
If there's some consensus on a better name, then we can change it before the
release. Please see the 2nd discussion link below for the discussion on the
"Result Cache" name. Author: David Rowley Reviewed-by: Andy Fan, Justin
Pryzby, Zhihong Yu, Hou Zhijie Tested-By: Konstantin Knizhnik Discussion:
[https://postgr.es/m/CAApHDvrPcQyQdWERGYWx8J%2B2DLUNgXu%2BfOSbQ1UscxrunyXyrQ%40mail.gmail.com](https://postgr.es/m/CAApHDvrPcQyQdWERGYWx8J%2B2DLUNgXu%2BfOSbQ1UscxrunyXyrQ%40mail.gmail.com)
Discussion:
[https://postgr.es/m/CAApHDvq=yQXr5kqhRviT2RhNKwToaWr9JAN5t+5_PzhuRJ3wvg@mail.gmail.com](https://postgr.es/m/CAApHDvq=yQXr5kqhRviT2RhNKwToaWr9JAN5t+5_PzhuRJ3wvg@mail.gmail.com)
[https://git.postgresql.org/pg/commitdiff/9eacee2e62d89cab7b004f97c206c4fba4f1d745](https://git.postgresql.org/pg/commitdiff/9eacee2e62d89cab7b004f97c206c4fba4f1d745)

- Attempt to fix unstable Result Cache regression tests. force_parallel_mode =
regress is causing a few more problems than I thought. It seems that both the
leader and the single worker can contribute to the execution. I had mistakenly
thought that only the worker process would do any work. Since it's not
deterministic as to which of the two processes will get a chance to work on
the plan, it seems just better to disable force_parallel_mode for these tests.
At least doing this seems better than changing to EXPLAIN only rather than
EXPLAIN ANALYZE. Additionally, I overlooked the fact that the number of
executions of the sub-plan below a Result Cache will execute a varying number
of times depending on cache eviction. 32-bit machines will use less memory
and evict fewer tuples from the cache. That results in the subnode being
executed fewer times on 32-bit machines. Let's just blank out the number of
loops in each node.
[https://git.postgresql.org/pg/commitdiff/a4fac4ffe8f8d543a10ac7debf1157e34963ece3](https://git.postgresql.org/pg/commitdiff/a4fac4ffe8f8d543a10ac7debf1157e34963ece3)

- Remove useless Asserts in Result Cache code. Testing if an unsigned variable
is >= 0 is pretty pointless. There's likely enough code in
remove_cache_entry() to verify the cache memory accounting is correct in
assert enabled builds. These Asserts were not adding much extra cover, even if
they had been checking >= 0 on a signed variable. Reported-by: Andres Freund
Discussion:
[https://postgr.es/m/20210402204734.6mo3nfacnljlicgn@alap3.anarazel.de](https://postgr.es/m/20210402204734.6mo3nfacnljlicgn@alap3.anarazel.de)
[https://git.postgresql.org/pg/commitdiff/1267d9862fc6a4f8cdc0ca38d1988b61f39da585](https://git.postgresql.org/pg/commitdiff/1267d9862fc6a4f8cdc0ca38d1988b61f39da585)

Peter Geoghegan pushed:

- PageAddItemExtended(): Add LP_UNUSED assertion. Assert that LP_UNUSED items
have no storage. If it's worth having defensive code in non-assert builds
then it's worth having an assertion as well.
[https://git.postgresql.org/pg/commitdiff/30aaab26e52144097a1a5bbb0bb66ea1ebc0cb81](https://git.postgresql.org/pg/commitdiff/30aaab26e52144097a1a5bbb0bb66ea1ebc0cb81)

Peter Eisentraut pushed:

- Reset standard_conforming_strings in strings test. After some tests relating
to standard_conforming_strings behavior, the value was not reset to the
default value. Therefore, the rest of the tests in that file ran with the
nondefault setting, which affected the results of some tests. For clarity,
reset the value and run the rest of the tests with the default setting again.
[https://git.postgresql.org/pg/commitdiff/ebedd0c78fc51c293abe56e99a18c67af14da0c9](https://git.postgresql.org/pg/commitdiff/ebedd0c78fc51c293abe56e99a18c67af14da0c9)

- Add unistr function. This allows decoding a string with Unicode escape
sequences. It is similar to Unicode escape strings, but offers some more
flexibility. Author: Pavel Stehule <pavel(dot)stehule(at)gmail(dot)com> Reviewed-by:
Asif Rehman <asifr(dot)rehman(at)gmail(dot)com> Discussion:
[https://www.postgresql.org/message-id/flat/CAFj8pRA5GnKT+gDVwbVRH2ep451H_myBt+NTz8RkYUARE9+qOQ(at)mail(dot)gmail(dot)com](https://www.postgresql.org/message-id/flat/CAFj8pRA5GnKT+gDVwbVRH2ep451H_myBt+NTz8RkYUARE9+qOQ(at)mail(dot)gmail(dot)com)
[https://git.postgresql.org/pg/commitdiff/f37fec837ce8bf7af408ba66d32099e5a0182402](https://git.postgresql.org/pg/commitdiff/f37fec837ce8bf7af408ba66d32099e5a0182402)

- Clean up date_part tests a bit. Some tests for timestamp and timestamptz were
in the date.sql test file. Move them to their appropriate files, or drop
tests cases that were already present there.
[https://git.postgresql.org/pg/commitdiff/efcc7572f532ea564fedc6359c2df43045ee7908](https://git.postgresql.org/pg/commitdiff/efcc7572f532ea564fedc6359c2df43045ee7908)

- Add upper boundary tests for timestamp and timestamptz types. The existing
regression tests only tested the lower boundary of the range supported by the
timestamp and timestamptz types because "The upper boundary differs between
integer and float timestamps, so no check". Since this is obsolete, add
similar tests for the upper boundary.
[https://git.postgresql.org/pg/commitdiff/bc9f1afdebc98b490d0a00468d75e8e4d080afb0](https://git.postgresql.org/pg/commitdiff/bc9f1afdebc98b490d0a00468d75e8e4d080afb0)

- Add tests for date_part of epoch near upper bound of timestamp range. This
exercises a special case in the implementations of date_part('epoch',
timestamp[tz]) that was previously not tested.
[https://git.postgresql.org/pg/commitdiff/6131ffc43ff3d2f566e93f017e56a09e4e717318](https://git.postgresql.org/pg/commitdiff/6131ffc43ff3d2f566e93f017e56a09e4e717318)

- doc: Remove Cyrillic from unistr example. Not supported by PDF build right
now, so let's do without it.
[https://git.postgresql.org/pg/commitdiff/287d2a97c1de07486e4525c8ad06258f04bd6268](https://git.postgresql.org/pg/commitdiff/287d2a97c1de07486e4525c8ad06258f04bd6268)

- Add errhint_plural() function and make use of it. Similar to existing
errmsg_plural() and errdetail_plural(). Some errhint() calls hadn't received
the proper plural treatment yet.
[https://git.postgresql.org/pg/commitdiff/91c5a8caaa61055959aa5fb68a00e5f690e39a34](https://git.postgresql.org/pg/commitdiff/91c5a8caaa61055959aa5fb68a00e5f690e39a34)

- Add p_names field to ParseNamespaceItem. ParseNamespaceItem had a wired-in
assumption that p_rte->eref describes the table and column aliases exposed by
the nsitem. This relaxes this by creating a separate p_names field in an
nsitem. This is mainly preparation for a patch for JOIN USING aliases, but it
saves one indirection in common code paths, so it's possibly a win on its own.
Author: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> Discussion:
[https://www.postgresql.org/message-id/785329.1616455091@sss.pgh.pa.us](https://www.postgresql.org/message-id/785329.1616455091@sss.pgh.pa.us)
[https://git.postgresql.org/pg/commitdiff/66392d396508c91c2ec07a61568bf96acb663ad8](https://git.postgresql.org/pg/commitdiff/66392d396508c91c2ec07a61568bf96acb663ad8)

- Allow an alias to be attached to a JOIN ... USING. This allows something like
SELECT ... FROM t1 JOIN t2 USING (a, b, c) AS x where x has the columns a, b,
c and unlike a regular alias it does not hide the range variables of the
tables being joined t1 and t2. Per SQL:2016 feature F404 "Range variable for
common column names". Reviewed-by: Vik Fearing <vik(dot)fearing(at)2ndquadrant(dot)com>
Reviewed-by: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> Discussion:
[https://www.postgresql.org/message-id/flat/454638cf-d563-ab76-a585-2564428062af(at)2ndquadrant(dot)com](https://www.postgresql.org/message-id/flat/454638cf-d563-ab76-a585-2564428062af(at)2ndquadrant(dot)com)
[https://git.postgresql.org/pg/commitdiff/055fee7eb4dcc78e58672aef146334275e1cc40d](https://git.postgresql.org/pg/commitdiff/055fee7eb4dcc78e58672aef146334275e1cc40d)

- Make extract(timetz) tests a bit more interesting. Use a time zone offset with
nonzero minutes to make the timezone_minute test meaningful.
[https://git.postgresql.org/pg/commitdiff/e2639a767bfa1afebaf1877515a1187feb393443](https://git.postgresql.org/pg/commitdiff/e2639a767bfa1afebaf1877515a1187feb393443)

- Fix internal extract(timezone_minute) formulas. Through various refactorings
over time, the extract(timezone_minute from time with time zone) and
extract(timezone_minute from timestamp with time zone) implementations ended
up with two different but equally nonsensical formulas by using
SECS_PER_MINUTE and MINS_PER_HOUR interchangeably. Since those two are of
course both the same number, the formulas do work, but for readability, fix
them to be semantically correct.
[https://git.postgresql.org/pg/commitdiff/91e7c903291116bd081abe7d4a058d40a2a06e16](https://git.postgresql.org/pg/commitdiff/91e7c903291116bd081abe7d4a058d40a2a06e16)

- Add support for NullIfExpr in eval_const_expressions. Author: Hou Zhijie
<houzj(dot)fnst(at)cn(dot)fujitsu(dot)com> Discussion:
[https://www.postgresql.org/message-id/flat/7ea5ce773bbc4eea9ff1a381acd3b102(at)G08CNEXMBPEKD05(dot)g08(dot)fujitsu(dot)local](https://www.postgresql.org/message-id/flat/7ea5ce773bbc4eea9ff1a381acd3b102(at)G08CNEXMBPEKD05(dot)g08(dot)fujitsu(dot)local)
[https://git.postgresql.org/pg/commitdiff/9c5f67fd6256246b2a788a8feb1d42b79dcd0448](https://git.postgresql.org/pg/commitdiff/9c5f67fd6256246b2a788a8feb1d42b79dcd0448)

Andrew Dunstan pushed:

- Allow matching the DN of a client certificate for authentication. Currently we
only recognize the Common Name (CN) of a certificate's subject to be matched
against the user name. Thus certificates with subjects '/OU=eng/CN=fred' and
'/OU=sales/CN=fred' will have the same connection rights. This patch provides
an option to match the whole Distinguished Name (DN) instead of just the CN.
On any hba line using client certificate identity, there is an option
'clientname' which can have values of 'DN' or 'CN'. The default is 'CN', the
current procedure. The DN is matched against the RFC2253 formatted DN, which
looks like 'CN=fred,OU=eng'. This facility of probably best used in
conjunction with an ident map. Discussion:
[https://postgr.es/m/92e70110-9273-d93c-5913-0bccb6562740@dunslane.net](https://postgr.es/m/92e70110-9273-d93c-5913-0bccb6562740@dunslane.net)
Reviewed-By: Michael Paquier, Daniel Gustafsson, Jacob Champion
[https://git.postgresql.org/pg/commitdiff/6d7a6feac48b1970c4cd127ee65d4c487acbb5e9](https://git.postgresql.org/pg/commitdiff/6d7a6feac48b1970c4cd127ee65d4c487acbb5e9)

- Fix typo in 6d7a6feac4. Per gripe from Daniel Gustafsson
[https://git.postgresql.org/pg/commitdiff/1877c9ac3acc05cc787dd6392d073202f8c8ee21](https://git.postgresql.org/pg/commitdiff/1877c9ac3acc05cc787dd6392d073202f8c8ee21)

Álvaro Herrera pushed:

- psql: call clearerr() just before printing. We were never doing clearerr() on
the output stream, which results in a message being printed after each result
once an EOF is seen: could not print result table: Success This message was
added by commit b03436994bcc (in the pg13 era); before that, the error
indicator would never be examined. So backpatch only that far back, even
though the actual bug (to wit: the fact that the error indicator is never
cleared) is older.
[https://git.postgresql.org/pg/commitdiff/8d645a116ef6e04bfb03e259149b8e163dbdf50c](https://git.postgresql.org/pg/commitdiff/8d645a116ef6e04bfb03e259149b8e163dbdf50c)

- Improve PQtrace() output format. Transform the PQtrace output format from its
ancient (and mostly useless) byte-level output format to a
logical-message-level output, making it much more usable. This implementation
allows the printing code to be written (as it indeed was) by looking at the
protocol documentation, which gives more confidence that the three (docs,
trace code and actual code) actually match. Author: 岩田 彩 (Aya Iwata)
<iwata(dot)aya(at)fujitsu(dot)com> Reviewed-by: 綱川 貴之 (Takayuki Tsunakawa)
<tsunakawa(dot)takay(at)fujitsu(dot)com> Reviewed-by: Kirk Jamison
<k(dot)jamison(at)fujitsu(dot)com> Reviewed-by: Kyotaro Horiguchi
<horikyota(dot)ntt(at)gmail(dot)com> Reviewed-by: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
Reviewed-by: 黒田 隼人 (Hayato Kuroda) <kuroda(dot)hayato(at)fujitsu(dot)com>
Reviewed-by: "Nagaura, Ryohei" <nagaura(dot)ryohei(at)jp(dot)fujitsu(dot)com> Reviewed-by:
Ryo Matsumura <matsumura(dot)ryo(at)fujitsu(dot)com> Reviewed-by: Greg Nancarrow
<gregn4422(at)gmail(dot)com> Reviewed-by: Jim Doty <jdoty(at)pivotal(dot)io> Reviewed-by:
Álvaro Herrera <alvherre(at)alvh(dot)no-ip(dot)org> Discussion:
[https://postgr.es/m/71E660EB361DF14299875B198D4CE5423DE3FBA4@g01jpexmbkw25](https://postgr.es/m/71E660EB361DF14299875B198D4CE5423DE3FBA4@g01jpexmbkw25)
[https://git.postgresql.org/pg/commitdiff/198b3716dba68544b55cb97bd120738a86d5df2d](https://git.postgresql.org/pg/commitdiff/198b3716dba68544b55cb97bd120738a86d5df2d)

- libpq_pipeline: add PQtrace() support and tests. The libpq_pipeline program
recently introduced by commit acb7e4eb6b1c is well equipped to test the
PQtrace() functionality, so let's make it do that. Author: Álvaro Herrera
<alvherre(at)alvh(dot)no-ip(dot)org> Discussion:
[https://postgr.es/m/20210327192812.GA25115@alvherre.pgsql](https://postgr.es/m/20210327192812.GA25115@alvherre.pgsql)
[https://git.postgresql.org/pg/commitdiff/7bebd0d00998a28449d83376f4bcdeec65d5eea6](https://git.postgresql.org/pg/commitdiff/7bebd0d00998a28449d83376f4bcdeec65d5eea6)

- Fix some libpq_pipeline test problems. Test pipeline_abort was not checking
that it got the rows it expected in one mode; make it do so. This doesn't fix
the actual problem (no idea what that is, yet) but at least it should make it
more obvious rather than being visible only as a difference in the trace
output. While at it, fix other infelicities in the test: * I reversed the
order of result vs. expected in like(). * The output traces from -t are being
put in the log dir, which means the buildfarm script uselessly captures them.
Put them in a separate dir tmp_check/traces instead, to avoid cluttering the
buildfarm results. * Test pipelined_insert was using too large a row count.
Reduce that a tad and add a filler column to make each insert a little
bulkier, while still keeping enough that a buffer is filled and we have to
switch mode.
[https://git.postgresql.org/pg/commitdiff/db973ffb3ca43e65a0bf15175a35184a53bf977d](https://git.postgresql.org/pg/commitdiff/db973ffb3ca43e65a0bf15175a35184a53bf977d)

- Disable force_parallel_mode in libpq_pipeline. Some buildfarm animals with
force_parallel_mode=regress were failing this test because the error is
reported in a parallel worker quicker than the rows that succeed. Take the
opportunity to move the SET of lc_messages out of the traced section, because
it's not very interesting. Diagnosed-by: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
Discussion:
[https://postgr.es/m/3304521.1617221724@sss.pgh.pa.us](https://postgr.es/m/3304521.1617221724@sss.pgh.pa.us)
[https://git.postgresql.org/pg/commitdiff/a6d3dea8e5e0c8a0df2f95d66b6c3903a4354ca0](https://git.postgresql.org/pg/commitdiff/a6d3dea8e5e0c8a0df2f95d66b6c3903a4354ca0)

- Initialize conn->Pfdebug to NULL when creating a connection. Failing to do
this can cause a crash, and I suspect is what has happened with a buildfarm
member reporting mysterious failures. This is an ancient bug, but I'm not
backpatching since evidently nobody cares about PQtrace in older releases.
Discussion:
[https://postgr.es/m/3333908.1617227066@sss.pgh.pa.us](https://postgr.es/m/3333908.1617227066@sss.pgh.pa.us)
[https://git.postgresql.org/pg/commitdiff/aba24b51cc1b045a9810458b4bb15fee2c182948](https://git.postgresql.org/pg/commitdiff/aba24b51cc1b045a9810458b4bb15fee2c182948)

- Remove setvbuf() call from PQtrace(). It's misplaced there -- it's not libpq's
output stream to tweak in that way. In particular, POSIX says that it has to
be called before any other operation on the file, so if a stream previously
used by the calling application, bad things may happen. Put setvbuf() in
libpq_pipeline for good measure. Also, reduce fopen(..., "w+") to just
fopen(..., "w") in libpq_pipeline.c. It's not clear that this fixes anything,
but we don't use w+ anywhere. Per complaints from Tom Lane. Discussion:
[https://postgr.es/m/3337422.1617229905@sss.pgh.pa.us](https://postgr.es/m/3337422.1617229905@sss.pgh.pa.us)
[https://git.postgresql.org/pg/commitdiff/6ec578e60101c3c02533f99715945a0400fb3286](https://git.postgresql.org/pg/commitdiff/6ec578e60101c3c02533f99715945a0400fb3286)

- libpq_pipeline: Must strdup(optarg) to avoid crash. I forgot to strdup() when
processing argv[]. Apparently many platforms hide this mistake from users,
but in those that don't you may get a program crash. Repair. Per buildfarm
member drongo, which is the only one in all the buildfarm manifesting a
problem here. While at it, move "numrows" processing out of the line of
special cases, and make it getopt's -r instead. (A similar thing could be
done to 'conninfo', but current use of the program doesn't warrant spending
time on that -- nowhere else we use conninfo in so simplistic a manner.)
Discussion:
[https://postgr.es/m/20210401124850.GA19247@alvherre.pgsql](https://postgr.es/m/20210401124850.GA19247@alvherre.pgsql)
[https://git.postgresql.org/pg/commitdiff/dde1a35aee6266dc8105717275335c46cd2b3650](https://git.postgresql.org/pg/commitdiff/dde1a35aee6266dc8105717275335c46cd2b3650)

- Fix setvbuf()-induced crash in libpq_pipeline. Windows doesn't like
`setvbuf(..., _IOLBF)` and crashes if you use it, which has been causing the
libpq_pipeline failures all along ... and our own port.h has known about it
for a long time: it offers PG_IOLBF that's defined to `_IONBF` on that platform.
Follow its advice. While at it, get rid of a bogus bitshift that used a
constant of the wrong size. Decorate the constant as LL to fix. While at it,
remove a pointless addition that only confused matters. All as diagnosed by
Tom Lane. Discussion:
[https://postgr.es/m/3458958.1617302154@sss.pgh.pa.us](https://postgr.es/m/3458958.1617302154@sss.pgh.pa.us)
[https://git.postgresql.org/pg/commitdiff/a68a894f0198aaeffa81b3027f135adcdaa8abf6](https://git.postgresql.org/pg/commitdiff/a68a894f0198aaeffa81b3027f135adcdaa8abf6)

Etsuro Fujita pushed:

- Update obsolete comment. Back-patch to all supported branches. Author: Etsuro
Fujita Discussion:
[https://postgr.es/m/CAPmGK17DwzaSf%2BB71dhL2apXdtG-OmD6u2AL9Cq2ZmAR0%2BzapQ%40mail.gmail.com](https://postgr.es/m/CAPmGK17DwzaSf%2BB71dhL2apXdtG-OmD6u2AL9Cq2ZmAR0%2BzapQ%40mail.gmail.com)
[https://git.postgresql.org/pg/commitdiff/bc2797ebb14bae663da1ee7845774dd98716c0d0](https://git.postgresql.org/pg/commitdiff/bc2797ebb14bae663da1ee7845774dd98716c0d0)

- Add support for asynchronous execution. This implements asynchronous
execution, which runs multiple parts of a non-parallel-aware Append
concurrently rather than serially to improve performance when possible.
Currently, the only node type that can be run concurrently is a ForeignScan
that is an immediate child of such an Append. In the case where such
ForeignScans access data on different remote servers, this would run those
ForeignScans concurrently, and overlap the remote operations to be performed
simultaneously, so it'll improve the performance especially when the
operations involve time-consuming ones such as remote join and remote
aggregation. We may extend this to other node types such as joins or
aggregates over ForeignScans in the future. This also adds the support for
postgres_fdw, which is enabled by the table-level/server-level option
"async_capable". The default is false. Robert Haas, Kyotaro Horiguchi,
Thomas Munro, and myself. This commit is mostly based on the patch proposed
by Robert Haas, but also uses stuff from the patch proposed by Kyotaro
Horiguchi and from the patch proposed by Thomas Munro. Reviewed by Kyotaro
Horiguchi, Konstantin Knizhnik, Andrey Lepikhov, Movead Li, Thomas Munro,
Justin Pryzby, and others. Discussion:
[https://postgr.es/m/CA%2BTgmoaXQEt4tZ03FtQhnzeDEMzBck%2BLrni0UWHVVgOTnA6C1w%40mail.gmail.com](https://postgr.es/m/CA%2BTgmoaXQEt4tZ03FtQhnzeDEMzBck%2BLrni0UWHVVgOTnA6C1w%40mail.gmail.com)
Discussion:
[https://postgr.es/m/CA%2BhUKGLBRyu0rHrDCMC4%3DRn3252gogyp1SjOgG8SEKKZv%3DFwfQ%40mail.gmail.com](https://postgr.es/m/CA%2BhUKGLBRyu0rHrDCMC4%3DRn3252gogyp1SjOgG8SEKKZv%3DFwfQ%40mail.gmail.com)
Discussion:
[https://postgr.es/m/20200228.170650.667613673625155850.horikyota.ntt%40gmail.com](https://postgr.es/m/20200228.170650.667613673625155850.horikyota.ntt%40gmail.com)
[https://git.postgresql.org/pg/commitdiff/27e1f14563cf982f1f4d71e21ef247866662a052](https://git.postgresql.org/pg/commitdiff/27e1f14563cf982f1f4d71e21ef247866662a052)

Amit Kapila pushed:

- Add a xid argument to the filter_prepare callback for output plugins. Along
with gid, this provides a different way to identify the transaction. The users
that use xid in some way to prepare the transactions can use it to filter
prepare transactions. The later commands COMMIT PREPARED or ROLLBACK PREPARED
carries both identifiers, providing an output plugin the choice of what to
use. Author: Markus Wanner Reviewed-by: Vignesh C, Amit Kapila Discussion:
[https://postgr.es/m/ee280000-7355-c4dc-e47b-2436e7be959c@enterprisedb.com](https://postgr.es/m/ee280000-7355-c4dc-e47b-2436e7be959c@enterprisedb.com)
[https://git.postgresql.org/pg/commitdiff/f64ea6dc5c8ccaec9a3d3d39695ca261febb6883](https://git.postgresql.org/pg/commitdiff/f64ea6dc5c8ccaec9a3d3d39695ca261febb6883)

- Doc: Use consistent terminology for tablesync slots. At some places in the
docs, we refer to them as tablesync slots and at other places as table
synchronization slots. For consistency, we refer to them as table
synchronization slots at all places. Author: Peter Smith Reviewed-by: Amit
Kapila Discussion:
[https://postgr.es/m/CAHut+PvzYNKCeZ=kKBDkh3dw-r=2D3fk=nNc9SXSW=CZGk69xg@mail.gmail.com](https://postgr.es/m/CAHut+PvzYNKCeZ=kKBDkh3dw-r=2D3fk=nNc9SXSW=CZGk69xg@mail.gmail.com)
[https://git.postgresql.org/pg/commitdiff/9f45631766bd0c51a74102770737ba3b0561977e](https://git.postgresql.org/pg/commitdiff/9f45631766bd0c51a74102770737ba3b0561977e)

- Remove extra semicolon in postgres_fdw tests. Author: Suraj Kharage
Reviewed-by: Bharath Rupireddy, Vignesh C Discussion:
[https://postgr.es/m/CAF1DzPWRfxUeH-wShz7P_pK5Tx6M_nEK+TkS8gn5ngvg07Q5=g@mail.gmail.com](https://postgr.es/m/CAF1DzPWRfxUeH-wShz7P_pK5Tx6M_nEK+TkS8gn5ngvg07Q5=g@mail.gmail.com)
[https://git.postgresql.org/pg/commitdiff/13cb5bd84657ed49021fe6fc4ce46601c315c9a5](https://git.postgresql.org/pg/commitdiff/13cb5bd84657ed49021fe6fc4ce46601c315c9a5)

- Ensure to send a prepare after we detect concurrent abort during decoding. It
is possible that while decoding a prepared transaction, it gets aborted
concurrently via a ROLLBACK PREPARED command. In that case, we were skipping
all the changes and directly sending Rollback Prepared when we find the same
in WAL. However, the downstream has no idea of the GID of such a transaction.
So, ensure to send prepare even when a concurrent abort is detected. Author:
Ajin Cherian Reviewed-by: Markus Wanner, Amit Kapila Discussion:
[https://postgr.es/m/f82133c6-6055-b400-7922-97dae9f2b50b@enterprisedb.com](https://postgr.es/m/f82133c6-6055-b400-7922-97dae9f2b50b@enterprisedb.com)
[https://git.postgresql.org/pg/commitdiff/4778826532a62fd6e4d3fdeef9532c943604c730](https://git.postgresql.org/pg/commitdiff/4778826532a62fd6e4d3fdeef9532c943604c730)

Tom Lane pushed:

- Further tweaking of pg_dump's handling of default_toast_compression. As
committed in bbe0a81db, pg_dump from a pre-v14 server effectively acts as
though you'd said --no-toast-compression. I think the right thing is for it
to act as though default_toast_compression is set to "pglz", instead, so that
the tables' toast compression behavior is preserved. You can always get the
other behavior, if you want that, by giving the switch. Discussion:
[https://postgr.es/m/1112852.1616609702@sss.pgh.pa.us](https://postgr.es/m/1112852.1616609702@sss.pgh.pa.us)
[https://git.postgresql.org/pg/commitdiff/54bb91c30e3964fd81059e6b02e377cc9dd2d64c](https://git.postgresql.org/pg/commitdiff/54bb91c30e3964fd81059e6b02e377cc9dd2d64c)

- Remove small inefficiency in ExecARDeleteTriggers/ExecARUpdateTriggers. Whilst
poking at nodeModifyTable.c, I chanced to notice that while its calls to
ExecBR*Triggers and ExecIR*Triggers are protected by tests to see if there are
any relevant triggers to fire, its calls to ExecAR*Triggers are not; the
latter functions do the equivalent tests themselves. This seems possibly
reasonable given the more complex conditions involved, but what's less
reasonable is that the ExecAR* functions aren't careful to do no work when
there is no work to be done. ExecARInsertTriggers gets this right, but the
other two will both force creation of a slot that the query may have no use
for. ExecARUpdateTriggers additionally performed a usually-useless
ExecClearTuple() on that slot. This is probably all pretty microscopic in
real workloads, but a cycle shaved is a cycle earned.
[https://git.postgresql.org/pg/commitdiff/65158f497a7d7523ad438b2034d01a560fafe6bd](https://git.postgresql.org/pg/commitdiff/65158f497a7d7523ad438b2034d01a560fafe6bd)

- Rework planning and execution of UPDATE and DELETE. This patch makes two
closely related sets of changes: 1. For UPDATE, the subplan of the
ModifyTable node now only delivers the new values of the changed columns
(i.e., the expressions computed in the query's SET clause) plus row identity
information such as CTID. ModifyTable must re-fetch the original tuple to
merge in the old values of any unchanged columns. The core advantage of this
is that the changed columns are uniform across all tables of an inherited or
partitioned target relation, whereas the other columns might not be. A
secondary advantage, when the UPDATE involves joins, is that less data needs
to pass through the plan tree. The disadvantage of course is an extra fetch
of each tuple to be updated. However, that seems to be very nearly free in
context; even worst-case tests don't show it to add more than a couple percent
to the total query cost. At some point it might be interesting to combine the
re-fetch with the tuple access that ModifyTable must do anyway to mark the old
tuple dead; but that would require a good deal of refactoring and it seems it
wouldn't buy all that much, so this patch doesn't attempt it. 2. For
inherited UPDATE/DELETE, instead of generating a separate subplan for each
target relation, we now generate a single subplan that is just exactly like a
SELECT's plan, then stick ModifyTable on top of that. To let ModifyTable know
which target relation a given incoming row refers to, a tableoid junk column
is added to the row identity information. This gets rid of the horrid hack
that was inheritance_planner(), eliminating O(N^2) planning cost and memory
consumption in cases where there were many unprunable target relations. Point
2 of course requires point 1, so that there is a uniform definition of the
non-junk columns to be returned by the subplan. We can't insist on uniform
definition of the row identity junk columns however, if we want to keep the
ability to have both plain and foreign tables in a partitioning hierarchy.
Since it wouldn't scale very far to have every child table have its own row
identity column, this patch includes provisions to merge similar row identity
columns into one column of the subplan result. In particular, we can merge the
whole-row Vars typically used as row identity by FDWs into one column by
pretending they are type RECORD. (It's still okay for the actual composite
Datums to be labeled with the table's rowtype OID, though.) There is more
that can be done to file down residual inefficiencies in this patch, but it
seems to be committable now. FDW authors should note several API changes: *
The argument list for AddForeignUpdateTargets() has changed, and so has the
method it must use for adding junk columns to the query. Call
add_row_identity_var() instead of manipulating the parse tree directly. You
might want to reconsider exactly what you're adding, too. *
PlanDirectModify() must now work a little harder to find the ForeignScan plan
node; if the foreign table is part of a partitioning hierarchy then the
ForeignScan might not be the direct child of ModifyTable. See postgres_fdw
for sample code. * To check whether a relation is a target relation, it's no
longer sufficient to compare its relid to root->parse->resultRelation.
Instead, check it against all_result_relids or leaf_result_relids, as
appropriate. Amit Langote and Tom Lane Discussion:
[https://postgr.es/m/CA+HiwqHpHdqdDn48yCEhynnniahH78rwcrv1rEX65-fsZGBOLQ@mail.gmail.com](https://postgr.es/m/CA+HiwqHpHdqdDn48yCEhynnniahH78rwcrv1rEX65-fsZGBOLQ@mail.gmail.com)
[https://git.postgresql.org/pg/commitdiff/86dc90056dfdbd9d1b891718d2e5614e3e432f35](https://git.postgresql.org/pg/commitdiff/86dc90056dfdbd9d1b891718d2e5614e3e432f35)

- Improve style of some replication-related error messages. Put the remote end's
error message into the primary error string, instead of relegating it to
errdetail(). Although this could end up being awkward if the remote sends us
a really long error message, it seems more in keeping with our message style
guidelines, and more helpful in situations where the errdetail could get
dropped. Peter Smith Discussion:
[https://postgr.es/m/CAHut+Ps-Qv2yQceCwobQDP0aJOkfDzRFrOaR6+2Op2K=WHGeWg@mail.gmail.com](https://postgr.es/m/CAHut+Ps-Qv2yQceCwobQDP0aJOkfDzRFrOaR6+2Op2K=WHGeWg@mail.gmail.com)
[https://git.postgresql.org/pg/commitdiff/6197db5340b8154adce1c6d07f6d3325547429c1](https://git.postgresql.org/pg/commitdiff/6197db5340b8154adce1c6d07f6d3325547429c1)

- Suppress compiler warning in libpq_pipeline.c. Some compilers seem to be
concerned about the possibility that recv_step is not any of the defined enum
values. Silence warnings about uninitialized cmdtag in a different way than I
did in 9fb9691a8.
[https://git.postgresql.org/pg/commitdiff/522d1a89f8d7ed45681988c60bd0a687332a4023](https://git.postgresql.org/pg/commitdiff/522d1a89f8d7ed45681988c60bd0a687332a4023)

- Don't prematurely cram a value into a short int. Since a4d75c86b, some
buildfarm members have been warning that Assert(attnum <= MaxAttrNumber);
is useless if attnum is an AttrNumber. I'm not certain how plausible it is
that the value coming out of the bitmap could actually exceed MaxAttrNumber,
but we seem to have thought that that was possible back in 7300a6995. Revert
the intermediate variable to int so that we have the same overflow protection
as before.
[https://git.postgresql.org/pg/commitdiff/c545e9524dcfcfce25c370f584b31562e8d7a4b7](https://git.postgresql.org/pg/commitdiff/c545e9524dcfcfce25c370f584b31562e8d7a4b7)

- Silence compiler warning in non-assert builds. Per buildfarm.
[https://git.postgresql.org/pg/commitdiff/8998e3cafa23632790787b8cc726998e84067259](https://git.postgresql.org/pg/commitdiff/8998e3cafa23632790787b8cc726998e84067259)

- Fix portability and safety issues in pqTraceFormatTimestamp. Remove confusion
between time_t and pg_time_t; neither gettimeofday() nor localtime() deal in
the latter. libpq indeed has no business using <pgtime.h> at all. Use
snprintf not sprintf, to ensure we can't overrun the supplied buffer.
(Unlikely, but let's be safe.) Per buildfarm.
[https://git.postgresql.org/pg/commitdiff/f1be740a991406d7885047beb971e1ff5dbe8b71](https://git.postgresql.org/pg/commitdiff/f1be740a991406d7885047beb971e1ff5dbe8b71)

- Fix unportable use of isprint(). We must cast the arguments of <ctype.h>
functions to unsigned char to avoid problems where char is signed. Speaking
of which, considering that this *is* a <ctype.h> function, it's rather
remarkable that we aren't seeing more complaints about not having included
that header. Per buildfarm.
[https://git.postgresql.org/pg/commitdiff/9e20406dd847d0f8c1cbd803786c6d0ad33bcbdd](https://git.postgresql.org/pg/commitdiff/9e20406dd847d0f8c1cbd803786c6d0ad33bcbdd)

- Fix pg_restore's misdesigned code for detecting archive file format. Despite
the clear comments pointing out that the duplicative code segments in
ReadHead() and `_discoverArchiveFormat()` needed to be in sync, they were not:
the latter did not bother to apply any of the sanity checks in the former.
We'd missed noticing this partly because none of those checks would fail in
scenarios we customarily test, and partly because the oversight would be
masked if both segments execute, which they would in cases other than needing
to autodetect the format of a non-seekable stdin source. However, in a case
meeting all these requirements --- for example, trying to read a
newer-than-supported archive format from non-seekable stdin --- pg_restore
missed applying the version check and would likely dump core or otherwise
misbehave. The whole thing is silly anyway, because there seems little reason
to duplicate the logic beyond the one-line verification that the file starts
with "PGDMP". There seems to have been an undocumented assumption that
multiple major formats (major enough to require separate reader modules) would
nonetheless share the first half-dozen fields of the custom-format header.
This seems unlikely, so let's fix it by just nuking the duplicate logic in
`_discoverArchiveFormat()`. Also get rid of the pointless attempt to seek back
to the start of the file after successful autodetection. That wastes cycles
and it means we have four behaviors to verify not two. Per bug #16951 from
Sergey Koposov. This has been broken for decades, so back-patch to all
supported versions. Discussion:
[https://postgr.es/m/16951-a4dd68cf0de23048@postgresql.org](https://postgr.es/m/16951-a4dd68cf0de23048@postgresql.org)
[https://git.postgresql.org/pg/commitdiff/ec03f2df17a8ba5b431b34dd924e020a0be729f6](https://git.postgresql.org/pg/commitdiff/ec03f2df17a8ba5b431b34dd924e020a0be729f6)

- Rethink handling of pass-by-value leaf datums in SP-GiST. The existing
convention in SP-GiST is that any pass-by-value datatype is stored in Datum
representation, i.e. it's of width sizeof(Datum) even when typlen is less than
that. This is okay, or at least it's too late to change it, for prefix datums
and node-label datums in inner (upper) tuples. But it's problematic for leaf
datums, because we'd prefer those to be stored in Postgres' standard on-disk
representation so that we can easily extend leaf tuples to carry additional
"included" columns. I believe, however, that we can get away with just up and
changing that. This would be an unacceptable on-disk-format break, but there
are two big mitigating factors: 1. It seems quite unlikely that there are any
SP-GiST opclasses out there that use pass-by-value leaf datatypes. Certainly
none of the ones in core do, nor has codesearch.debian.net heard of any.
Given what SP-GiST is good for, it's hard to conceive of a use-case where the
leaf-level values would be both small and fixed-width. (As an example, if you
wanted to index text values with the leaf level being just a byte, then every
text string would have to be represented with one level of inner tuple per
preceding byte, which would be horrendously space-inefficient and slow to
access. You always want to use as few inner-tuple levels as possible, leaving
as much as possible in the leaf values.) 2. Even granting that you have such
an index, this change only breaks things on big-endian machines. On
little-endian, the high order bytes of the Datum format will now just appear
to be alignment padding space. So, change the code to store pass-by-value
leaf datums in their usual on-disk form. Inner-tuple datums are not touched.
This is extracted from a larger patch that intends to add support for
"included" columns. I'm committing it separately for visibility in our commit
logs. Pavel Borisov and Tom Lane, reviewed by Andrey Borodin Discussion:
[https://postgr.es/m/CALT9ZEFi-vMp4faht9f9Junb1nO3NOSjhpxTmbm1UGLMsLqiEQ@mail.gmail.com](https://postgr.es/m/CALT9ZEFi-vMp4faht9f9Junb1nO3NOSjhpxTmbm1UGLMsLqiEQ@mail.gmail.com)
[https://git.postgresql.org/pg/commitdiff/1ebdec8c03294e55a9fdb6e676a9e8de680231cc](https://git.postgresql.org/pg/commitdiff/1ebdec8c03294e55a9fdb6e676a9e8de680231cc)

- Strip file names reported in error messages on Windows, too. Commit dd136052b
established a policy that error message FILE items should include only the
base name of the reporting source file, for uniformity and succinctness. We
now observe that some Windows compilers use backslashes in __FILE__ strings,
so truncate at backslashes as well. This is expected to fix some platform
variation in the results of the new libpq_pipeline test module. Discussion:
[https://postgr.es/m/3650140.1617372290@sss.pgh.pa.us](https://postgr.es/m/3650140.1617372290@sss.pgh.pa.us)
[https://git.postgresql.org/pg/commitdiff/53aafdb9ff6a561c7dea0f428a7c168f2b7e0f16](https://git.postgresql.org/pg/commitdiff/53aafdb9ff6a561c7dea0f428a7c168f2b7e0f16)

- Improve psql's behavior when the editor is exited without saving. When editing
the previous query buffer, if the editor is exited without modifying the temp
file then clear the query buffer, rather than re-loading (and probably
re-executing) the previous query buffer. This reduces the probability of
accidentally re-executing something you didn't intend to. Similarly, in "\e
file", if the file isn't actually modified then don't load it into the query
buffer. And in "\ef" and "\ev", if no changes are made then clear the query
buffer instead of loading the function or view definition into it. Cases
where we fail to invoke the editor at all, or it returns a nonzero status, are
treated like the no-file-modification case. Laurenz Albe, reviewed by Jacob
Champion Discussion:
[https://postgr.es/m/0ba3f2a658bac6546d9934ab6ba63a805d46a49b.camel@cybertec.at](https://postgr.es/m/0ba3f2a658bac6546d9934ab6ba63a805d46a49b.camel@cybertec.at)
[https://git.postgresql.org/pg/commitdiff/55873a00e3c3349664e7215077dca74ccea08b4d](https://git.postgresql.org/pg/commitdiff/55873a00e3c3349664e7215077dca74ccea08b4d)

- Fix confusion in SP-GiST between attribute type and leaf storage type.
According to the documentation, the attType passed to the opclass config
function (and also relied on by the core code) is the type of the heap column
or expression being indexed. But what was actually being passed was the type
stored for the index column. This made no difference for user-defined SP-GiST
opclasses, because we weren't allowing the STORAGE clause of CREATE OPCLASS to
be used, so the two types would be the same. But it's silly not to allow
that, seeing that the built-in poly_ops opclass has a different value for
opckeytype than opcintype, and that if you want to do lossy storage then the
types must really be different. (Thus, user-defined opclasses doing lossy
storage had to lie about what type is in the index.) Hence, remove the
restriction, and make sure that we use the input column type not opckeytype
where relevant. For reasons of backwards compatibility with existing
user-defined opclasses, we can't quite insist that the specified leafType
match the STORAGE clause; instead just add an amvalidate() warning if they
don't match. Also fix some bugs that would only manifest when trying to
return index entries when attType is different from attLeafType. It's not too
surprising that these have not been reported, because the only usual reason
for such a difference is to store the leaf value lossily, rendering index-only
scans impossible. Add a src/test/modules module to exercise cases where
attType is different from attLeafType and yet index-only scan is supported.
Discussion:
[https://postgr.es/m/3728741.1617381471@sss.pgh.pa.us](https://postgr.es/m/3728741.1617381471@sss.pgh.pa.us)
[https://git.postgresql.org/pg/commitdiff/ac9099fc1dd460bffaafec19272159dd7bc86f5b](https://git.postgresql.org/pg/commitdiff/ac9099fc1dd460bffaafec19272159dd7bc86f5b)

Stephen Frost pushed:

- Use a WaitLatch for vacuum/autovacuum sleeping. Instead of using pg_usleep()
in vacuum_delay_point(), use a WaitLatch. This has the advantage that we will
realize if the postmaster has been killed since the last time we decided to
sleep while vacuuming. Reviewed-by: Thomas Munro Discussion:
[https://postgr.es/m/CAFh8B=kcdk8k-Y21RfXPu5dX=bgPqJ8TC3p_qxR_ygdBS=JN5w@mail.gmail.com](https://postgr.es/m/CAFh8B=kcdk8k-Y21RfXPu5dX=bgPqJ8TC3p_qxR_ygdBS=JN5w@mail.gmail.com)
[https://git.postgresql.org/pg/commitdiff/4753ef37e0eda4ba0af614022d18fcbc5a946cc9](https://git.postgresql.org/pg/commitdiff/4753ef37e0eda4ba0af614022d18fcbc5a946cc9)

- Add a docs section for obsoleted and renamed functions and settings. The new
appendix groups information on renamed or removed settings, commands, etc into
an out-of-the-way part of the docs. The original id elements are retained in
each subsection to ensure that the same filenames are produced for HTML docs.
This prevents /current/ links on the web from breaking, and allows users of
the web docs to follow links from old version pages to info on the changes in
the new version. Prior to this change, a link to /current/ for renamed
sections like the recovery.conf docs would just 404. Similarly if someone
searched for recovery.conf they would find the pg11 docs, but there would be
no /12/ or /current/ link, so they couldn't easily find out that it was
removed in pg12 or how to adapt. Index entries are also added so that there's
a breadcrumb trail for users to follow when they know the old name, but not
what we changed it to. So a user who is trying to find out how to set
standby_mode in PostgreSQL 12+, or where pg_resetxlog went, now has more
chance of finding that information. Craig Ringer and Stephen Frost
Reviewed-by: Euler Taveira Discussion:
[https://postgr.es/m/CAGRY4nzPNOyYQ_1-pWYToUVqQ0ThqP5jdURnJMZPm539fdizOg%40mail.gmail.com](https://postgr.es/m/CAGRY4nzPNOyYQ_1-pWYToUVqQ0ThqP5jdURnJMZPm539fdizOg%40mail.gmail.com)
Backpatch-through: 10
[https://git.postgresql.org/pg/commitdiff/3b0c647bbfc52894d979976f1e6d60e40649bba7](https://git.postgresql.org/pg/commitdiff/3b0c647bbfc52894d979976f1e6d60e40649bba7)

- Rename Default Roles to Predefined Roles. The term 'default roles' wasn't
quite apt as these roles aren't able to be modified or removed after
installation, so rename them to be 'Predefined Roles' instead, adding an entry
into the newly added Obsolete Appendix to help users of current releases find
the new documentation. Bruce Momjian and Stephen Frost Discussion:
[https://postgr.es/m/157742545062.1149.11052653770497832538%40wrigleys.postgresql.org](https://postgr.es/m/157742545062.1149.11052653770497832538%40wrigleys.postgresql.org)
and
[https://www.postgresql.org/message-id/20201120211304.GG16415@tamriel.snowman.net](https://www.postgresql.org/message-id/20201120211304.GG16415@tamriel.snowman.net)
[https://git.postgresql.org/pg/commitdiff/c9c41c7a337d3e2deb0b2a193e9ecfb865d8f52b](https://git.postgresql.org/pg/commitdiff/c9c41c7a337d3e2deb0b2a193e9ecfb865d8f52b)

Bruce Momjian pushed:

- In messages, use singular nouns for -1, like we do for +1. This outputs "-1
year", not "-1 years". Reported-by: neverov(dot)max(at)gmail(dot)com Bug: 16939
Discussion:
[https://postgr.es/m/16939-cceeb03fb72736ee@postgresql.org](https://postgr.es/m/16939-cceeb03fb72736ee@postgresql.org)
[https://git.postgresql.org/pg/commitdiff/5da9868ed983f95cc1cff80dcd81252a513774f8](https://git.postgresql.org/pg/commitdiff/5da9868ed983f95cc1cff80dcd81252a513774f8)

- adjust dblink regression expected output for commit 5da9868ed9. Seems the
-1/singular output is used in the dblink regression tests. Reported-by:
Álvaro Herrera Discussion:
[https://postgr.es/m/20210330231506.GA10666@alvherre.pgsql](https://postgr.es/m/20210330231506.GA10666@alvherre.pgsql)
[https://git.postgresql.org/pg/commitdiff/9ee7d533dacf8594057ced2d016250f09056c284](https://git.postgresql.org/pg/commitdiff/9ee7d533dacf8594057ced2d016250f09056c284)

- doc: mention that intervening major releases can be skipped. Also mention
that you should read the intervening major releases notes. This change was
also applied to the website. Discussion:
[https://postgr.es/m/20210330144949.GA8259@momjian.us](https://postgr.es/m/20210330144949.GA8259@momjian.us)
Backpatch-through: 9.6
[https://git.postgresql.org/pg/commitdiff/2bda93f813919b58225f5a0e282e10b98d7633d4](https://git.postgresql.org/pg/commitdiff/2bda93f813919b58225f5a0e282e10b98d7633d4)

- Use macro MONTHS_PER_YEAR instead of '12' in /ecpg/pgtypeslib. All other
places already use MONTHS_PER_YEAR appropriately. Backpatch-through: 9.6
[https://git.postgresql.org/pg/commitdiff/84bc2b17523ef485f102be7f00f7affb88f00f18](https://git.postgresql.org/pg/commitdiff/84bc2b17523ef485f102be7f00f7affb88f00f18)

Michaël Paquier pushed:

- Add support for --extension in pg_dump. When specified, only extensions
matching the given pattern are included in dumps. Similarly to --table and
--schema, when --strict-names is used, a perfect match is required. Also,
like the two other options, this new option offers no guarantee that dependent
objects have been dumped, so a restore may fail on a clean database. Tests
are added in test_pg_dump/, checking after a set of positive and negative
cases, with or without an extension's contents added to the dump generated.
Author: Guillaume Lelarge Reviewed-by: David Fetter, Tom Lane, Michael
Paquier, Asif Rehman, Julien Rouhaud Discussion:
[https://postgr.es/m/CAECtzeXOt4cnMU5+XMZzxBPJ_wu76pNy6HZKPRBL-j7yj1E4+g@mail.gmail.com](https://postgr.es/m/CAECtzeXOt4cnMU5+XMZzxBPJ_wu76pNy6HZKPRBL-j7yj1E4+g@mail.gmail.com)
[https://git.postgresql.org/pg/commitdiff/6568cef26e0f40c25ae54b8e20aad8d1410a854b](https://git.postgresql.org/pg/commitdiff/6568cef26e0f40c25ae54b8e20aad8d1410a854b)

- Fix comment in parsenodes.h. CreateStmt->inhRelations is a list of RangeVars,
but a comment was incorrect about that. Author: Julien Rouhaud Discussion:
[https://postgr.es/m/20210330123015.yzekhz5sweqbgxdr@nol](https://postgr.es/m/20210330123015.yzekhz5sweqbgxdr@nol)
[https://git.postgresql.org/pg/commitdiff/7ef64e7e72a65f191fc2f7d4bbe220f53dd8d5de](https://git.postgresql.org/pg/commitdiff/7ef64e7e72a65f191fc2f7d4bbe220f53dd8d5de)

- Move some client-specific routines from SSLServer to PostgresNode.
test_connect_ok() and test_connect_fails() have always been part of the SSL
tests, and check if a connection to the backend should work or not, and there
are sanity checks done on specific error patterns dropped by libpq if the
connection fails. This was fundamentally wrong on two aspects. First,
SSLServer.pm works mostly on setting up and changing the SSL configuration of
a PostgresNode, and has really nothing to do with the client. Second, the
situation became worse in light of b34ca595, where the SSL tests would finish
by using a psql command that may not come from the same installation as the
node set up. This commit moves those client routines into PostgresNode,
making easier the refactoring of SSLServer to become more SSL-implementation
aware. This can also be reused by the ldap, kerberos and authentication test
suites for connection checks, and a follow-up patch should extend those
interfaces to match with backend log patterns. Author: Michael Paquier
Reviewed-by: Andrew Dunstan, Daniel Gustafsson, Álvaro Herrera Discussion:
[https://postgr.es/m/YGLKNBf9zyh6+WSt@paquier.xyz](https://postgr.es/m/YGLKNBf9zyh6+WSt@paquier.xyz)
[https://git.postgresql.org/pg/commitdiff/0d1a33438d3a88938264e12e94c22818307d2f4d](https://git.postgresql.org/pg/commitdiff/0d1a33438d3a88938264e12e94c22818307d2f4d)

- doc: Clarify use of ACCESS EXCLUSIVE lock in various sections. Some sections
of the documentation used "exclusive lock" to describe that an ACCESS
EXCLUSIVE lock is taken during a given operation. This can be confusing to
the reader as ACCESS SHARE is allowed with an EXCLUSIVE lock is used, but that
would not be the case with what is described on those parts of the
documentation. Author: Greg Rychlewski Discussion:
[https://postgr.es/m/CAKemG7VptD=7fNWckFMsMVZL_zzvgDO6v2yVmQ+ZiBfc_06kCQ@mail.gmail.com](https://postgr.es/m/CAKemG7VptD=7fNWckFMsMVZL_zzvgDO6v2yVmQ+ZiBfc_06kCQ@mail.gmail.com)
Backpatch-through: 9.6
[https://git.postgresql.org/pg/commitdiff/ffd3391ea94165fbb5adc9534894c62d41138505](https://git.postgresql.org/pg/commitdiff/ffd3391ea94165fbb5adc9534894c62d41138505)

- Improve stability of test with vacuum_truncate in reloptions.sql. This test
has been using a simple VACUUM with pg_relation_size() to check if a relation
gets physically truncated or not, but forgot the fact that some concurrent
activity, like checkpoint buffer writes, could cause some pages to be skipped.
The second test enabling vacuum_truncate could fail, seeing a non-empty
relation. The first test would not have failed, but could finish by testing a
behavior different than the one aimed for. Both tests gain a FREEZE option,
to make the vacuums more aggressive and prevent page skips. This is similar
to the issues fixed in c2dc1a7. Author: Arseny Sher Reviewed-by: Masahiko
Sawada Discussion:
[https://postgr.es/m/87tuotr2hh.fsf@ars-thinkpad](https://postgr.es/m/87tuotr2hh.fsf@ars-thinkpad)
backpatch-through: 12
[https://git.postgresql.org/pg/commitdiff/fe246d1c111d43fd60a1b0afff25ed09b7ae11eb](https://git.postgresql.org/pg/commitdiff/fe246d1c111d43fd60a1b0afff25ed09b7ae11eb)

- doc: Clarify how to generate backup files with non-exclusive backups. The
current instructions describing how to write the backup_label and
tablespace_map files are confusing. For example, opening a file in text mode
on Windows and copy-pasting the file's contents would result in a failure at
recovery because of the extra CRLF characters generated. The documentation
was not stating that clearly, and per discussion this is not considered as a
supported scenario. This commit extends a bit the documentation to mention
that it may be required to open the file in binary mode before writing its
data. Reported-by: Wang Shenhao Author: David Steele Reviewed-by: Andrew
Dunstan, Magnus Hagander Discussion:
[https://postgr.es/m/8373f61426074f2cb6be92e02f838389@G08CNEXMBPEKD06.g08.fujitsu.local](https://postgr.es/m/8373f61426074f2cb6be92e02f838389@G08CNEXMBPEKD06.g08.fujitsu.local)
Backpatch-through: 9.6
[https://git.postgresql.org/pg/commitdiff/6fb66c268df2de1112cac3cf0a6cf0a8b96ceaf0](https://git.postgresql.org/pg/commitdiff/6fb66c268df2de1112cac3cf0a6cf0a8b96ceaf0)

- Refactor HMAC implementations. Similarly to the cryptohash implementations,
this refactors the existing HMAC code into a single set of APIs that can be
plugged with any crypto libraries PostgreSQL is built with (only OpenSSL
currently). If there is no such libraries, a fallback implementation is
available. Those new APIs are designed similarly to the existing cryptohash
layer, so there is no real new design here, with the same logic around buffer
bound checks and memory handling. HMAC has a dependency on cryptohashes, so
all the cryptohash types supported by `cryptohash{_openssl}.c` can be used with
HMAC. This refactoring is an advantage mainly for SCRAM, that included its
own implementation of HMAC with SHA256 without relying on the existing crypto
libraries even if PostgreSQL was built with their support. This code has been
tested on Windows and Linux, with and without OpenSSL, across all the versions
supported on HEAD from 1.1.1 down to 1.0.1. I have also checked that the
implementations are working fine using some sample results, a custom extension
of my own, and doing cross-checks across different major versions with SCRAM
with the client and the backend. Author: Michael Paquier Reviewed-by: Bruce
Momjian Discussion:
[https://postgr.es/m/X9m0nkEJEzIPXjeZ@paquier.xyz](https://postgr.es/m/X9m0nkEJEzIPXjeZ@paquier.xyz)
[https://git.postgresql.org/pg/commitdiff/e6bdfd9700ebfc7df811c97c2fc46d7e94e329a2](https://git.postgresql.org/pg/commitdiff/e6bdfd9700ebfc7df811c97c2fc46d7e94e329a2)

- Use more verbose matching patterns for errors in SSL TAP tests. The TAP tests
of src/test/ssl/ have been using rather generic matching patterns to check
some failure scenarios, like "SSL error" or just "FATAL". These have been
introduced in 081bfc1. Those messages are not wrong per se, but when working
on the integration of new SSL libraries it becomes hard to know if those
errors are legit or not, and existing scenarios may fail in incorrect ways.
This commit makes all those messages more verbose by adding the information
generated by OpenSSL. Fortunately, the same error messages are used for all
the versions supported on HEAD (checked that after running the tests from
1.0.1 to 1.1.1), so the change is straight-forward. Reported-by: Jacob
Champion, Álvaro Herrera Discussion:
[https://postgr.es/m/YGU3AxQh0zBMMW8m@paquier.xyz](https://postgr.es/m/YGU3AxQh0zBMMW8m@paquier.xyz)
[https://git.postgresql.org/pg/commitdiff/8d3a4c3eae5367fba60ab77c159814defba784fe](https://git.postgresql.org/pg/commitdiff/8d3a4c3eae5367fba60ab77c159814defba784fe)

Noah Misch pushed:

- Accept slightly-filled pages for tuples larger than fillfactor. We always
inserted a larger-than-fillfactor tuple into a newly-extended page, even when
existing pages were empty or contained nothing but an unused line pointer.
This was unnecessary relation extension. Start tolerating page usage up to
1/8 the maximum space that could be taken up by line pointers. This is
somewhat arbitrary, but it should allow more cases to reuse pages. This has
no effect on tables with fillfactor=100 (the default). John Naylor and Floris
van Nee. Reviewed by Matthias van de Meent. Reported by Floris van Nee.
Discussion:
[https://postgr.es/m/6e263217180649339720afe2176c50aa@opammb0562.comp.optiver.com](https://postgr.es/m/6e263217180649339720afe2176c50aa@opammb0562.comp.optiver.com)
[https://git.postgresql.org/pg/commitdiff/0ff8bbdee19a9db2794a95d439c946ab017d0acd](https://git.postgresql.org/pg/commitdiff/0ff8bbdee19a9db2794a95d439c946ab017d0acd)

Joe Conway pushed:

- Fix has_column_privilege function corner case. According to the comments, when
an invalid or dropped column oid is passed to has_column_privilege(), the
intention has always been to return NULL. However, when the caller had table
level privilege the invalid/missing column was never discovered, because table
permissions were checked first. Fix that by introducing extended versions of
pg_attribute_acl(check|mask) and pg_class_acl(check|mask) which take a new
argument, is_missing. When is_missing is NULL, the old behavior is preserved.
But when is_missing is passed by the caller, no ERROR is thrown for dropped or
missing columns/relations, and is_missing is flipped to true. This in turn
allows has_column_privilege to check for column privileges first, providing
the desired semantics. Not backpatched since it is a user visible behavioral
change with no previous complaints, and the fix is a bit on the invasive side.
Author: Joe Conway Reviewed-By: Tom Lane Reported by: Ian Barwick Discussion:
[https://postgr.es/m/flat/9b5f4311-157b-4164-7fe7-077b4fe8ed84%40joeconway.com](https://postgr.es/m/flat/9b5f4311-157b-4164-7fe7-077b4fe8ed84%40joeconway.com)
[https://git.postgresql.org/pg/commitdiff/b12bd4869b5e64b742a69ca07915e2f77f85a9ae](https://git.postgresql.org/pg/commitdiff/b12bd4869b5e64b742a69ca07915e2f77f85a9ae)

- Clarify documentation of RESET ROLE. Command-line options, or previous "ALTER
(ROLE|DATABASE) ... SET ROLE ..." commands, can change the value of the
default role for a session. In the presence of one of these, RESET ROLE will
change the current user identifier to the default role rather than the session
user identifier. Fix the documentation to reflect this reality. Backpatch to
all supported versions. Author: Nathan Bossart Reviewed-By: Laurenz Albe,
David G. Johnston, Joe Conway Reported by: Nathan Bossart Discussion:
[https://postgr.es/m/flat/925134DB-8212-4F60-8AB1-B1231D750CB4%40amazon.com](https://postgr.es/m/flat/925134DB-8212-4F60-8AB1-B1231D750CB4%40amazon.com)
Backpatch-through: 9.6
[https://git.postgresql.org/pg/commitdiff/174edbe9f9c1538ab3347474e96d176223591cd1](https://git.postgresql.org/pg/commitdiff/174edbe9f9c1538ab3347474e96d176223591cd1)

Heikki Linnakangas pushed:

- Add 'noError' argument to encoding conversion functions. With the 'noError'
argument, you can try to convert a buffer without knowing the character
boundaries beforehand. The functions now need to return the number of input
bytes successfully converted. This is is a backwards-incompatible change, if
you have created a custom encoding conversion with CREATE CONVERSION. This
adds a check to pg_upgrade for that, refusing the upgrade if there are any
user-defined encoding conversions. Custom conversions are very rare, there are
no commonly used extensions that I know of that uses that feature. No other
objects can depend on conversions, so if you do have one, you can fairly
easily drop it before upgrading, and recreate it after the upgrade with an
updated version. Add regression tests for built-in encoding conversions. This
doesn't cover every conversion, but it covers all the internal functions in
conv.c that are used to implement the conversions. Reviewed-by: John Naylor
Discussion:
[https://www.postgresql.org/message-id/e7861509-3960-538a-9025-b75a61188e01%40iki.fi](https://www.postgresql.org/message-id/e7861509-3960-538a-9025-b75a61188e01%40iki.fi)
[https://git.postgresql.org/pg/commitdiff/ea1b99a6619cd9dcfd46b82ac0d926b0b80e0ae9](https://git.postgresql.org/pg/commitdiff/ea1b99a6619cd9dcfd46b82ac0d926b0b80e0ae9)

- Do COPY FROM encoding conversion/verification in larger chunks. This gives a
small performance gain, by reducing the number of calls to the
conversion/verification function, and letting it work with larger inputs.
Also, reorganizing the input pipeline makes it easier to parallelize the input
parsing: after the input has been converted to the database encoding, the next
stage of finding the newlines can be done in parallel, because there cannot be
any newline chars "embedded" in multi-byte characters in the encodings that we
support as server encodings. This changes behavior in one corner case: if
client and server encodings are the same single-byte encoding (e.g. latin1),
previously the input would not be checked for zero bytes ('\0'). Any fields
containing zero bytes would be truncated at the zero. But if encoding
conversion was needed, the conversion routine would throw an error on the
zero. After this commit, the input is always checked for zeros. Reviewed-by:
John Naylor Discussion:
[https://www.postgresql.org/message-id/e7861509-3960-538a-9025-b75a61188e01%40iki.fi](https://www.postgresql.org/message-id/e7861509-3960-538a-9025-b75a61188e01%40iki.fi)
[https://git.postgresql.org/pg/commitdiff/f82de5c46bdf8cd65812a7b04c9509c218e1545d](https://git.postgresql.org/pg/commitdiff/f82de5c46bdf8cd65812a7b04c9509c218e1545d)

Robert Haas pushed:

- amcheck: Fix verify_heapam's tuple visibility checking rules. We now follow
the order of checks from `HeapTupleSatisfies*` more closely to avoid coming to
erroneous conclusions. Mark Dilger and Robert Haas Discussion:
[http://postgr.es/m/CA+Tgmob6sii0yTvULYJ0Vq4w6ZBmj7zUhddL3b+SKDi9z9NA7Q@mail.gmail.com](http://postgr.es/m/CA+Tgmob6sii0yTvULYJ0Vq4w6ZBmj7zUhddL3b+SKDi9z9NA7Q@mail.gmail.com)
[https://git.postgresql.org/pg/commitdiff/3b6c1259f9ca8e21860aaf24ec6735a8e5598ea0](https://git.postgresql.org/pg/commitdiff/3b6c1259f9ca8e21860aaf24ec6735a8e5598ea0)

Fujii Masao pushed:

- Fix typos in comments. Author: Masahiko Sawada Discussion:
[https://postgr.es/m/CAD21AoA1YL7t0nzVSEySx6zOaE7xO3r0jyu8hkitGL2_XbaMxQ@mail.gmail.com](https://postgr.es/m/CAD21AoA1YL7t0nzVSEySx6zOaE7xO3r0jyu8hkitGL2_XbaMxQ@mail.gmail.com)
[https://git.postgresql.org/pg/commitdiff/98e5bd103f887326e381c509c2fbe879ba3ea2f3](https://git.postgresql.org/pg/commitdiff/98e5bd103f887326e381c509c2fbe879ba3ea2f3)

- Fix pgstat_report_replslot() to use proper data types for its arguments. The
caller of pgstat_report_replslot() passes int64 values to the function. Also
the function stores those values in PgStat_Counter (i.e., int64) fields of
PgStat_MsgReplSlot struct. But previously the function used "int" as the data
types of some arguments for those values, which could lead to the overflow of
values. To avoid this risk, this commit fixes pgstat_report_replslot() to use
PgStat_Counter type for the arguments. Since they are the statistics counters,
PgStat_Counter, the data type used for counters, is used for them instead of
int64. Reported-by: Vignesh C Author: Vignesh C Reviewed-by: Jeevan Ladhe,
Fujii Masao Discussion:
[https://postgr.es/m/CALDaNm080OpG=ZwOb0i8EyChH5SyHAMFWJCKaKTXmrfvJLbgaA@mail.gmail.com](https://postgr.es/m/CALDaNm080OpG=ZwOb0i8EyChH5SyHAMFWJCKaKTXmrfvJLbgaA@mail.gmail.com)
[https://git.postgresql.org/pg/commitdiff/96bdb7e19de80a0c9521c5696455bca2a685c919](https://git.postgresql.org/pg/commitdiff/96bdb7e19de80a0c9521c5696455bca2a685c919)

- postgres_fdw: Add option to control whether to keep connections open. This
commit adds a new option keep_connections that controls whether postgres_fdw
keeps the connections to the foreign server open so that the subsequent
queries can re-use them. This option can only be specified for a foreign
server. The default is on. If set to off, all connections to the foreign
server will be discarded at the end of transaction. Closed connections will be
re-established when they are necessary by future queries using a foreign
table. This option is useful, for example, when users want to prevent the
connections from eating up the foreign servers connections capacity. Author:
Bharath Rupireddy Reviewed-by: Alexey Kondratov, Vignesh C, Fujii Masao
Discussion:
[https://postgr.es/m/CALj2ACVvrp5=AVp2PupEm+nAC8S4buqR3fJMmaCoc7ftT0aD2A@mail.gmail.com](https://postgr.es/m/CALj2ACVvrp5=AVp2PupEm+nAC8S4buqR3fJMmaCoc7ftT0aD2A@mail.gmail.com)
[https://git.postgresql.org/pg/commitdiff/b1be3074ac719ce8073fba35d4c8b52fb4ddd0c3](https://git.postgresql.org/pg/commitdiff/b1be3074ac719ce8073fba35d4c8b52fb4ddd0c3)

- pg_checksums: Fix progress reporting. pg_checksums uses two counters, total
size and current size, to calculate the progress. Previously the progress that
pg_checksums reported could not reach 100% at the end. The cause of this issue
was that the sizes of only pages excluding new ones in each file were counted
as the current size while the size of each file is counted as the total size.
That is, the total size of all new pages could be reported as the difference
between the total size and current size. This commit fixes this issue by
making pg_checksums count the sizes of all pages including new ones in each
file as the current size. Back-patch to v12 where progress reporting was
added to pg_checksums. Reported-by: Shinya Kato Author: Shinya Kato
Reviewed-by: Fujii Masao Discussion:
[https://postgr.es/m/TYAPR01MB289656B1ACA0A5E7CAD07BE3C47A9@TYAPR01MB2896.jpnprd01.prod.outlook.com](https://postgr.es/m/TYAPR01MB289656B1ACA0A5E7CAD07BE3C47A9@TYAPR01MB2896.jpnprd01.prod.outlook.com)
[https://git.postgresql.org/pg/commitdiff/2eb1fc8b1ae8b974007e85636fc7336a9b5d7222](https://git.postgresql.org/pg/commitdiff/2eb1fc8b1ae8b974007e85636fc7336a9b5d7222)

Thomas Munro pushed:

- Detect POLLHUP/POLLRDHUP while running queries. Provide a new GUC
check_client_connection_interval that can be used to check whether the client
connection has gone away, while running very long queries. It is disabled by
default. For now this uses a non-standard Linux extension (also adopted by at
least one other OS). POLLRDHUP is not defined by POSIX, and other OSes don't
have a reliable way to know if a connection was closed without actually trying
to read or write. In future we might consider trying to send a
no-op/heartbeat message instead, but that could require protocol changes.
Author: Sergey Cherkashin <s(dot)cherkashin(at)postgrespro(dot)ru> Author: Thomas Munro
<thomas(dot)munro(at)gmail(dot)com> Reviewed-by: Thomas Munro <thomas(dot)munro(at)gmail(dot)com>
Reviewed-by: Tatsuo Ishii <ishii(at)sraoss(dot)co(dot)jp> Reviewed-by: Konstantin
Knizhnik <k(dot)knizhnik(at)postgrespro(dot)ru> Reviewed-by: Zhihong Yu
<zyu(at)yugabyte(dot)com> Reviewed-by: Andres Freund <andres(at)anarazel(dot)de>
Reviewed-by: Maksim Milyutin <milyutinma(at)gmail(dot)com> Reviewed-by: Tsunakawa,
Takayuki/綱川 貴之 <tsunakawa(dot)takay(at)fujitsu(dot)com> Reviewed-by: Tom Lane
<tgl(at)sss(dot)pgh(dot)pa(dot)us> (much earlier version) Discussion:
[https://postgr.es/m/77def86b27e41f0efcba411460e929ae%40postgrespro.ru](https://postgr.es/m/77def86b27e41f0efcba411460e929ae%40postgrespro.ru)
[https://git.postgresql.org/pg/commitdiff/c30f54ad732ca5c8762bb68bbe0f51de9137dd72](https://git.postgresql.org/pg/commitdiff/c30f54ad732ca5c8762bb68bbe0f51de9137dd72)

Andres Freund pushed:

- Split wait event related code from pgstat.[ch] into wait_event.[ch]. The wait
event related code is independent from the rest of the pgstat.[ch] code, of
nontrivial size and changes on a regular basis. Put it into its own set of
files. As there doesn't seem to be a good pre-existing directory for code
like this, add src/backend/utils/activity. Reviewed-By: Robert Haas
<robertmhaas(at)gmail(dot)com> Discussion:
[https://postgr.es/m/20210316195440.twxmlov24rr2nxrg@alap3.anarazel.de](https://postgr.es/m/20210316195440.twxmlov24rr2nxrg@alap3.anarazel.de)
[https://git.postgresql.org/pg/commitdiff/a333476b925134f6185037eaff3424c07a9f466f](https://git.postgresql.org/pg/commitdiff/a333476b925134f6185037eaff3424c07a9f466f)

- Do not rely on pgstat.h to indirectly include storage/ headers. An upcoming
patch might remove the (now indirect) proc.h include (which in turn includes
other headers), and it's cleaner for the modified files to include their
dependencies directly anyway... Discussion:
[https://postgr.es/m/20210402194458.2vu324hkk2djq6ce@alap3.anarazel.de](https://postgr.es/m/20210402194458.2vu324hkk2djq6ce@alap3.anarazel.de)
[https://git.postgresql.org/pg/commitdiff/1d9c5d0ce2dcac05850401cf266a9df10a68de49](https://git.postgresql.org/pg/commitdiff/1d9c5d0ce2dcac05850401cf266a9df10a68de49)

- Split backend status and progress related functionality out of pgstat.c.
Backend status (supporting pg_stat_activity) and command progress (supporting
`pg_stat_progress*`) related code is largely independent from the rest of
pgstat.[ch] (supporting views like pg_stat_all_tables that accumulate data
over time). See also a333476b925. This commit doesn't rename the function
names to make the distinction from the rest of pgstat_ clearer - that'd be
more invasive and not clearly beneficial. If we were to decide to do such a
rename at some point, it's better done separately from moving the code as
well. Robert's review was of an earlier version. Reviewed-By: Robert Haas
<robertmhaas(at)gmail(dot)com> Discussion:
[https://postgr.es/m/20210316195440.twxmlov24rr2nxrg@alap3.anarazel.de](https://postgr.es/m/20210316195440.twxmlov24rr2nxrg@alap3.anarazel.de)
[https://git.postgresql.org/pg/commitdiff/e1025044cd4e7f33f7304aed54d5778b8a82cd5d](https://git.postgresql.org/pg/commitdiff/e1025044cd4e7f33f7304aed54d5778b8a82cd5d)

- Improve efficiency of wait event reporting, remove proc.h dependency.
pgstat_report_wait_start() and pgstat_report_wait_end() required two
conditional branches so far. One to check if MyProc is NULL, the other to
check if pgstat_track_activities is set. As wait events are used around
comparatively lightweight operations, and are inlined (reducing branch
predictor effectiveness), that's not great. The dependency on MyProc has a
second disadvantage: Low-level subsystems, like storage/file/fd.c, report wait
events, but architecturally it is preferable for them to not depend on
inter-process subsystems like proc.h (defining PGPROC). After this change
including pgstat.h (nor obviously its sub-components like backend_status.h,
wait_event.h, ...) does not pull in IPC related headers anymore. These goals,
efficiency and abstraction, are achieved by having
pgstat_report_wait_start/end() not interact with MyProc, but instead a new
my_wait_event_info variable. At backend startup it points to a local variable,
removing the need to check for MyProc being NULL. During process
initialization my_wait_event_info is redirected to MyProc->wait_event_info. At
shutdown this is reversed. Because wait event reporting now does not need to
know about where the wait event is stored, it does not need to know about
PGPROC anymore. The removal of the branch for checking
pgstat_track_activities is simpler: Don't check anymore. The cost due to the
branch are often higher than the store - and even if not,
pgstat_track_activities is rarely disabled. The main motivator to commit this
work now is that removing the (indirect) pgproc.h include from pgstat.h
simplifies a patch to move statistics reporting to shared memory (which still
has a chance to get into 14). Author: Andres Freund <andres(at)anarazel(dot)de>
Discussion:
[https://postgr.es/m/20210402194458.2vu324hkk2djq6ce@alap3.anarazel.de](https://postgr.es/m/20210402194458.2vu324hkk2djq6ce@alap3.anarazel.de)
[https://git.postgresql.org/pg/commitdiff/225a22b19ed2960acc8e9c0b7ae53e0e5b0eac87](https://git.postgresql.org/pg/commitdiff/225a22b19ed2960acc8e9c0b7ae53e0e5b0eac87)

Tomáš Vondra pushed:

- Fix BRIN minmax-multi distance for interval type. The distance calculation for
interval type was treating months as having 31 days, which is inconsistent
with the interval comparator (using 30 days). Due to this it was possible to
get negative distance (b-a) when `(a<b)`, trigerring an assert. Fixed by
adopting the same logic as interval_cmp_value. Reported-by: Jaime Casanova
Discussion:
[https://postgr.es/m/CAJKUy5jKH0Xhneau2mNftNPtTy-BVgQfXc8zQkEvRvBHfeUThQ%40mail.gmail.com](https://postgr.es/m/CAJKUy5jKH0Xhneau2mNftNPtTy-BVgQfXc8zQkEvRvBHfeUThQ%40mail.gmail.com)
[https://git.postgresql.org/pg/commitdiff/2b10e0e3c2ca14d732521479123e5d5e2094e143](https://git.postgresql.org/pg/commitdiff/2b10e0e3c2ca14d732521479123e5d5e2094e143)

- Fix BRIN minmax-multi distance for timetz type. The distance calculation
ignored the time zone, so the result of (b-a) might have ended negative even
if `(b > a)`. Fixed by considering the time zone difference. Reported-by: Jaime
Casanova Discussion:
[https://postgr.es/m/CAJKUy5jLZFLCxyxfT%3DMfK5mtPfSzHA1rVLowR-j4RRsFVvKm7A%40mail.gmail.com](https://postgr.es/m/CAJKUy5jLZFLCxyxfT%3DMfK5mtPfSzHA1rVLowR-j4RRsFVvKm7A%40mail.gmail.com)
[https://git.postgresql.org/pg/commitdiff/7262f2421a1e099a631356f7b80ad198e34e2a8a](https://git.postgresql.org/pg/commitdiff/7262f2421a1e099a631356f7b80ad198e34e2a8a)

- Fix BRIN minmax-multi distance for inet type. The distance calculation ignored
the mask, unlike the inet comparator, which resulted in negative distance in
some cases. Fixed by applying the mask in brin_minmax_multi_distance_inet.
I've considered simply calling inetmi() to calculate the delta, but that does
not consider mask either. Reviewed-by: Zhihong Yu Discussion:
[https://postgr.es/m/1a0a7b9d-9bda-e3a2-7fa4-88f15042a051%40enterprisedb.com](https://postgr.es/m/1a0a7b9d-9bda-e3a2-7fa4-88f15042a051%40enterprisedb.com)
[https://git.postgresql.org/pg/commitdiff/e1fbe1181c86247eaf8b4b142b81361ce4efcc66](https://git.postgresql.org/pg/commitdiff/e1fbe1181c86247eaf8b4b142b81361ce4efcc66)

- Fix order of parameters in BRIN minmax-multi calls. The BRIN minmax-multi
consistent function incorrectly assumed it can lookup an operator, and then
swap the arguments to get the commutator. For example <(a,b) would be called
as <(b,a) to get >(a,b). This works when the arguments are of the same type,
but with cross-type opclasses this fails. We can't swap <(float4,float8)
arguments, for example. Fixed by passing arguments in the right order.
Discussion:
[https://postgr.es/m/CAJKUy5jLZFLCxyxfT%3DMfK5mtPfSzHA1rVLowR-j4RRsFVvKm7A%40mail.gmail.com](https://postgr.es/m/CAJKUy5jLZFLCxyxfT%3DMfK5mtPfSzHA1rVLowR-j4RRsFVvKm7A%40mail.gmail.com)
[https://git.postgresql.org/pg/commitdiff/1dad2a5ea3d14dd205603c31cc94ec088183ab2a](https://git.postgresql.org/pg/commitdiff/1dad2a5ea3d14dd205603c31cc94ec088183ab2a)

- Add regression test for minmax-multi macaddr8 type. The regression test for
BRIN minmax-multi opclasses tested almost all supported data types, with the
exception of macaddr8. So this adds it.
[https://git.postgresql.org/pg/commitdiff/4908684ddab35135869efa2af6b49c4d67c422f9](https://git.postgresql.org/pg/commitdiff/4908684ddab35135869efa2af6b49c4d67c422f9)

- Fix bug in brin_minmax_multi_union. When calling sort_expanded_ranges() we
need to remember the return value, because the function sorts and also
deduplicates the ranges. So the number of ranges may decrease.
brin_minmax_multi_union failed to do that, which resulted in crashes due to
bogus ranges (equal minval/maxval but not marked as compacted). Reported-by:
Jaime Casanova Discussion:
[https://postgr.es/m/20210404052550.GA4376%40ahch-to](https://postgr.es/m/20210404052550.GA4376%40ahch-to)
[https://git.postgresql.org/pg/commitdiff/d9c5b9a9eeb9e3061ae139e0e564ce5358c94001](https://git.postgresql.org/pg/commitdiff/d9c5b9a9eeb9e3061ae139e0e564ce5358c94001)

# Pending Patches

James Hilliard sent in another revision of a patch to fix detection of
preadv/pwritev support for OSX.

Mark Rofail sent in another revision of a patch to implement foreign key arrays.

Tomáš Vondra sent in a patch to combine statistics from child relations using a
new subcommand, ANALYZE (MERGE).

Zeng Wenjing sent in another revision of a patch to implement global temporary
tables.

Marcus Wanner sent in four more revisions of a patch to add an xid argument to
the filter_prepare callback for output plugins.

Euler Taveira de Oliveira sent in another revision of a patch to add row
filtering specified by a WHERE clause for logical replication.

Peter Smith sent in another revision of a patch to add support for prepared
transactions to built-in logical replication.

Arne Roland sent in two more revisions of a patch to make ALTER TRIGGER ...
RENAME TO work on partitioned tables.

Tang sent in a patch to update the copyright year for nbtsearch.c.

Paul Guo sent in another revision of a patch to support node initialization from
backup with tablespaces, fix the replay of create database records on standby,
and fix database create/drop wal description.

Masahiro Ikeda sent in two more revisions of a patch to speed up reporting of
WAL stats.

Daniil Zakhlystov sent in two more revisions of a patch to add zlib and zstd
streaming compression, and implement libpq compression.

Atsushi Torikoshi and Fujii Masao traded patches to get memory contexts of an
arbitrary backend process.

John Naylor sent in two revisions of a patch to document the recently added
date_bin() function.

Dean Rasheed and Fabien COELHO traded patches to add a pseudo-random permutation
function to pgbench.

Isaac Moreland sent in a patch to add an abs(interval) function and the related
@ operator.

Kyotaro HORIGUCHI sent in a patch to make the box type's description clearer.

Vigneshwaran C sent in another revision of a patch to fail a prepared
transaction if it has locked system tables/user catalog tables.

Douglas Hirn sent in another revision of a patch to allow multiple linear
recursive self-references in WITH RECURSIVE.

Sait Talha Nisanci sent in a patch intended to fix a bug that manifested as
crash in record_type_typmod_compare.

Tomáš Vondra sent in a patch to use extended statistics to improve join
estimates.

Stephen Frost sent in another revision of a patch to rename default roles to
predefined roles.

Vigneshwaran C sent in three revisions of a patch to handle the overwriting of
replication slot statistic issue, and add total txns and total txn bytes to
replication statistics.

Peter Geoghegan sent in two more revisions of a patch to simplify the state
managed by VACUUM, refactor lazy_scan_heap(), remove the tupgone special case
from vacuumlazy.c, truncate line pointer array during VACUUM, and bypass index
vacuuming in some cases.

Peter Geoghegan and Matthias van de Meent traded patches to truncate a page's
line pointer array when it has trailing unused ItemIds, and clobber free page
space in PageRepairFragmentation.

Tang sent in another revision of a patch to support tab completion with a query
result for upper character inputs in psql.

Fujii Masao sent in another revision of a patch to fix an assertion failure in
walreciever.

John Naylor sent in another revision of a patch to replace pg_utf8_verifystr()
with two faster implementations: one for Intel-ish processors that uses the
SSE-4.1 instruction set, the other which uses a bespoke fallback function rather
than one that relies on pg_utf8_verifychar() and pg_utf8_isvalid().

Peter Eisentraut sent in another revision of a patch to change the return type
of EXTRACT to numeric.

Stephen Frost sent in a patch to add pg_read_all_data and pg_write_all_data
roles.

Thomas Munro sent in a patch to use POSIX_NAMED_SEMAPHORES on OpenBSD.

Fujii Masao and Bharath Rupireddy traded patches to add a postgres_fdw server
level option, keep_connections to not cache connection.

Heikki Linnakangas sent in a patch to simplify COPY FROM parsing by forcing
lookahead.

Daniel Gustafsson sent in two more revisions of a patch to support NSS as a
libpq TLS backend.

Yuzuko Hosoya and Álvaro Herrera traded patches to fix autovacuum on partitioned
tables.

Bharath Rupireddy sent in a patch to emit a warning when a partitioned table's
persistence is changed.

Amit Langote sent in another revision of a patch to create foreign key triggers
in partitioned tables too, and use same to enforce foreign key correctly during
cross-partition updates.

Euler Taveira de Oliveira sent in another revision of a patch to refactor the
parse_output_parameters function to use the struct PGOutputData that
encapsulates all pgoutput options instead of using multiple parameters, and use
same to add logical decoding message support to pgoutput.

Peter Eisentraut sent in another revision of a patch to implement the
SQL-standard function body.

Justin Pryzby sent in another revision of a patch to implement CLUSTER of
partitioned tables.

Amit Langote sent in two more revisions of a patch to export
get_partition_for_tuple(), and use same to avoid using SPI for some RI checks.

Julien Rouhaud sent in three more revisions of a patch to move
pg_stat_statements query jumbling to core, and use same to expose queryid in
pg_stat_activity, log_line_prefix, and verbose explain.

Joel Jacobson sent in a patch to add a MotD function.

Bharath Rupireddy sent in another revision of a patch to implement ALTER
SUBSCRIPTION ... ADD/DROP PUBLICATION ...

Erik Rijkers sent in another revision of a patch to fix an old confusing JSON
example.

Kazutaka Onishi sent in six more revisions of a patch to implement TRUNCATE on
foreign tables.

Thomas Munro sent in another revision of a patch to add a buffer mapping table
for SLRUs, and make all SLRU buffer sizes configurable.

Takamichi Osumi sent in two more revisions of a patch to Safeguard for archive
recovery not to miss data. This disables the server to start up when it detects
WAL generated with wal_level=minimal during archive recovery. This should be
done regardless of the value of EnableHotStandby, because we don't think the
scenario to go through the period of wal_level=minimal happens. The motivation
of this patch is to protect user ends up with getting replica that could miss
data in standby mode and getting the server to miss data in recovery mode.

Amit Langote sent in another revision of a patch to set
ForeignScanState.resultRelInfo lazily, and initialize result relation
information lazily.

Justin Pryzby sent in a patch to make track_activity_query_size a
STATS_COLLECTOR category, make sure log_autovacuum_min_duration is LOGGING_WHAT,
make track_commit_timestamp REPLICATION_SENDING, and change force_parallel_mode
to a DEVELOPER GUC, and remove it from sample the configuration.

Pavel Stěhule sent in another revision of a patch to implement schema variables.

Anton Voloshin sent in a patch to fix a typo in collationcmds.c.

Zhihong Yu sent in a patch to remove an unused variable from AttrDefaultFetch.

Amit Langote sent in another revision of a patch to allow batching of inserts
during cross-partition updates.

Anton Voloshin sent in a patch to use repalloc() instead of palloc() in
icu_convert_case(), as the structure in question might already have been
palloc()ed.

Tom Lane sent in a patch to fix some adbin inconsistencies.