== PostgreSQL Weekly News - March 3, 2019 ==

== PostgreSQL Weekly News - March 3, 2019 ==

== PostgreSQL Jobs for March ==

http://archives.postgresql.org/pgsql-jobs/2019-03/

== PostgreSQL Local ==

PostgreSQL(at)SCaLE is a two day, two track event which takes place on
March 7-8, 2019, at Pasadena Convention Center, as part of SCaLE 17X.
https://www.socallinuxexpo.org/scale/17x/postgresscale

pgDay Paris 2019 will be held in Paris, France on March 12, 2019
at 199bis rue Saint-Martin.
http://2019.pgday.paris/

Nordic PGDay 2019 will be held in Copenhagen, Denmark, at the
Copenhagen Marriott Hotel, on March 19, 2019.
https://2019.nordicpgday.org/

PGConf APAC 2019 will be held in Singapore March 19-21, 2019.
http://2019.pgconfapac.org/

The German-speaking PostgreSQL Conference 2019 will take place on May 10, 2019
in Leipzig.
http://2019.pgconf.de/

PGDay.IT 2019 will take place May 16th and May 17th in Bologna, Italy.
https://2019.pgday.it/en/

PGCon 2019 will take place in Ottawa on May 28-31, 2019.
https://www.pgcon.org/2019

Swiss PGDay 2019 will take place in Rapperswil (near Zurich) on June 28, 2019.
The CfP is open through April 18, 2019, and registration is open.
http://www.pgday.ch/2019/

PostgresLondon 2019 will be July 2-3, 2019 with an optional training day on
July 1. The CfP is open at https://goo.gl/forms/hsvZKAmq0c96XQ4l2 through March
15, 2019.
http://postgreslondon.org

PGConf.Brazil 2019 is on August 1-3 2019 in São Paulo.
http://pgconf.com.br

== PostgreSQL in the News ==

Planet PostgreSQL: http://planet.postgresql.org/

PostgreSQL Weekly News is brought to you this week by David Fetter

Submit news and announcements by Sunday at 3:00pm PST8PDT to david(at)fetter(dot)org(dot)

== Applied Patches ==

Thomas Munro pushed:

- Fix inconsistent out-of-memory error reporting in dsa.c. Commit 16be2fd1
introduced the flag DSA_ALLOC_NO_OOM to control whether the DSA allocator
would raise an error or return InvalidDsaPointer on failure to allocate. One
edge case was not handled correctly: if we fail to allocate an internal "span"
object for a large allocation, we would always return InvalidDsaPointer
regardless of the flag; a caller not expecting that could then dereference a
null pointer. This is a plausible explanation for a one-off report of a
segfault. Remove a redundant pair of braces so that all three stanzas that
handle DSA_ALLOC_NO_OOM match in style, for visual consistency. While fixing
inconsistencies, if FreePageManagerGet() can't supply the pages that our
book-keeping says it should be able to supply, then we should always report a
FATAL error. Previously we treated that as a regular allocation failure in
one code path, but as a FATAL condition in another. Back-patch to 10, where
dsa.c landed. Author: Thomas Munro Reported-by: Jakub Glapa Discussion:
https://postgr.es/m/CAEepm=2oPqXxyWQ-1o60tpOLrwkw=VpgNXqqF1VN2EyO9zKGQw@mail.gmail.com
https://git.postgresql.org/pg/commitdiff/29ddb548f683e15429cd5464029c01d83c40bb40

Michaël Paquier pushed:

- Make release of 2PC identifier and locks consistent in COMMIT PREPARED. When
preparing a transaction in two-phase commit, a dummy PGPROC entry holding the
GID used for the transaction is registered, which gets released once COMMIT
PREPARED is run. Prior releasing its shared memory state, all the locks taken
in the prepared transaction are released using a dedicated set of callbacks
(pgstat and multixact having similar callbacks), which may cause the locks to
be released before the GID is set free. Hence, there is a small window where
lock conflicts could happen, for example: - Transaction A releases its locks,
still holding its GID in shared memory. - Transaction B held a lock which
conflicted with locks of transaction A. - Transaction B continues its
processing, reusing the same GID as transaction A. - Transaction B fails
because of a conflicting GID, already in use by transaction A. This commit
changes the shared memory state release so as post-commit callbacks and
predicate lock cleanup happen consistently with the shared memory state
cleanup for the dummy PGPROC entry. The race window is small and 2PC had this
issue from the start, so no backpatch is done. On top if that fixes discussed
involved ABI breakages, which are not welcome in stable branches.
Reported-by: Oleksii Kliukin, Ildar Musin Diagnosed-by: Oleksii Kliukin, Ildar
Musin Author: Michael Paquier Reviewed-by: Masahiko Sawada, Oleksii Kliukin
Discussion:
https://postgr.es/m/BF9B38A4-2BFF-46E8-BA87-A2D00A8047A6@hintbits.com
https://git.postgresql.org/pg/commitdiff/effe7d9552dd41fd6119404e39eecd2653cd966c

- Simplify some code in pg_rewind when syncing target directory. 9a4059d
simplified the flush of target data folder when finishing processing, and
could have done a bit more. Discussion:
https://postgr.es/m/20190131064759.GA13429@paquier.xyz
https://git.postgresql.org/pg/commitdiff/6e52209eb1f830552a5ec220c2bfc3b517535837

- Fix memory leak when inserting tuple at relation creation for CTAS. The leak
has been introduced by 763f2ed which has addressed the problem for transient
tables, and forgot CREATE TABLE AS which shares a similar logic when receiving
a new tuple to store into the newly-created relation. Author: Jeff Janes
Discussion:
https://postgr.es/m/CAMkU=1xZXtz3mziPEPD2Fubbas4G2RWkZm5HHABtfKVcbu1=Sg@mail.gmail.com
https://git.postgresql.org/pg/commitdiff/414a9d3cf34c7aff1c63533df4c40ebb63bd0840

- Fix SCRAM authentication via SSL when mixing versions of OpenSSL. When using a
libpq client linked with OpenSSL 1.0.1 or older to connect to a backend linked
with OpenSSL 1.0.2 or newer, the server would send SCRAM-SHA-256-PLUS and
SCRAM-SHA-256 as valid mechanisms for the SASL exchange, and the client would
choose SCRAM-SHA-256-PLUS even if it does not support channel binding, leading
to a confusing error. In this case, what the client ought to do is switch to
SCRAM-SHA-256 so as the authentication can move on and succeed. So for a
SCRAM authentication over SSL, here are all the cases present and how we deal
with them using libpq: 1) Server supports channel binding, it sends
SCRAM-SHA-256-PLUS and SCRAM-SHA-256 as allowed mechanisms. 1-1) Client
supports channel binding, chooses SCRAM-SHA-256-PLUS. 1-2) Client does not
support channel binding, chooses SCRAM-SHA-256. 2) Server does not support
channel binding, sends SCRAM-SHA-256 as allowed mechanism. 2-1) Client
supports channel binding, still it has no choice but to choose SCRAM-SHA-256.
2-2) Client does not support channel binding, it chooses SCRAM-SHA-256. In all
these scenarios the connection should succeed, and the one which was handled
incorrectly prior this commit is 1-2), causing the connection attempt to fail
because client chose SCRAM-SHA-256-PLUS over SCRAM-SHA-256. Reported-by: Hugh
Ranalli Diagnosed-by: Peter Eisentraut Author: Michael Paquier Reviewed-by:
Peter Eisentraut Discussion:
https://postgr.es/m/CAAhbUMO89SqUk-5mMY+OapgWf-twF2NA5sCucbHEzMfGbvcepA@mail.gmail.com
Backpatch-through: 11
https://git.postgresql.org/pg/commitdiff/87c346a35ee869e401e736163ec9d21632cc3b0d

- Improve documentation of data_sync_retry. Reflecting an updated parameter
value requires a server restart, which was not mentioned in the documentation
and in postgresql.conf.sample. Reported-by: Thomas Poty Discussion:
https://postgr.es/m/15659-0cd812f13027a2d8@postgresql.org
https://git.postgresql.org/pg/commitdiff/b3a156858ab03cb5f9c55dedcaf85a6249b3ec64

- Make pg_partition_tree return no rows on unsupported and undefined objects.
The function was tweaked so as it returned one row full of NULLs when working
on an unsupported relkind or an undefined object as of cc53123, and after
discussion with Amit and Álvaro it looks more natural to make it return no
rows. Author: Michael Paquier Reviewed-by: Álvaro Herrera, Amit Langote
Discussion: https://postgr.es/m/20190227184808.GA17357@alvherre.pgsql
https://git.postgresql.org/pg/commitdiff/0f3cdf873e7d81fbf61965e13685595c8a23c322

- Consider only relations part of partition trees in partition functions. This
changes the partition functions so as tables and indexes which are not part of
partition trees are handled the same way as what is done for undefined objects
and unsupported relkinds: pg_partition_tree() returns no rows and
pg_partition_root() returns a NULL result. Hence, partitioned tables,
partitioned indexes and relations whose flag pg_class.relispartition is set
are considered as valid objects to process. Previously, tables and indexes
not included in a partition tree were processed the same way as a partition or
a partitioned table, which caused the functions to return inconsistent results
for inherited tables, especially when inheriting from multiple tables.
Reported-by: Álvaro Herrera Author: Amit Langote, Michael Paquier Reviewed-by:
Tom Lane Discussion: https://postgr.es/m/20190228193203.GA26151@alvherre.pgsql
https://git.postgresql.org/pg/commitdiff/3422955735d9b2623532c60c53cb885c9cb6e3b1

Peter Eisentraut pushed:

- Remove unnecessary use of PROCEDURAL. Remove some unnecessary, legacy-looking
use of the PROCEDURAL keyword before LANGUAGE. We mostly don't use this
anymore, so some of these look a bit old. There is still some use in pg_dump,
which is harder to remove because it's baked into the archive format, so I'm
not touching that. Discussion:
https://www.postgresql.org/message-id/2330919b-62d9-29ac-8de3-58c024fdcb96@2ndquadrant.com
https://git.postgresql.org/pg/commitdiff/bc09d5e4cc1813c9af60c4537fe7d70ed1baae11

- psql: Remove obsolete code. The check in create_help.pl for a null end tag
(</>) has been obsolete since the conversion from SGML to XML, since XML does
not allow that anymore.
https://git.postgresql.org/pg/commitdiff/b6926dee0e1304368964e1161bfd60dd4b01b25d

- Set cluster_name for PostgresNode.pm instances. This can help identifying test
instances more easily at run time, and it also provides some minimal test
coverage for the cluster_name feature. Reviewed-by: Euler Taveira
<euler(at)timbira(dot)com(dot)br> Discussion:
https://www.postgresql.org/message-id/flat/1257eaee-4874-e791-e83a-46720c72cac7(at)2ndquadrant(dot)com
https://git.postgresql.org/pg/commitdiff/538ecc17c478e87cc3155c2b9cda7a1b1812d767

- Set fallback_application_name for a walreceiver to cluster_name. By default,
the fallback_application_name for a physical walreceiver is "walreceiver".
This means that multiple standbys cannot be distinguished easily on a primary,
for example in pg_stat_activity or synchronous_standby_names. If cluster_name
is set, use that for fallback_application_name in the walreceiver. (If it's
not set, it remains "walreceiver".) If someone set cluster_name to identify
their instance, we might as well use that by default to identify the node
remotely as well. It's still possible to specify another application_name in
primary_conninfo explicitly. Reviewed-by: Euler Taveira
<euler(at)timbira(dot)com(dot)br> Discussion:
https://www.postgresql.org/message-id/flat/1257eaee-4874-e791-e83a-46720c72cac7(at)2ndquadrant(dot)com
https://git.postgresql.org/pg/commitdiff/6ae578a91e653ce3f4cba8cab96ce41e0cc7103f

- Remove unused macro. It has never been used as long as hstore has been in the
tree.
https://git.postgresql.org/pg/commitdiff/da35d14806249c4619b2b5efda061211768a27ce

- Update comment. for ff11e7f4b9ae017585c3ba146db7ba39c31f209a
https://git.postgresql.org/pg/commitdiff/22c0d52b8dc175fe46ebb6e0ab5b2be35e02be4e

- Remove unnecessary unused MATCH PARTIAL code. ri_triggers.c spends a lot of
space catering to a not-yet-implemented MATCH PARTIAL option. An actual
implementation would probably not use the existing code structure anyway, so
let's just simplify this for now. First, have ri_FetchConstraintInfo() check
that riinfo->confmatchtype is valid. Then we don't have to repeat that
everywhere. In the various referential action functions, we don't need to pay
attention to the match type at all right now, so remove all that code. A
future MATCH PARTIAL implementation would probably have some conditions added
to the present code, but it won't need an entirely separate switch branch in
each case. In RI_FKey_fk_upd_check_required(), reorganize the code to make it
much simpler. Reviewed-by: Corey Huinker <corey(dot)huinker(at)gmail(dot)com>
Discussion:
https://www.postgresql.org/message-id/flat/0ccdd3e1-10b0-dd05-d8a7-183507c11eb1%402ndquadrant.com
https://git.postgresql.org/pg/commitdiff/0afdecc1e5d70f86d87fdc125b83ed4b7e3e88dd

- Compact for loops. Declare loop variable in for loop, for readability and to
save space. Reviewed-by: Corey Huinker <corey(dot)huinker(at)gmail(dot)com> Discussion:
https://www.postgresql.org/message-id/flat/0ccdd3e1-10b0-dd05-d8a7-183507c11eb1%402ndquadrant.com
https://git.postgresql.org/pg/commitdiff/554ebf687852d045f0418d3242b978b49f160f44

- Reduce comments. Reduce the vertical space used by comments in ri_triggers.c,
making the file longer and more tedious to read than it needs to be. Update
some comments to use a more common style. Reviewed-by: Corey Huinker
<corey(dot)huinker(at)gmail(dot)com> Discussion:
https://www.postgresql.org/message-id/flat/0ccdd3e1-10b0-dd05-d8a7-183507c11eb1%402ndquadrant.com
https://git.postgresql.org/pg/commitdiff/05d604718f7fec7db33548374ecde3f11916ec15

- Clean up some variable names in ri_triggers.c. There was a mix of
old_slot/oldslot, new_slot/newslot. Since we've changed everything from row
to slot, we might as well take this opportunity to clean this up. Also update
some more comments for the slot change.
https://git.postgresql.org/pg/commitdiff/0a271df705c98c1cc617078ebd069cf080fa5a16

- Merge near-duplicate code in RI triggers. Merge ri_setnull and ri_setdefault
into one function ri_set. These functions were to a large part identical.
This is a continuation in spirit of 4797f9b519995ceca5d6b8550b5caa2ff6d19347.
Author: Corey Huinker <corey(dot)huinker(at)gmail(dot)com> Discussion:
https://www.postgresql.org/message-id/flat/0ccdd3e1-10b0-dd05-d8a7-183507c11eb1%402ndquadrant.com
https://git.postgresql.org/pg/commitdiff/3f61999cc9da68890174dd8cf97f2cb96b223e81

- Fix whitespace.
https://git.postgresql.org/pg/commitdiff/aad21d4c3cd84e7c2a28d239048d2e2f494be47e

Peter Geoghegan pushed:

- Correct obsolete nbtree page deletion comment. Commit efada2b8e92, which made
the nbtree page deletion algorithm more robust, removed _bt_getstackbuf()
calls from _bt_pagedel(). It failed to update a comment that referenced the
earlier approach. Update the comment to explain that the _bt_getstackbuf()
page deletion call site mirrors the only other remaining _bt_getstackbuf()
call site, which is reached during page splits.
https://git.postgresql.org/pg/commitdiff/067786cea0f150ccf9350abb9a5ac3e4ad328804

- Remove unneeded argument from _bt_getstackbuf(). _bt_getstackbuf() is called
at exactly two points following commit efada2b8e92 (one call site is concerned
with page splits, while the other is concerned with page deletion). The
parent buffer returned by _bt_getstackbuf() is write-locked in both cases.
Remove the 'access' argument and make _bt_getstackbuf() assume that callers
require a write-lock.
https://git.postgresql.org/pg/commitdiff/2ab23445bc6af517bddb40d8429146d8ff8d7ff4

Michael Meskes pushed:

- Hopefully fixing memory handling issues in ecpglib that Coverity found.
https://git.postgresql.org/pg/commitdiff/0cc0507940c1cf4153fc25c8f6f4f75adada86d0

- Free memory in ecpg bytea regression test. While not really a problem it's
easier to run tools like valgrind against it when fixed.
https://git.postgresql.org/pg/commitdiff/42ccbe4351e4c0b1ad1684323c2fea79c86ae573

Robert Haas pushed:

- Change lock acquisition order in expand_inherited_rtentry. Previously, this
function acquired locks in the order using find_all_inheritors(), which locks
the children of each table that it processes in ascending OID order, and which
processes the inheritance hierarchy as a whole in a breadth-first fashion.
Now, it processes the inheritance hierarchy in a depth-first fashion, and at
each level it proceeds in the order in which tables appear in the
PartitionDesc. If table inheritance rather than table partitioning is used,
the old order is preserved. This change moves the locking of any given
partition much closer to the code that actually expands that partition. This
seems essential if we ever want to allow concurrent DDL to add or remove
partitions, because if the set of partitions can change, we must use the same
data to decide which partitions to lock as we do to decide which partitions to
expand; otherwise, we might expand a partition that we haven't locked. It
should hopefully also facilitate efforts to postpone inheritance expansion or
locking for performance reasons, because there's really no way to postpone
locking some partitions if we're blindly locking them all using
find_all_inheritors(). The only downside of this change which is known to me
is that it further deviates from the principle that we should always lock the
inheritance hierarchy in find_all_inheritors() order to avoid deadlock risk.
However, we've already crossed that bridge in commit
9eefba181f7782d27d85d7e94e6028371e7ab2d7 and there are futher patches pending
that make similar changes, so this isn't really giving up anything that we
haven't surrendered already -- and it seems entirely worth it, given the
performance benefits some of those changes seem likely to bring. Patch by me;
thanks to David Rowley for discussion of these issues. Discussion:
http://postgr.es/m/CAKJS1f_eEYVEq5tM8sm1k-HOwG0AyCPwX54XG9x4w0zy_N4Q_Q@mail.gmail.com
Discussion:
http://postgr.es/m/CA+TgmoZUwPf_uanjF==gTGBMJrn8uCq52XYvAEorNkLrUdoawg@mail.gmail.com
https://git.postgresql.org/pg/commitdiff/f4b6341d5fcb627d51b3fc89527aa5eaa659fb2e

Andres Freund pushed:

- Add ExecStorePinnedBufferHeapTuple. This allows to avoid an unnecessary
pin/unpin cycle when storing a tuple in an already pinned buffer into a slot,
when the pin isn't further needed at the call site. Only a single caller for
now (to ensure coverage), but upcoming patches will increase use of the new
function. Author: Andres Freund Discussion:
https://postgr.es/m/20180703070645.wchpu5muyto5n647@alap3.anarazel.de
https://git.postgresql.org/pg/commitdiff/8aa02b52db11039925191912eca71e3584b68860

- Allow to use HeapTupleData embedded in [Buffer]HeapTupleTableSlot. That avoids
having to care about the lifetime of the HeapTupleHeaderData passed to
ExecStore[Buffer]HeapTuple(). That doesn't make a huge difference for a plain
HeapTupleTableSlot, but for BufferHeapTupleTableSlot it can be a significant
advantage, avoiding the need to materialize slots where it's inconvenient to
provide a HeapTupleData with appropriate lifetime to point to the on-disk
tuple. It's quite possible that we'll want to add support functions for
constructing HeapTuples using that embedded HeapTupleData, but for now callers
do so themselves. Author: Andres Freund Discussion:
https://postgr.es/m/20180703070645.wchpu5muyto5n647@alap3.anarazel.de
https://git.postgresql.org/pg/commitdiff/5408e233f0667478e7f2a3e4b914e14217e20729

- Store table oid and tuple's tid in tuple slots directly. After the
introduction of tuple table slots all table AMs need to support returning the
table oid of the tuple stored in a slot created by said AM. It does not make
sense to re-implement that in every AM, therefore move handling of table OIDs
into the TupleTableSlot structure itself. It's possible that we, at a later
date, might want to get rid of HeapTupleData.t_tableOid entirely, but doing so
before the abstractions for table AMs are integrated turns out to be too hard,
so delay that for now. Similarly, every AM needs to support the concept of a
tuple identifier (tid / item pointer) for its tuples. It's quite possible that
we'll generalize the exact form of a tid at a future point (to allow for
things like index organized tables), but for now many parts of the code know
about tids, so there's not much point in abstracting tids away. Therefore also
move into slot (rather than providing API to set/get the tid associated with
the tuple in a slot). Once table AM includes insert/updating/deleting tuples,
the responsibility to set the correct tid after such an action will move into
that. After that change, code doing such modifications, should not have to
deal with HeapTuples directly anymore. Author: Andres Freund, Haribabu Kommi
and Ashutosh Bapat Discussion:
https://postgr.es/m/20180703070645.wchpu5muyto5n647@alap3.anarazel.de
https://git.postgresql.org/pg/commitdiff/b8d71745eac0a12740a70dc78cbcdedadade37f8

- Use slots in trigger infrastructure, except for the actual invocation. In
preparation for abstracting table storage, convert trigger.c to track tuples
in slots. Which also happens to make code calling triggers simpler. As the
calling interface for triggers themselves is not changed in this patch,
HeapTuples still are extracted from the slot at that time. But that's handled
solely inside trigger.c, not visible to callers. It's quite likely that we'll
want to revise the external trigger interface, but that's a separate large
project. As part of this work the slots used for old/new/return tuples are
moved from EState into ResultRelInfo, as different updated tables might need
different slots. The slots are now also now created on-demand, which is good
both from an efficiency POV, but also makes the modifying code simpler.
Author: Andres Freund, Amit Khandekar and Ashutosh Bapat Discussion:
https://postgr.es/m/20180703070645.wchpu5muyto5n647@alap3.anarazel.de
https://git.postgresql.org/pg/commitdiff/ff11e7f4b9ae017585c3ba146db7ba39c31f209a

- Initialize variable to silence compiler warning. After ff11e7f4b9ae Tom's
compiler warns about accessing a potentially uninitialized rInfo. That's not
actually possible, but it's understandable the compiler would get this wrong.
NULL initialize too. Reported-By: Tom Lane Discussion:
https://postgr.es/m/11199.1551285318@sss.pgh.pa.us
https://git.postgresql.org/pg/commitdiff/5963b29e03d9b93011acb3954ccc3565d217acc3

- Allow buffer tuple table slots to materialize after ExecStoreVirtualTuple().
While not common, it can be useful to store a virtual tuple into a buffer
tuple table slot, and then materialize that slot. So far we've asserted out,
which surprisingly wasn't a problem for anything in core. But that seems
fragile, and it also breaks redis_fdw after ff11e7f4b9. Thus, allow
materializing a virtual tuple stored in a buffer tuple table slot. Author:
Andres Freund Discussion:
https://postgr.es/m/20190227181621.xholonj7ff7ohxsg@alap3.anarazel.de
https://postgr.es/m/20180703070645.wchpu5muyto5n647@alap3.anarazel.de
https://git.postgresql.org/pg/commitdiff/f414abd62da7aa3cfd75e9047affd94c029c7c04

- Don't superfluously materialize slot after DELETE from an FDW. Previously that
was needed to safely store the table oid, but after b8d71745eac0a127 that's
not necessary anymore. Author: Andres Freund
https://git.postgresql.org/pg/commitdiff/253655116bb437d649203a5a87105e885e4b08f9

- Don't force materializing when copying a buffer tuple table slot. After
5408e233f0667478 it's not necessary to force materializing the target slot
when copying from one buffer slot to another. Previously that was required
because the HeapTupleData portion of the source slot wasn't guaranteed to stay
valid long enough, but now we can simply copy that part into the destination
slot's tupdata. Author: Andres Freund
https://git.postgresql.org/pg/commitdiff/8f0577386e68c3b1c83a07b95756e5ee3f4ae73f

- Store tuples for EvalPlanQual in slots, rather than as HeapTuples. For the
upcoming pluggable table access methods it's quite inconvenient to store
tuples as HeapTuples, as that'd require converting tuples from a their native
format into HeapTuples. Instead use slots to manage epq tuples. To fit into
that scheme, change the foreign data wrapper callback RefetchForeignRow, to
store the tuple in a slot. Insist on using the caller provided slot, so it
conveniently can be stored in the corresponding EPQ slot. As there is no in
core user of RefetchForeignRow, that change was done blindly, but we plan to
test that soon. To avoid duplicating that work for row locks, move row locks
to just directly use the EPQ slots - it previously temporarily stored tuples
in LockRowsState.lr_curtuples, but that doesn't seem beneficial, given we'd
possibly end up with a significant number of additional slots. The behaviour
of es_epqTupleSet[rti -1] is now checked by es_epqTupleSlot[rti -1] != NULL,
as that is distinguishable from a slot containing an empty tuple. Author:
Andres Freund, Haribabu Kommi, Ashutosh Bapat Discussion:
https://postgr.es/m/20180703070645.wchpu5muyto5n647@alap3.anarazel.de
https://git.postgresql.org/pg/commitdiff/ad0bda5d24ea2bcc72b5e50020e3c79bab10836b

- Use a virtual rather than a heap slot in two places where that suffices.
Author: Andres Freund Discussion:
https://postgr.es/m/20180703070645.wchpu5muyto5n647@alap3.anarazel.de
https://git.postgresql.org/pg/commitdiff/70b9bda65f8174e24eef5d8967705485a314a82e

Tom Lane pushed:

- Standardize some more loops that chase down parallel lists. We have forboth()
and forthree() macros that simplify iterating through several parallel lists,
but not everyplace that could reasonably use those was doing so. Also invent
forfour() and forfive() macros to do the same for four or five parallel lists,
and use those where applicable. The immediate motivation for doing this is to
reduce the number of ad-hoc lnext() calls, to reduce the footprint of a WIP
patch. However, it seems like good cleanup and error-proofing anyway; the
places that were combining forthree() with a manually iterated loop seem
particularly illegible and bug-prone. There was some speculation about
restructuring related parsetree representations to reduce the need for
parallel list chasing of this sort. Perhaps that's a win, or perhaps not, but
in any case it would be considerably more invasive than this patch; and it's
not particularly related to my immediate goal of improving the List
infrastructure. So I'll leave that question for another day. Patch by me;
thanks to David Rowley for review. Discussion:
https://postgr.es/m/11587.1550975080@sss.pgh.pa.us
https://git.postgresql.org/pg/commitdiff/c94fb8e8acc05c4b5f9f5b2a595ce7930827c2be

- Teach optimizer's predtest.c more things about ScalarArrayOpExpr. In
particular, make it possible to prove/refute "x IS NULL" and "x IS NOT NULL"
predicates from a clause involving a ScalarArrayOpExpr even when we are unable
or unwilling to deconstruct the expression into an AND/OR tree. This avoids a
former unexpected degradation of plan quality when the size of an ARRAY[]
expression or array constant exceeded the arbitrary MAX_SAOP_ARRAY_SIZE limit.
For IS-NULL proofs, we don't really care about the values of the individual
array elements; at most, we care whether there are any, and for some common
cases we needn't even know that. The main user-visible effect of this is to
let the optimizer recognize applicability of partial indexes with "x IS NOT
NULL" predicates to queries with "x IN (array)" clauses in some cases where it
previously failed to recognize that. The structure of predtest.c is such that
a bunch of related proofs will now also succeed, but they're probably much
less useful in the wild. James Coleman, reviewed by David Rowley Discussion:
https://postgr.es/m/CAAaqYe8yKSvzbyu8w-dThRs9aTFMwrFxn_BkTYeXgjqe3CbNjg@mail.gmail.com
https://git.postgresql.org/pg/commitdiff/65ce07e0202f2ef0953be9d085d3e5df7ad353a4

- Check we don't misoptimize a NOT IN where the subquery returns no rows.
Future-proofing against a common mistake in attempts to optimize NOT IN. We
don't have such an optimization right now, but attempts to do so are in the
works, and some of 'em are buggy. Add a regression test case covering the
point. David Rowley Discussion:
https://postgr.es/m/CAKJS1f90E9agVZryVyUpbHQbjTt5ExqS2Fsodmt5_A7E_cEyVA@mail.gmail.com
https://git.postgresql.org/pg/commitdiff/3396138a6de329fa7b5e0dda79219b4ae82622dc

Álvaro Herrera pushed:

- pg_dump: Fix ArchiveEntry handling of some empty values. Commit f831d4acc
changed what pg_dump emits for some empty fields: they were output as empty
strings before, NULL pointer afterwards. That makes old pg_restore unable to
work (crash) with such files, which is unacceptable. Return to the original
representation by explicitly setting those struct members to "" where needed;
remove some no longer needed checks for NULL input. We can declutter the code
a little by returning to NULLs when we next update the archive version, so add
a note to remind us later. Discussion:
https://postgr.es/m/20190225074539.az6j3u464cvsoxh6@depesz.com Reported-by:
hubert depesz lubaczewski Author: Dmitry Dolgov
https://git.postgresql.org/pg/commitdiff/19455c9f5606072a191962271c35ad659fbab526

- Improve docs for ALTER TABLE .. SET TABLESPACE. Discussion:
https://postgr.es/m/20190220173815.GA7959@alvherre.pgsql Reviewed-by: Robert
Haas
https://git.postgresql.org/pg/commitdiff/2c31825fb9a95914418f1e376917f60841a26c78

Joe Conway pushed:

- Make get_controlfile not leak file descriptors. When backend functions were
added to expose controldata via SQL, reading of pg_control was consolidated
under src/common so that both frontend and backend could share the same code.
That move from frontend-only to shared frontend-backend failed to recognize
the risk (and coding standards violation) of using a bare open(). In
particular, it risked leaking file descriptors if transient errors occurred
while reading the file. Fix that by using OpenTransientFile() instead in the
backend case, which is purpose-built for this type of usage. Since there have
been no complaints from the field, and an intermittent failure low risk, no
backpatch. Hard failure would of course be bad, but in that case these
functions are probably the least of your worries. Author: Joe Conway
Reviewed-By: Michael Paquier Reported by: Michael Paquier Discussion:
https://postgr.es/m/20190227074728.GA15710@paquier.xyz
https://git.postgresql.org/pg/commitdiff/4598a99cf22de0dbab975d9c87ce16fd53146aca

Amit Kapila pushed:

- Clear the local map when not used. After commit b0eaa4c51b, we use a local map
of pages to find the required space for small relations. We do clear this map
when we have found a block with enough free space, when we extend the
relation, or on transaction abort so that it can be used next time. However,
we miss to clear it when we didn't find any pages to try from the map which
leads to an assertion failure when we later tried to use it after relation
extension. In the passing, I have improved some comments in this area.
Reported-by: Tom Lane based on buildfarm results Author: Amit Kapila
Reviewed-by: John Naylor Tested-by: Kuntal Ghosh Discussion:
https://postgr.es/m/32368.1551114120@sss.pgh.pa.us
https://git.postgresql.org/pg/commitdiff/9c32e4c35026bd52aaf340bfe7594abc653e42f0

Andrew Dunstan pushed:

- Add --exclude-database option to pg_dumpall. This option functions similarly
to pg_dump's --exclude-table option, but for database names. The option can be
given once, and the argument can be a pattern including wildcard characters.
Author: Andrew Dunstan. Reviewd-by: Fabien Coelho and Michael Paquier
Discussion:
https://postgr.es/m/43a54a47-4aa7-c70e-9ca6-648f436dd6e6@2ndQuadrant.com
https://git.postgresql.org/pg/commitdiff/f092de0503cd77a8463e7089c249d5e0586a120b

- Add extra descriptive headings in pg_dumpall. Headings are added for the User
Configurations and Databases sections, and for each user configuration and
database in the output. Author: Fabien Coelho Discussion:
https://postgr.es/m/alpine.DEB.2.21.1812272222130.32444@lancre
https://git.postgresql.org/pg/commitdiff/6cbdbd9e8d8f2986fde44f2431ed8d0c8fce7f5d

- Remove tests for pg_dumpall --exclude-database missing argument. It turns out
that different getopt implementations spell the error for missing arguments
different ways. This test is of fairly marginal value, so instead of trying to
keep up with the different error messages just remove the test.
https://git.postgresql.org/pg/commitdiff/97b6f2eb759ce898ad632fd7b5db29f542b93454

- Avoid accidental wildcard expansion in msys shell. Commit f092de05 added a
test for pg_dumpall --exclude-database including the wildcard pattern '*dump*'
which matches some files in the source directory. The test library on msys
uses the shell which expands this and thus the program gets incorrect
arguments. This doesn't happen if the pattern doesn't match any files, so here
the pattern is set to '*dump_test*' which is such a pattern. Per buildfarm
animal jacana.
https://git.postgresql.org/pg/commitdiff/78b408a20a5a19486f0ed833466d0de342c7e471

Dean Rasheed pushed:

- Further fixing for multi-row VALUES lists for updatable views. Previously,
rewriteTargetListIU() generated a list of attribute numbers from the
targetlist, which were passed to rewriteValuesRTE(), which expected them to
contain the same number of entries as there are columns in the VALUES RTE, and
to be in the same order. That was fine when the target relation was a table,
but for an updatable view it could be broken in at least three different ways
--- rewriteTargetListIU() could insert additional targetlist entries for view
columns with defaults, the view columns could be in a different order from the
columns of the underlying base relation, and targetlist entries could be
merged together when assigning to elements of an array or composite type. As a
result, when recursing to the base relation, the list of attribute numbers
generated from the rewritten targetlist could no longer be relied upon to
match the columns of the VALUES RTE. We got away with that prior to 41531e42d3
because it used to always be the case that rewriteValuesRTE() did nothing for
the underlying base relation, since all DEFAULTS had already been replaced
when it was initially invoked for the view, but that was incorrect because it
failed to apply defaults from the base relation. Fix this by examining the
targetlist entries more carefully and picking out just those that are simple
Vars referencing the VALUES RTE. That's sufficient for the purposes of
rewriteValuesRTE(), which is only responsible for dealing with DEFAULT items
in the VALUES RTE. Any DEFAULT item in the VALUES RTE that doesn't have a
matching simple-Var-assignment in the targetlist is an error which we complain
about, but in theory that ought to be impossible. Additionally, move this
code into rewriteValuesRTE() to give a clearer separation of concerns between
the 2 functions. There is no need for rewriteTargetListIU() to know about the
details of the VALUES RTE. While at it, fix the comment for
rewriteValuesRTE() which claimed that it doesn't support array element and
field assignments --- that hasn't been true since a3c7a993d5 (9.6 and later).
Back-patch to all supported versions, with minor differences for the pre-9.6
branches, which don't support array element and field assignments to the same
column in multi-row VALUES lists. Reviewed by Amit Langote. Discussion:
https://postgr.es/m/15623-5d67a46788ec8b7f@postgresql.org
https://git.postgresql.org/pg/commitdiff/ed4653db8ca7a70ba7a4d329a44812893f8e59c2

== Pending Patches ==

Masahiko Sawada sent in another revision of a patch to add a function to copy
replication slots.

Peter Eisentraut sent in a patch to remove volatile from latch API.

Etsuro Fujita sent in another revision of a patch to fix another oddity in
costing aggregate pushdown paths.

Kyotaro HORIGUCHI sent in another revision of a patch to move the temporary
storage used by the stats collector from files to shared memory.

Mike Palmiotto sent in a patch to change StartChildProcess to take a struct with
data for forking/execing each different process.

David Rowley sent in a patch to increase the default vacuum_cost_limit from 200
to 2000.

Peter Eisentraut sent in another revision of a patch to implement GENERATED
columns in two flavors: virtual (generated at query time) and stored.

Peter Eisentraut sent in two more revisions of a patch to psql to add a
documentation URL to the \help output.

Mike Palmiotto sent in two revisions of a patch to add a "partition pruning"
hook.

Takeshi Ideriha sent in a patch to protect syscache from bloating with negative
cache entries.

Konstantin Knizhnik and Michaël Paquier traded patches to fix the readdir
implementation on Windows.

Takayuki Tsunakawa sent in two more revisions of a patch to disable vacuum
truncation.

Nagaura Ryohei sent in five more revisions of a patch to add TCP timeout
parameters.

Haribabu Kommi sent in a patch to move the current_logfiles file into
log_directory.

Andrew Gierth sent in a patch to ensure that JIT works on FreeBSD/ARMv7.

Aleksey Kondratov sent in a patch to remove some redundant tests from
portalmem.c.

Noah Misch sent in a patch to make it possible to change a column's type from
timestamp to timestamptz without a full-table rewrite.

Michael Banck sent in another revision of a patch to make it possible to enable
page checksums offline.

Thomas Munro sent in a patch to report bgworker launch failure during smart
shutdown.

Álvaro Herrera sent in two more revisions of a patch to create a function
pg_partition_ancestors() and use same to display foreign key relationships in
psql for partitioned tables.

Simon Riggs sent in two revisions of a patch to replace the hard-wired value of
10 in pgbench with a MAX_ARGS parameter.

Robert Haas sent in two more revisions of a patch to implement ATTACH/DETACH
PARTITION CONCURRENTLY.

Amit Kapila sent in three revisions of a patch to ensure that when a block is
not obtained from the local map, it is cleared.

Etsuro Fujita sent in a patch to fix an oddity with parallel safety test for
scan/join target in grouping_planner.

Peter Geoghegan sent in another revision of a patch to make all nbtree entries
unique by having heap TIDs participate in comparisons.

Justin Pryzby sent in another revision of a patch to avoid repetitive log of
PREPARE during EXECUTE of prepared statements.

Haribabu Kommi sent in another revision of a patch to add libpq support to
connect to standby server as priority.

Dean Rasheed sent in a patch to fix an inconsistent use of default for updatable
view.

Etsuro Fujita sent in a patch to remove some unneeded parallel safety tests from
grouping_planner.

Shawn Debnath sent in another revision of a patch to refactor the fsync
mechanism to support future SMGR implementations.

David Rowley and Tom Lane traded patches to turn use arrays instead of the List
structure.

Álvaro Herrera sent in another revision of a patch to take advantage of CoW
filesystems for WAL.

Alexander Kuzmenkov sent in another revision of a patch to remove unneeded
self-joins.

Ildus Kurbangaliev sent in another revision of a patch to make it possible to
use custom compression methods.

Antonin Houska sent in another revision of a patch to enable aggregate pushdown.

Joe Conway sent in a patch to allow for redacting all client messages for
functions marked as leakproof.

Jeff Janes sent in another revision of a patch to fix the costing for Bloom
indexes.

Peter Moser sent in another revision of a patch to implement temporal queries
using range types.

Takamichi Osumi sent in a patch to implement CREATE OR REPLACE TRIGGER.

David Steele sent in two revisions of a patch to add exclusive backup
deprecation notes to documentation.

Álvaro Herrera sent in another revision of a patch to make it possible for
partitioned tables to be referenced by foreign keys.

Surafel Temesgen sent in two more revisions of a patch to implement FETCH FIRST
... PERCENT.

Masahiko Sawada sent in another revision of a patch to add a DISABLE_INDEX_CLEANUP
option to VACUUM and a corresponding --disable-index-cleanup option to vacuumdb.

Thomas Munro sent in a patch to drop the "smgr" type.

Tomáš Vondra and David Rowley traded patches to implement multivariate MCV lists
and histograms.

Christoph Berg and Tom Lane traded patches to fix an issue which manifested as
errors mentioning incomplete startup packets.

Nikita Glukhov sent in another revision of a patch to add the SQL/JSON
functions.

Nikita Glukhov sent in a patch to fix some memory leaks and mistaken error
handling in jsonb_plpython.

Kyotaro HORIGUCHI sent in another revision of a patch to prevent syscache from
bloating with negative cache entries.

David Rowley sent in another revision of a patch to make ALTER TABLE ... SET NOT
NULL operate by constraints alone.

Michael Meskes sent in a patch to make PREPARE work in ECPG.

Amit Langote sent in another revision of a patch to speed up planning with
partitions.

Etsuro Fujita sent in a patch to revive the modify_in_place parameter to
apply_scanjoin_target_to_paths.

Nikita Glukhov sent in another revision of a patch to implement JSON_TABLE.

Adrien Nayrat sent in another revision of a patch to make it possible to log a
sample of transactions.

Tatsuro Yamada sent in another revision of a patch to implement a progress
monitor for CLUSTER.

Michael Banck and Fabien COELHO traded patches to make it possible to enable
page checksums online.

Nikita Glukhov and Alexander Korotkov traded patches to implement jsonpath.

Sergei Kornilov sent in another revision of a patch to refactor
WaitForWALToBecomeAvailable to gracefully restart source.

Fabien COELHO sent in another revision of a patch to pgbench to add a
pseudo-random permutation function.

David Rowley sent in two more revisions of a patch to turn some NOT IN queries
into antijoins.

Pavel Stěhule sent in another revision of a patch to implement schema variables.