== PostgreSQL Weekly News - February 3, 2019 ==

== PostgreSQL Weekly News - February 3, 2019 ==

== PostgreSQL Product News ==

pg2arrow, a utility to query a PostgreSQL database and dump the query results in
Apache Arrow format, released.
https://github.com/heterodb/pg2arrow

PostgreSQL Automatic Failover (PAF) v2.2.1 released.
https://github.com/ClusterLabs/PAF/releases/latest

check_pgactivity 2.4, a Nagios remote agent for monitoring PostgreSQL, released.
https://github.com/OPMDG/check_pgactivity/releases

pg_snakeoil 1.0, an anti-virus extension for PostgreSQL, released.
https://github.com/credativ/pg_snakeoil/releases/tag/v1.0

== PostgreSQL Jobs for February ==

http://archives.postgresql.org/pgsql-jobs/2019-02/

== PostgreSQL Local ==

Prague PostgreSQL Developer Day 2019 (P2D2 2019) will be held on February 13-14,
2019 in Prague, Czech Republic.
http://www.p2d2.cz/

PGConf India 2019 will be on February 13-15, 2019 in Bengaluru, Karnataka.
http://pgconf.in/

PostgreSQL(at)SCaLE is a two day, two track event which takes place on
March 7-8, 2019, at Pasadena Convention Center, as part of SCaLE 17X.
https://www.socallinuxexpo.org/scale/17x/postgresscale

pgDay Paris 2019 will be held in Paris, France on March 12, 2019
at 199bis rue Saint-Martin.
http://2019.pgday.paris/

Nordic PGDay 2019 will be held in Copenhagen, Denmark, at the
Copenhagen Marriott Hotel, on March 19, 2019.
https://2019.nordicpgday.org/

PGConf APAC 2019 will be held in Singapore March 19-21, 2019.
http://2019.pgconfapac.org/

The German-speaking PostgreSQL Conference 2019 will take place on May 10, 2019
in Leipzig. The CfP is open until February 26, 2019 at http://2019.pgconf.de/cfp
http://2019.pgconf.de/

PGDay.IT 2019 will take place May 16th and May 17th in Bologna, Italy.
https://2019.pgday.it/en/

PGCon 2019 will take place in Ottawa on May 28-31, 2019.
https://www.pgcon.org/2019

Swiss PGDay 2019 will take place in Rapperswil (near Zurich) on June 28, 2019.
The CfP is open through April 18, 2019, and registration is open.
http://www.pgday.ch/2019/

== PostgreSQL in the News ==

Planet PostgreSQL: http://planet.postgresql.org/

PostgreSQL Weekly News is brought to you this week by David Fetter

Submit news and announcements by Sunday at 3:00pm PST8PDT to david(at)fetter(dot)org(dot)

== Applied Patches ==

Amit Kapila pushed:

- In bootstrap mode, don't allow the creation of files if they don't
already.exist. In commit's b9d01fe288 and 3908473c80, we have added some code
where we allowed the creation of files during mdopen even if they didn't exist
during the bootstrap mode. The later commit obviates the need for same. This
was harmless code till now but with an upcoming feature where we don't allow
to create FSM for small tables, this will needlessly create FSM files.
Author: John Naylor Reviewed-by: Amit Kapila Discussion:
https://www.postgresql.org/message-id/CAJVSVGWvB13PzpbLEecFuGFc5V2fsO736BsdTakPiPAcdMM5tQ@mail.gmail.com
https://www.postgresql.org/message-id/CAA4eK1KsET6sotf+rzOTQfb83pzVEzVhbQi1nxGFYVstVWXUGw@mail.gmail.com
https://git.postgresql.org/pg/commitdiff/d66e3664b8baf41908865ad363c6ba943e6f9c4e

- Avoid creation of the free space map for small heap relations. Previously, all
heaps had FSMs. For very small tables, this means that the FSM took up more
space than the heap did. This is wasteful, so now we refrain from creating the
FSM for heaps with 4 pages or fewer. If the last known target block has
insufficient space, we still try to insert into some other page before giving
up and extending the relation, since doing otherwise leads to table bloat.
Testing showed that trying every page penalized performance slightly, so we
compromise and try every other page. This way, we visit at most two pages. Any
pages with wasted free space become visible at next relation extension, so we
still control table bloat. As a bonus, directly attempting one or two pages
can even be faster than consulting the FSM would have been. Once the FSM is
created for a heap we don't remove it even if somebody deletes all the rows
from the corresponding relation. We don't think it is a useful optimization
as it is quite likely that relation will again grow to the same size. Author:
John Naylor with design inputs and some code contribution by Amit Kapila
Reviewed-by: Amit Kapila Tested-by: Mithun C Y Discussion:
https://www.postgresql.org/message-id/CAJVSVGWvB13PzpbLEecFuGFc5V2fsO736BsdTakPiPAcdMM5tQ@mail.gmail.com
https://git.postgresql.org/pg/commitdiff/ac88d2962a96a9c7e83d5acfc28fe49a72812086

- Revert "Avoid creation of the free space map for small heap relations.". This
reverts commit ac88d2962a96a9c7e83d5acfc28fe49a72812086.
https://git.postgresql.org/pg/commitdiff/a23676503b746b7f1588cd2ab0c60411032d32da

- Avoid possible deadlock while locking multiple heap pages. To avoid deadlock,
backend acquires a lock on heap pages in block number order. In certain
cases, lock on heap pages is dropped and reacquired. In this case, the locks
are dropped for reading in corresponding VM page/s. The issue is we re-acquire
locks in bufferId order whereas the intention was to acquire in blockid order.
This commit ensures that we will always acquire locks on heap pages in blockid
order. Reported-by: Nishant Fnu Author: Nishant Fnu Reviewed-by: Amit Kapila
and Robert Haas Backpatch-through: 9.4 Discussion:
https://postgr.es/m/5883C831-2ED1-47C8-BFAC-2D5BAE5A8CAE@amazon.com
https://git.postgresql.org/pg/commitdiff/0b8bdb3c3ec092f18813d5b6fc738d7ec450289b

Michaël Paquier pushed:

- Add tab completion for ALTER INDEX ALTER COLUMN in psql. The completion here
consists of attribute numbers, which is specific to this grammar. Author:
Tatsuro Yamada Reviewed-by: Peter Eisentraut Discussion:
https://portgr.es/m/b58a78fa-81ce-186f-f0bc-c1aa93c46cbf@lab.ntt.co.jp
https://git.postgresql.org/pg/commitdiff/23349b18d92d0dd522da17917bb5231f37a87d88

- Use catalog query to discover tables to process in vacuumdb. vacuumdb would
use a catalog query only when the command caller does not define a list of
tables. Switching to a catalog table represents two advantages: - Relation
existence check can happen before running any VACUUM or ANALYZE query. Before
this change, if multiple relations are defined using --table, the utility
would fail only after processing the firstly-defined ones, which may be a long
some depending on the size of the relation. This adds checks for the relation
names, and does nothing, at least yet, for the attribute names. - More
filtering options can become available for the utility user. These options,
which may be introduced later on, are based on the relation size or the
relation age, and need to be made available even if the user does not list any
specific table with --table. Author: Nathan Bossart Reviewed-by: Michael
Paquier, Masahiko Sawada Discussion:
https://postgr.es/m/FFE5373C-E26A-495B-B5C8-911EC4A41C5E@amazon.com
https://git.postgresql.org/pg/commitdiff/e0c2933a767c652429ddef674622b4656fa43092

- Do not filter by relkind in vacuumdb's catalog query if --table is used. If a
user specifies a relation name which cannot be processed, then the backend can
warn directly about what is wrong with it. This fixes an oversight from
e0c2933. Author: Nathan Bossart Discussion:
https://postgr.es/m/32049A78-C429-4742-AEC1-941C9ABDE7B8@amazon.com
https://git.postgresql.org/pg/commitdiff/b8f73df0f845d865823ef72669024dc150282392

- Add --min-xid-age and --min-mxid-age options to vacuumdb. These two new
options can be used to improve the selectivity of relations to vacuum or
analyze even further depending on the age of respectively their transaction ID
or multixact ID, so as it is possible to prioritize tables to prevent
wraparound of one or the other. Combined with --table, it is possible to
target a subset of tables to choose as potential processing targets. Author:
Nathan Bossart Reviewed-by: Michael Paquier, Masahiko Sawada Discussion:
https://postgr.es/m/FFE5373C-E26A-495B-B5C8-911EC4A41C5E@amazon.com
https://git.postgresql.org/pg/commitdiff/00d1e88d36687ceae1be2317fac90e967941c085

- Fix use of dangling pointer in heap_delete() when logging replica identity.
When logging the replica identity of a deleted tuple, XLOG_HEAP_DELETE records
include references of the old tuple. Its data is stored in an intermediate
variable used to register this information for the WAL record, but this
variable gets away from the stack when the record gets actually inserted.
Spotted by clang's AddressSanitizer. Author: Stas Kelvish Discussion:
https://postgr.es/m/085C8825-AD86-4E93-AF80-E26CDF03D1EA@postgrespro.ru
Backpatch-through: 9.4
https://git.postgresql.org/pg/commitdiff/eb8c9f0bc394160efd6951c69e644551e835a486

- Adjust comment about timeout when waiting for WAL at recovery. A timeout of 5s
is used when waiting for WAL to become available at recovery so as the startup
process is able to react promptly if a trigger file shows up. However this
missed the fact that the startup process also relies on the timeout to check
periodically the status of any active WAL receiver. Discussion:
https://postgr.es/m/20190131070956.GE13429@paquier.xyz
https://git.postgresql.org/pg/commitdiff/c93001b3f957994942ea18ed261ae06ac507ea95

- Improve installation instructions with pg_ctl in documentation. The
documentation includes sections to be able to initialize and start Postgres
via a couple of commands. Some of its recommendations involve using directly
"postgres", which is inconsistent with the recommendation given by initdb. At
the same time make some other command calls more consistent with the rest, by
using an absolute path when creating a database. Author: Andreas Scherbaum
Reviewed-by: Michael Banck, Ryan Lambert
https://git.postgresql.org/pg/commitdiff/3e938a83b2ee3b47d47cdbdf58df09946877d4fd

- Add PG_CFLAGS, PG_CXXFLAGS, and PG_LDFLAGS variables to PGXS. Add PG_CFLAGS,
PG_CXXFLAGS, and PG_LDFLAGS variables to pgxs.mk which will be appended or
prepended to the corresponding make variables. Notably, there was previously
no way to pass custom CXXFLAGS to third party extension module builds, COPT
and PROFILE supporting only CFLAGS and LDFLAGS. Backpatch all the way down to
ease integration with existing extensions. Author: Christoph Berg
Reviewed-by: Andres Freund, Tom Lane, Michael Paquier Discussion:
https://postgr.es/m/20181113104005.GA32154@msg.credativ.de Backpatch-through:
9.4
https://git.postgresql.org/pg/commitdiff/ac3a9afdbefd76de51fa8f864288f2d2372ca4e9

Peter Eisentraut pushed:

- doc: Add link from sslinfo to pg_stat_ssl. Reviewed-by: Kyotaro HORIGUCHI
<horiguchi(dot)kyotaro(at)lab(dot)ntt(dot)co(dot)jp> Discussion:
https://www.postgresql.org/message-id/flat/398754d8-6bb5-c5cf-e7b8-22e5f0983caf(at)2ndquadrant(dot)com/
https://git.postgresql.org/pg/commitdiff/bcf3f00be515c1da76b1d88923d01a78862467d3

- psql: Remove unused tab completion query. This was used for the old CLUSTER
syntax, has been unused since e55c8e36ae44677dca4420bed07ad09d191fdf6c.
https://git.postgresql.org/pg/commitdiff/d4316b87bb8ec7a0ea04fb5fbc18f34a6799fe3f

- Make SSL tests more robust. Someone running these test could have key or
certificate files in their ~/.postgresql/, which would interfere with the
tests. The way to override that is to specify sslcert=invalid and/or
sslrootcert=invalid if no actual certificate is used for a particular test.
Document that and fix up one test that had a risk of failing in these
circumstances. Discussion:
https://www.postgresql.org/message-id/flat/398754d8-6bb5-c5cf-e7b8-22e5f0983caf(at)2ndquadrant(dot)com/
https://git.postgresql.org/pg/commitdiff/bdd6e9ba171765ae34d320e1be17717620cb491c

- Add tests for pg_stat_ssl system view. Reviewed-by: Kyotaro HORIGUCHI
<horiguchi(dot)kyotaro(at)lab(dot)ntt(dot)co(dot)jp> Discussion:
https://www.postgresql.org/message-id/flat/398754d8-6bb5-c5cf-e7b8-22e5f0983caf(at)2ndquadrant(dot)com/
https://git.postgresql.org/pg/commitdiff/18059543e76aeb6b3cf3a5513e69daf222d3558e

- Fix pg_stat_ssl.clientdn. Return null if there is no client certificate. This
is how it has always been documented, but in reality it returned an empty
string. Reviewed-by: Kyotaro HORIGUCHI <horiguchi(dot)kyotaro(at)lab(dot)ntt(dot)co(dot)jp>
Discussion:
https://www.postgresql.org/message-id/flat/398754d8-6bb5-c5cf-e7b8-22e5f0983caf(at)2ndquadrant(dot)com/
https://git.postgresql.org/pg/commitdiff/e77cfa54d700557ea700d47454c9e570f20f1841

- Fix a crash in logical replication. The bug was that determining which columns
are part of the replica identity index using RelationGetIndexAttrBitmap()
would run eval_const_expressions() on index expressions and predicates across
all indexes of the table, which in turn might require a snapshot, but there
wasn't one set, so it crashes. There were actually two separate bugs, one on
the publisher and one on the subscriber. To trigger the bug, a table that is
part of a publication or subscription needs to have an index with a predicate
or expression that lends itself to constant expressions simplification. The
fix is to avoid the constant expressions simplification in
RelationGetIndexAttrBitmap(), so that it becomes safe to call in these
contexts. The constant expressions simplification comes from the calls to
RelationGetIndexExpressions()/RelationGetIndexPredicate() via
BuildIndexInfo(). But RelationGetIndexAttrBitmap() calling BuildIndexInfo()
is overkill. The latter just takes pg_index catalog information, packs it
into the IndexInfo structure, which former then just unpacks again and throws
away. We can just do this directly with less overhead and skip the
troublesome calls to eval_const_expressions(). This also removes the awkward
cross-dependency between relcache.c and index.c. Bug: #15114 Reported-by:
Петър Славов <pet(dot)slavov(at)gmail(dot)com> Reviewed-by: Noah Misch
<noah(at)leadboat(dot)com> Reviewed-by: Michael Paquier <michael(at)paquier(dot)xyz>
Discussion:
https://www.postgresql.org/message-id/flat/152110589574(dot)1223(dot)17983600132321618383(at)wrigleys(dot)postgresql(dot)org/
https://git.postgresql.org/pg/commitdiff/dfa774ff9ac9de33d8cb98b91906ada55a2ab1df

- postmaster: Start syslogger earlier. When the syslogger was originally added
(bdf8ef6925de6ea1a9330fa1ce32e1a315d07eb2), nothing was normally logged before
the point where it was started. But since
f9dfa5c9776649f769d537dd0923003b35f128de, the creation of sockets causes
messages of level LOG to be written routinely, so those don't go to the
syslogger now. To improve that, arrange the sequence in PostmasterMain()
slightly so that the syslogger is started early enough to capture those
messages. Discussion:
https://www.postgresql.org/message-id/d5d50936-20b9-85f1-06bc-94a01c5040c1%402ndquadrant.com
Reviewed-by: Christoph Berg <christoph(dot)berg(at)credativ(dot)de>
https://git.postgresql.org/pg/commitdiff/57431a911d3a650451d198846ad3194900666152

- Log PostgreSQL version number on startup. Logging the PostgreSQL version on
startup is useful for two reasons: There is a clear marker in the log file
that a new postmaster is beginning, and it's useful for tracking the server
version across startup while upgrading. Author: Christoph Berg
<christoph(dot)berg(at)credativ(dot)de> Discussion:
https://www.postgresql.org/message-id/flat/20181121144611(dot)GJ15795(at)msg(dot)credativ(dot)de/
https://git.postgresql.org/pg/commitdiff/689d15e95e3509554fbc1afd231b41f4a1fee669

- Add more columns to pg_stat_ssl. Add columns client_serial and issuer_dn to
pg_stat_ssl. These allow uniquely identifying the client certificate. Rename
the existing column clientdn to client_dn, to make the naming more consistent
and easier to read. Discussion:
https://www.postgresql.org/message-id/flat/398754d8-6bb5-c5cf-e7b8-22e5f0983caf(at)2ndquadrant(dot)com/
https://git.postgresql.org/pg/commitdiff/f60a0e96778854ed0b7fd4737488ba88022e47bd

Andres Freund pushed:

- Move page initialization from RelationAddExtraBlocks() to use. Previously we
initialized pages when bulk extending in RelationAddExtraBlocks(). That has a
major disadvantage: It ties RelationAddExtraBlocks() to heap, as other types
of storage are likely to need different amounts of special space, have
different amount of free space (previously determined by
PageGetHeapFreeSpace()). That we're relying on initializing pages, but not
WAL logging the initialization, also means the risk for getting "WARNING:
relation \"%s\" page %u is uninitialized --- fixing" style warnings in vacuums
after crashes/immediate shutdowns, is considerably higher. The warning sounds
much more serious than what they are. Fix those two issues together by not
initializing pages in RelationAddExtraPages() (but continue to do so in
RelationGetBufferForTuple(), which is linked much more closely to heap), and
accepting uninitialized pages as normal in vacuumlazy.c. When vacuumlazy
encounters an empty page it now adds it to the FSM, but does nothing else. We
chose to not issue a debug message, much less a warning in that case - it
seems rarely useful, and quite likely to scare people unnecessarily. For now
empty pages aren't added to the VM, because standbys would not re-discover
such pages after a promotion. In contrast to other sources for empty pages,
there's no corresponding WAL records triggering FSM updates during replay.
Author: Andres Freund Reviewed-By: Tom Lane Discussion:
https://postgr.es/m/20181219083945.6khtgm36mivonhva@alap3.anarazel.de
https://git.postgresql.org/pg/commitdiff/e6799d5a53011985d916fdb48fe014a4ae70422e

- Install JIT related headers. There's no reason not to install these, and jit.h
can be useful for users of e.g. planner hooks. Author: Donald Dong
Reviewed-By: Andres Freund Discussion:
https://postgr.es/m/296D405F-7F95-49F1-B565-389D6AA78505@csumb.edu Backpatch:
11-, where JIT compilation was introduced
https://git.postgresql.org/pg/commitdiff/5c1186751214416fdf88f33a89c3dc88391d2d60

- Fix race condition between relation extension and vacuum. In e6799d5a5301 I
removed vacuumlazy.c trickery around re-checking whether a page is actually
empty after acquiring an extension lock on the relation, because the page is
not PageInit()ed anymore, and entries in the FSM ought not to lead to
user-visible errors. As reported by various buildfarm animals that is not
correct, given the way to code currently stands: If vacuum processes a page
that's just been newly added by either RelationGetBufferForTuple() or
RelationAddExtraBlocks(), it could add that page to the FSM and it could be
reused by other backends, before those two functions check whether the newly
added page is actually new. That's a relatively narrow race, but several
buildfarm machines appear to be able to hit it. While it seems wrong that the
FSM, given it's lack of durability and approximative nature, can trigger
errors like this, that seems better fixed in a separate commit. Especially
given that a good portion of the buildfarm is red, and this is just
re-introducing logic that existed a few hours ago. Author: Andres Freund
Discussion:
https://postgr.es/m/20190128222259.zhi7ovzgtkft6em6@alap3.anarazel.de
https://git.postgresql.org/pg/commitdiff/fc02e6724f3ce069b33284bce092052ab55bd751

- Revert "Move page initialization from RelationAddExtraBlocks() to use.". This
reverts commit fc02e6724f3ce069b33284bce092052ab55bd751 and
e6799d5a53011985d916fdb48fe014a4ae70422e. Parts of the buildfarm error out
with ERROR: page %u of relation "%s" should be empty but is not errors, and so
far I/we do not know why. fc02e672 didn't fix the issue. As I cannot
reproduce the issue locally, it seems best to get the buildfarm green again,
and reproduce the issue without time pressure.
https://git.postgresql.org/pg/commitdiff/684200543b4cbfe1ac002c9962e90683d4ea4691

- Fix LLVM related headers to compile standalone (to fix cpluspluscheck).
Previously llvmjit.h #error'ed when USE_LLVM was not defined, to prevent it
from being included from code not having #ifdef USE_LLVM guards - but that's
not actually that useful after, during the development of JIT support, LLVM
related code was moved into a separately compiled .so. Having that #error
means cpluspluscheck doesn't work when llvm support isn't enabled, which isn't
great. Similarly add USE_LLVM guards to llvmjit_emit.h, and additionally make
sure it compiles standalone. Per complaint from Tom Lane. Author: Andres
Freund Discussion: https://postgr.es/m/19808.1548692361@sss.pgh.pa.us
Backpatch: 11, where JIT support was added
https://git.postgresql.org/pg/commitdiff/da05eb51debd5d4b0284bcafa728244183c303ae

- Move page initialization from RelationAddExtraBlocks() to use, take 2.
Previously we initialized pages when bulk extending in
RelationAddExtraBlocks(). That has a major disadvantage: It ties
RelationAddExtraBlocks() to heap, as other types of storage are likely to need
different amounts of special space, have different amount of free space
(previously determined by PageGetHeapFreeSpace()). That we're relying on
initializing pages, but not WAL logging the initialization, also means the
risk for getting "WARNING: relation \"%s\" page %u is uninitialized ---
fixing" style warnings in vacuums after crashes/immediate shutdowns, is
considerably higher. The warning sounds much more serious than what they are.
Fix those two issues together by not initializing pages in
RelationAddExtraPages() (but continue to do so in RelationGetBufferForTuple(),
which is linked much more closely to heap), and accepting uninitialized pages
as normal in vacuumlazy.c. When vacuumlazy encounters an empty page it now
adds it to the FSM, but does nothing else. We chose to not issue a debug
message, much less a warning in that case - it seems rarely useful, and quite
likely to scare people unnecessarily. For now empty pages aren't added to the
VM, because standbys would not re-discover such pages after a promotion. In
contrast to other sources for empty pages, there's no corresponding WAL
records triggering FSM updates during replay. Previously when extending the
relation, there was a moment between extending the relation, and acquiring an
exclusive lock on the new page, in which another backend could lock the page.
To avoid new content being put on that new page, vacuumlazy needed to acquire
the extension lock for a brief moment when encountering a new page. A second
corner case, only working somewhat by accident, was that
RelationGetBufferForTuple() sometimes checks the last page in a relation for
free space, without consulting the FSM; that only worked because
PageGetHeapFreeSpace() interprets the zero page header in a new page as no
free space. The lack of handling this properly required reverting the
previous attempt in 684200543b. This issue can be solved by using
RBM_ZERO_AND_LOCK when extending the relation, thereby avoiding this window.
There's some added complexity when RelationGetBufferForTuple() is called with
another buffer (for updates), to avoid deadlocks, but that's rarely hit at
runtime. Author: Andres Freund Reviewed-By: Tom Lane Discussion:
https://postgr.es/m/20181219083945.6khtgm36mivonhva@alap3.anarazel.de
https://git.postgresql.org/pg/commitdiff/0d1fe9f74e369a6066b18fcbc0a25d87cbdff7ce

Tom Lane pushed:

- In the planner, replace an empty FROM clause with a dummy RTE. The fact that
"SELECT expression" has no base relations has long been a thorn in the side of
the planner. It makes it hard to flatten a sub-query that looks like that, or
is a trivial VALUES() item, because the planner generally uses relid sets to
identify sub-relations, and such a sub-query would have an empty relid set if
we flattened it. prepjointree.c contains some baroque logic that works around
this in certain special cases --- but there is a much better answer. We can
replace an empty FROM clause with a dummy RTE that acts like a table of one
row and no columns, and then there are no such corner cases to worry about.
Instead we need some logic to get rid of useless dummy RTEs, but that's
simpler and covers more cases than what was there before. For really trivial
cases, where the query is just "SELECT expression" and nothing else, there's a
hazard that adding the extra RTE makes for a noticeable slowdown; even though
it's not much processing, there's not that much for the planner to do overall.
However testing says that the penalty is very small, close to the noise level.
In more complex queries, this is able to find optimizations that we could not
find before. The new RTE type is called RTE_RESULT, since the "scan" plan
type it gives rise to is a Result node (the same plan we produced for a
"SELECT expression" query before). To avoid confusion, rename the old
ResultPath path type to GroupResultPath, reflecting that it's only used in
degenerate grouping cases where we know the query produces just one grouped
row. (It wouldn't work to unify the two cases, because there are different
rules about where the associated quals live during query_planner.) Note:
although this touches readfuncs.c, I don't think a catversion bump is
required, because the added case can't occur in stored rules, only plans.
Patch by me, reviewed by David Rowley and Mark Dilger Discussion:
https://postgr.es/m/15944.1521127664@sss.pgh.pa.us
https://git.postgresql.org/pg/commitdiff/4be058fe9ec5e630239b656af21fc083371f30ed

- Make some small planner API cleanups. Move a few very simple node-creation and
node-type-testing functions from the planner's clauses.c to nodes/makefuncs
and nodes/nodeFuncs. There's nothing planner-specific about them, as evidenced
by the number of other places that were using them. While at it, rename
and_clause() etc to is_andclause() etc, to clarify that they are
node-type-testing functions not node-creation functions. And use "static
inline" implementations for the shortest ones. Also, modify
flatten_join_alias_vars() and some subsidiary functions to take a Query not a
PlannerInfo to define the join structure that Vars should be translated
according to. They were only using the "parse" field of the PlannerInfo
anyway, so this just requires removing one level of indirection. The
advantage is that now parse_agg.c can use flatten_join_alias_vars() without
the horrid kluge of creating an incomplete PlannerInfo, which will allow that
file to be decoupled from relation.h in a subsequent patch. Discussion:
https://postgr.es/m/11460.1548706639@sss.pgh.pa.us
https://git.postgresql.org/pg/commitdiff/a1b8c41e990ec0f083e9b684700a07640d5a356a

- Refactor planner's header files. Create a new header optimizer/optimizer.h,
which exposes just the planner functions that can be used "at arm's length",
without need to access Paths or the other planner-internal data structures
defined in nodes/relation.h. This is intended to provide the whole planner
API seen by most of the rest of the system; although FDWs still need to use
additional stuff, and more thought is also needed about just what selfuncs.c
should rely on. The main point of doing this now is to limit the amount of
new #include baggage that will be needed by "planner support functions", which
I expect to introduce later, and which will be in relevant datatype modules
rather than anywhere near the planner. This commit just moves relevant
declarations into optimizer.h from other header files (a couple of which go
away because everything got moved), and adjusts #include lists to match.
There's further cleanup that could be done if we want to decide that some
stuff being exposed by optimizer.h doesn't belong in the planner at all, but
I'll leave that for another day. Discussion:
https://postgr.es/m/11460.1548706639@sss.pgh.pa.us
https://git.postgresql.org/pg/commitdiff/f09346a9c6218dd239fdf3a79a729716c0d305bd

- Rename nodes/relation.h to nodes/pathnodes.h. The old name of this file was
never a very good indication of what it was for. Now that there's also
access/relation.h, we have a potential confusion hazard as well, so let's
rename it to something more apropos. Per discussion, "pathnodes.h" is
reasonable, since a good fraction of the file is Path node definitions. While
at it, tweak a couple of other headers that were gratuitously importing
relation.h into modules that don't need it. Discussion:
https://postgr.es/m/7719.1548688728@sss.pgh.pa.us
https://git.postgresql.org/pg/commitdiff/fa2cf164aaf91e074be653c28e035f65d80eb666

- Allow RECORD and RECORD[] to be specified in function coldeflists. We can't
allow these pseudo-types to be used as table column types, because storing an
anonymous record value in a table would result in data that couldn't be
understood by other sessions. However, it seems like there's no harm in
allowing the case in a column definition list that's specifying what a
function-returning-record returns. The data involved is all local to the
current session, so we should be just as able to resolve its actual tuple type
as we are for the function-returning-record's top-level tuple output. Elvis
Pranskevichus, with cosmetic changes by me Discussion:
https://postgr.es/m/11038447.kQ5A9Uj5xi@hammer.magicstack.net
https://git.postgresql.org/pg/commitdiff/5f5c01459053c0975b89b5aa1a3d705e677348ae

Tomáš Vondra pushed:

- Separate per-batch and per-tuple memory contexts in COPY. In batching mode,
COPY was using the same (per-tuple) memory context for allocations with longer
lifetime. This was confusing but harmless, until commit 31f3817402 added COPY
FROM ... WHERE feature, introducing a risk of memory leak. The "per-tuple"
memory context was reset only when starting new batch, but as the rows may be
filtered out by the WHERE clauses, that may not happen at all. The WHERE
clause however has to be evaluated for all rows, before filtering them out.
This commit separates the per-tuple and per-batch contexts, removing the
ambiguity. Expressions (both defaults and WHERE clause) are evaluated in the
per-tuple context, while tuples are formed in the batch context. This allows
resetting the contexts at appropriate times. The main complexity is related
to partitioning, in which case we need to reset the batch context after
forming the tuple (which happens before routing to leaf partition). Instead
of switching between two contexts as before, we simply copy the last tuple
aside, reset the context and then copy the tuple back. The performance impact
is negligible, and juggling with two contexts is not free either. Discussion:
https://www.postgresql.org/message-id/flat/CALAY4q_DdpWDuB5-Zyi-oTtO2uSk8pmy+dupiRe3AvAc++1imA(at)mail(dot)gmail(dot)com
https://git.postgresql.org/pg/commitdiff/36a1281f86c0f805462e9f05cacbe64779009845

Etsuro Fujita pushed:

- postgres_fdw: Fix test for cached costs in estimate_path_cost_size().
estimate_path_cost_size() failed to re-use cached costs when the cached
startup/total cost was 0, so it calculated the costs redundantly. This is an
oversight in commit aa09cd242f; but apply the patch to HEAD only because there
are no reports of actual trouble from that. Author: Etsuro Fujita Discussion:
https://postgr.es/m/5C4AF3F3.4060409%40lab.ntt.co.jp
https://git.postgresql.org/pg/commitdiff/449d0a85507b07be3040df78a93e2966ed0908c8

Magnus Hagander pushed:

- Improve wording about WAL files in tar mode of pg_basebackup. Author: Alex
Kliukin Reviewed-By: Michael Paquier, Magnus Hagander
https://git.postgresql.org/pg/commitdiff/9745b528f7cb03be3afad0f91a634072c13ea91f

Michael Meskes pushed:

- Make sure that ecpglib's statement variable has a defined value no matter
what.
https://git.postgresql.org/pg/commitdiff/5c04630ad05e6e33bc18a6846b4086f9794d99e8

- Make some ecpg test cases more robust against unexpected errors that
happen.during development. Test cases themselves should not hang or segfault.
https://git.postgresql.org/pg/commitdiff/e2f731cdba9b7a79cddc64325990a8f51818877b

- Change error handling of out of scope variables in ecpg. The function called
can result in an out of memory error that subsequently was disregarded.
Instead it should set the appropriate SQL error variables and be checked by
whatever whenever statement is defined.
https://git.postgresql.org/pg/commitdiff/7ea38f045dad6bbb7fbe807f2486df7370bc0b0f

Álvaro Herrera pushed:

- Move building of child base quals out into a new function. An upcoming patch
which changes how inheritance planning works requires adding a new function
that does a similar job to set_append_rel_size() but for child target
relations. To save it from having to duplicate the qual building code, move
that to a separate function first. Here we also change things so that we
never attempt to build security quals after detecting some const false child
quals. We needlessly used to do this just before we marked the child relation
as a dummy rel. In passing, this also moves the partition pruned check to
before the qual building code. We don't need to build the child quals before
we check if the partition has been pruned. Author: David Rowley Discussion:
https://postgr.es/m/CAKJS1f_i+jrrD+if8qC7KPuTAAWsd=dtepgY_7u=P86GDEwm7A@mail.gmail.com
https://git.postgresql.org/pg/commitdiff/80579f9bb171350fccdd5f1d793c538254d9de62

- Add ArchiveOpts to pass options to ArchiveEntry. The ArchiveEntry function has
a number of arguments that can be considered optional. Split them out into a
separate struct, to make the API more flexible for changes. Author: Dmitry
Dolgov Discussion:
https://postgr.es/m/CA+q6zcXRxPE+qp6oerQWJ3zS061WPOhdxeMrdc-Yf-2V5vsrEw@mail.gmail.com
https://git.postgresql.org/pg/commitdiff/f831d4accda00b9144bc647ede2e2f848b59f39d

- Renaming for new subscripting mechanism. Over at patch
https://commitfest.postgresql.org/21/1062/ Dmitry wants to introduce a more
generic subscription mechanism, which allows subscripting not only arrays but
also other object types such as JSONB. That functionality is introduced in a
largish invasive patch, out of which this internal renaming patch was
extracted. Author: Dmitry Dolgov Reviewed-by: Tom Lane, Arthur Zakirov
Discussion:
https://postgr.es/m/CA+q6zcUK4EqPAu7XRRO5CCjMwhz5zvg+rfWuLzVoxp_5sKS6=w@mail.gmail.com
https://git.postgresql.org/pg/commitdiff/558d77f20e4e9ed18101d5d057b770ae22ece855

Thomas Munro pushed:

- Add combining characters to unaccent.rules. Strip certain classes of combining
characters, so that accents encoded this way are removed. Author: Hugh
Ranalli Discussion:
https://postgr.es/m/15548-cef1b3f8de190d4f%40postgresql.org
https://git.postgresql.org/pg/commitdiff/456e3718e7b72efe4d2639437fcbca2e4ad83099

- Add shared_memory_type GUC. Since 9.3 we have used anonymous shared mmap for
our main shared memory region, except in EXEC_BACKEND builds. Provide a GUC
so that users can opt for System V shared memory once again, like in 9.2 and
earlier. A later patch proposes to add huge/large page support for AIX, which
requires System V shared memory and provided the motivation to revive this
possibility. It may also be useful on some BSDs. Author: Andres Freund
(revived and documented by Thomas Munro) Discussion:
https://postgr.es/m/HE1PR0202MB28126DB4E0B6621CC6A1A91286D90%40HE1PR0202MB2812.eurprd02.prod.outlook.com
Discussion:
https://postgr.es/m/2AE143D2-87D3-4AD1-AC78-CE2258230C05%40FreeBSD.org
https://git.postgresql.org/pg/commitdiff/f1bebef60ec8f557324cd3bfc1671da1318de968

== Pending Patches ==

Nagaura Ryohei sent in another revision of a patch to add GUCs and supporting
infrastructure for TCP timeouts.

Thomas Munro sent in a patch to fix an infelicity in pgindent's handling of
function declarations.

Etsuro Fujita sent in a patch to PostgreSQL FDW to Fix test for cached costs in
estimate_path_cost_size().

Alexey Klyukin sent in a patch to clarify the documentation of WAL methods in
pg_basebackup.

Andrey Borodin and Andreas Karlsson traded patches to implement covering GiST
indexes.

Elvis Pranskevichus and Tom Lane traded patches to allow anonymous rowtypes in
function return column definitions.

Dmitry Dolgov sent in another revision of a patch to support access methods in
psql and pg_dump.

Tom Lane sent in a patch to refactor the planner header files.

Nathan Bossart and Michaël Paquier traded patches to add --min-xid-age and
--min-mxid-age options to vacuumdb.

Fnu Nishant and Amit Kapila traded patches to fix a possible deadlock locking
multiple heap pages.

Amit Kapila and David Rowley traded patches to speed up planning with
partitions.

Jesper Pedersen sent in five more revisions of a patch to highlight the fact
that the --jobs option isn't passed down to vacuumdb by default.

Takashi Menjo sent in a patch to remove the no-longer-used static global
variable openLogOff from xlog.c.

Evgeniy Efimkin sent in another revision of a patch to add a FOR TABLES clause
to CREATE SUBSCRIPTION.

Álvaro Herrera sent in another revision of a patch to move hash_any to
hashutils.h and move move strat nums to stratnum.h.

Dmitry Belyavsky sent in a patch to extend the sets of symbols allowed in ltree
labels.

Konstantin Knizhnik sent in another revision of a patch to implement
autoprepare.

Peter Eisentraut and Michaël Paquier traded patches to implement REINDEX
CONCURRENTLY.

Alexander Korotkov sent in another revision of a patch to implement JSONPATH.

Kyotaro HORIGUCHI sent in four revisions of a patch to implement ALTER SESSION.

Robert Haas sent in three more revisions of a patch to implement ATTACH/DETACH
PARTITION CONCURRENTLY.

Kyotaro HORIGUCHI sent in another revision of a patch to add a TAP test for the
copy-truncation optimization, write WAL for empty nbtree index build, add
infrastructure for the WAL-logging skip feature, and fix the WAL-skipping
feature.

Kyotaro HORIGUCHI sent in another revision of a patch to add a WAL relief vent
for replication slots.

Takashi Menjo sent in another revision of a patch to apply PMDK to WAL
operations for persistent memory.

Etsuro Fujita sent in another revision of a patch to the PostgreSQL FDW to
perform, the UPPERREL_ORDERED and UPPERREL_FINAL steps remotely.

Konstantin Knizhnik and Álvaro Herrera traded patches to make it possible to
launch the WAL receiver before the replica reaches the end of WAL.

Pavel Stěhule sent in another revision of a patch to implement commontype and
commontypearray.

Pavel Stěhule sent in two more revisions of a patch to implement schema
variables.

Dmitry Dolgov sent in another revision of a patch to implement index skip scan.

David Rowley sent in another revision of a patch to add --rows-per-insert to
pg_dump.

David Rowley sent in another revision of a patch to allow Append to be used in
place of MergeAppend for some cases.

David Rowley sent in another revision of a patch to forgo generating
single-subpath Append and MergeAppend paths.

Arseny Sher sent in a patch to remove the assertion in reorderbuffer that cmax
is stable.

Michaël Paquier sent in another revision of a patch to add pg_partition_root to
get top-most parent of a partition tree.

Michaël Paquier sent in a patch to add a flag to format_type_extended to enforce
NULL-ness, refactor format procedure and operator APIs to be more modular, and
eliminate user-visible cache lookup errors for objaddr SQL functions.

Michaël Paquier sent in a patch to clean up some unused parameters.

Amit Khandekar sent in a patch to add some test scenarios for testing the
dependency of various object types on table access methods.

Etsuro Fujita sent in another revision of a patch to improve the partition
matching algorithm for partition-wise join.

Kyotaro HORIGUCHI sent in a patch to stop recovery before target.

Lætitia Avrot sent in two more revisions of a patch to implement log10 and
hyperbolic functions per the SQL standard.

Takayuki Tsunakawa sent in two more revisions of a patch to add a reloption to
prevent VACUUM from truncating empty pages at the end of a relation.

Surafel Temesgen sent in another revision of a patch to implement a PERCENT
options for FETCH FIRST.

Dmitry Dolgov sent in another revision of a patch to generic type subscripting.

Matwey V. Kornilov sent in a patch to add an SP-GiST quadtree @<(point,circle)
operator.

Andrew Gierth sent in four more revisions of a patch to apply the Ryu
optimization for floating point output.

Andres Freund sent in a patch to speed up composite_to_json() by deforming the
entire tuple rather than doing it one column at a time.

Peter Geoghegan sent in a patch to avoid amcheck TOAST compression
inconsistencies.

Noah Misch sent in a patch to fix a spurious "apparent wraparound" caused by
SimpleLruTruncate() rounding.

Masahiko Sawada sent in another revision of a patch to implement block-level
parallel VACUUM.

Tom Lane sent in a patch to refactor the IndexPath representation of index
conditions.

Michaël Paquier sent in a patch to clarifying the fact that Windows does not
have meaningful support for initdb --allow-group-access.

Tom Lane sent in another revision of a patch to inline CTEs.

Sergei Kornilov sent in another revision of a patch to make it possible to
change primary_conninfo online.

Noah Misch sent in a patch to synchronize with the upstream imath library.