== PostgreSQL Weekly News - January 27, 2019 ==

== PostgreSQL Weekly News - January 27, 2019 ==

== PostgreSQL Product News ==

datasketches 1.0.0, an extension that provides approximate algorithms for big data
analysis based on the Datasketches library, released.
https://pgxn.org/dist/datasketches/1.0.0/

== PostgreSQL Jobs for January ==

http://archives.postgresql.org/pgsql-jobs/2019-01/

== PostgreSQL Local ==

FOSDEM PGDay 2019, a one day conference held before the main FOSDEM event will
be held in Brussels, Belgium, on Feb 1st, 2019.
https://2019.fosdempgday.org/

Prague PostgreSQL Developer Day 2019 (P2D2 2019) is a two-day
conference that will be held on February 13-14, 2019 in Prague, Czech Republic.
http://www.p2d2.cz/

PGConf India 2019 will be on February 13-15, 2019 in Bengaluru, Karnataka.
http://pgconf.in/

PostgreSQL(at)SCaLE is a two day, two track event which takes place on
March 7-8, 2019, at Pasadena Convention Center, as part of SCaLE 17X.
https://www.socallinuxexpo.org/scale/17x/postgresscale

pgDay Paris 2019 will be held in Paris, France on March 12, 2019
at 199bis rue Saint-Martin.
http://2019.pgday.paris/

PGConf APAC 2019 will be held in Singapore March 19-21, 2019.
http://2019.pgconfapac.org/

The German-speaking PostgreSQL Conference 2019 will take place on May 10, 2019
in Leipzig. The CfP is open until February 26, 2019 at http://2019.pgconf.de/cfp
http://2019.pgconf.de/

PGDay.IT 2019 will take place May 16th and May 17th in Bologna, Italy.
https://2019.pgday.it/en/

PGCon 2019 will take place in Ottawa on May 28-31, 2019.
https://www.pgcon.org/2019

Swiss PGDay 2019 will take place in Rapperswil (near Zurich) on June 28, 2019.
The CfP is open through April 18, 2019, and registration is open.
http://www.pgday.ch/2019/

== PostgreSQL in the News ==

Planet PostgreSQL: http://planet.postgresql.org/

PostgreSQL Weekly News is brought to you this week by David Fetter

Submit news and announcements by Sunday at 3:00pm PST8PDT to david(at)fetter(dot)org(dot)

== Applied Patches ==

Etsuro Fujita pushed:

- Postpone generating tlists and EC members for inheritance dummy children.
Previously, in set_append_rel_size(), we generated tlists and EC members for
dummy children for possible use by partition-wise join, even if partition-wise
join was disabled or the top parent was not a partitioned table, but adding
such EC members causes noticeable planning speed degradation for queries with
certain kinds of join quals like "(foo.x + bar.y) = constant" where foo and
bar are partitioned tables in cases where there are lots of dummy children, as
the EC members lists grow huge, especially for the ECs derived from such join
quals, which makes the search for the parent EC members in
add_child_rel_equivalences() very time-consuming. Postpone the work until
such children are actually involved in a partition-wise join. Reported-by:
Sanyo Capobiango Analyzed-by: Justin Pryzby and Alvaro Herrera Author: Amit
Langote, with a few additional changes by me Reviewed-by: Ashutosh Bapat
Backpatch-through: v11 where partition-wise join was added Discussion:
https://postgr.es/m/CAO698qZnrxoZu7MEtfiJmpmUtz3AVYFVnwzR%2BpqjF%3DrmKBTgpw%40mail.gmail.com
https://git.postgresql.org/pg/commitdiff/8d8dcead1295a0d718b08e84d62913b587501425

- postgres_fdw: Account for tlist eval costs in estimate_path_cost_size().
Previously, estimate_path_cost_size() didn't account for tlist eval costs,
except when costing a foreign-grouping path using local statistics, but such
costs should be accounted for when costing that path using remote estimates,
because some of the tlist expressions might be evaluated locally. Also, such
costs should be accounted for in the case of a foreign-scan or foreign-join
path, because the tlist might contain PlaceHolderVars, which postgres_fdw
currently evaluates locally. This also fixes an oversight in my commit
f8f6e44676. Like that commit, apply this to HEAD only to avoid destabilizing
existing plan choices. Author: Etsuro Fujita Discussion:
https://postgr.es/m/5BFD3EAD.2060301%40lab.ntt.co.jp
https://git.postgresql.org/pg/commitdiff/fd1afdbafd4fbb0ce23a3f319adc177e4cf8fe99

Tom Lane pushed:

- Revert "Fix under-quoted filename pattern in pgbench TAP test.". This reverts
commit 458a1244f1fcf407874482a93b7631ecf5303d6e. It has portability problems
on Windows, which will require a little bit of research to fix. Discussion:
https://postgr.es/m/20202.1548035461@sss.pgh.pa.us
https://git.postgresql.org/pg/commitdiff/f4593bd2ff0376fbc56dd1b384c7d51e1a7062f0

- Avoid thread-safety problem in ecpglib. ecpglib attempts to force the
LC_NUMERIC locale to "C" while reading server output, to avoid problems with
strtod() and related functions. Historically it's just issued setlocale()
calls to do that, but that has major problems if we're in a threaded
application. setlocale() itself is not required by POSIX to be thread-safe
(and indeed is not, on recent OpenBSD). Moreover, its effects are
process-wide, so that we could cause unexpected results in other threads, or
another thread could change our setting. On platforms having uselocale(),
which is required by POSIX:2008, we can avoid these problems by using
uselocale() instead. Windows goes its own way as usual, but we can make it
safe by using _configthreadlocale(). Platforms having neither continue to use
the old code, but that should be pretty much nobody among current systems.
This should get back-patched, but let's see what the buildfarm thinks of it
first. Michael Meskes and Tom Lane; thanks also to Takayuki Tsunakawa.
Discussion: https://postgr.es/m/31420.1547783697@sss.pgh.pa.us
https://git.postgresql.org/pg/commitdiff/8eb4a9312c95be56cdb31f5411eddc2cb2ba89be

- Sort the dependent objects before recursing in findDependentObjects().
Historically, the notices output by DROP CASCADE tended to come out in
uncertain order, and in some cases you might get different claims about which
object depends on which other one. This is because we just traversed the
dependency tree in the order in which pg_depend entries are seen, and nbtree
has never promised anything about the order of equal-keyed index entries.
We've put up with that for years, hacking regression tests when necessary to
prevent them from emitting unstable output. However, it's a problem for
pending work that will change nbtree's behavior for equal keys, as that causes
unexpected changes in the regression test results. Hence, adjust
findDependentObjects to sort the results of each indexscan before processing
them. The sort is on descending OID of the dependent objects, hence more or
less reverse creation order. While this rule could still result in bogus
regression test failures if an OID wraparound occurred mid-test, that seems
unlikely to happen in any plausible development or packaging-test scenario.
This is enough to ensure output stability for ordinary DROP CASCADE commands,
but not for DROP OWNED BY, because that has a different code path with the
same problem. We might later choose to sort in the DROP OWNED BY code as
well, but this patch doesn't do so. I've also not done anything about
reverting the existing hacks to suppress unstable DROP CASCADE output in
specific regression tests. It might be worth undoing those, but it seems like
a distinct question. The first indexscan loop in findDependentObjects is not
touched, meaning there is a hazard of unstable error reports from that too.
However, said hazard is not the fault of that code: it was designed on the
assumption that there could be at most one "owning" object to complain about,
and that assumption does not seem unreasonable. The recent patch that added
the possibility of multiple DEPENDENCY_INTERNAL_AUTO links broke that
assumption, but we should fix that situation not band-aid around it. That's a
matter for another patch, though. Discussion:
https://postgr.es/m/12244.1547854440@sss.pgh.pa.us
https://git.postgresql.org/pg/commitdiff/f1ad067fc3ae3d34dc6b8826913d454cc3fe354f

- Fix sepgsql regression test. Message order in the expected output changes due
to commit f1ad067fc. Per buildfarm. Discussion:
https://postgr.es/m/20190121201134.dyx6anto6akflh5d@alap3.anarazel.de
https://git.postgresql.org/pg/commitdiff/071e11898dd42c89dc68ddc4e4fe6a35049729d6

- Second try at fixing ecpglib thread-safety problem. While Windows (allegedly)
has _configthreadlocale() pretty far back, it seems MinGW didn't acquire
support for that till more recently. Fortunately, we can use an autoconf probe
on that toolchain, instead of guessing whether it's there. (Hm, I wonder
whether Cygwin will need this also.) Per buildfarm. Discussion:
https://postgr.es/m/20190121193512.tdmcnic2yjxlufaw@alap3.anarazel.de
https://git.postgresql.org/pg/commitdiff/ee27584c4a40620fb26d1729e9cc449d54d62b07

- Remove useless bms_copy step in RelationGetIndexAttrBitmap. Seems to be from a
bad case of copy-and-paste-itis in commit 665d1fad9. It wouldn't be quite so
annoying if it didn't contradict the comment half a dozen lines above. David
Rowley Discussion:
https://postgr.es/m/CAKJS1f95Dyf8Qkdz4W+PbCmT-HTb54tkqUCC8isa2RVgSJ_pXQ@mail.gmail.com
https://git.postgresql.org/pg/commitdiff/8f9e934ab746907abbeeed06a1a9b995e5b4d339

- Blind attempt to fix _configthreadlocale() failures on MinGW. Apparently, some
builds of MinGW contain a version of _configthreadlocale() that always returns
-1, indicating failure. Rather than treating that as a curl-up-and-die
condition, soldier on as though the function didn't exist. This leaves us
without thread safety on such MinGW versions, but we didn't have it anyway.
Discussion:
https://postgr.es/m/d06a16bc-52d6-9f0d-2379-21242d7dbe81@2ndQuadrant.com
https://git.postgresql.org/pg/commitdiff/2cf91ccb73ce888c44e3751548fb7c77e87335f2

- Fix portability problem in pgbench. The pgbench regression test supposed that
srandom() with a specific value would result in deterministic output from
random(), as required by POSIX. It emerges however that OpenBSD is too smart
to be constrained by mere standards, so their random() emits nondeterministic
output anyway. While a workaround does exist, what seems like a better fix is
to stop relying on the platform's srandom()/random() altogether, so that what
you get from --random-seed=N is not merely deterministic but platform
independent. Hence, use a separate pg_jrand48() random sequence in place of
random(). Also adjust the regression test case that's supposed to detect
nondeterminism so that it's more likely to detect it; the original choice of
random_zipfian parameter tended to produce the same output all the time even
if the underlying behavior wasn't deterministic. In passing, improve
pgbench's docs about random_zipfian(). Back-patch to v11 where this code was
introduced. Fabien Coelho and Tom Lane Discussion:
https://postgr.es/m/4615.1547792324@sss.pgh.pa.us
https://git.postgresql.org/pg/commitdiff/e6c3ba7fbfd59ceabcf9bdaf52d71b44831b09d2

- Remove infinite-loop hazards in ecpg test suite. A report from Andrew Dunstan
showed that an ecpglib breakage that causes repeated query failures could lead
to infinite loops in some ecpg test scripts, because they contain "while(1)"
loops with no exit condition other than successful test completion. That
might be all right for manual testing, but it seems entirely unacceptable for
automated test environments such as our buildfarm. We don't want buildfarm
owners to have to intervene manually when a test goes wrong. To fix, just
change all those while(1) loops to exit after at most 100 iterations (which is
more than any of them expect to iterate). This seems sufficient since we'd see
discrepancies in the test output if any loop executed the wrong number of
times. I tested this by dint of intentionally breaking ecpg_do_prologue to
always fail, and verifying that the tests still got to completion. Back-patch
to all supported branches, since the whole point of this exercise is to
protect the buildfarm against future mistakes. Discussion:
https://postgr.es/m/18693.1548302004@sss.pgh.pa.us
https://git.postgresql.org/pg/commitdiff/d5a1fde397269bd60b99e70a4eb2934b75fd31ec

- Remove _configthreadlocale() calls in ecpg test suite. This essentially
reverts commits a772624b1 and 04fbe0e45, which added
"_configthreadlocale(_ENABLE_PER_THREAD_LOCALE)" calls to the thread-related
ecpg test programs. That was nothing but a hack, because we shouldn't expect
that ecpg-using applications have done that for us; and now that we've
inserted such calls into ecpglib, the tests should still pass without it. (If
they don't, it would be good to know that.) HEAD only; there seems no big
need to change this in the back branches. Discussion:
https://postgr.es/m/22937.1548307384@sss.pgh.pa.us
https://git.postgresql.org/pg/commitdiff/e3565fd61cee16414a682d2be0d44c112a80a30c

- Fix possibly-uninitialized-variable warning from commit 9556aa01c. Heikki's
compiler doesn't complain about end_ptr, apparently, but mine does. In
passing, I failed to resist the temptation to remove the no-longer-used fldnum
variable, and relocate chunk_len's declaration to a narrower scope.
https://git.postgresql.org/pg/commitdiff/6119060d01ab2dfb9e2c1c101595f2de7890965d

- Teach nulltestsel() that system columns are never NULL. While it's perhaps
unlikely that users would write an explicit test like "ctid IS NULL", this
function is also used in range estimation, and an incorrect answer can throw
off the results for tight ranges. Anyway it's not much code so we might as
well do it. Edmund Horner Discussion:
https://postgr.es/m/CAMyN-kCa3BFUFrCTtQeprxTU1anCd3Pua7zXstGCKq4pXgjukw@mail.gmail.com
https://git.postgresql.org/pg/commitdiff/ff750ce2d82979e9588c629955e161a9379b05f3

- Split QTW_EXAMINE_RTES flag into QTW_EXAMINE_RTES_BEFORE/_AFTER. This change
allows callers of query_tree_walker() to choose whether to visit an RTE before
or after visiting the contents of the RTE (i.e., prefix or postfix tree
order). All existing users of QTW_EXAMINE_RTES want the
QTW_EXAMINE_RTES_BEFORE behavior, but an upcoming patch will want
QTW_EXAMINE_RTES_AFTER, and it seems like a potentially useful change on its
own. Andreas Karlsson (extracted from CTE inlining patch) Discussion:
https://postgr.es/m/8810.1542402910@sss.pgh.pa.us
https://git.postgresql.org/pg/commitdiff/18c0da88a5d9da566c3bfac444366b73bd0b57da

- Allow UNLISTEN in hot-standby mode. Since LISTEN is (still) disallowed,
UNLISTEN must be a no-op in a hot-standby session, and so there's no harm in
allowing it. This change allows client code to not worry about whether it's
connected to a primary or standby server when performing session-state-reset
type activities. (Note that DISCARD ALL, which includes UNLISTEN, was already
allowed, making it inconsistent to reject UNLISTEN.) Per discussion,
back-patch to all supported versions. Shay Rojansky, reviewed by Mi Tar
Discussion:
https://postgr.es/m/CADT4RqCf2gA_TJtPAjnGzkC3ZiexfBZiLmA-mV66e4UyuVv8bA@mail.gmail.com
https://git.postgresql.org/pg/commitdiff/ebfe20dc706bd3238a9bdf3b44cd8f82337e86a8

- Fix psql's "\g target" meta-command to work with COPY TO STDOUT. Previously,
\g would successfully execute the COPY command, but the target specification
if any was ignored, so that the data was always dumped to the regular query
output target. This seems like a clear bug, so let's not just fix it but
back-patch it. While at it, adjust the documentation for \copy to recommend
"COPY ... TO STDOUT \g foo" as a plausible alternative. Back-patch to 9.5.
The problem exists much further back, but the code associated with \g was
refactored enough in 9.5 that we'd need a significantly different patch for
9.4, and it doesn't seem worth the trouble. Daniel Vérité, reviewed by Fabien
Coelho Discussion:
https://postgr.es/m/15dadc39-e050-4d46-956b-dcc4ed098753@manitou-mail.org
https://git.postgresql.org/pg/commitdiff/6d3ede5f1c654f923b2767b0b0c3b09569adaa18

- Allow for yet another crash symptom in 013_crash_restart.pl. Given the right
timing, psql could emit "connection to server was lost" rather than one of the
other messages that this test script checked for. It looks like commit
4247db625 may have made this more likely, but I don't really believe it was
impossible before then. Rather than stress about it, just add that spelling
as one of the crash-successfully- detected cases. Discussion:
https://postgr.es/m/19344.1548554028@sss.pgh.pa.us
https://git.postgresql.org/pg/commitdiff/d6f6f0fc2d84906985b21d959622e4cab6f8f9b1

Andres Freund pushed:

- Adjust some more comments for WITH OIDS removal. I missed these in
578b229718e8f. Author: Andres Freund
https://git.postgresql.org/pg/commitdiff/93507e67c9ca54026019ebec3026de35d30370f9

- Fix ALTER TRIGGER ... RENAME, broken in WITH OIDS removal. I (Andres) broke
this in 578b229718e. Author: Rushabh Lathia Discussion:
https://postgr.es/m/CAGPqQf04PywZX3sVQaF6H=oLiW9GJncRW+=e78vTy4MokEWcZw@mail.gmail.com
https://git.postgresql.org/pg/commitdiff/8cc157b23413c21f229e376cb2982a2086655fc9

- Introduce access/{table.h, relation.h}, for generic functions from heapam.h.
access/heapam contains functions that are very storage specific (say
heap_insert() and a lot of lower level functions), and fairly generic
infrastructure like relation_open(), heap_open() etc. In the upcoming
pluggable storage work we're introducing a layer between table accesses in
general and heapam, to allow for different storage methods. For a bit cleaner
separation it thus seems advantageous to move generic functions like the
aforementioned to their own headers. access/relation.h will contain
relation_open() etc, and access/table.h will contain table_open() (formerly
known as heap_open()). I've decided for table.h not to include relation.h, but
we might change that at a later stage. relation.h already exists in another
directory, but the other plausible name (rel.h) also conflicts. It'd be nice
if there were a non-conflicting name, but nobody came up with a suggestion.
It's possible that the appropriate way to address the naming conflict would be
to rename nodes/relation.h, which isn't particularly well named. To avoid
breaking a lot of extensions that just use heap_open() etc, table.h has macros
mapping the old names to the new ones, and heapam.h includes relation,
table.h. That also allows to keep the bulk renaming of existing callers in a
separate commit. Author: Andres Freund Discussion:
https://postgr.es/m/20190111000539.xbv7s6w7ilcvm7dp@alap3.anarazel.de
https://git.postgresql.org/pg/commitdiff/4b21acf522d751ba5b6679df391d5121b6c4a35f

- Replace uses of heap_open et al with the corresponding table_* function.
Author: Andres Freund Discussion:
https://postgr.es/m/20190111000539.xbv7s6w7ilcvm7dp@alap3.anarazel.de
https://git.postgresql.org/pg/commitdiff/e0c4ec07284db817e1f8d9adfb3fffc952252db0

- Replace heapam.h includes with {table, relation}.h where applicable. A lot of
files only included heapam.h for relation_open, heap_open etc - replace the
heapam.h include in those files with the narrower header. Author: Andres
Freund Discussion:
https://postgr.es/m/20190111000539.xbv7s6w7ilcvm7dp@alap3.anarazel.de
https://git.postgresql.org/pg/commitdiff/111944c5ee567f1c45bf0f1ecfdec682af467aa6

- Remove superfluous tqual.h includes. Most of these had been obsoleted by
568d4138c / the SnapshotNow removal. This is is preparation for moving most
of tqual.[ch] into either snapmgr.h or heapam.h, which in turn is in
preparation for pluggable table AMs. Author: Andres Freund Discussion:
https://postgr.es/m/20180703070645.wchpu5muyto5n647@alap3.anarazel.de
https://git.postgresql.org/pg/commitdiff/e7cc78ad43eb1a6ea43dbb46c83264d2d00e402c

- Fix "Remove superfluous tqual.h includes" by adding back one include. I
removed one include too many in e7cc78ad43eb, not sure why that escaped my
test script. Author: Andres Freund
https://git.postgresql.org/pg/commitdiff/527114e51e45dbd91d46171fa4a111355f0dfc3b

- Change snapshot type to be determined by enum rather than callback. This is in
preparation for allowing the same snapshot be used for different table AMs.
With the current callback based approach we would need one callback for each
supported AM, which clearly would not be extensible. Thus add a new
Snapshot->snapshot_type field, and move the dispatch into
HeapTupleSatisfiesVisibility() (which is now a function). Later work will then
dispatch calls to HeapTupleSatisfiesVisibility() and other AMs visibility
functions depending on the type of the table. The central SnapshotType enum
also seems like a good location to centralize documentation about the intended
behaviour of various types of snapshots. As tqual.h isn't included by
bufmgr.h any more (as HeapTupleSatisfies* isn't referenced by
TestForOldSnapshot() anymore) a few files now need to include it directly.
Author: Andres Freund, loosely based on earlier work by Haribabu Kommi
Discussion:
https://postgr.es/m/20180703070645.wchpu5muyto5n647@alap3.anarazel.de
https://postgr.es/m/20160812231527.GA690404@alvherre.pgsql
https://git.postgresql.org/pg/commitdiff/63746189b23815415cacc715fdc4f6b991f1a5e7

- Move generic snapshot related code from tqual.h to snapmgr.h. The code in
tqual.c is largely heap specific. Due to the upcoming pluggable storage work,
it therefore makes sense to move it into access/heap/ (as the file's header
notes, the tqual name isn't very good). But the various statically allocated
snapshot and snapshot initialization functions are now (see previous commit)
generic and do not depend on functions declared in tqual.h anymore. Therefore
move. Also move XidInMVCCSnapshot as that's useful for future AMs, and already
used outside of tqual.c. Author: Andres Freund Discussion:
https://postgr.es/m/20180703070645.wchpu5muyto5n647@alap3.anarazel.de
https://git.postgresql.org/pg/commitdiff/b7eda3e0e33402a7727253e6f307ca96b92eb23d

- Rephrase references to "time qualification". Now that the relevant code has,
for other reasons, moved out of tqual.[ch], it seems time to refer to
visiblity rather than time qualification. Author: Andres Freund Discussion:
https://postgr.es/m/20180703070645.wchpu5muyto5n647@alap3.anarazel.de
https://git.postgresql.org/pg/commitdiff/ebcc7bf949bae614cccc6b86e3ef9b926a6e2f99

- Move remaining code from tqual.[ch] to heapam.h / heapam_visibility.c. Given
these routines are heap specific, and that there will be more generic
visibility support in via table AM, it makes sense to move the prototypes to
heapam.h (routines like HeapTupleSatisfiesVacuum will not be exposed in a
generic fashion, because they are too storage specific). Similarly, the code
in tqual.c is specific to heap, so moving it into access/heap/ makes sense.
Author: Andres Freund Discussion:
https://postgr.es/m/20180703070645.wchpu5muyto5n647@alap3.anarazel.de
https://git.postgresql.org/pg/commitdiff/c91560defc57f89f7e88632ea14ae77b5cec78ee

- Rename RelationData.rd_amroutine to rd_indam. The upcoming table AM support
makes rd_amroutine to generic, as its only about index AMs. The new name makes
that clear, and is shorter to boot. Author: Andres Freund Discussion:
https://postgr.es/m/20180703070645.wchpu5muyto5n647@alap3.anarazel.de
https://git.postgresql.org/pg/commitdiff/346ed70b0ad7efc574711a97812692dab4542712

- llvm: Fix file-ending in IDENTIFICATION comments. Author: Amit Langote
Discussion:
https://postgr.es/m/9a54dcef-c799-ce89-2e47-0a7fc12d5fc2@lab.ntt.co.jp
Backpatch: 11-, where llvm was introduced.
https://git.postgresql.org/pg/commitdiff/005881033d4e34c0b2e0a157464dc1929db52e17

- Change function call information to be variable length. Before this change
FunctionCallInfoData, the struct arguments etc for V1 function calls are
stored in, always had space for FUNC_MAX_ARGS/100 arguments, storing datums
and their nullness in two arrays. For nearly every function call 100
arguments is far more than needed, therefore wasting memory. Arg and argnull
being two separate arrays also guarantees that to access a single argument,
two cachelines have to be touched. Change the layout so there's a single
variable-length array with pairs of value / isnull. That drastically reduces
memory consumption for most function calls (on x86-64 a two argument function
now uses 64bytes, previously 936 bytes), and makes it very likely that
argument value and its nullness are on the same cacheline. Arguments are
stored in a new NullableDatum struct, which, due to padding, needs more memory
per argument than before. But as usually far fewer arguments are stored, and
individual arguments are cheaper to access, that's still a clear win. It's
likely that there's other places where conversion to NullableDatum arrays
would make sense, e.g. TupleTableSlots, but that's for another commit.
Because the function call information is now variable-length allocations have
to take the number of arguments into account. For heap allocations that can be
done with SizeForFunctionCallInfoData(), for on-stack allocations there's a
new LOCAL_FCINFO(name, nargs) macro that helps to allocate an appropriately
sized and aligned variable. Some places with stack allocation function call
information don't know the number of arguments at compile time, and currently
variably sized stack allocations aren't allowed in postgres. Therefore allow
for FUNC_MAX_ARGS space in these cases. They're not that common, so for now
that seems acceptable. Because of the need to allocate FunctionCallInfo of
the appropriate size, older extensions may need to update their code. To avoid
subtle breakages, the FunctionCallInfoData struct has been renamed to
FunctionCallInfoBaseData. Most code only references FunctionCallInfo, so that
shouldn't cause much collateral damage. This change is also a prerequisite
for more efficient expression JIT compilation (by allocating the function call
information on the stack, allowing LLVM to optimize it away); previously the
size of the call information caused problems inside LLVM's optimizer. Author:
Andres Freund Reviewed-By: Tom Lane Discussion:
https://postgr.es/m/20180605172952.x34m5uz6ju6enaem@alap3.anarazel.de
https://git.postgresql.org/pg/commitdiff/a9c35cf85ca1ff72f16f0f10d7ddee6e582b62b8

Álvaro Herrera pushed:

- Add 'id' to Acknowledgments section. Per note from Erik Rijkers Discussion:
https://postgr.es/m/3db724af16ee009ab7f812a6a1d9354e@xs4all.nl
https://git.postgresql.org/pg/commitdiff/fcea1e10904856bbc77b701a0195d0524b9705d6

- Flush relcache entries when their FKs are meddled with. Back in commit
100340e2dcd0, we made relcache entries keep lists of the foreign keys applying
to the relation -- but we forgot to update CacheInvalidateHeapTuple to flush
those entries when new FKs got created or existing ones updated/deleted. No
bugs appear to have been reported that would be explained by this ommission,
but I noticed the problem while working on an unrelated bugfix which clearly
showed it. Fix by adding relcache flush on relevant foreign key changes.
Backpatch to 9.6, like the aforementioned commit. Discussion:
https://postgr.es/m/201901211927.7mmhschxlejh@alvherre.pgsql Reviewed-by: Tom
Lane
https://git.postgresql.org/pg/commitdiff/175544093524bef9c28c59c8d49c77b1fcf4fb92

- Create action triggers when partitions are detached. Detaching a partition
from a partitioned table that's constrained by foreign keys requires
additional action triggers on the referenced side; otherwise, DELETE/UPDATE
actions there fail to notice rows in the table that was partition, and so are
incorrectly allowed through. With this commit, those triggers are now
created. Conversely, when a table that has a foreign key is attached as a
partition to a table that also has the same foreign key, those action triggers
are no longer needed, so we remove them. Add a minimal test case verifying
(part of) this. Authors: Amit Langote, Álvaro Herrera Discussion:
https://postgr.es/m/f2b8ead5-4131-d5a8-8016-2ea0a31250af@lab.ntt.co.jp
https://git.postgresql.org/pg/commitdiff/0464fdf07f69452dd67dd798c026d82c956bda33

- Detach constraints when partitions are detached. I (Álvaro) forgot to do this
in eb7ed3f30634, leading to undroppable constraints after partitions are
detached. Repair. Reported-by: Amit Langote Author: Amit Langote Discussion:
https://postgr.es/m/c1c9b688-b886-84f7-4048-1e4ebe9b1d06@lab.ntt.co.jp
https://git.postgresql.org/pg/commitdiff/ae366aa57762ad0e6a1a0885a7644e79541afe39

- Simplify coding to detach constraints when detaching partition. The original
coding was too baroque and led to an use-after-release mistake, noticed by
buildfarm member prion. Discussion:
https://postgr.es/m/21693.1548305934@sss.pgh.pa.us
https://git.postgresql.org/pg/commitdiff/19184fcc09739abf75ccdada965ed6135c6d07c3

- Fix droppability of constraints upon partition detach. We were failing to set
conislocal correctly for constraints in partitions after partition detach,
leading to those constraints becoming undroppable. Fix by setting the flag
correctly. Existing databases might contain constraints with the conislocal
wrongly set to false, for partitions that were detached; this situation should
be fixable by applying an UPDATE on pg_constraint to set conislocal true.
This problem should otherwise be innocuous and should disappear across a
dump/restore or pg_upgrade. Secondarily, when constraint drop was attempted
in a partitioned table, ATExecDropConstraint would try to recurse to
partitions after doing performDeletion() of the constraint in the partitioned
table itself; but since the constraint in the partitions are dropped by the
initial call of performDeletion() (because of following dependencies), the
recursion step would fail since it would not find the constraint, causing the
whole operation to fail. Fix by preventing recursion. Reported-by: Amit
Langote Diagnosed-by: Amit Langote Author: Amit Langote, Álvaro Herrera
Discussion:
https://postgr.es/m/f2b8ead5-4131-d5a8-8016-2ea0a31250af@lab.ntt.co.jp
https://git.postgresql.org/pg/commitdiff/efd9366dcedec8db9fc85e77d071b0c5a6badea2

Michaël Paquier pushed:

- Adjust documentation for vacuumdb --disable-page-skipping. This makes the
description more consistent with the other options, and the mapping with
VACUUM is intuitive. Author: Nathan Bossart Discussion:
https://postgr.es/m/FFE5373C-E26A-495B-B5C8-911EC4A41C5E@amazon.com
https://git.postgresql.org/pg/commitdiff/efab708997dcd7a78cce289bcd9d3643a3455465

- Make vacuumdb test regex more modular for its query output. This is in
preparation for always using a catalog query to discover tables, where the
ANALYZE and VACUUM queries get completed with relation names. Author: Nathan
Bossart Discussion: https://postgr.es/m/20190122060730.GD8719@paquier.xyz
https://git.postgresql.org/pg/commitdiff/adaaacae658e37a3e3c836266681774395b437fe

- Fix typo in pgbench.c. Author: Moon, Insung Discussion:
https://postgr.es/m/008001d4b2db$1f772170$5e656450$@lab.ntt.co.jp
https://git.postgresql.org/pg/commitdiff/1699e6dd1fa7fbc7c3d114b121828eb27eb523b1

- Remove argument isprimary from index_build(). The flag was introduced in
3fdeb18, but f66e8bf actually forgot to finish the cleanup as
index_update_stats() has simplified its interface. Author: Michael Paquier
Discussion: https://postgr.es/m/20190122080852.GB3873@paquier.xyz
https://git.postgresql.org/pg/commitdiff/289198c0d93a3d591f379c4134e90275bd426426

- Simplify restriction handling of two-phase commit for temporary objects. There
were two flags used to track the access to temporary tables and to the
temporary namespace of a session which are used to restrict PREPARE
TRANSACTION, however the first control flag is a concept included in the
second. This removes the flag for temporary table tracking, keeping around
only the one at namespace level. Author: Michael Paquier Reviewed-by: Álvaro
Herrera Discussion: https://postgr.es/m/20190118053126.GH1883@paquier.xyz
https://git.postgresql.org/pg/commitdiff/c9b75c5838feeae73dbae00bce9d8f650b80ba38

- Add TAP tests for vacuumdb with column lists. vacuumdb generates by itself SQL
queries to run ANALYZE or VACUUM on the backend, but we never actually checked
for query patterns with column lists defined. Author: Michael Paquier
Reviewed-by: Nathan Bossart Discussion:
https://postgr.es/m/FFE5373C-E26A-495B-B5C8-911EC4A41C5E@amazon.com
https://git.postgresql.org/pg/commitdiff/0803b0ae1ed6e5825eee4d1ba227e96cd8a328d8

Tomáš Vondra pushed:

- Fix handling of volatile expressions in COPY FROM ... WHERE. The checking for
calls to volatile functions in the COPY FROM ... WHERE expression was treating
all WHERE clauses as if containing such calls. While that does not produce
incorrect results, this disables batching which may result in significant
performance regression. Discussion:
https://www.postgresql.org/message-id/flat/CALAY4q_DdpWDuB5-Zyi-oTtO2uSk8pmy+dupiRe3AvAc++1imA(at)mail(dot)gmail(dot)com
https://git.postgresql.org/pg/commitdiff/4a8283d0ec5a6781b9e0d4ba16b44da8a0856d02

Tatsuo Ishii pushed:

- Doc: fix typo in URL of OASIS group web site. In other places that has been
changed from http://www.oasis-open.org/ https://www.oasis-open.org/ but
there's a place where the change was missed. Discussion:
https://postgr.es/m/20190121.222844.399814306477973879.t-ishii%40sraoss.co.jp
https://git.postgresql.org/pg/commitdiff/78855e798325f64e36e2775031137ea83ad0c3ef

Heikki Linnakangas pushed:

- Fix misc typos in comments. Spotted mostly by Fabien Coelho. Discussion:
https://www.postgresql.org/message-id/alpine.DEB.2.21.1901230947050.16643@lancre
https://git.postgresql.org/pg/commitdiff/95931133a959404cf7320b49058b2f1c2e4606b9

- Fix comments that claimed that mblen() only looks at first byte. GB18030's
mblen() function looks at the first and the second byte of the multibyte
character, to determine its length. copy.c had made the assumption that
mblen() only looks at the first byte, but it turns out to work out fine,
because of the way the GB18030 encoding works. COPY will see a 4-byte encoded
character as two 2-byte encoded characters, which is enough for COPY's
purposes. It cannot mix those up with delimiter or escaping characters,
because only single-byte ASCII characters are supported as delimiters or
escape characters. Discussion:
https://www.postgresql.org/message-id/7704d099-9643-2a55-fb0e-becd64400dcb%40iki.fi
https://git.postgresql.org/pg/commitdiff/a5be6e9a1dfe820807f9ccb21dec5144982618e6

- Use single-byte Boyer-Moore-Horspool search even with multibyte encodings. The
old implementation first converted the input strings to arrays of wchars, and
performed the conversion on those. However, the conversion is expensive, and
for a large input string, consumes a lot of memory. Allocating the large
arrays also meant that these functions could not be used on strings larger 1
GB / pg_encoding_max_length() (256 MB for UTF-8). Avoid the conversion, and
instead use the single-byte algorithm even with multibyte encodings. That can
get fooled, if there is a matching byte sequence in the middle of a multi-byte
character, so to eliminate false positives like that, we verify any matches by
walking the string character by character with pg_mblen(). Also, if the caller
needs the position of the match, as a character-offset, we also need to walk
the string to count the characters. Performance testing shows that walking
the whole string with pg_mblen() is somewhat slower than converting the whole
string to wchars. It's still often a win, though, because we don't need to do
it if there is no match, and even when there is, we only need to walk up to
the point where the match is, not the whole string. Even in the worst case,
there would be room for optimization: Much of the CPU time in the current loop
with pg_mblen() is function call overhead, and could be improved by inlining
pg_mblen() and/or the encoding-specific mblen() functions. But I didn't
attempt to do that as part of this patch. Most of the callers of
text_position_setup/next functions were actually not interested in the
position of the match, counted in characters. To cater for them, refactor the
text_position_next() interface into two parts: searching for the next match
(text_position_next()), and returning the current match's position as a
pointer (text_position_get_match_ptr()) or as a character offset
(text_position_get_match_pos()). Getting the pointer to the match is a more
convenient API for many callers, and with UTF-8, it allows skipping the
character-walking step altogether, because UTF-8 can't have false matches even
when treated like raw byte strings. Reviewed-by: John Naylor Discussion:
https://www.postgresql.org/message-id/3173d989-bc1c-fc8a-3b69-f24246f73876%40iki.fi
https://git.postgresql.org/pg/commitdiff/9556aa01c69a26ca726d8dda8e395acc7c1e30fc

Peter Eisentraut pushed:

- Fix whitespace.
https://git.postgresql.org/pg/commitdiff/bf2fb2e03ebd897575d7428c73ae6ba7b1f91fc7

- PL/pgSQL: Add statement ID to statement structures. This can be used by a
profiler as the index for an array of per-statement metrics. Author: Pavel
Stehule <pavel(dot)stehule(at)gmail(dot)com> Reviewed-by: Peter Eisentraut
<peter(dot)eisentraut(at)2ndquadrant(dot)com> Discussion:
https://www.postgresql.org/message-id/flat/CAFj8pRDRCjN6rpM9ZccU7Ta_afsNX7mg9=n34F+r445Nt9v2tA(at)mail(dot)gmail(dot)com/
https://git.postgresql.org/pg/commitdiff/bbd5c207b960dd072445a3547f16abea0c27f726

- Allow generalized expression syntax for partition bounds. Previously, only
literals were allowed. This change allows general expressions, including
functions calls, which are evaluated at the time the DDL command is executed.
Besides offering some more functionality, it simplifies the parser structures
and removes some inconsistencies in how the literals were handled. Author:
Kyotaro Horiguchi, Tom Lane, Amit Langote Reviewed-by: Peter Eisentraut
<peter(dot)eisentraut(at)2ndquadrant(dot)com> Discussion:
https://www.postgresql.org/message-id/flat/9f88b5e0-6da2-5227-20d0-0d7012beaa1c(at)lab(dot)ntt(dot)co(dot)jp/
https://git.postgresql.org/pg/commitdiff/7c079d7417a8f2d4bf5144732e2f85117db9214f

- Make regression test output locale-independent. In some locales, letters sort
before numbers, so change the object naming to not depend on that. Introduced
by commit 7c079d7417a8f2d4bf5144732e2f85117db9214f.
https://git.postgresql.org/pg/commitdiff/1e4730c639acb967b8e40e6f33e81ebe1721ea5f

Bruce Momjian pushed:

- SQL comment: remove extra word in heading comment. Reported-by: Daniel
Gustafsson Discussion:
https://postgr.es/m/431D5BC1-9696-43FA-B54C-39D5503EB753@yesql.se
Backpatch-through: master
https://git.postgresql.org/pg/commitdiff/df4c9044406f1e907268930dd12ba6c3642d21dd

== Pending Patches ==

Masahiko Sawada sent in a patch to add a DISABLE_INDEX_CLEANUP option to VACUUM.

Fnu Nishant sent in another revision of a patch to ensure that locks for heap
pages are acquired in a consistent order.

Aleksey Kondratov sent in another revision of a patch to pg_rewind to add
options to use restore_command from postgresql.conf or command line.

Peter Geoghegan sent in a patch to demonstrate how to adjust the harmless
objects-to-drop output from DROP ROLE to make relevant tests pass, and another
to make the regression tests actively ignore visible manifestations of the
current DEPENDENCY_INTERNAL_AUTO bug.

Andreas Karlsson sent in two more revisions of a patch to implement temporary
materialized views.

Kyotaro HORIGUCHI sent in two more revisions of a patch to reimplement the stats
collector so it uses shared memory instead of files.

Haribabu Kommi sent in two more revisions of a patch to reduce the use of
HeapTuple t_tableOid and remove scan_update_snapshot.

Amit Langote sent in two more revisions of a patch to speed up planning with
partitions.

Kyotaro HORIGUCHI sent in two more revisions of a patch to remove old catcache
entries, collect syscache statistics and expose them in a new view, and make
catcache remote-settable.

David Rowley sent in a patch to fix a mismatch between the comments and code in
RelationGetIndexAttrBitmap().

Hubert Zhang sent in another revision of a patch to add smgr*_hook() hook points
to extend the logic of storage management.

Dilip Kumar sent in another revision of a patch to implement logical decoding
for operations on zheap tables.

Fabien COELHO sent in another revision of a patch to fix an issue where Perl
special characters in a path name were being treated differently from other path
names.

Etsuro Fujita sent in another revision of a patch to improve the algorithm for
partition-wise JOINs.

James Coleman sent in two more revisions of a patch to enable proving IS NOT
NULL inference for ScalarArrayOpExpr's.

Pavel Stěhule sent in another revision of a patch to implement schema
variables.

David Rowley sent in another revision of a patch to allow lock acquisitions for
partitions to be delayed.

Álvaro Herrera sent in another revision of a patch to make it possible to point
foreign key constraints at partitioned tables.

Tomáš Vondra sent in two revisions of a patch to separate the per-tuple and
per-batch contexts in COPY ... FROM ... WHERE.

Michaël Paquier sent in a patch to remove the isprimary argument from
index_build().

Surafel Temesgen and David Rowley traded patches to make it possible for
pg_dump's --inserts and --column-inserts options to emit multi-VALUES INSERTs.

Fabien COELHO sent in another revision of a patch to fix usage of PRNGs in
pgbench.

Fabien COELHO sent in another revision of a patch to add a Zipf distribution to
pgbench.

David Rowley sent in another revision of a patch to delay locking of partitions
during INSERT and UPDATE.

Alexander Korotkov and Nikita Glukhov traded patches to implement JSONPATH.

Andreas Karlsson and Tom Lane traded patches to make CTEs inlined by default,
with controls to get different behaviors including the legacy "run exactly 0 or
1 times all the way through" behavior.

Chris Travers sent in another revision of a patch to clarify the documentation
for signal detection.

Corey Huinker sent in a patch to change referential integrity check triggers
from per-row to per-statement, speeding up the process by up to 25%.

Dmitry Dolgov sent in two revisions of a patch to factor out arguments for
ArchiveEntry into a new ArchiveOpts structure, to make it more flexible for
changes we anticipate making fairly frequently in the near term.

Amit Kapila sent in two more revisions of a patch to prevent bootstrap mode from
creating files if they don't already exist, and avoid creating FSM files for
small heap relations.

Peter Geoghegan sent in another revision of a patch to make all nbtree entries
unique by having heap TIDs participate in comparisons.

Masahiko Sawada sent in another revision of a patch to implement block-level
parallel vacuum.

Kyotaro HORIGUCHI sent in a patch to make it possible to control whether symbols
in external modules are exported.

Dilip Kumar sent in another revision of a patch to implement an UNDO worker and
ROLLBACK on zheap that can go through same.

Konstantin Knizhnik sent in another revision of a patch to implement a built-in
connection pooler.

Tom Lane sent in two revisions of a patch to enable extensions to supply
operator- and function-specific info.

Surafel Temesgen sent in another revision of a patch to implement FETCH FIRST
... PERCENT.

Corey Huinker sent in a patch to implement \describe in psql, expanding the
somewhat cryptic \d syntax into an optional form that's more descriptive.

Etsuro Fujita sent in a patch to fix a cost size estimate.

John Naylor and Amit Kapila traded patches to conditionally skip transfer of
FSMs during pg_upgrade.

Takashi Menjo sent in another revision of a patch to add a configure option for
PMDK, read and write WALs using PMDK, and enable walreceiver use PMDK.

Peter Eisentraut sent in another revision of a patch to implement collations
with nondeterministic comparison, of which case-insensitive collation is a
special instance.

Alexey Klyukin sent in another revision of a patch to add a max_wal_senders GUC.

Chapman Flack sent in two more revisions of a patch to fix some infelicities in
handling of XML.

Adrien Nayrat sent in another revision of a patch to make it possible to log a
sample of transactions.

Dmitry Dolgov sent in two more revisions of a patch to implement index skip
scan a.k.a. "loose index scan."

Lætitia Avrot sent in two revisions of a patch to implement log10 and hyperbolic
functions for SQL:2016 compliance.

Alexey Bashtanov sent in another revision of a patch to log bind parameter
values on error.