== PostgreSQL Weekly News - July 14, 2019 ==

== PostgreSQL Weekly News - July 14, 2019 ==

Nordic PGDay 2020 will be held in Helsinki, Finland at the Hilton Helsinki
Strand Hotel on March 24, 2020. The CfP is open through December 31, 2019 at
https://2020.nordicpgday.org/cfp/

== PostgreSQL Jobs for July ==

http://archives.postgresql.org/pgsql-jobs/2019-07/

== PostgreSQL Local ==

PGConf.Brazil 2019 will take place August 1-3, 2019 in São Paulo.
http://pgconf.com.br

The first Austrian pgDay, will take place September 6, 2019 at the Hilton Garden
Inn in Wiener Neustadt.
https://pgday.at/en/

PostgresOpen will be September 11th - 13th, 2019 in Orlando, Florida at the
Rosen Centre Hotel. The CfP is open at https://2019.postgresopen.org/callforpapers/
https://2019.postgresopen.org/

PostgresConf South Africa 2019 will take place in Johannesburg on October 8-9, 2019
https://postgresconf.org/conferences/SouthAfrica2019

pgDay Paris 2020 will be held in Paris, France on March 26, 2020
at Espace Saint-Martin.
http://2020.pgday.paris/

== PostgreSQL in the News ==

Planet PostgreSQL: http://planet.postgresql.org/

PostgreSQL Weekly News is brought to you this week by David Fetter

Submit news and announcements by Sunday at 3:00pm PST8PDT to david(at)fetter(dot)org(dot)

== Applied Patches ==

Andrew Gierth pushed:

- Repair logic for reordering grouping sets optimization. The logic in
reorder_grouping_sets to order grouping set elements to match a pre-specified
sort ordering was defective, resulting in unnecessary sort nodes (though the
query output would still be correct). Repair, simplifying the code a little,
and add a test. Per report from Richard Guo, though I didn't use their patch.
Original bug seems to have been my fault. Backpatch back to 9.5 where
grouping sets were introduced. Discussion:
https://postgr.es/m/CAN_9JTzyjGcUjiBHxLsgqfk7PkdLGXiM=pwM+=ph2LsWw0WO1A@mail.gmail.com
https://git.postgresql.org/pg/commitdiff/da53be23d1c5e529675e958eaee54cfed4e4dbac

Noah Misch pushed:

- Don't read fields of a misaligned ExpandedObjectHeader or AnyArrayType. UBSan
complains about this. Instead, cast to a suitable type requiring only 4-byte
alignment. DatumGetAnyArrayP() already assumes one can cast between
AnyArrayType and ArrayType, so this doesn't introduce a new assumption.
Back-patch to 9.5, where AnyArrayType was introduced. Reviewed by Tom Lane.
Discussion: https://postgr.es/m/20190629210334.GA1244217@rfd.leadboat.com
https://git.postgresql.org/pg/commitdiff/459c3cdb4ad8323bfe6bcae0186a3831e0c89c94

Michaël Paquier pushed:

- Fix many typos and inconsistencies. Author: Alexander Lakhin Discussion:
https://postgr.es/m/af27d1b3-a128-9d62-46e0-88f424397f44@gmail.com
https://git.postgresql.org/pg/commitdiff/c74d49d41c9e0fabeb1179dcabfb3137833ed831

- Refactor code of reindexdb for query generation. This merges the portion
related to REINDEX SYSTEM into the routine already available for all the other
reindex types, making the query generation cleaner. While on it, change the
handling of the reindex types using an enum, which allows to get rid of the
hardcoded strings used directly in the query generation present for the same
purpose (aka "TABLE", "DATABASE", etc.). Per discussion with Julien Rouhaud,
Tom Lane, Alvaro Herrera and me. Author: Julien Rouhaud Discussion:
https://postgr.es/m/CAOBaU_bSmSik_WRK9niDnm-3NkNZky6+uKxkmQwvthZvMWpS5A@mail.gmail.com
https://git.postgresql.org/pg/commitdiff/9adda24543e354317abf5400d7e7d3961a93bce6

- Add support for Visual Studio 2019 in build scripts. This fixes at the same
time a set of inconsistencies in the documentation and the scripts related to
the versions of Windows SDK supported. Author: Haribabu Kommi Reviewed-by:
Andrew Dunstan, Juan José Santamaría Flecha, Michael Paquier Discussion:
https://postgr.es/m/CAJrrPGcfqXhfPyMrny9apoDU7M1t59dzVAvoJ9AeAh5BJi+UzA@mail.gmail.com
https://git.postgresql.org/pg/commitdiff/2b1394fc2b52a2573d08aa626e7b49568f27464e

- Simplify TAP tests of pg_dump for connection strings. The last set of
scenarios did an initialization of nodes followed by an extra command to set
up the authentication policy with pg_regress --config-auth. This
configuration step can be integrated directly using the option auth_extra from
PostgresNode::init when initializing the node, saving from one extra command.
On Windows, this also restricts more pg_ident.conf for the SSPI user mapping
by removing the entry of the OS user running the test, which is not needed
anyway. Note that IPC::Run mishandles double quotes, hence the restore user
name is changed to map with that. This was already done in the test as a
later step, but not in a consistent way, causing the switch to use auth_extra
to fail. Found while reviewing ca129e5. Discussion:
https://postgr.es/m/20190703062024.GD3084@paquier.xyz
https://git.postgresql.org/pg/commitdiff/d5ab9a891cb590aad4278026b2edda685f2524a2

- Introduce safer encoding and decoding routines for base64.c. This is a
follow-up refactoring after 09ec55b and b674211, which has proved that the
encoding and decoding routines used by SCRAM have a poor interface when it
comes to check after buffer overflows. This adds an extra argument in the
shape of the length of the result buffer for each routine, which is used for
overflow checks when encoding or decoding an input string. The original idea
comes from Tom Lane. As a result of that, the encoding routine can now fail,
so all its callers are adjusted to generate proper error messages in case of
problems. On failure, the result buffer gets zeroed. Author: Michael Paquier
Reviewed-by: Daniel Gustafsson Discussion:
https://postgr.es/m/20190623132535.GB1628@paquier.xyz
https://git.postgresql.org/pg/commitdiff/cfc40d384ae51ea2886d599d2008ae57b529e6ea

- Update hardcoded DH parameters to IANA standards. The source defining the
current fallback and hardcoded DH parameters has disappeared from the web a
long time ago, and RFC 3526 defines the most current Diffie-Hellman MODP
groups, so update to those new values. Author: Daniel Gustafsson Reviewed-by:
Peter Eisentraut, Michael Paquier Discussion:
https://postgr.es/m/5E60AC9A-CB10-4851-9EF2-7209490A164C@yesql.se
https://git.postgresql.org/pg/commitdiff/8a810a177c80909b71e9fb3760a1d56ed988638a

- Add min() and max() aggregates for pg_lsn. This is useful for monitoring, when
it comes for example to calculations of WAL retention with replication slots
and delays with a set of standbys. Bump catalog version. Author: Fabrízio de
Royes Mello Reviewed-by: Surafel Temesgen Discussion:
https://postgr.es/m/CAFcNs+oc8ZoHhowA4rR1GGCgG8QNgK_TOwPRVYQo5rYy8_PXzA@mail.gmail.com
https://git.postgresql.org/pg/commitdiff/313f87a17155a6dbd27a3ce687cf59bd171fe75e

- Remove more unreferenced function declarations. Author: Masahiko Sawada
Discussion:
https://postgr.es/m/CAD21AoDuAYsRb3Q9aobkFZ6DZMWxsyg4HOmgkwgeWNfSkTwGxw@mail.gmail.com
https://git.postgresql.org/pg/commitdiff/d1a040543b49e0aad273e7766cd7e2fcf2b781fa

Peter Eisentraut pushed:

- Add missing serial commas.
https://git.postgresql.org/pg/commitdiff/1b29e990e352885642149262859b83c19d362405

- Convert some stragglers to new frontend logging API.
https://git.postgresql.org/pg/commitdiff/95bbe5d82e428db342fa3ec60b95f1b9873741e5

- Remove support for non-ELF BSD systems. This is long obsolete. Discussion:
https://www.postgresql.org/message-id/8eacdc0d-123f-dbca-bacf-0a68766a4889@2ndquadrant.com
https://git.postgresql.org/pg/commitdiff/c72f9b9502eadb6b84c6681cdb3bff12b35d3c8a

- Remove redundant newlines from error messages. These are no longer
needed/allowed with the new logging API.
https://git.postgresql.org/pg/commitdiff/24c7000f64487323fedb52b8aeadf2c84274dcf5

- Fix accidentally swapped error message arguments. Author: Alexey Kondratov
<a(dot)kondratov(at)postgrespro(dot)ru>
https://git.postgresql.org/pg/commitdiff/84c41ae81bdf15cac71cc5ae0af69b4815594522

- Sync our Snowball stemmer dictionaries with current upstream. The main change
is a new stemmer for Greek. There are minor changes in the Danish and French
stemmers. Author: Panagiotis Mavrogiorgos <pmav99(at)gmail(dot)com>
https://git.postgresql.org/pg/commitdiff/7b925e12703652fef63a2fbbb28d3407b2971d6e

- Clean up whitespace a bit.
https://git.postgresql.org/pg/commitdiff/dedb6e0143554e76d4d11376d65c0aa68f8412d4

- Unwind some workarounds for lack of portable int64 format specifier. Because
there is no portable int64/uint64 format specifier and we can't stick macros
like INT64_FORMAT into the middle of a translatable string, we have been using
various workarounds that put the number to be printed into a string buffer
first. Now that we always use our own sprintf(), we can rely on %lld and %llu
to work, so we can use those. This patch undoes this workaround in a few
places where it was egregiously verbose. Reviewed-by: Tom Lane
<tgl(at)sss(dot)pgh(dot)pa(dot)us> Discussion:
https://www.postgresql.org/message-id/flat/CAH2-Wz%3DWbNxc5ob5NJ9yqo2RMJ0q4HXDS30GVCobeCvC9A1L9A%40mail.gmail.com
https://git.postgresql.org/pg/commitdiff/6a1cd8b9236dcfa91b40af3a8337859e16ba7113

- Remove unnecessary casts from size_t to int. We can use the %zu format
specifier directly, no need to cast to int.
https://git.postgresql.org/pg/commitdiff/d5ab9df7774b4570ff50e64b7fa3ba8295596d06

- doc: Spell checking.
https://git.postgresql.org/pg/commitdiff/594df378ffb04a72b713a13cc0a7166b3bced7b7

- psql: Fix logging output format. In normal interactive mode, psql's log
messages accidentally got a "psql:" prefix that was not supposed to be there.
This only happened if there was no .psqlrc file being read, so it wasn't
discovered for a while. Fix this by adding the appropriate logging format
configuration call in the right code path. Discussion:
https://www.postgresql.org/message-id/7586.1560540361@sss.pgh.pa.us
https://git.postgresql.org/pg/commitdiff/3f3542621f131379e32e9283d40853cb6d03a97f

- Add missing source files to nls.mk.
https://git.postgresql.org/pg/commitdiff/b33283c36409aef7eddb5ba92bdd9300dd45d974

- Use consistent style for checking return from system calls. Use if
(something() != 0) error ... instead of just if (something)
error ... The latter is not incorrect, but it's a bit confusing and not the
common style. Discussion:
https://www.postgresql.org/message-id/flat/5de61b6b-8be9-7771-0048-860328efe027%402ndquadrant.com
https://git.postgresql.org/pg/commitdiff/7e9a4c5c3dca0d9637812d8991e96fc8f46800d9

David Rowley pushed:

- Fix missing call to table_finish_bulk_insert during COPY. 86b85044e abstracted
calls to heap functions in COPY FROM to support a generic table AM. However,
when performing a copy into a partitioned table, this commit neglected to call
table_finish_bulk_insert for each partition. Before 86b85044e, when we always
called the heap functions, there was no need to call heapam_finish_bulk_insert
for partitions since it only did any work when performing a copy without WAL.
For partitioned tables, this was unsupported anyway, so there was no issue.
With pluggable storage, we can't make any assumptions about what the table AM
might want to do in its equivalent function, so we'd better ensure we always
call table_finish_bulk_insert each partition that's received a row. For now,
we make the table_finish_bulk_insert call whenever we evict a
CopyMultiInsertBuffer out of the CopyMultiInsertInfo. This does mean that
it's possible that we call table_finish_bulk_insert multiple times per
partition, which is not a problem other than being an inefficiency. Improving
this requires a more invasive patch, so let's leave that for another day. In
passing, move the table_finish_bulk_insert for the target of the COPY command
so that it's only called when we're actually performing bulk inserts. We
don't need to call this when inserting 1 row at a time. Reported-by: Robert
Haas Discussion:
https://postgr.es/m/CA+TgmoYK=6BpxiJ0tN-p9wtH0BTAfbdxzHhwou0mdud4+BkYuQ@mail.gmail.com
https://git.postgresql.org/pg/commitdiff/4de60244e2fcdeb100afc7f8bf5c28b32923a637

- Remove surplus call to table_finish_bulk_insert. 4de60244e added the call to
table_finish_bulk_insert to the CopyMultiInsertBufferCleanup function. We use
a CopyMultiInsertBuffer even for non-partitioned tables, so having the cleanup
do that meant we would call table_finsh_bulk_insert twice when performing COPY
FROM with a non-partitioned table. Here we can just remove the direct call in
CopyFrom and let CopyMultiInsertBufferCleanup handle the call instead.
https://git.postgresql.org/pg/commitdiff/b2d69806d8cc902802a7b56ba37fb7c55e58b0ed

- Revert fix missing call to table_finish_bulk_insert during COPY. This reverts
commits 4de60244e and b2d69806d. Further thought is required to make this work
properly.
https://git.postgresql.org/pg/commitdiff/f5db56fc4d6e95c582b61c99328ea0702b869fa0

- Don't remove surplus columns from GROUP BY for inheritance parents. d4c3a156c
added code to remove columns that were not part of a table's PRIMARY KEY
constraint from the GROUP BY clause when all the primary key columns were
present in the group by. This is fine to do since we know that there will
only be one row per group coming from this relation. However, the logic failed
to consider inheritance parent relations. These can have child relations
without a primary key, but even if they did, they could duplicate one of the
parent's rows or one from another child relation. In this case, those
additional GROUP BY columns are required. Fix this by disabling the
optimization for inheritance parent tables. In v11 and beyond, partitioned
tables are fine since partitions cannot overlap and before v11 partitioned
tables could not have a primary key. Reported-by: Manuel Rigger Discussion:
http://postgr.es/m/CA+u7OA7VLKf_vEr6kLF3MnWSA9LToJYncgpNX2tQ-oWzYCBQAw@mail.gmail.com
Backpatch-through: 9.6
https://git.postgresql.org/pg/commitdiff/a5be4062f7bf2ae9487c5a31ee337a56425cdc84

- Use appendStringInfoString and appendPQExpBufferStr where possible. This
changes various places where appendPQExpBuffer was used in places where it was
possible to use appendPQExpBufferStr, and likewise for appendStringInfo and
appendStringInfoString. This is really just a stylistic improvement, but
there are also small performance gains to be had from doing this. Discussion:
http://postgr.es/m/CAKJS1f9P=M-3ULmPvr8iCno8yvfDViHibJjpriHU8+SXUgeZ=w@mail.gmail.com
https://git.postgresql.org/pg/commitdiff/8abc13a88938ef473b8a486186f1b96630450728

Tom Lane pushed:

- Doc: remove obsolete description of oid column as being "hidden". Looks like
one copy of this text didn't get updated. Justin Pryzby Discussion:
https://postgr.es/m/20190701155932.GA22866@telsasoft.com
https://git.postgresql.org/pg/commitdiff/145b18688c04963efcab8ef3e7e10659bf783eb1

- pgindent run prior to branching v12. pgperltidy and reformat-dat-files too,
though the latter didn't find anything to change.
https://git.postgresql.org/pg/commitdiff/9e1c9f959422192bbe1b842a2a1ffaf76b080196

- Stamp HEAD as 13devel. Let the hacking begin ...
https://git.postgresql.org/pg/commitdiff/615cebc94b5ef8fbe353e3c8b838b1e97bcdfd49

- Simplify psql \d's rule for ordering the indexes of a table. The previous rule
was "primary key (if any) first, then other unique indexes in name order, then
all other indexes in name order". But the preference for unique indexes seems
a bit obsolete since the introduction of exclusion constraints. It's no
longer the case that unique indexes are the only ones that constrain what data
can be in the table, and it's hard to see what other rationale there is for
separating out unique indexes. Other new features like the possibility for
some indexes to be INVALID (hence, not constraining anything) make this even
shakier. Hence, simplify the sort order to be "primary key (if any) first,
then all other indexes in name order". No documentation change, since this
was never documented anyway. A couple of existing regression test cases change
output, though. Discussion:
https://postgr.es/m/14422.1561474929@sss.pgh.pa.us
https://git.postgresql.org/pg/commitdiff/4d6603f28dfc4a1cab0d7d317855d935e314297a

- Fix tab completion of "SET variable TO|=" to not offer bogus completions.
Don't think that the context "UPDATE tab SET var =" is a GUC-setting command.
If we have "SET var =" but the "var" is not a known GUC variable, don't offer
any completions. The most likely explanation is that we've misparsed the
context and it's not really a GUC-setting command. Per gripe from Ken Tanzer.
Back-patch to 9.6. The issue exists further back, but before 9.6 the code
looks very different and it doesn't actually know whether the "var" name
matches anything, so I desisted from trying to fix it. Discussion:
https://postgr.es/m/CAD3a31XpXzrZA9TT3BqLSHghdTK+=cXjNCE+oL2Zn4+oWoc=qA@mail.gmail.com
https://git.postgresql.org/pg/commitdiff/0ec3e13c69779117c8cfa39adcc6863631dedd44

- Don't treat complete_from_const as equivalent to complete_from_list. Commit
4f3b38fe2 supposed that complete_from_const() is equivalent to the
one-element-list case of complete_from_list(), but that's not really true at
all. complete_from_const() supposes that the completion is certain enough to
justify wiping out whatever the user typed, while complete_from_list() will
only provide completions that match the word-so-far. In practice, given the
lame parsing technology used by tab-complete.c, it's fairly hard to believe
that we're *ever* certain enough about a completion to justify auto-correcting
user input that doesn't match. Hence, remove the inappropriate unification of
the two cases. As things now stand, complete_from_const() is used only for the
situation where we have no matches and we need to keep readline from applying
its default complete-with-file-names behavior. This (mis?) behavior actually
exists much further back, but I'm hesitant to change it in released branches.
It's not too late for v12, though, especially seeing that the aforesaid commit
is new in v12. Per gripe from Ken Tanzer. Discussion:
https://postgr.es/m/CAD3a31XpXzrZA9TT3BqLSHghdTK+=cXjNCE+oL2Zn4+oWoc=qA@mail.gmail.com
https://git.postgresql.org/pg/commitdiff/b4771d7c7f37d19e2879b18e288f681849d55806

- Show table persistence in psql's \dt+ and related commands. In verbose mode,
listTables() now emits a "Persistence" column showing whether the
table/index/view/etc is permanent, temporary, or unlogged. David Fetter,
reviewed by Fabien Coelho and Rafia Sabih Discussion:
https://postgr.es/m/20190423005642.GZ28936@fetter.org
https://git.postgresql.org/pg/commitdiff/9a2ea618323a4cf8ca7eb6a828b08c6e39b95cdd

- Doc: document table persistence display in \dt+. Forgotten in commit
9a2ea6183.
https://git.postgresql.org/pg/commitdiff/03e7b302b1d5a67758e756b1f64686c29d37558f

- Ensure plpgsql result tuples have the right composite type marking. A function
that is declared to return a named composite type must return tuple datums
that are physically marked as having that type. The plpgsql code path that
allowed directly returning an expanded-record datum forgot to check that, so
that an expanded record marked as type RECORDOID could be returned if it had a
physically-compatible tupdesc. This'd be harmless, I think, if the record
value never escaped the current session --- but it's possible for it to get
stored into a table, and then subsequent sessions can't interpret the
anonymous record type. Fix by flattening the record into a tuple datum and
overwriting its type/typmod fields, if its declared type doesn't match the
function's declared type. (In principle it might be possible to just change
the expanded record's stored type ID info, but there are enough tricky
consequences that I didn't want to mess with that, especially not in a
back-patched bug fix.) Per bug report from Steve Rogerson. Back-patch to v11
where the bug was introduced. Discussion:
https://postgr.es/m/cbaecae6-7b87-584e-45f6-4d047b92ca2a@yewtc.demon.co.uk
https://git.postgresql.org/pg/commitdiff/5683b34956b4e8da9dccadc2e3a53b86104ebb33

- Add \warn command to psql. This is like \echo except that the text is sent to
stderr not stdout. In passing, fix a pre-existing bug in \echo and \qecho:
per documentation the -n switch should only be recognized when it is the first
argument, but actually any argument matching "-n" was treated as a switch.
(Should we back-patch that?) David Fetter (bug fix by me), reviewed by Fabien
Coelho Discussion: https://postgr.es/m/20190421183115.GA4311@fetter.org
https://git.postgresql.org/pg/commitdiff/02e95a5049f7933cbde1dacf401604ea3fc02aa5

- Remove dead encoding-conversion functions. The code for conversions SQL_ASCII
<-> MULE_INTERNAL and SQL_ASCII <-> UTF8 was unreachable, because we long ago
changed the wrapper functions pg_do_encoding_conversion() et al so that they
have hard-wired behaviors for conversions involving SQL_ASCII. (At least some
of those fast paths date back to 2002, though it looks like we may not have
been totally consistent about this until later.) Given the lack of
complaints, nobody is dissatisfied with this state of affairs. Hence, let's
just remove the unreachable code. Also, change CREATE CONVERSION so that it
rejects attempts to define such conversions. Since we consider that SQL_ASCII
represents lack of knowledge about the encoding in use, such a conversion
would be semantically dubious even if it were reachable. Adjust a couple of
regression test cases that had randomly decided to rely on these conversion
functions rather than any other ones. Discussion:
https://postgr.es/m/41163.1559156593@sss.pgh.pa.us
https://git.postgresql.org/pg/commitdiff/0ab1a2e39b6f65b0f6a5879605ddbf12f9f50de4

- Remove unreferenced function declarations. These seem to be leftovers from old
patches, perhaps. Masahiko Sawada Discussion:
https://postgr.es/m/CAD21AoDuAYsRb3Q9aobkFZ6DZMWxsyg4HOmgkwgeWNfSkTwGxw@mail.gmail.com
https://git.postgresql.org/pg/commitdiff/79b94716e72086b07549b1c867a8ecdea6bae77e

- Add some test cases to improve test coverage of parse_expr.c. I chanced to
notice while thumbing through lcov reports that we had exactly no coverage of
BETWEEN SYMMETRIC, nor of current_time(N) and localtime(N). Improve that.
parse_expr.c still has a pretty awful coverage number, but a large part of
that is due to lack of coverage of the operator_precedence_warning logic. I
have zero desire to write tests for that; I think ripping it out would be more
sensible at this point.
https://git.postgresql.org/pg/commitdiff/cf20cc00a99155a8e41a1bb2a1e498624c86db29

- In pg_log_generic(), be more paranoid about preserving errno. This code failed
to account for the possibility that malloc() would change errno, resulting in
wrong output for %m, not to mention the possibility of message truncation.
Such a change is obviously expected when malloc fails, but there's reason to
fear that on some platforms even a successful malloc call can modify errno.
Discussion: https://postgr.es/m/2576.1527382833@sss.pgh.pa.us
https://git.postgresql.org/pg/commitdiff/fb30c9c1c5c36989d6b93906986358cb96936d64

Peter Geoghegan pushed:

- Remove obsolete nbtree "get root" comment. Remove a very old Berkeley era
comment that doesn't seem to have anything to do with the current locking
considerations within _bt_getroot(). Discussion:
https://postgr.es/m/CAH2-WzmA2H+rL-xxF5o6QhMD+9x6cJTnz2Mr3Li_pbPBmqoTBQ@mail.gmail.com
https://git.postgresql.org/pg/commitdiff/66c5bd3a6fd8a4c317412838ab3870ab251833b6

Michael Meskes pushed:

- Fix small memory leak in ecpglib ecpg_update_declare_statement() is called
the.second time. Author: "Zhang, Jie" <zhangjie2(at)cn(dot)fujitsu(dot)com>
https://git.postgresql.org/pg/commitdiff/e72489e101b21c328e3d10ca64e5367c60f424a5

- Made ecpg compatibility mode and run-time behaviour options case insensitive.
https://git.postgresql.org/pg/commitdiff/75220fb62b1387b61f92c42b1bd147cb30607012

- Use strtoint() instead of strtol() in pgtypeslib where the result is stored
in.an int variable. Author: Yang Xiao <YangX92(at)hotmail(dot)com>
https://git.postgresql.org/pg/commitdiff/8372e3c98fbbd529e4545c4d7982e278e118958e

Etsuro Fujita pushed:

- postgres_fdw: Remove redundancy in postgresAcquireSampleRowsFunc().
Previously, in the loop in postgresAcquireSampleRowsFunc() to iterate fetching
rows from a given remote table, we redundantly 1) determined the fetch size by
parsing the table's server/table-level options and then 2) constructed the
fetch command; remove that redundancy. Author: Etsuro Fujita Reviewed-by:
Julien Rouhaud Discussion:
https://postgr.es/m/CAPmGK17_urk9qkLV65_iYMFw64z5qhdfhY=tMVV6Jg4KNYx8+w@mail.gmail.com
https://git.postgresql.org/pg/commitdiff/2a1612104cadbfdc739ff0370f279779f323c3b5

Tomáš Vondra pushed:

- Speed-up build of MCV lists with many distinct values. When building
multi-column MCV lists, we compute base frequency for each item, i.e. a
product of per-column frequencies for values from the item. As a value may be
in multiple groups, the code was scanning the whole array of groups while
adding items to the MCV list. This works fine as long as the number of
distinct groups is small, but it's easy to trigger trigger O(N^2) behavior,
especially after increasing statistics target. This commit precomputes
frequencies for values in all columns, so that when computing the base
frequency it's enough to make a simple bsearch lookup in the array. Backpatch
to 12, where multi-column MCV lists were introduced. Discussion:
https://postgr.es/m/20190618205920.qtlzcu73whfpfqne@development
https://git.postgresql.org/pg/commitdiff/e365a581c246a8e18f38cc530013391329dcdb02

- Fix pg_mcv_list_items() to produce text[]. The function pg_mcv_list_items()
returns values stored in MCV items. The items may contain columns with
different data types, so the function was generating text array-like
representation, but in an ad-hoc way without properly escaping various
characters etc. Fixed by simply building a text[] array, which also makes it
easier to use from queries etc. Requires changes to pg_proc entry, so bump
catversion. Backpatch to 12, where multi-column MCV lists were introduced.
Author: Tomas Vondra Reviewed-by: Dean Rasheed Discussion:
https://postgr.es/m/20190618205920.qtlzcu73whfpfqne@development
https://git.postgresql.org/pg/commitdiff/4d66285adc6bb4f9e4fd394d478d663cbccb5fc8

- Simplify pg_mcv_list (de)serialization. The serialization format of
multivariate MCV lists included alignment in order to allow direct access to
part of the serialized data, but despite multiple fixes (see for example
commits d85e0f366a and ea4e1c0e8f) this proved to be problematic. This commit
abandons alignment in the serialized format, and just copies everything during
deserialization. We now also track amount of memory needed after
deserialization (including alignment), which allows us to deserialize the MCV
list in a single pass. Bump catversion, as this affects contents of
pg_statistic_ext_data. Backpatch to 12, where multi-column MCV lists were
introduced. Author: Tomas Vondra Reviewed-by: Tom Lane Discussion:
https://postgr.es/m/2201.1561521148@sss.pgh.pa.us
https://git.postgresql.org/pg/commitdiff/08aa131c7a72195113ab3a7b191fe8014dd3a898

- Remove unused variable in statext_mcv_serialize(). The itemlen variable used
to be referenced in multiple places, but since reworking the serialization
code it's used only in one assert. Fixed by removing the variable and calling
the macro from the assert directly. Backpatch to 12, where this code was
introduced. Reported-by: Jeff Janes Discussion:
https://postgr.es/m/CAMkU=1zc_ovH9NZd_9ovuiEWkF9yX06URUDdXCmgDydf-bqB5A@mail.gmail.com
https://git.postgresql.org/pg/commitdiff/ef777cb093e8cb45dd3ae9d3f1499c765147c1dd

Thomas Munro pushed:

- Improve comment in postgresql.conf.sample. The Unix manual section that "man
tcp" appears in varies, so let's just leave it out of the command to run.
https://git.postgresql.org/pg/commitdiff/e8fdcacc6cbeed7d1a2175c5eddf0b162e0cb7c4

Amit Kapila pushed:

- Add missing assertions for required table am callbacks. Reported-by: Ashwin
Agrawal Author: Ashwin Agrawal Reviewed-by: Amit Kapila Backpatch-through: 12,
where it was introduced Discussion:
https://postgr.es/m/CALfoeisgdZhYDrJOukaBzvXfJOK2FQ0szVMK7dzmcy6w93iDUA@mail.gmail.com
https://git.postgresql.org/pg/commitdiff/78d41f6c9b0e1c4bd28f9b80cd07c7530660312f

== Pending Patches ==

Tomáš Vondra sent in two more revisions of a patch to implement incremental
sort. James Coleman sent in a patch to parallelize same.

Hao Wu sent in a patch to add certificates, keys and test cases for
contrib/sslinfo.

Pavel Stěhule sent in another revision of a patch to implement dropdb --force.

Etsuro Fujita sent in another revision of a patch to improve the partition
matching algorithms.

Ildus Kurbangaliev sent in another revision of a patch to add custom compression
methods.

Tomáš Vondra sent in another revision of a patch to implement ALTER STATISTICS.

Daniel Gustafsson sent in a patch to add an Assertion for logically decoding
multi inserts into the catalog.

Robert Haas sent in another revision of a patch to refactor TOAST so table AMs
other than heap can use it.

Surafel Temesgen sent in another revision of a patch to implement FETCH FIRST
... PERCENT.

Dmitry Belyavsky sent in three more revisions of a patch to improve the ltree
syntax.

Julien Rouhaud sent in three more revisions of a patch to export vacuumdb's
parallel infrastructure and use same to add parallel processing to reindexdb.

Pavel Stěhule sent in another revision of a patch to make it possible to sort by
size in psql.

Konstantin Knizhnik sent in another revision of a patch to implement a built-in
connection pooler.

Dhruv Goel sent in another revision of a patch to avoid deadlock errors in
CREATE INDEX CONCURRENTLY.

Kirk Jamison sent in another revision of a patch to speed up truncates of
relation forks.

Masahiko Sawada sent in another revision of a patch to fix a bug that manifested
as duplicated LSN in ReorderBuffer.

Antonin Houska sent in another revision of a patch to factor out reading of XLOG
pages into a single code path.

Daniel Gustafsson sent in another revision of a patch to use heap_multi_insert()
for pg_attribute/depend insertions.

Justin Pryzby sent in another revision of a patch to fix some infelicities in
the docs.

David Cramer sent in a patch to make lc_monetary, lc_numeric and search_path
GUC_REPORT so that we get these in the startup messages and whenever someone
changes them

Patrick McHardy and Álvaro Herrera traded patches to fix trigger argument
propagation to child partitions.

Konstantin Knizhnik sent in another revision of a patch to implement
autoprepare.

Takamichi Osumi sent in another revision of a patch to implement CREATE OR
REPLACE TRIGGER.

Michaël Paquier sent in another revision of a patch to fix hash joins.

Tatsuro Yamada sent in another revision of a patch to implement a progress
report for ANALYZE.

Dmitry Dolgov and David Fetter traded patches to implement generic type
subscripting.

Jesper Pedersen sent in a patch to add a UniqueKey planner node.

Tom Lane and John Naylor traded patches to reduce the number of states in the
core scanner table and merge in the ECPG scanner states for C-style comments.

Antonin Houska sent in two more revisions of a patch to implement aggregate
pushdown.

Mike Palmiotto and Kyotaro HORIGUCHI traded patches to add a hook for partition
pruning.

Dilip Kumar sent in another revision of a patch to clean up orphan files using
the UNDO log.

Fabien COELHO sent in another revision of a patch to pgbench which adds \aset to
store results of a combined query.

Kyotaro HORIGUCHI sent in a patch to fix an issue where "make clean" removes too
much.

Fabien COELHO sent in a patch to make the libpq documentation navigable between
functions.

Magnus Hagander sent in another revision of a patch to add barrier wait events.

Binguo Bao sent in another revision of a patch to optimize partial TOAST
decompression.

Sehrope Sarkuni sent in a patch to refactor the syslogger piping to simplify
adding new log destinations.

Takuma Hoshiai sent in another revision of a patch to implement incremental
matview maintenance.

Jesper Pedersen sent in another revision of a patch to highlight the fact that
pg_receivewal doesn't apply WAL, and as such synchronous-commit needs to be
remote_write or lower.

Binguo Bao sent in three more revisions of a patch to de-TOAST using an
iterator.

Anastasia Lubennikova and Jeevan Chalke traded patches to implement incremental
backup.

Kyotaro HORIGUCHI sent in another revision of a patch to remove the page-read
callback from XLogReaderState.

Kyotaro HORIGUCHI sent in two more revisions of a patch to fix the WAL-skipping
feature.

Amit Khandekar sent in another revision of a patch to implement minimal logical
decoding on standbys.

Peter Geoghegan sent in a patch to fix a pathological page split issue for
nbtrees.

Kyotaro HORIGUCHI sent in another revision of a patch to implement a
shared-memory-based stats collector.

David Cramer sent in a patch to add a special startup parameter _pq_.guc_report
to add GUC_REPORT dynamically per session.

Julien Rouhaud sent in a patch to make it possible to filter the indexes being
reindexed by REINDEX.

Surafel Temesgen sent in another revision of a patch to implement conflict
handling in COPY FROM.

Thomas Munro sent in another revision of a patch to add SmgrId to smgropen() and
BufferTag and move tablespace directory creation from smgr.c to md.c.

Ashwin Agrawal sent in a patch to remove IndexBuildCallback's dependency on
HeapTuple.

Thomas Munro sent in a patch to add a XLogRecGetFullXid() function, which can
work during replay.

Brad DeJong sent in a patch to fix some more typos in tableam.h along with a few
grammar changes.

Jeff Davis sent in another revision of a patch to implement memory-bounded hash
aggregation.

Luis Carril sent in a patch to make it possible to alter foreign tables in
pg_dump.

Luis Carril sent in another revision of a patch to support dumping the data from
foreign tables in pg_dump.

Nikolay Shaplov sent in another revision of a patch to replace StdRdOptions with
individual binary reloptions representation for each relation kind.

Nikita Glukhov sent in another revision of a patch to implement kNN for B-Tree.

Kyotaro HORIGUCHI sent in another revision of a patch to fix comments in
config_default.pl.

Kyotaro HORIGUCHI sent in a patch to allow only IMMUTABLE functions in CHECK
constraints.

Fabien COELHO sent in another revision of a patch to refactor the str2*int64 so
they're all in one place.

Julien Rouhaud sent in another revision of a patch to update the recommendations
for initdb.

Tomáš Vondra sent in another revision of a patch to fix a crash in
mcv_get_match_bitmap.

Alexander Lakhin sent in another revision of a patch to fix typos and
inconsistencies in HEAD.

Fabien COELHO sent in another revision of a patch to implement a strict
TPC-B-like benchmark in pgbench.

Karl O. Pinc sent in another revision of a patch to document base64 decoding.