== PostgreSQL Weekly News - October 21, 2018 ==

== PostgreSQL Weekly News - October 21, 2018 ==

PostgreSQL 11 released!
https://www.postgresql.org/about/news/1894/

== PostgreSQL Product News ==

sqlite_fdw 1.0.0 released.
https://github.com/pgspider/sqlite_fdw

== PostgreSQL Jobs for October ==

http://archives.postgresql.org/pgsql-jobs/2018-10/

== PostgreSQL Local ==

PostgreSQL Conference Europe 2018 will be held on October 23-26, 2018 at the
Lisbon Marriott Hotel in Lisbon, Portugal.
https://2018.pgconf.eu/

pgday.Seoul 2018 will be held in Seoul, South Korea on November 3, 2018 in
Seoul. Korean language information is here:
http://pgday.postgresql.kr/

2Q PGConf will be on December 4-5, 2018 in Chicago, IL.
http://www.2qpgconf.com/

PGConf.ASIA 2018 will take place on December 10-12, 2018 in Akihabara, Tokyo,
Japan.
http://www.pgconf.asia/EN/2018/

pgDay Paris 2019 will be held in Paris, France on March 12, 2019
at 199bis rue Saint-Martin. The CfP is open until November 30, 2018.
http://2019.pgday.paris/callforpapers/

PGConf APAC 2019 will be held in Singapore March 19-21, 2019. The CfP is open
at http://2019.pgconfapac.org/cfp through November 16, 2018.
http://2019.pgconfapac.org/

PGDay.IT 2019 will take place May 16th and May 17th in Bologna, Italy. The CfP
is open at https://2019.pgday.it/en/blog/cfp and the Call for Workshops is at
https://2019.pgday.it/en/blog/cfw until January 15, 2019.
https://2019.pgday.it/en/

== PostgreSQL in the News ==

Planet PostgreSQL: http://planet.postgresql.org/

PostgreSQL Weekly News is brought to you this week by David Fetter

Submit news and announcements by Sunday at 3:00pm PST8PDT to david(at)fetter(dot)org(dot)

== Applied Patches ==

Peter Eisentraut pushed:

- Fixes for "Glyph not available" warnings from FOP. With the PostgreSQL 11
release notes acknowledgments list, FOP reported WARNING: Glyph "?" (0x144,
nacute) not available in font "Times-Roman". WARNING: Glyph "?" (0x15e,
Scedilla) not available in font "Times-Roman". WARNING: Glyph "?" (0x15f,
scedilla) not available in font "Times-Roman". WARNING: Glyph "?" (0x131,
dotlessi) not available in font "Times-Roman". This is because we have some
new contributors whose names use letters that we haven't used before, and
apparently FOP can't handle them out of the box. For now, just fix this by
"unaccenting" those names. In the future, maybe this can be fixed better with
a different font configuration. There is also another warning WARNING: Glyph
"?" (0x3c0, pi) not available in font "Times-Roman". but that existed in
previous releases and is not touched here.
https://git.postgresql.org/pg/commitdiff/9274c577f7351c5af8e4c85f4a6031db11c93992

- Make spelling of "acknowledgment" consistent. I used the preferred U.S.
spelling, as we do in other cases.
https://git.postgresql.org/pg/commitdiff/35584fd05f9b104f057c700360985991f111a847

- pgbench: Report errors during run better. When an error occurs during a
benchmark run, exit with a nonzero exit code and write a message at the end.
Previously, it would just print the error message when it happened but then
proceed to print the run summary normally and exit with status 0. To still
allow distinguishing setup from run-time errors, we use exit status 2 for the
new state, whereas existing errors during pgbench initialization use exit
status 1. Reviewed-by: Fabien COELHO <coelho(at)cri(dot)ensmp(dot)fr>
https://git.postgresql.org/pg/commitdiff/5b75a4f8266a4f187a47929fed0582ee778c703f

- Fix crash in multi-insert COPY. A bug introduced in
0d5f05cde011512e605bb2688d9b1fbb5b3ae152 considered the *previous* partition's
triggers when deciding whether multi-insert can be used. Rearrange the code
so that the current partition is considered. Author: Ashutosh Sharma
<ashu(dot)coek88(at)gmail(dot)com>
https://git.postgresql.org/pg/commitdiff/a7a1b44516e7db89104c06440f759c2831fcb0ff

Tom Lane pushed:

- Check for stack overrun in standard_ProcessUtility(). ProcessUtility can
recurse, and indeed can be driven to infinite recursion, so it ought to have a
check_stack_depth() call. This covers the reported bug (portal trying to
execute itself) and a bunch of other cases that could perhaps arise somewhere.
Per bug #15428 from Malthe Borch. Back-patch to all supported branches.
Discussion: https://postgr.es/m/15428-b3c2915ec470b033@postgresql.org
https://git.postgresql.org/pg/commitdiff/d48da369ab22a8326d4d3d2b05b574d581057193

- Avoid statically allocating gmtsub()'s timezone workspace. localtime.c's
"struct state" is a rather large object, ~23KB. We were statically allocating
one for gmtsub() to use to represent the GMT timezone, even though that
function is not at all heavily used and is never reached in most backends.
Let's malloc it on-demand, instead. This does pose the question of how to
handle a malloc failure, but there's already a well-defined error report
convention here, ie set errno and return NULL. We have but one caller of
pg_gmtime in HEAD, and two in back branches, neither of which were troubling
to check for error. Make them do so. The possible errors are sufficiently
unlikely (out-of-range timestamp, and now malloc failure) that I think elog()
is adequate. Back-patch to all supported branches to keep our copies of the
IANA timezone code in sync. This particular change is in a stanza that
already differs from upstream, so it's a wash for maintenance purposes --- but
only as long as we keep the branches the same. Discussion:
https://postgr.es/m/20181015200754.7y7zfuzsoux2c4ya@alap3.anarazel.de
https://git.postgresql.org/pg/commitdiff/3dfef0c519de40a196dc26b76604151347bf5454

- Improve stability of recently-added regression test case. Commit b5febc1d1
added a contrib/btree_gist test case that has been observed to fail in the
buildfarm as a result of background auto-analyze updating stats and changing
the selected plan. Forestall that by forcibly analyzing in foreground,
instead. The new plan choice is just as good for our purposes, since we
really only care that an index-only plan does not get selected. Back-patch to
9.5, like the previous patch. Discussion:
https://postgr.es/m/14643.1539629304@sss.pgh.pa.us
https://git.postgresql.org/pg/commitdiff/17d6a8fb764c95536c08742a7f94705e9774db05

- Make PostgresNode.pm's poll_query_until() more chatty about failures.
Reporting only the stderr is unhelpful when the problem is that the server
output we're getting doesn't match what was expected. So we should report the
query output too; and just for good measure, let's print the query we used and
the output we expected. Back-patch to 9.5 where poll_query_until was
introduced. Discussion: https://postgr.es/m/17913.1539634756@sss.pgh.pa.us
https://git.postgresql.org/pg/commitdiff/c015ccb306ec81bca3023818c9cf0113cae25be1

- Avoid statically allocating formatting.c's format string caches. This
eliminates circa 120KB of static data from Postgres' memory footprint. In
some usage patterns that space will get allocated anyway, but in many
processes it never will be allocated. We can improve matters further by
allocating only as many cache entries as we actually use, rather than
allocating the whole array on first use. However, to avoid wasting lots of
space due to palloc's habit of rounding requests up to power-of-2 sizes, tweak
the maximum cacheable format string length to make the struct sizes be powers
of 2 or just less. The sizes I chose make the maximums a little bit less than
they were before, but I doubt it matters much. While at it, rearrange struct
FormatNode to avoid wasting quite so much padding space. This change actually
halves the size of that struct on 64-bit machines. Discussion:
https://postgr.es/m/20181015200754.7y7zfuzsoux2c4ya@alap3.anarazel.de
https://git.postgresql.org/pg/commitdiff/fd85e9f78d44f0f36776e644380ada077451c992

- Avoid rare race condition in privileges.sql regression test. We created a
temp table, then switched to a new session, leaving the old session to clean
up its temp objects in background. If that took long enough, the eventual
attempt to drop the user that owns the temp table could fail, as exhibited
today by sidewinder. Fix by dropping the temp table explicitly when we're
done with it. It's been like this for quite some time, so back-patch to all
supported branches. Report:
https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=sidewinder&dt=2018-10-16%2014%3A45%3A00
https://git.postgresql.org/pg/commitdiff/f7a953c2d870185499faaabd06983526a83eb387

- Be smarter about age-counter overflow in formatting.c caches. The previous
code here simply threw away whatever it knew about cache entry ages whenever a
counter overflow occurred. Since the counter is int width and will be bumped
once per format function execution, overflows are not really so rare as to not
be worth thinking about. Instead, let's deal with the situation by halving
all the age values, essentially rescaling the age metric. In that way, we
retain a pretty accurate (if not quite perfect) idea of which entries are
oldest.
https://git.postgresql.org/pg/commitdiff/2c300c680767a45450bf7afa4075095f40502dc4

- Back off using -isysroot on Darwin. Rethink the solution applied in commit
5e2217131 to get PL/Tcl to build on macOS Mojave. I feared that adding
-isysroot globally might have undesirable consequences, and sure enough Jakob
Egger reported one: it complicates building extensions with a different Xcode
version than was used for the core server. (I find that a risky proposition
in general, but apparently it works most of the time, so we shouldn't break it
if we don't have to.) We'd already adopted the solution for PL/Perl of
inserting the sysroot path directly into the -I switches used to find Perl's
headers, and we can do the same thing for PL/Tcl by changing the -iwithsysroot
switch that Apple's tclConfig.sh reports. This restricts the risks to PL/Perl
and PL/Tcl themselves and directly-dependent extensions, which is a lot more
pleasing in general than a global -isysroot switch. Along the way, tighten
the test to see if we need to inject the sysroot path into $perl_includedir,
as I'd speculated about upthread but not gotten round to doing. As before,
back-patch to all supported versions. Discussion:
https://postgr.es/m/20840.1537850987@sss.pgh.pa.us
https://git.postgresql.org/pg/commitdiff/68fc227dd0737f38295f3dee0c10268ad11a5150

- Formatting cleanup in ecpglib/prepare.c. Looking at this code made my head
hurt. Format the comments more like the way it's done elsewhere, break a few
overly long lines. No actual code changes in this commit.
https://git.postgresql.org/pg/commitdiff/92dff341160ae896dcfff2dac59fd0c860892f26

- Avoid statically allocating statement cache in ecpglib/prepare.c. This
removes a megabyte of storage that isn't used at all in ecpglib's default
operating mode --- you have to enable auto-prepare to get any use out of it.
Seems well worth the trouble to allocate on demand. Discussion:
https://postgr.es/m/20181015200754.7y7zfuzsoux2c4ya@alap3.anarazel.de
https://git.postgresql.org/pg/commitdiff/e15aae829e3eb686bdf03dea9022a61f8039a7ef

- Improve tzparse's handling of TZDEFRULES ("posixrules") zone data. In the
IANA timezone code, tzparse() always tries to load the zone file named by
TZDEFRULES ("posixrules"). Previously, we'd hacked that logic to skip the
load in the "lastditch" code path, which we use only to initialize the default
"GMT" zone during GUC initialization. That's critical for a couple of
reasons: since we do not support leap seconds, we *must not* allow "GMT" to
have leap seconds, and since this case runs before the GUC subsystem is fully
alive, we'd really rather not take the risk of pg_open_tzfile throwing any
errors. However, that still left the code reading TZDEFRULES on every other
call, something we'd noticed to the extent of having added code to cache the
result so it was only done once per process not a lot of times. Andres Freund
complained about the static data space used up for the cache; but as long as
the logic was like this, there was no point in trying to get rid of that
space. We can improve matters by looking a bit more closely at what the IANA
code actually needs the TZDEFRULES data for. One thing it does is that if
"posixrules" is a leap-second-aware zone, the leap-second behavior will be
absorbed into every POSIX-style zone specification. However, that's a
behavior we'd really prefer to do without, since for our purposes the end
effect is to render every POSIX-style zone name unsupported. Otherwise, the
TZDEFRULES data is used only if the POSIX zone name specifies DST but doesn't
include a transition date rule (e.g., "EST5EDT" rather than
"EST5EDT,M3.2.0,M11.1.0"). That is a minority case for our purposes --- in
particular, it never happens when tzload() invokes tzparse() to interpret a
transition date rule string found in a tzdata zone file. Hence, if we
legislate that we're going to ignore leap-second data from "posixrules", we
can postpone the TZDEFRULES load into the path where we actually need to
substitute for a missing date rule string. That means it will never happen at
all in common scenarios, making it reasonable to dynamically allocate the
cache space when it does happen. Even when the data is already loaded, this
saves some cycles in the common code path since we avoid a memcpy of 23KB or
so. And, IMO at least, this is a less ugly hack on the IANA logic than what
we had before, since it's not messing with the lastditch-vs-regular code
paths. Back-patch to all supported branches, not so much because this is a
critical change as that I want to keep all our copies of the IANA timezone
code in sync. Discussion:
https://postgr.es/m/20181015200754.7y7zfuzsoux2c4ya@alap3.anarazel.de
https://git.postgresql.org/pg/commitdiff/e7eb07f7095185e64d7b9c8ff5f3e1588d6e508a

- Minor additional improvements for ecpglib/prepare.c. Avoid allocating
never-used entries in stmtCacheEntries[], other than the intentionally-unused
zero'th entry. Tie the array size directly to the bucket count and size,
rather than having undocumented dependencies between three magic constants.
Fix the hash calculation to be platform-independent --- notably, it was
sensitive to the signed'ness of "char" before, not to mention having an
unnecessary hard-wired dependency on the existence and size of type "long
long". (The lack of complaints says it's been a long time since anybody tried
to build PG on a compiler without "long long", and certainly with the
requirement for C99 this isn't a live bug anymore. But it's still not per
project coding style.) Fix ecpg_auto_prepare's new-cache-entry path so that
it increments the exec count for the new cache entry not the dummy zero'th
entry. The last of those is an actual bug, though one of little consequence;
the rest is mostly future-proofing and neatnik-ism. Doesn't seem necessary to
back-patch.
https://git.postgresql.org/pg/commitdiff/d8cc1616b5547d49d899be7b68713272ce334708

- Fix minor bug in isolationtester. If the lock wait query failed,
isolationtester would report the PQerrorMessage from some other connection,
meaning there would be no message or an unrelated one. This seems like a
pretty unlikely occurrence, but if it did happen, this bug could make it
really difficult/confusing to figure out what happened. That seems to justify
patching all the way back. In passing, clean up another place where the
"wrong" conn was used for an error report. That one's not actually buggy
because it's a different alias for the same connection, but it's still
confusing to the reader.
https://git.postgresql.org/pg/commitdiff/9958b2b2a8cc25e8d791fc965fff3f0496163be6

- Const-ify a few more large static tables. Per research by Andres.
Discussion:
https://postgr.es/m/20181015200754.7y7zfuzsoux2c4ya@alap3.anarazel.de
https://git.postgresql.org/pg/commitdiff/48d818ede15b3e43546e8d49dc19a3021bc773cf

- Improve some comments related to executor result relations.
es_leaf_result_relations doesn't exist; perhaps this was an old name for
es_tuple_routing_result_relations, or maybe this comment has gone unmaintained
through multiple rounds of whacking the code around. Related comment in
execnodes.h was both obsolete and ungrammatical.
https://git.postgresql.org/pg/commitdiff/26cb82030fa83e222f5744fb8254ec9073162273

- Still further rethinking of build changes for macOS Mojave. To avoid the
sorts of problems complained of by Jakob Egger, it'd be best if configure
didn't emit any references to the sysroot path at all. In the case of PL/Tcl,
we can do that just by keeping our hands off the TCL_INCLUDE_SPEC string
altogether. In the case of PL/Perl, we need to substitute -iwithsysroot for
-I in the compile commands, which is easily handled if we change to using a
configure output variable that includes the switch not only the directory
name. Since PL/Tcl and PL/Python already do it like that, this seems like
good consistency cleanup anyway. Hence, this replaces the advice given to
Perl-related extensions in commit 5e2217131; instead of writing
"-I$(perl_archlibexp)/CORE", they should just write "$(perl_includespec)".
(The old way continues to work, but not on recent macOS.) It's still the case
that configure needs to be aware of the sysroot path internally, but that's
cleaner than what we had before. As before, back-patch to all supported
versions. Discussion: https://postgr.es/m/20840.1537850987@sss.pgh.pa.us
https://git.postgresql.org/pg/commitdiff/e74dd00f53cd6dc1887f76b9672e5f6dcf0fd8a2

- Add missing quote_identifier calls for CREATE TRIGGER ... REFERENCING.
Mixed-case names for transition tables weren't dumped correctly. Oversight in
commit 8c48375e5, per bug #15440 from Karl Czajkowski. In passing, I couldn't
resist a bit of code beautification. Back-patch to v10 where this was
introduced. Discussion:
https://postgr.es/m/15440-02d1468e94d63d76@postgresql.org
https://git.postgresql.org/pg/commitdiff/350410be459ccda7eeeea35b56b5f2d24391f90c

- Update time zone data files to tzdata release 2018f. DST law changes in
Chile, Fiji, and Russia (Volgograd). Historical corrections for China, Japan,
Macau, and North Korea. Note: like the previous tzdata update, this involves
a depressingly large amount of semantically-meaningless churn in tzdata.zi.
That is a consequence of upstream's data compression method assigning unstable
abbreviations to DST rulesets. I complained about that to them last time, and
this version now uses an assignment method that pays some heed to not changing
abbreviations unnecessarily. So hopefully, that'll be better going forward.
https://git.postgresql.org/pg/commitdiff/13877d30f2ec93f6043937f76af207dcc614a4e7

- Sync our copy of the timezone library with IANA release tzcode2018f. About
half of this is purely cosmetic changes to reduce the diff between our code
and theirs, like inserting "const" markers where they have them. The other
half is tracking actual code changes in zic.c and localtime.c. I don't think
any of these represent near-term compatibility hazards, but it seems best to
stay up to date. I also fixed longstanding bugs in our code for producing the
known_abbrevs.txt list, which by chance hadn't been exposed before, but which
resulted in some garbage output after applying the upstream changes in zic.c.
Notably, because upstream removed their old phony transitions at the Big Bang,
it's now necessary to cope with TZif files containing no DST transition times
at all.
https://git.postgresql.org/pg/commitdiff/12bfb778ce688fc662a6cb35f6298734fcf4856f

- Client-side fixes for delayed NOTIFY receipt. PQnotifies() is defined to just
process already-read data, not try to read any more from the socket. (This is
a debatable decision, perhaps, but I'm hesitant to change longstanding library
behavior.) The documentation has long recommended calling PQconsumeInput()
before PQnotifies() to ensure that any already-arrived message would get
absorbed and processed. However, psql did not get that memo, which explains
why it's not very reliable about reporting notifications promptly. Also, most
(not quite all) callers called PQconsumeInput() just once before a
PQnotifies() loop. Taking this recommendation seriously implies that we
should do PQconsumeInput() before each call. This is more important now that
we have "payload" strings in notification messages than it was before; that
increases the probability of having more than one packet's worth of notify
messages. Hence, adjust code as well as documentation examples to do it like
that. Back-patch to 9.5 to match related server fixes. In principle we could
probably go back further with these changes, but given lack of field
complaints I doubt it's worthwhile. Discussion:
https://postgr.es/m/CAOYf6ec-TmRYjKBXLLaGaB-jrd=mjG1Hzn1a1wufUAR39PQYhw@mail.gmail.com
https://git.postgresql.org/pg/commitdiff/4247db62522fafd1d4c045c5a432f50f2f05c0e0

- Server-side fix for delayed NOTIFY and SIGTERM processing. Commit 4f85fde8e
introduced some code that was meant to ensure that we'd process cancel, die,
sinval catchup, and notify interrupts while waiting for client input. But
there was a flaw: it supposed that the process latch would be set upon arrival
at secure_read() if any such interrupt was pending. In reality, we might well
have cleared the process latch at some earlier point while those flags
remained set -- particularly notifyInterruptPending, which can't be handled as
long as we're within a transaction. To fix the NOTIFY case, also attempt to
process signals (except ProcDiePending) before trying to read. Also, if we
see that ProcDiePending is set before we read, forcibly set the process latch
to ensure that we will handle that signal promptly if no data is available. I
also made it set the process latch on the way out, in case there is similar
logic elsewhere. (It remains true that we won't service ProcDiePending here
unless we need to wait for input.) The code for handling ProcDiePending during
a write needs those changes, too. Also be a little more careful about when to
reset whereToSendOutput, and improve related comments. Back-patch to 9.5
where this code was added. I'm not entirely convinced that older branches
don't have similar issues, but the complaint at hand is just about the >= 9.5
code. Jeff Janes and Tom Lane Discussion:
https://postgr.es/m/CAOYf6ec-TmRYjKBXLLaGaB-jrd=mjG1Hzn1a1wufUAR39PQYhw@mail.gmail.com
Branch ------ master src/backend/libpq/be-secure.c | 20 ++++++-----
src/backend/tcop/postgres.c | 77 ++++++++++++++++++++++++++++---------------
2 files changed, 62 insertions(+), 35 deletions(-)

- Silence perlcritic warning about missing return. Per buildfarm member crake.
https://git.postgresql.org/pg/commitdiff/e65e8f8218e3b99534f01bd5e67b1e6ec99e6f7e

Thomas Munro pushed:

- Move the replication lag tracker into heap memory. Andres Freund complained
about the 128KB of .bss occupied by LagTracker. It's only needed in the
walsender process, so allocate it in heap memory there. Author: Thomas Munro
Discussion:
https://postgr.es/m/20181015200754.7y7zfuzsoux2c4ya%40alap3.anarazel.de
https://git.postgresql.org/pg/commitdiff/e73ca79fc78db4eb13b7f513f16b7164c7b05ba7

- Refactor pid, random seed and start time initialization. Background workers,
including parallel workers, were generating the same sequence of numbers in
random(). This showed up as DSM handle collisions when Parallel Hash created
multiple segments, but any code that calls random() in background workers
could be affected if it cares about different backends generating different
numbers. Repair by making sure that all new processes initialize the seed at
the same time as they set MyProcPid and MyStartTime in a new function
InitProcessGlobals(), called by the postmaster, its children and also
standalone processes. Also add a new high resolution MyStartTimestamp as a
potentially useful by-product, and remove SessionStartTime from struct Port as
it is now redundant. No back-patch for now, as the known consequences so far
are just a bunch of harmless shm_open(O_EXCL) collisions. Author: Thomas
Munro Reviewed-by: Tom Lane Discussion:
https://postgr.es/m/CAEepm%3D2eJj_6%3DB%2B2tEpGu2nf1BjthCf9nXXUouYvJJ4C5WSwhg%40mail.gmail.com
https://git.postgresql.org/pg/commitdiff/197e4af9d5da180190a0f2be851b095dba57d9cd

Andres Freund pushed:

- Move TupleTableSlots boolean member into one flag variable. There's several
reasons for this change: 1) It reduces the total size of TupleTableSlot /
reduces alignment padding, making the commonly accessed members fit into a
single cacheline (but we currently do not force proper alignment, so that's
not yet guaranteed to be helpful) 2) Combining the booleans into a flag allows
to combine read/writes from memory. 3) With the upcoming slot abstraction
changes, it allows to have core and extended flags, in a memory efficient way.
Author: Ashutosh Bapat and Andres Freund Discussion:
https://postgr.es/m/20180220224318.gw4oe5jadhpmcdnm@alap3.anarazel.de
https://git.postgresql.org/pg/commitdiff/c5257345ef61922468cd9abd887c3cb6c38792cb

- Move generic slot support functions from heaptuple.c into execTuples.c.
heaptuple.c was never a particular good fit for slot_getattr(),
slot_getsomeattrs() and slot_getmissingattrs(), but in upcoming changes slots
will be made more abstract (allowing slots that contain different types of
tuples), making it clearly the wrong place. Note that slot_deform_tuple()
remains in it's current place, as it clearly deals with a HeapTuple.
getmissingattrs() also remains, but it's less clear that that's correct - but
execTuples.c wouldn't be the right place. Author: Ashutosh Bapat.
Discussion:
https://postgr.es/m/20180220224318.gw4oe5jadhpmcdnm@alap3.anarazel.de
https://git.postgresql.org/pg/commitdiff/9d906f1119de893a4ca533c5e7b97207a3aa963b

- Correct constness of a few variables. This allows the compiler / linker to
mark affected pages as read-only. There's other cases, but they're a bit more
invasive, and should go through some review. These are easy. They were found
with objdump -j .data -t src/backend/postgres|awk '{print $4, $5, $6}'|sort
-r|less Discussion:
https://postgr.es/m/20181015200754.7y7zfuzsoux2c4ya@alap3.anarazel.de
https://git.postgresql.org/pg/commitdiff/62649bad831fb69b5d9644470acc413a35cddea6

- Correct constness of system attributes in heap.c & prerequisites. This allows
the compiler / linker to mark affected pages as read-only. There's a fair
number of pre-requisite changes, to allow the const properly be propagated.
Most of consts were already required for correctness anyway, just not
represented on the type-level. Arguably we could be more aggressive in using
consts in related code, but.. This requires using a few of the types
underlying typedefs that removes pointers (e.g. const NameData *) as declaring
the typedefed type constant doesn't have the same meaning (it makes the
variable const, not what it points to). Discussion:
https://postgr.es/m/20181015200754.7y7zfuzsoux2c4ya@alap3.anarazel.de
https://git.postgresql.org/pg/commitdiff/02a30a09f9e57a29f7bda82f5f4bfc214eed3980

- Add macro to cast away const without allowing changes to underlying type. The
new unconsitify(underlying_type, var) macro allows to cast constness away from
a variable, but doesn't allow changing the underlying type. Enforcement of
the latter currently only works for gcc like compilers. Please note IT IS NOT
SAFE to cast constness away if the variable will ever be modified (it would be
undefined behaviour). Doing so anyway can cause compiler misoptimizations or
runtime crashes (modifying readonly memory). It is only safe to use when the
the variable will not be modified, but API design or language restrictions
prevent you from declaring that (e.g. because a function returns both const
and non-const variables). This'll be used in an upcoming change, but seems
like it's independent infrastructure. Author: Andres Freund Discussion:
https://postgr.es/m/20181015200754.7y7zfuzsoux2c4ya@alap3.anarazel.de
https://git.postgresql.org/pg/commitdiff/d1211c63f010c4e0c22d6070872e80b8c8b4df82

- Mark constantly allocated dest receiver as const. This allows the compiler /
linker to mark affected pages as read-only. Doing so requires casting
constness away, as CreateDestReceiver() returns both constant and non-constant
dest receivers. That's fine though, as any modification of the statically
allocated receivers would already have been a bug (and would now be caught on
some platforms). Discussion:
https://postgr.es/m/20181015200754.7y7zfuzsoux2c4ya@alap3.anarazel.de
https://git.postgresql.org/pg/commitdiff/93ca02e00560ad6553fbd8ebabb7d5349e98ade8

- Reorder FmgrBuiltin members, saving 25% in size. That's worth it, as
fmgr_builtins is frequently accessed, and as fmgr_builtins is one of the
biggest constant variables in a backend. On most 64bit systems this will
change the size of the struct from 32byte to 24bytes. While that could make
indexing into the array marginally more expensive, the higher cache hit ratio
is worth more, especially because these days fmgr_builtins isn't searched with
a binary search anymore (c.f. 212e6f34d5). Discussion:
https://postgr.es/m/20181016201145.aa2dfeq54rhqzron@alap3.anarazel.de
https://git.postgresql.org/pg/commitdiff/28d750c0cd5b3cea0dfbc84cce24e30f5c2eb966

Andrew Dunstan pushed:

- Lower privilege level of programs calling regression_main. On Windows this
mean that the regression tests can now safely and successfully run as
Administrator, which is useful in situations like Appveyor. Elsewhere it's a
no-op. Backpatch to 9.5 - this is harder in earlier branches and not worth
the trouble. Discussion:
https://postgr.es/m/650b0c29-9578-8571-b1d2-550d7f89f307@2ndQuadrant.com
https://git.postgresql.org/pg/commitdiff/ce5d3424d6411f7a7228fd4463242cb382af3e0c

- Don't try to test files named with a trailing dot on Windows. The
pg_verify_checksums test tries to create files with corrupt data named "123."
and "123_." But on Windows a file name with a trailing dot is the same as a
file without the trailing dot. In the first case this will create a file with
a "valid" name, which causes the test to fail in an unexpected way, and in the
secongd case this will be redandant as the test already creates a file named
"123_". Bug discovered by buildfarm animal bowerbird.
https://git.postgresql.org/pg/commitdiff/c468bd5c087051e2edc573118a42d73a8ae3c3a4

Michaël Paquier pushed:

- Use whitelist to choose files scanned with pg_verify_checksums The original
implementation of pg_verify_checksums used a blacklist to decide which files
should be skipped for scanning as they do not include data checksums, like
pg_internal.init or pg_control. However, this missed two things: - Some files
are created within builds of EXEC_BACKEND and these were not listed, causing
failures on Windows. - Extensions may create custom files in data folders,
causing the tool. to equally fail. This commit switches to a whitelist-like
method instead by checking if the files to scan are authorized relation files.
This is close to a reverse-engineering of what is defined in relpath.c in
charge of building the relation paths, and we could consider refactoring what
this patch does so as all routines are in a single place. This is left for
later. This is based on a suggestion from Andres Freund. TAP tests are
updated so as multiple file patterns are tested. The bug has been spotted by
various buildfarm members as a result of b34e84f which has introduced the TAP
tests of pg_verify_checksums. Author: Michael Paquier Reviewed-by: Andrew
Dunstan, Michael Banck Discussion:
https://postgr.es/m/20181012005614.GC26424@paquier.xyz Backpatch-through: 11
https://git.postgresql.org/pg/commitdiff/d55241af705667d4503638e3f77d3689fd6be31d

== Pending Patches ==

Tom Lane sent in another revision of a patch to get rid of empty jointrees.

Thomas Munro sent in a patch to refactor the checkpointer request queue.

Hironobu SUZUKI and Fabien COELHO traded patches to add a pseudo-random
permutation function to pgbench.

Fabien COELHO sent in another revision of a patch to make it possible to store
the results of pgbench SELECT statements into variables.

Alexander Korotkov sent in another revision of a patch to make LWLocks more
fair.

Alexander Korotkov sent in another revision of a patch to remove some unneeded
snapshot functions from TableAmRoutine and add a costing function to the tableam
interface.

Laurenz Albe sent in three more revisions of a patch to add pg_promote() to
promote standby servers.

John Naylor sen in another revision of a patch to avoid creation of the free
space map for small tables and add an nblocks parameter to
RecordPageWithFreeSpace().

Dmitry Dolgov sent in another revision of a patch to implement index skip scans.

Andrey Klychkov sent in a patch to make heap_insert() and heap_update() faster.

Surafel Temesgen sent in another revision of a patch to add a
--multi-row-inserts option to pg_dump.

Victor Wagner sent in a patch to make it possible to build a 64-bit PostgreSQL
with GSS support on Windows.

Richard Guo sent in a patch to pull up sublink of type 'NOT NOT (expr)'.

James Coleman sent in a patch to add a tuple_data_record() to the pageinspect
contrib extension.

Peter Eisentraut sent in a patch to add columns clientserial and issuerdn to
pg_catalog.pg_stat_ssl.

Ants Aasma sent in a patch to move logging of checkpoint start out of a critical
section.

Thomas Munro sent in three more revisions of a patch to fix a thinko in the
handling of corrupted DSM control areas by ensuring that dsm_postmaster_startup
is called during crash restarts.

Alexander Kuzmenkov sent in another revision of a patch to remove unneeded
self-joins.

Ildar Musin sent in a patch to remove a redundant context switch in the
PostgreSQL FDW.

Peter Eisentraut sent in another revision of a patch to add an option to use
file cloning in pg_dump.

Peter Eisentraut sent in a patch to remove get_attidentity() and
get_atttypmod(), the former on grounds of inefficiency, the latter on grounds of
disuse.

Thomas Munro sent in another revision of a patch to ensure that the storage
manager remembers fsync() requests so that future attempts retry them should
they fail and make failed fsync() PANIC on the operating systems where that's
the sensible behavior.

Kyotaro HORIGUCHI sent in another revision of a patch to remove catcache entries
that haven't been used for the time set in the new syscache_prune_min_age GUC,
track syscache usage with a new track_syscache_usage_interval GUC, add a
non-transactional GUC-setting feature, and implements a way to set GUCs
remotely.

Amit Langote sent in two more revisions of a patch to set relhassubclass on
index parents.

Amit Langote sent in another revision of a patch to add a pg_partition_tree to
display information about partitions.

Masahiko Sawada sent in another revision of a patch to add support for atomic
commits among multiple foreign servers and enable same for the PostgreSQL FDW.

Heikki Linnakangas sent in a patch to speed up text_position_setup/next() in
some common cases with multibyte encodings.

Aleksey Kondratov sent in a patch to make it possible for pg_rewind to use
restore_command from recovery.conf or from the command line.

Aleksandr Parfenov sent in another revision of a patch to optimize the use of
immutable functions as relations.

Renato dos Santos sent in a patch to make psql handle SIGQUIT the same way it
does SIGINT.

Michaël Paquier sent in a patch to ensure that pg_verify_checksums and checksum
verification for base backups use the same logic, namely a whitelist-based
approach.