== PostgreSQL Weekly News - April 03 2016 ==

== PostgreSQL Weekly News - April 03 2016 ==

Security Updates 9.5.2, 9.4.7, 9.3.12, 9.2.16, and 9.1.21, released.
Upgrade ASAP!
http://www.postgresql.org/about/news/1656/

The first meetup of the Islamabad (Pakistan) PUG will be on April 8.
Details and RSVP below:
http://www.meetup.com/Islamabad-PostgreSQL-User-Group/events/229935189/

== PostgreSQL Product News ==

Postgres-XL 9.5 R1 Beta2 released.
http://2ndquadrant.com/en-us/resources/postgres-xl/

PL/Java 1.5.0 is out, now with security policy changes.
http://tada.github.io/pljava/

== PostgreSQL Jobs for April ==

http://archives.postgresql.org/pgsql-jobs/2016-04/

== PostgreSQL Local ==

The 8th PostgreSQL Session will be held on April 6th, 2016, in Lyon,
France.
<AT> postgresql-sessions <DOT> org.

The first meeting of PostgreSQL Israel will be taking place in Tel
Aviv on April 7, 2016.
http://www.meetup.com/PostgreSQL-Israel/events/229430729/

PGConf US 2016 will take place April 18-20, 2016 in NYC. Registration
is open.
http://www.pgconf.us/2016/

LinuxFest Northwest will take place April 23-24, 2016 at Bellingham
Technical College in Bellingham, Washington, USA. The CfP is now
open.
http://www.linuxfestnorthwest.org/2016/present

FOSS4G NA, will be held May 2-5, 2016 in Raleigh, North Carolina.
The CfP is still open.
https://2016.foss4g-na.org/cfp

PGCon 2016 will be held May 17-21, 2016 in Ottawa.
http://www.pgcon.org/

This year's Swiss PGDay will be held on June 24, 2016 at the
University of Applied Sciences in Rapperswil (Switzerland).
The CfP is open.
http://www.pgday.ch/

"5432 ... Meet us!", will take place in Milan, Italy on June 28-29, 2016.
The CfP is open until March 14, 2016.
http://5432meet.us/

PG Day UK 2016 - Call for Papers is open until 5th April 2016. The
conference will be 5th July 2016. Please submit proposals to
Simon(dot)Riggs(at)2ndquadrant(dot)com <Simon AT 2ndquadrant DOT com> or via the
website at:
http://www.pgconf.uk/papers

PgConf Silicon Valley 2016 will be held on November 14-16, 2016.
http://www.pgconfsv.com/

== PostgreSQL in the News ==

Planet PostgreSQL: http://planet.postgresql.org/

PostgreSQL Weekly News is brought to you this week by David Fetter

Submit news and announcements by Sunday at 3:00pm Pacific time.
Please send English language ones to david(at)fetter(dot)org, German language
to pwn(at)pgug(dot)de, Italian language to pwn(at)itpug(dot)org(dot) Spanish language
to pwn(at)arpug(dot)com(dot)ar(dot)

== Applied Patches ==

Andres Freund pushed:

- pg_rewind: Close backup_label file descriptor. This was a
relatively harmless leak, as createBackupLabel() is only called once
per pg_rewind invocation. Author: Michael Paquier Reported-By:
Michael Paquier Discussion:
CAB7nPqRnOw30gOXe2_SPLjh37bgm4V+txbYAPwoXb97nGQ297w(at)mail(dot)gmail(dot)com
Backpatch: 9.5, where pg_rewind was introduced
http://git.postgresql.org/pg/commitdiff/a6c845946dac5c1f26cf8729cf61f1d852f75484

- Fix LWLockReportWaitEnd() parameter list to be (void). Previously
it was an "old style" function declaration.
http://git.postgresql.org/pg/commitdiff/9f7c527af308dcdaba2f0ff9d362d672e8886fb1

- pg_rewind: fsync target data directory. Previously pg_rewind did
not fsync any files. That's problematic, given that the target
directory is modified. If the database was started afterwards,
2ce439f33 luckily already caused the data directory to be synced to
disk at postmaster startup; reducing the scope of the problem. To
fix, use initdb -S, at the end of the pg_rewind run. It doesn't seem
worthwhile to duplicate the code into pg_rewind, and initdb -S is
already used that way by pg_upgrade. Reported-By: Andres Freund
Author: Michael Paquier, somewhat edited by me Discussion:
20160310034352(dot)iuqgvpmg5qmnxtkz(at)alap3(dot)anarazel(dot)de
CAB7nPqSytVG1o4S3S2pA1O=692ekurJ+fckW2PywEG3sNw54Ow(at)mail(dot)gmail(dot)com
Backpatch: 9.5, where pg_rewind was introduced
http://git.postgresql.org/pg/commitdiff/408f0438531eec17ac62f91fc23f72bcfc48dd36

Tom Lane pushed:

- Clamp adjusted ndistinct to positive integer in
estimate_hash_bucketsize(). This avoids a possible divide-by-zero
in the following calculation, and rounding the number to an integer
seems like saner behavior anyway. Assuming IEEE math, the division
would yield +Infinity which would get replaced by 1.0 at the bottom
of the function, so nothing really interesting would ensue; but
avoiding divide-by-zero seems like a good idea on general
principles. Per report from Piotr Stefaniak. No back-patch since
this seems mostly cosmetic.
http://git.postgresql.org/pg/commitdiff/fa09f8935156533584b4e215bdf70ec1ff968dad

- Guard against zero vardata.rel->tuples in
estimate_hash_bucketsize(). If the referenced rel was proven empty,
we'd compute 0/0 here, which results in the function returning NaN.
That's a bit more serious than the other zero-divide case. Still,
it only seems to be possible in HEAD, so no back-patch. Per report
from Piotr Stefaniak. I looked through the rest of selfuncs.c and
found no other likely trouble spots.
http://git.postgresql.org/pg/commitdiff/d65b665d524a67273b075f468bf3d60ce31f4040

- Release notes for 9.5.2, 9.4.7, 9.3.12, 9.2.16, 9.1.21.
http://git.postgresql.org/pg/commitdiff/499a50571c72f41bb1365970d55dae5c8afcb6ba

- Code and docs review for commit
3187d6de0e5a9e805b27c48437897e8c39071d45. Fix up check for
high-bit-set characters, which provoked "comparison is always true
due to limited range of data type" warnings on some compilers, and
was unlike the way we do it elsewhere anyway. Fix omission of "$"
from the set of valid identifier continuation characters. Get rid
of sanitize_text(), which was utterly inconsistent with any other
error report anywhere in the system, and wasn't even well designed
on its own terms (double-quoting the result string without escaping
contained double quotes doesn't seem very well thought out). Fix up
error messages, which didn't follow the message style guidelines
very well, and were overly specific in situations where the actual
mistake might not be what they said. Improve documentation. (I
started out just intending to fix the compiler warning, but the more
I looked at the patch the less I liked it.)
http://git.postgresql.org/pg/commitdiff/d12e5bb79bb535c2df13b76cd7d01f0bb8dc8e4d

- Document errhidecontext() where it ought to be documented. Seems to
have been missed when this function was added. Noted while looking
at David Steele's proposal to add another similar function.
http://git.postgresql.org/pg/commitdiff/e5a4dea80f2506a7a565508e48aaa52296ff410a

- Sync our copy of the timezone library with IANA release tzcode2016c.
We hadn't done this in about six years, which proves to have been a
mistake because there's been a lot of code churn upstream, making
the merge rather painful. But putting it off any further isn't
going to lessen the pain, and there are at least two incompatible
changes that we need to absorb before someone starts complaining
that --with-system-tzdata doesn't work at all on their platform, or
we get blindsided by a tzdata release that our out-of-date zic can't
compile. Last week's "time zone abbreviation differs from POSIX
standard" mess was a wake-up call in that regard. This is a
sufficiently large patch that I'm afraid to back-patch it
immediately, though the foregoing considerations imply that we
probably should do so eventually. For the moment, just put it in
HEAD so that it can get some testing. Maybe we can wait till the
end of the 9.6 beta cycle before deeming it okay.
http://git.postgresql.org/pg/commitdiff/1c1a7cbd6a1600c97dfcd9b5dc78a23b5db9bbf6

- Fix MSVC build for changes in zic. zic now only needs zic.c, but I
didn't realize knowledge about it was hardwired into Mkvcbuild.pm.
Per buildfarm.
http://git.postgresql.org/pg/commitdiff/f5f15ea6aad1b75c1c133a914cf29f9831089a6e

- Sync tzload() and tzparse() APIs with IANA release tzcode2016c.
This brings us a bit closer to matching upstream, but since it
affects files outside src/timezone/, we might choose not to
back-patch it. Hence keep it separate from the main update patch.
http://git.postgresql.org/pg/commitdiff/1f4e9da624a0caf78bcb526f6b05f5993e26f2c7

- Fix portability issues in 86c43f4e22c0771fd0cc6bce2799802c894ee2ec.
INT64_MIN/MAX should be spelled PG_INT64_MIN/MAX, per well
established convention in our sources. Less obviously, a symbol
named DOUBLE causes problems on Windows builds, so rename that to
DOUBLE_CONST; and rename INTEGER to INTEGER_CONST for consistency.
Also, get rid of incorrect/obsolete hand-munging of yycolumn, and
fix the grammar for float constants to handle expected cases such as
".1". First two items by Michael Paquier, second two by me.
http://git.postgresql.org/pg/commitdiff/656ee8489053aafc85324b9ef7e91b645674ffb9

- Fix zic for Windows. The new coding of dolink() is dependent on
link() returning an on-point errno when it fails; but the quick-hack
implementation of link() that we'd put in for Windows didn't bother
with setting errno. Fix that. Analysis and patch by Christian
Ullrich.
http://git.postgresql.org/pg/commitdiff/6d257e732b358ee601a114fe3d1640a46317e554

- Protect zic's symlink() call with #ifdef HAVE_SYMLINK. The IANA
crew seem to think that symlink() exists everywhere nowadays, and
they may well be right. But we use #ifdef HAVE_SYMLINK elsewhere so
for consistency we should do it here too. Noted by Michael Paquier.
http://git.postgresql.org/pg/commitdiff/534da37927f97ae7cb1b468963ba9bca747209ea

- Avoid possibly-unsafe use of Windows' FormatMessage() function.
Whenever this function is used with the FORMAT_MESSAGE_FROM_SYSTEM
flag, it's good practice to include FORMAT_MESSAGE_IGNORE_INSERTS as
well. Otherwise, if the message contains any %n insertion markers,
the function will try to fetch argument strings to substitute ---
which we are not passing, possibly leading to a crash. This is
exactly analogous to the rule about not giving printf() a format
string you're not in control of. Noted and patched by Christian
Ullrich. Back-patch to all supported branches.
http://git.postgresql.org/pg/commitdiff/7abc1571652a924ba4258bda0a26df2de03b790e

- Allow to_timestamp(float8) to convert float infinity to timestamp
infinity. With the original SQL-function implementation, such cases
failed because we don't support infinite intervals. Converting the
function to C lets us bypass the interval representation, which
should be a bit faster as well as more flexible. Vitaly Burovoy,
reviewed by Anastasia Lubennikova
http://git.postgresql.org/pg/commitdiff/e511d878f3bbc205cd260a79740e646eea3c1cd3

- Fix interval_mul() to not produce insane results. interval_mul()
attempts to prevent its calculations from producing silly results,
but it forgot that zero times infinity yields NaN in IEEE
arithmetic. Hence, a case like '1 second'::interval *
'infinity'::float8 produced a NaN for the months product, which
didn't trigger the range check, resulting in bogus and possibly
platform-dependent output. This isn't terribly obvious to the naked
eye because if you try that exact case, you get "interval out of
range" which is what you expect --- but if you look closer, the
error is coming from interval_out not interval_mul. interval_mul
has allowed a bogus value into the system. Fix by adding isnan
tests. Noted while testing Vitaly Burovoy's fix for infinity input
to to_timestamp(). Given the lack of field complaints, I doubt this
is worth a back-patch.
http://git.postgresql.org/pg/commitdiff/a898b409f66f956e99694710f537829db02652c0

- Remove TZ environment-variable entry from postgres reference page.
The server hasn't paid attention to the TZ environment variable
since commit ca4af308c32d03db, but that commit missed removing this
documentation reference, as did commit d883b916a947a3c6 which added
the reference where it now belongs (initdb). Back-patch to 9.2
where the behavior changed. Also back-patch d883b916a947a3c6 as
needed. Matthew Somerville
http://git.postgresql.org/pg/commitdiff/c3834ef9e8abaca54ae542eac960f96e9fecc9a8

- Remove just-added tests for to_timestamp(float8) with out-of-range
inputs. Reporting the specific out-of-range input value produces
platform-dependent results. We could skip reporting the value, but
that's contrary to our message style guidelines and unhelpful to
users. Or we could add a separate expected-output file for Windows,
but that would be a substantial maintenance burden, and these test
cases seem unlikely to be worth it. Per buildfarm.
http://git.postgresql.org/pg/commitdiff/c53ab8a3af46029b72634ec0643e78661b252f62

- Suppress uninitialized-variable warnings. My compiler doesn't like
the lack of initialization of "flag", and I think it's right: if
there were zero keys we'd have an undefined result. The AND of zero
items is TRUE, so initialize to TRUE.
http://git.postgresql.org/pg/commitdiff/818e59373625d194bdec89631b661c4355d15f13

- Improve portability of I/O behavior for the geometric types.
Formerly, the geometric I/O routines such as box_in and point_out
relied directly on strtod() and sprintf() for conversion of the
float8 component values of their data types. However, the behavior
of those functions is pretty platform-dependent, especially for
edge-case values such as infinities and NaNs. This was exposed by
commit acdf2a8b372aec1d, which added test cases involving boxes with
infinity endpoints, and immediately failed on Windows and AIX
buildfarm members. We solved these problems years ago in the main
float8in and float8out functions, so let's fix it by making the
geometric types use that code instead of depending directly on the
platform-supplied functions. To do this, refactor the float8in code
so that it can be used to parse just part of a string, and as a
convenience make the guts of float8out usable without going through
DirectFunctionCall. While at it, get rid of geo_ops.c's fairly
shaky assumptions about the maximum output string length for a
double, by having it build results in StringInfo buffers instead of
fixed-length strings. In passing, convert all the "invalid input
syntax for type foo" messages in this area of the code into "invalid
input syntax for type %s" to reduce the number of distinct
translatable strings, per recent discussion. We would have needed a
fair number of the latter anyway for code-sharing reasons, so we
might as well just go whole hog. Note: this patch is by no means
intended to guarantee that the geometric types uniformly behave
sanely for infinity or NaN component values. But any bugs we have
in that line were there all along, they were just harder to reach in
a platform-independent way.
http://git.postgresql.org/pg/commitdiff/50861cd683e86d5ef2dc1cb669fb503225e4eb98

- Last-minute updates for release notes. Security: CVE-2016-2193,
CVE-2016-3065
http://git.postgresql.org/pg/commitdiff/4c46f83386a7e3556856d1e4c9f0c294d16b0dcc

- Support using index-only scans with partial indexes in more cases.
Previously, the planner would reject an index-only scan if any
restriction clause for its table used a column not available from
the index, even if that restriction clause would later be dropped
from the plan entirely because it's implied by the index's
predicate. This is a fairly common situation for partial indexes
because predicates using columns not included in the index are often
the most useful kind of predicate, and we have to duplicate (or at
least imply) the predicate in the WHERE clause in order to get the
index to be considered at all. So index-only scans were essentially
unavailable with such partial indexes. To fix, we have to do
detection of implied-by-predicate clauses much earlier in the
planner. This patch puts it in check_index_predicates (nee
check_partial_indexes), meaning it gets done for every partial
index, whereas we previously only considered this issue at
createplan time, so that the work was only done for an index
actually selected for use. That could result in a noticeable
planning slowdown for queries against tables with many partial
indexes. However, testing suggested that there isn't really a
significant cost, especially not with reasonable numbers of partial
indexes. We do get a small additional benefit, which is that
cost_index is more accurate since it correctly discounts the
evaluation cost of clauses that will be removed. We can also avoid
considering such clauses as potential indexquals, which saves
useless matching cycles in the case where the predicate columns
aren't in the index, and prevents generating bogus plans that
double-count the clause's selectivity when the columns are in the
index. Tomas Vondra and Kyotaro Horiguchi, reviewed by Kevin
Grittner and Konstantin Knizhnik, and whacked around a little by me
http://git.postgresql.org/pg/commitdiff/f9aefcb91fc1f73fc43e384f660c120e515af931

- Another zic portability fix. I should have remembered that we can't
use INT64_MODIFIER with sscanf(): configure chooses that to work
with snprintf(), but it might be for our src/port/snprintf.c
implementation and so not compatible with the platform's sscanf().
This appears to be the explanation for buildfarm member frogmouth's
continuing unhappiness with the tzcode update. Fortunately, in all
of the places where zic is attempting to read into an int64
variable, it's reading a year which certainly will fit just fine
into an int. So make it read into an int with %d, and then cast or
copy as necessary.
http://git.postgresql.org/pg/commitdiff/c202ecf9023ac3571709c274b326038ae39e90a7

- Fix oversight in getParamDescriptions(), and improve comments. When
getParamDescriptions was changed to handle out-of-memory better by
cribbing error recovery logic from
getRowDescriptions/getAnotherTuple, somebody omitted to copy the
stanza about checking for excess data in the message. But you need
to do that, since continue'ing out of the switch in pqParseInput3
means no such check gets applied there anymore. Noted while looking
at Michael Paquier's patch that made yet another copy of this
advance_and_error logic. (This whole business desperately needs
refactoring, because I sure don't want to see a dozen copies of this
code, but that's where we seem to be headed. What's more, the
"suspend parsing on EOF return" convention is a holdover from
protocol 2 and shouldn't exist at all in protocol 3, because we
don't process partial messages anymore. But for now, just fix the
obvious bug.) Also, fix some wrong/missing comments about what the
API spec is for these three functions. This doesn't seem worthy of
back-patching, even though it's a bug; the case shouldn't ever arise
in the field.
http://git.postgresql.org/pg/commitdiff/2306696004dc6b9259a45e76522c01d6ee5d2ee7

- Get rid of minus zero in box regression test. Commit acdf2a8b added
a test case involving minus zero as a box endpoint. This is not
very portable, as evidenced by the several older buildfarm members
that are failing on the test because they print minus zero as just
"0". If there were any significant reason to test this behavior, we
could consider carrying a separate expected-file; but it doesn't
look to me like there's adequate justification to accept such a
maintenance burden. Just change the test to use plain zero,
instead.
http://git.postgresql.org/pg/commitdiff/a067b50470cf7fda77dd28b03519f2483c2322bf

- Omit null rows when applying the Haas-Stokes estimator for
ndistinct. Previously, we included null rows in the values of n and
N that went into the formula, which amounts to considering null as a
value in its own right; but the d and f1 values do not include
nulls. This is inconsistent, and it contributes to significant
underestimation of ndistinct when the column is mostly nulls. In
any case stadistinct is defined as the number of distinct non-null
values, so we should exclude nulls when doing this computation.
This is an aboriginal bug in our application of the Haas-Stokes
formula, but we'll refrain from back-patching for fear of
destabilizing plan choices in released branches. While at it, make
the code a bit more readable by omitting unnecessary casts and
intermediate variables. Observation and original patch by Tomas
Vondra, adjusted to fix both uses of the formula by Alex Shulgin,
cosmetic improvements by me
http://git.postgresql.org/pg/commitdiff/be4b4dc75955318e763f5b2e3a990e35366ac797

- Omit null rows when setting the threshold for what's a most-common
value. As with the previous patch, large numbers of null rows could
skew this calculation unfavorably, causing us to discard values that
have a legitimate claim to be MCVs, since our definition of MCV is
that it's most common among the non-null population of the column.
Hence, make the numerator of avgcount be the number of non-null
sample values not the number of sample rows; likewise for
maxmincount in the compute_scalar_stats variant. Also, make the
denominator be the number of distinct values actually observed in
the sample, rather than reversing it back out of the computed
stadistinct. This avoids depending on the accuracy of the
Haas-Stokes approximation, and really it's what we want anyway; the
threshold should depend only on what we see in the sample, not on
what we extrapolate about the contents of the whole column. Alex
Shulgin, reviewed by Tomas Vondra and myself
http://git.postgresql.org/pg/commitdiff/3d3bf62f30200500637b24fdb7b992a99f9704c3

- Suppress compiler warning. Some buildfarm members are showing
"comparison is always false due to limited range of data type"
complaints on this test, so #ifdef it out on machines with 32-bit
int.
http://git.postgresql.org/pg/commitdiff/45aae8e78967b37f285e99617b919319bf2bf536

- Add missing "static". Per buildfarm member pademelon.
http://git.postgresql.org/pg/commitdiff/5a5b917184b630529635db2e037d298ad90c355d

- Make all the declarations of WaitEventSetWaitBlock be marked
"inline". The inconsistency here triggered compiler warnings on
some buildfarm members, and it's surely pretty pointless.
http://git.postgresql.org/pg/commitdiff/a1953f3a60cc7d1b8516d0b2c7e82ae8e9242de3

- Add psql \errverbose command to see last server error at full
verbosity. Often, upon getting an unexpected error in psql, one's
first wish is that the verbosity setting had been higher; for
example, to be able to see the schema-name field or the server code
location info. Up to now the only way has been to adjust the
VERBOSITY variable and repeat the failing query. That's a pain, and
it doesn't work if the error isn't reproducible. This commit adds a
psql feature that redisplays the most recent server error at full
verbosity, without needing to make any variable changes or
re-execute the failed command. We just need to hang onto the latest
error PGresult in case the user executes \errverbose, and then apply
libpq's new PQresultVerboseErrorMessage() function to it. This will
consume some trivial amount of psql memory, but otherwise the cost
when the feature isn't used should be negligible. Alex Shulgin,
reviewed by Daniel Vérité, some improvements by me
http://git.postgresql.org/pg/commitdiff/3cc38ca7d21255721d600eb75d7cc6708c14764b

- Add libpq support for recreating an error message with different
verbosity. Often, upon getting an unexpected error in psql, one's
first wish is that the verbosity setting had been higher; for
example, to be able to see the schema-name field or the server code
location info. Up to now the only way has been to adjust the
VERBOSITY variable and repeat the failing query. That's a pain, and
it doesn't work if the error isn't reproducible. This commit adds
support in libpq for regenerating the error message for an existing
error PGresult at any desired verbosity level. This is almost just
a matter of refactoring the existing code into a subroutine, but
there is one bit of possibly-needed information that was not getting
put into PGresults: the text of the last query sent to the server.
We must add that string to the contents of an error PGresult. But
we only need to save it if it might be used, which with the existing
error-formatting code only happens if there is a
PG_DIAG_STATEMENT_POSITION error field, which is probably pretty
rare for errors in production situations. So really the overhead
when the feature isn't used should be negligible. Alex Shulgin,
reviewed by Daniel Vérité, some improvements by me
http://git.postgresql.org/pg/commitdiff/e3161b231cfaadd4b6438eff2fc1f6cd086f41a9

- Clean up some stuff in new contrib/bloom module. Coverity
complained about implicit sign-extension in the
BloomPageGetFreeSpace macro, probably because sizeOfBloomTuple isn't
wide enough for size calculations. No overflow is really possible
as long as maxoff and sizeOfBloomTuple are small enough to represent
a realistic situation, but it seems like a good idea to declare
sizeOfBloomTuple as Size not int32. Add missing check on
BloomPageAddItem() result, again from Coverity. Avoid core dump due
to not allocating so->sign array when scan->numberOfKeys is zero.
Also thanks to Coverity. Use FLEXIBLE_ARRAY_MEMBER rather than
declaring an array as size 1 when it isn't necessarily. Very minor
beautification of related code. Unfortunately, none of the
Coverity-detected mistakes look like they could account for the
remaining buildfarm unhappiness with this module. It's barely
possible that the FLEXIBLE_ARRAY_MEMBER mistake does account for
that, if it's enabling bogus compiler optimizations; but I'm not
terribly optimistic. We probably still have bugs to find here.
http://git.postgresql.org/pg/commitdiff/a9284849b48b04fa2836aaf704659974c13e610d

- Fix contrib/bloom to not fail under CLOBBER_CACHE_ALWAYS. The code
was supposing that rd_amcache wouldn't disappear from under it
during a scan; which is wrong. Copy the data out of the relcache
rather than trying to reference it there.
http://git.postgresql.org/pg/commitdiff/8f75fd1f402acbc30bc15dbf51eb6dec1bbec600

Teodor Sigaev pushed:

- Fix comment in pg_dump. It was missed in
473b93287040b20017cc25a157cffdc5b978c254, CREATE ACCESS METHOD
Alexander Korotkov
http://git.postgresql.org/pg/commitdiff/dabd255d584785c61b1187f10b7bdbf99022ae2d

- psql tab-complete for CREATE/DROP ACCESS METHOD Alexander Korotkov
http://git.postgresql.org/pg/commitdiff/559e7a0a6d4450c09825055e3d255d30ee869c67

- Fix support of digits in email/hostnames. When tsearch was
implemented I did several mistakes in hostname/email definition
rules: 1) allow underscore in hostname what prohibited by RFC 2)
forget to allow leading digits separated by hyphen (like 123-x.com)
in hostname 3) do no allow underscore/hyphen after leading digits in
localpart of email Artur's patch resolves two last issues, but by
the way allows hosts name like 123_x.com together with 123-x.com.
RFC forbids underscore usage in hostname but pg allows that since
initial tsearch version in core, although only for non-digits. Patch
syncs support digits and nondigits in both hostname and email.
Forbidding underscore in hostname may break existsing usage of
tsearch and, anyhow, it should be done by separate patch. Author:
Artur Zakirov BUG: #13964
http://git.postgresql.org/pg/commitdiff/61d66c44f18c73094a50a2ef97d26cc03e171dc0

- Introduce traversalValue for SP-GiST scan During scan sometimes it
would be very helpful to know some information about parent node or
all ancestor nodes. Right now reconstructedValue could be used but
it's not a right usage of it (range opclass uses that).
traversalValue is arbitrary piece of memory in separate
MemoryContext while reconstructedVale should have the same type as
indexed column. Subsequent patches for range opclass and quad4d
tree will use it. Author: Alexander Lebedev, Teodor Sigaev
http://git.postgresql.org/pg/commitdiff/ccd6eb49a4ae924290ab7eba5540218f5beb48b8

- Use traversalValue in SP-GiST range opclass. Author: Alexander
Lebedev
http://git.postgresql.org/pg/commitdiff/87545f541278ffde4fa1ccf396affbc4e9d59a23

- Introduce SP-GiST operator class over box. Patch implements
quad-tree over boxes, naive approach of 2D quad tree will not work
for any non-point objects because splitting space on node is not
efficient. The idea of pathc is treating 2D boxes as 4D points, so,
object will not overlap (in 4D space). The performance tests reveal
that this technique especially beneficial with too much overlapping
objects, so called "spaghetti data". Author: Alexander Lebedev with
editorization by Emre Hasegeli and me
http://git.postgresql.org/pg/commitdiff/acdf2a8b372aec1da09370fca77ff7dccac7646d

- Bump catalog version, forget in
acdf2a8b372aec1da09370fca77ff7dccac7646d
http://git.postgresql.org/pg/commitdiff/2d02a856e8331329121ac2fa2c0b5dab0d106ca0

- Add Generic WAL interface This interface is designed to give an
access to WAL for extensions which could implement new access
method, for example. Previously it was impossible because restoring
from custom WAL would need to access system catalog to find a redo
custom function. This patch suggests generic way to describe changes
on page with standart layout. Bump XLOG_PAGE_MAGIC because of new
record type. Author: Alexander Korotkov with a help of Petr
Jelinek, Markus Nullmeier and minor editorization by my Reviewers:
Petr Jelinek, Alvaro Herrera, Teodor Sigaev, Jim Nasby, Michael
Paquier
http://git.postgresql.org/pg/commitdiff/65578341af1ae50e52e0f45e691ce88ad5a1b9b1

- Fix typo in generic wal docs Markus Nullmeier
http://git.postgresql.org/pg/commitdiff/4e56e5a6de766a6983ce723b1945d68a4e098a06

- Fix English in bloom module documentation Author: Erik Rijkers
http://git.postgresql.org/pg/commitdiff/a361c22ebfa8c87683f07f83f55a8456e7892e6d

- Bloom index contrib module Module provides new access method. It is
actually a simple Bloom filter implemented as pgsql's index. It
could give some benefits on search with large number of columns.
Module is a single way to test generic WAL interface committed
earlier. Author: Teodor Sigaev, Alexander Korotkov Reviewers:
Aleksander Alekseev, Michael Paquier, Jim Nasby Branch ------ master
Details -------
http://git.postgresql.org/pg/commitdiff/9ee014fc899a28a198492b074e32b60ed8915ea9
Fixes in bloom contrib module missed during review: Macros like (var
& FLAG) are changed to ((var & FLAG) != 0). Do not copy
uninitialized part of notFullPage array to page.
http://git.postgresql.org/pg/commitdiff/27f3bbfad46e694fcbf2fa39cb621110d46ec95c

- Fixes in bloom contrib module Looking at result of buildfarm member
jaguarundi it seems to me that BloomOptions isn't inited sometime,
but I don't see yet how it's possible. Nevertheless, check of
signature length's is missed, so, add a limit of it. Also add missed
GenericXLogAbort() in case of already deleted page in vacuum + minor
code refactoring.
http://git.postgresql.org/pg/commitdiff/80afb62db0f118f70367a357cef0509a11112ebb

- Prevent mark as deleted and as 'has free space' page in bloom module
Vacuum might put page into list of pages with some free space and
mark as deleted at the same time.
http://git.postgresql.org/pg/commitdiff/e9e441c9fac6cbc0510cded6abb9d0e6b646ecaf

- Fix condition in e9e441c9fac6cbc0510cded6abb9d0e6b646ecaf Comment is
right, but if - not.
http://git.postgresql.org/pg/commitdiff/9c50372d20aabdb23963f8d28651728d546acefb

Álvaro Herrera pushed:

- Fix minor leak in pg_dump for ACCESS METHOD. Bug reported by
Coverity. Author: Michaël Paquier
http://git.postgresql.org/pg/commitdiff/37732a2555f109b09b7eedfc44a9de22e22268a4

- pg_rewind: Improve internationalization. This is mostly cosmetic
since two of the three changes are debug messages, and the third one
is just a progress indicator. Author: Michaël Paquier
http://git.postgresql.org/pg/commitdiff/cad3edef4f51c37c4b4d8667a2e76a81ca53f9e7

- Update expected file from quoting change. I neglected to update
this in 59a2111b23f. Per buildfarm
http://git.postgresql.org/pg/commitdiff/4b746f0d07c762cf4b421b59a14dfd053eda1761

- Mention BRIN as able to do multi-column indexes. Documentation
mentioned B-tree, GiST and GIN as able to do multicolumn indexes; I
failed to add BRIN to the list. Author: Petr Jediný Reviewed-By:
Fujii Masao, Emre Hasegeli
http://git.postgresql.org/pg/commitdiff/80b986cf528c912f4efc2b0e9f03611f0f15f4df

- PostgresNode: initialize $timed_out if passed. Corrects an
oversight in 2c83f435a3 where the $timed_out reference var isn't
initialized; using it would require the caller to initialize it
beforehand, which is cumbersome. Author: Craig Ringer
http://git.postgresql.org/pg/commitdiff/9bd61311bd48ea53b18bfecb9adddfd844acbffa

- pgbench: allow a script weight of zero. This refines the previous
weight range and allows a script to be "turned off" by passing a
zero weight, which is useful when scripting multiple pgbench runs.
I did not apply the suggested warning when a script uses zero
weight; we use the principle elsewhere that if there's nothing to be
done, do nothing quietly. Adjust docs accordingly. Author: Jeff
Janes, Fabien Coelho
http://git.postgresql.org/pg/commitdiff/a1c935d3b71e44ba36530d47c3ccab6cc9b9eafe

- I forgot the alternate expected file in previous commit. Without
this, the test_slot_timelines modules fails "make installcheck"
because the required feature is not enabled in a stock server. Per
buildfarm
http://git.postgresql.org/pg/commitdiff/3a3b309041b0f30066f0b6cb6640563b6ea27cde

- Blind attempt at fixing Win32 issue on 24c5f1a103c. As best as I
can tell, MyReplicationSlot needs to be PGDLLIMPORT in order for the
new test_slot_timelines test module to compile. Per buildfarm
http://git.postgresql.org/pg/commitdiff/3dd0792ae014c9ceb2c2ade43d0a3132cfeb4fc5

- Fix broken variable declaration Author: Konstantin Knizhnik
http://git.postgresql.org/pg/commitdiff/3501f71c21e31b275b7816551b06a666d9c0c9c9

- Add missing checks to some of pageinspect's BRIN functions.
brin_page_type() and brin_metapage_info() did not enforce being
called by superuser, like other pageinspect functions that take
bytea do. Since they don't verify the passed page thoroughly, it is
possible to use them to read the server memory with a carefully
crafted bytea value, up to a file kilobytes from where the input
bytea is located. Have them throw errors if called by a
non-superuser. Report and initial patch: Andreas Seltenreich
Security: CVE-2016-3065
http://git.postgresql.org/pg/commitdiff/3e1338475ffc2eac25de60a9de9ce689b763aced

- Fix recovery_min_apply_delay test. Previously this test was relying
too much on WAL replay to occur in the exact configured interval,
which was unreliable on slow or overly busy servers. Use a custom
loop instead of poll_query_until, which is hopefully more reliable.
Per continued failures on buildfarm member hamster (which is
probably the only one running this test suite) Author: Michaël
Paquier
http://git.postgresql.org/pg/commitdiff/61608d38361f911a741d4a7df63afe3c7221437e

- Enable logical slots to follow timeline switches. When decoding
from a logical slot, it's necessary for xlog reading to be able to
read xlog from historical (i.e. not current) timelines; otherwise,
decoding fails after failover, because the archives are in the
historical timeline. This is required to make "failover logical
slots" possible; it currently has no other use, although
theoretically it could be used by an extension that creates a slot
on a standby and continues to replay from the slot when the standby
is promoted. This commit includes a module in src/test/modules with
functions to manipulate the slots (which is not otherwise possible
in SQL code) in order to enable testing, and a new test in
src/test/recovery to ensure that the behavior is as expected.
Author: Craig Ringer Reviewed-By: Oleksii Kliukin, Andres Freund,
Petr Jelínek
http://git.postgresql.org/pg/commitdiff/24c5f1a103ce6656a5cb430d9a996c34e61ab2a5

- Type names should not be quoted. Our actual convention, contrary to
what I said in 59a2111b23f, is not to quote type names, as evidenced
by unquoted use of format_type_be() result value in error messages.
Remove quotes from recently tweaked messages accordingly. Per note
from Tom Lane
http://git.postgresql.org/pg/commitdiff/f402b9950120358d1870aacc10070e121d8a17de

- Improve internationalization of messages involving type names.
Change the slightly different variations of the message function FOO
must return type BAR to a single wording, removing the variability
in type name so that they all create a single translation entry;
since the type name is not to be translated, there's no point in it
being part of the message anyway. Also, change them all to use the
same quoting convention, namely that the function name is not to be
quoted but the type name is. (I'm not quite sure why this is so,
but it's the clear majority.) Some similar messages such as
"encoding conversion function FOO must ..." are also changed.
http://git.postgresql.org/pg/commitdiff/59a2111b23f6ceec4c777d68e20c1027d3c57c6f

- Fix logical_decoding_timelines test crashes. In the
test_slot_timelines test module, we were abusing passing NULL values
which was received as zeroes in x86, but this breaks in ARM
(buildfarm member hamster) by crashing instead. Fix the breakage by
marking these functions as STRICT; the InvalidXid value that was
previously implicit in NULL values (on x86 at least) can now be
passed as 0. Failing to follow the fmgr protocol to check for NULLs
beforehand was causing ARM to fail, as evidenced by segmentation
faults in buildfarm member hamster. In order to use the new
functionality in the test script, use COALESCE in the right spot to
avoid forwarding NULL values. This was diagnosed from the hamster
crash by Craig Ringer, who also proposed a different patch (checking
for NULL values explicitely in the C function code, and keeping the
non-strictness in the C functions). I decided to go with this
approach instead.
http://git.postgresql.org/pg/commitdiff/82c83b337202fa0f5b235bdfaeb992a5cee40ed5

- pgbench: Remove unused parameter. For some reason this parameter
was introduced as unused in 3da0dfb4b146, and has never been used
for anything. Remove it. Author: Fabien Coelho
http://git.postgresql.org/pg/commitdiff/5cb882675ae239db9d00b16a9467c4f900fb10b6

- test_slot_timelines: Fix alternate expected output
http://git.postgresql.org/pg/commitdiff/f07d18b6e94da6ef93dc4e00096f1e7542814fdb

- XLogReader general code cleanup. Some minor tweaks and comment
additions, for cleanliness sake and to avoid having the upcoming
timeline-following patch be polluted with unrelated cleanup.
Extracted from a larger patch by Craig Ringer, reviewed by Andres
Freund, with some additions by myself.
http://git.postgresql.org/pg/commitdiff/3b02ea4f0780ccce7dc116010201dad7ee50a401

Robert Haas pushed:

- pgbench: Support double constants and functions. The new functions
are pi(), random(), random_exponential(), random_gaussian(), and
sqrt(). I was worried that this would be slower than before, but,
if anything, it actually turns out to be slightly faster, because we
now express the built-in pgbench scripts using fewer lines; each
\setrandom can be merged into a subsequent \set. Fabien Coelho
http://git.postgresql.org/pg/commitdiff/86c43f4e22c0771fd0cc6bce2799802c894ee2ec

- Fix typo in comment. Thomas Munro
http://git.postgresql.org/pg/commitdiff/bd0f206f5588767aac2456ebf6a21f7a6344cd58

- On all Windows platforms, not just Cygwin, use _timezone and
_tzname. Up until now, we've been using timezone and tzname, but
Visual Studio 2015 (for which we wish to add support) no longer
declares those symbols. All versions since Visual Studio 2003
apparently support the underscore-equipped names, and we don't
support anything older than Visual Studio 2005, so this should work
OK everywhere. But let's see what the buildfarm thinks. Michael
Paquier, reviewed by Petr Jelinek
http://git.postgresql.org/pg/commitdiff/868628e4fd44d75987d6c099ac63613cc5417629

- Don't require a user mapping for FDWs to work. Commit
fbe5a3fb73102c2cfec11aaaa4a67943f4474383 accidentally changed this
behavior; put things back the way they were, and add some regression
tests. Report by Andres Freund; patch by Ashutosh Bapat, with a bit
of kibitzing by me.
http://git.postgresql.org/pg/commitdiff/5d4171d1c70edfe3e9be1de9e66603af28e3afe1

- Rework custom scans to work more like the new extensible node stuff.
Per discussion, the new extensible node framework is thought to be
better designed than the custom path/scan/scanstate stuff we added
in PostgreSQL 9.5. Rework the latter to be more like the former.
This is not backward-compatible, but we generally don't promise that
for C APIs, and there probably aren't many people using this yet
anyway. KaiGai Kohei, reviewed by Petr Jelinek and me. Some
further cosmetic changes by me.
http://git.postgresql.org/pg/commitdiff/f9143d102ffd0947ca904c62b1d3d6fd587e0c80

- pgbench: Remove \setrandom. You can now do the same thing via \set
using the appropriate function, either random(), random_gaussian(),
or random_exponential(), depending on the desired distribution.
This is not backward-compatible, but per discussion, it's worth it
to avoid having the old syntax hang around forever. Fabien Coelho,
reviewed by Michael Paquier, and adjusted by me.
http://git.postgresql.org/pg/commitdiff/ad9566470b1ba63167d1dc7ae2cb52d88a448f76

- Fix pgbench documentation error. The description of what the
per-transaction log file says for skipped transactions is just plain
wrong. Report and patch by Tomas Vondra, reviewed by Fabien Coelho
and modified by me.
http://git.postgresql.org/pg/commitdiff/d797bf7da2cc954f7b5cd2776b65c6e91cd0cb04

- Improve pgbench docs regarding per-transaction logging. The old
documentation didn't know about the new -b flag, only about -f.
Fabien Coelho
http://git.postgresql.org/pg/commitdiff/7f0a2c85fb221bae6908fb2fddad21a4c6d14438

- Allow aggregate transition states to be serialized and deserialized.
This is necessary infrastructure for supporting parallel aggregation
for aggregates whose transition type is "internal". Such values
can't be passed between cooperating processes, because they are just
pointers. David Rowley, reviewed by Tomas Vondra and by me.
http://git.postgresql.org/pg/commitdiff/5fe5a2cee91117673e04617aeb1a38e305dcd783

- Fix bug in aggregate (de)serialization commit. resulttypeLen and
resulttypeByVal must be set correctly when serializing aggregates,
not just when finalizing them. This was in David's final patch but
I downloaded the wrong version by mistake and failed to spot the
error. David Rowley
http://git.postgresql.org/pg/commitdiff/96f8373cad5d6066baeb7a1c5a88f6f5c9661974

- Add new replication mode synchronous_commit = 'remote_apply'. In
this mode, the master waits for the transaction to be applied on the
remote side, not just written to disk. That means that you can
count on a transaction started on the standby to see all commits
previously acknowledged by the master. To make this work, the
standby sends a reply after replaying each commit record generated
with synchronous_commit >= 'remote_apply'. This introduces a small
inefficiency: the extra replies will be sent even by standbys that
aren't the current synchronous standby. But previously-existing
synchronous_commit levels make no attempt at all to optimize which
replies are sent based on what the primary cares about, so this is
no worse, and at least avoids any extra replies for people not using
the feature at all. Thomas Munro, reviewed by Michael Paquier and
by me. Some additional tweaks by me.
http://git.postgresql.org/pg/commitdiff/314cbfc5da988eff8998655158f84c9815ecfbcd

Magnus Hagander pushed:

- Add missing gss option to msvc config template. Michael Paquier
http://git.postgresql.org/pg/commitdiff/3063e7a84026ced2aadd2262f75eebbe6240f85b

Fujii Masao pushed:

- Use proper format specifier %X/%X for LSN.
http://git.postgresql.org/pg/commitdiff/cee31f5fee950fed4159a4e093dab60b70ef215a

Stephen Frost pushed:

- Reset plan->row_security_env and planUserId. In the plancache, we
check if the environment we planned the query under has changed in a
way which requires us to re-plan, such as when the user for whom the
plan was prepared changes and RLS is being used (and, therefore,
there may be different policies to apply). Unfortunately, while
those values were set and checked, they were not being reset when
the query was re-planned and therefore, in cases where we change
role, re-plan, and then change role again, we weren't re-planning
again. This leads to potentially incorrect policies being applied
in cases where role-specific policies are used and a given query is
planned under one role and then executed under other roles, which
could happen under security definer functions or when a common user
and query is planned initially and then re-used across multiple SET
ROLEs. Further, extensions which made use of CopyCachedPlan() may
suffer from similar issues as the RLS-related fields were not
properly copied as part of the plan and therefore
RevalidateCachedQuery() would copy in the current settings without
invalidating the query. Fix by using the same approach used for
'search_path', where we set the correct values in
CompleteCachedPlan(), check them early on in RevalidateCachedQuery()
and then properly reset them if re-planning. Also, copy through the
values during CopyCachedPlan(). Pointed out by Ashutosh Bapat.
Reviewed by Michael Paquier. Back-patch to 9.5 where RLS was
introduced. Security: CVE-2016-2193
http://git.postgresql.org/pg/commitdiff/86ebf30fd6d8964bbd5d48db053b0a7ff709a0d7

- Fix typo in pg_regress.c. s/afer/after Pointed out by Andreas 'ads'
Scherbaum
http://git.postgresql.org/pg/commitdiff/62b5cd234ba982f71f2501f405a26ed80c92a229

Noah Misch pushed:

- Copyedit comments and documentation.
http://git.postgresql.org/pg/commitdiff/4ad6f13500cd9a3ca0652e43232f9515319e55e2

- Refer to a TOKEN_USER payload as a "token user," not as a "user
token". This corrects messages for can't-happen errors. The
corresponding "user token" appears in the HANDLE argument of
GetTokenInformation().
http://git.postgresql.org/pg/commitdiff/c22650cd6450854e1a75064b698d7dcbb4a8821a

Simon Riggs pushed:

- Avoid pin scan for replay of XLOG_BTREE_VACUUM in all cases. Replay
of XLOG_BTREE_VACUUM during Hot Standby was previously thought to
require complex interlocking that matched the requirements on the
master. This required an O(N) operation that became a significant
problem with large indexes, causing replication delays of seconds or
in some cases minutes while the XLOG_BTREE_VACUUM was replayed.
This commit skips the pin scan that was previously required, by
observing in detail when and how it is safe to do so, with full
documentation. The pin scan is skipped only in replay; the VACUUM
code path on master is not touched here and WAL is identical. The
current commit applies in all cases, effectively replacing commit
687f2cd7a0150647794efe432ae0397cb41b60ff.
http://git.postgresql.org/pg/commitdiff/3e4b7d87988f0835f137f15f5c1a40598dd21f3d

== Rejected Patches (for now) ==

No one was disappointed this week :-)

== Pending Patches ==

Andres Freund and Michaël Paquier traded patches to remove a race
condition that could have caused fsync to have been assumed to have
worked by not actually to have been checked.

SAWADA Masahiko sent in three more revisions of a patch to allow
having N>1 synchronous standby servers.

Alexander Korotkov sent in a patch to prevent a false alarm about
xid wraparound.

Anastasia Lubennikova sent in another revision of a patch to implement
covering + unique indexes.

Peter Geoghegan sent in a patch to add an amcheck extension to
contrib.

Kyotaro HORIGUCHI sent in a patch for an issue in psql to fix the
condition to decide whether to add schema names.

Etsuro Fujita sent in a patch to add GetFdwScanTupleExtraData() and
FillFdwScanTupleSysAttr().

Thomas Munro sent in four more revisions of a patch to implement
"causal reads."

Dilip Kumar and Robert Haas traded patches to help scale relation
extension.

Kyotaro HORIGUCHI and Artur Zakirov traded patches to add tab
completion support for IF [NOT] EXISTS to psql.

Alexander Korotkov sent in five more revisions of a patch to move
PinBuffer and UnpinBuffer to atomics.

Thomas Munro sent in another revision of a patch to add kqueue
support.

Christian Ullrich sent in another revision of a patch to fix an SSPI
authentication failure where the wrong realm name was used.

Fabrízio de Royes Mello and Petr Jelínek traded patches to add a
sequence access method and implement gapless sequences using same.

Teodor Sigaev sent in another revision of a patch to add ICU support.

Alexander Korotkov sent in another revision of a patch to add partial
sort.

Tom Lane sent in a patch to fix an issue that manifested as pg_restore
casting check constraints differently.

Amit Langote sent in a patch to perform constraint name uniqueness
check for index constraints.

Michaël Paquier sent in a patch to add a missing mention of GSSAPI in
MSVC's config_default.pl.

Dmitry Dolgov sent in two more revisions of a patch to add
jsonb_insert().

Ian Lawrence Barwick sent in a patch to fix a replication slot
creation error message in 9.6.

Andres Freund and Amit Kapila traded patches to speed up clog Access
by increasing CLOG buffers.

Tomas Vondra and Dean Rasheed traded patches to improve GROUP BY
estimation.

Pavel Stěhule sent in another revision of a patch to add a raw mode
for COPY.

Aleksander Alekseev sent in a patch to implement a
--disable-setproctitle flag.

Teodor Sigaev and Dmitry Ivanov traded patches to add a phrase search
option to tsearch.

Bernd Helmle sent in a patch to fix an issue where a standalone
backend can PANIC during recovery.

Andres Freund sent in another revision of a patch to avoid the use of
a separate spinlock to protect LWLock's wait queue.

Peter Eisentraut sent in a patch to add an optional SSL indicator to
the psql prompt.

Abhijit Menon-Sen sent in another revision of a patch to work around
some issues in extension dependencies.

Stephen Frost sent in a patch to add new catalog called pg_init_privs,
change pg_dump to use a bitmap to represent what to include, split
"dump" into "dump" and "dump_contains", include pg_catalog and
extension ACLs, if changed, and use the GRANT system to manage access
to sensitive functions.

Kevin Grittner sent in another revision of a patch to implement
"snapshot too old," configured by time.

David Rowley sent in two more revisions of a patch to improve
performance for joins where outer side is unique.

Noah Misch sent in another revision of a patch to refer to a
TOKEN_USER as "token user"

Fabien COELHO sent in a patch to patch which adds a bunch of operators
(bitwise: & | ^ ~, comparisons: =/== <>/!= < <= > >=, logical: and/&&
or/|| xor/^^ not/!) and functions (exp ln if) to pgbench.

Tom Lane sent in a patch to make a better MCV cutoff.