== PostgreSQL Weekly News - August 02 2015 ==

== PostgreSQL Weekly News - August 02 2015 ==

== PostgreSQL Product News ==

cstore_fdw 1.3, a columnar store extension for PostgreSQL, released.
https://www.citusdata.com/blog/18-hadi/184-cstore-fdw-1-3-released

pgbouncer 1.6, a light-weight connection pooler for PostgreSQL, released.
http://pgbouncer.github.io/2015/08/pgbouncer-1-6/

pgCluu 2.4, a Perl program to audit PostgreSQL performance, released.
http://pgcluu.darold.net/

2UDA, a platform for data analytics, released.
http://2ndquadrant.com/2uda

== PostgreSQL Jobs for August ==

http://archives.postgresql.org/pgsql-jobs/2015-08/

== PostgreSQL Local ==

PGDay Campinas 2015 will take place in Campinas on August 7.
http://pgdaycampinas.com.br/english/

PostgresOpen 2015 will being held in Dallas, Texas September 16-18.
http://2015.postgresopen.org/

PostgreSQL Session #7, will be held September 24th, 2015 in Paris,
France,
http://www.postgresql-sessions.org/7/about

PGDay.IT 2015 will take place in Prato on October 23, 2015. The
International Call For Papers is open until August 8.
http://pgday.it

PostgreSQL Conference Europe 2015 will be held on October 27-30 in the
Vienna Marriott Hotel, in Vienna, Austria. The CfP is open until
August 7.
http://2015.pgconf.eu/

PGConf Silicon Valley 2015 is November 17-18 at the South San
Francisco Convention Center.
http://www.pgconfsv.com

PGBR2015 (The Brazilian PostgreSQL Conference) will take place in Porto
Alegre, Rio Grande do Sul, on November 18, 19 and 20.
http://pgbr.postgresql.org.br/2015/en/#call-for-papers

== PostgreSQL in the News ==

Planet PostgreSQL: http://planet.postgresql.org/

PostgreSQL Weekly News is brought to you this week by David Fetter

Submit news and announcements by Sunday at 3:00pm Pacific time.
Please send English language ones to david(at)fetter(dot)org, German language
to pwn(at)pgug(dot)de, Italian language to pwn(at)itpug(dot)org(dot) Spanish language
to pwn(at)arpug(dot)com(dot)ar(dot)

== Applied Patches ==

Heikki Linnakangas pushed:

- Remove false comment about speculative insertion. There is no full
discussion of speculative insertions in the executor README. There
is a high-level explanation in execIndexing.c, but it doesn't seem
necessary to refer it from here. Peter Geoghegan
http://git.postgresql.org/pg/commitdiff/e3a9a194b7cb70a8c01deeff9f876a2631f1be06

- Reuse all-zero pages in GIN. In GIN, an all-zeros page would be
leaked forever, and never reused. Just add them to the FSM in
vacuum, and they will be reinitialized when grabbed from the FSM. On
master and 9.5, attempting to access the page's opaque struct also
caused an assertion failure, although that was otherwise harmless.
Reported by Jeff Janes. Backpatch to all supported versions.
http://git.postgresql.org/pg/commitdiff/334445179c82ba304480ecbd00ae4237587e4baf

- Avoid calling PageGetSpecialPointer() on an all-zeros page. That
was otherwise harmless, but tripped the new assertion in
PageGetSpecialPointer(). Reported by Amit Langote. Backpatch to
9.5, where the assertion was added.
http://git.postgresql.org/pg/commitdiff/65c384c5abee7df6d27f98135790ea24c1b9578b

- Fix memory leak in xlogreader facility. XLogReaderFree failed to
free the per-block data buffers, when they happened to not be used
by the latest read WAL record. Michael Paquier. Backpatch to 9.5,
where the per-block buffers were added.
http://git.postgresql.org/pg/commitdiff/61a65c53bd3e48e7ff7661a528d1791dfd956957

- Don't assume that PageIsEmpty() returns true on an all-zeros page.
It does currently, and I don't see us changing that any time soon,
but we don't make that assumption anywhere else. Per Tom Lane's
suggestion. Backpatch to 9.2, like the previous patch that added
this assumption.
http://git.postgresql.org/pg/commitdiff/820d1ced1b308702b3f811647810b4030f974d89

- Fix handling of all-zero pages in SP-GiST vacuum. SP-GiST
initialized an all-zeros page at vacuum, but that was not
WAL-logged, which is not safe. You might get a torn page write, when
it gets flushed to disk, and end-up with a half-initialized index
page. To fix, leave it in the all-zeros state, and add it to the
FSM. It will be initialized when reused. Also don't set the
page-deleted flag when recycling an empty page. That was also not
WAL-logged, and a torn write of that would cause the page to have an
invalid checksum. Backpatch to 9.2, where SP-GiST indexes were
added.
http://git.postgresql.org/pg/commitdiff/023430abf72eb7d335430e241065d5ed19ddd94b

- Fix memory leaks in pg_rewind. Several PQclear() calls were missing.
Originally reported by Vladimir Borodin in the pg_rewind github
project, patch by Michael Paquier.
http://git.postgresql.org/pg/commitdiff/d7fd22a38ff48372c6813983317fc8d1e07fcf87

- Don't assume that 'char' is signed. On some platforms, notably ARM
and PowerPC, 'char' is unsigned by default. This fixes an assertion
failure at WAL replay on such platforms. Reported by Noah Misch.
Backpatch to 9.5, where this was broken.
http://git.postgresql.org/pg/commitdiff/5533a272ddecff18f16e9b6c307dcb2e130538f4

- Another attempt at fixing memory leak in xlogreader. max_block_id
is also reset between reading records. Michael Paquier
http://git.postgresql.org/pg/commitdiff/5e65f45c6eff8278e2901511da61d3a265adc874

- Fix bug in collecting total_latencies from all threads in pgbench.
This was broken in 1bc90f7a, which removed the thread-emulation.
With modest -j and -c settings the result were usually close enough
that you wouldn't notice it easily, but with a high enough thread
count it would access uninitialized memory and crash. Per report
from Andres Freund offlist.
http://git.postgresql.org/pg/commitdiff/b2ed8edeecd715c8a23ae462da843578e5cde7f4

- Fix typo in comment. Amit Langote
http://git.postgresql.org/pg/commitdiff/a309ebd6b9940ca466c7748eeefc93c86ba2c9d4

- Fix TAP tests with "make installcheck". I neglected that the
prove_installcheck rule also needs to also define PG_REGRESS, like
prove_check does.
http://git.postgresql.org/pg/commitdiff/c7f0b28c7a67a8602c65d6b38d40b38d07548fbd

- Make TAP tests work on Windows. On Windows, use
listen_address=127.0.0.1 to allow TCP connections. We were already
using "pg_regress --config-auth" to set up HBA appropriately. The
standard_initdb helper function now sets up the server's
unix_socket_directories or listen_addresses in the config file, so
that they don't need to be specified in the pg_ctl command line
anymore. That way, the pg_ctl invocations in test programs don't
need to differ between Windows and Unix. Add another helper
function to configure the server's pg_hba.conf to allow replication
connections. The configuration is done similarly to "pg_regress
--config-auth": trust on domain sockets on Unix, and SSPI
authentication on Windows. Replace calls to "cat" and "touch"
programs with built-in perl code, as those programs don't normally
exist on Windows. Add instructions in the docs on how to install
IPC::Run on Windows. Adjust vcregress.pl to not replace PERL5LIB
completely in vcregress.pl, because otherwise cannot install
IPC::Run in a non-standard location easily. Michael Paquier,
reviewed by Noah Misch, some additional tweaking by me.
http://git.postgresql.org/pg/commitdiff/13d856e177e69083f543d6383eeda9e12ce3c55c

- Update ax_pthread.m4 to an experimental draft version from upstream.
The current version is adding a spurious -pthread option on some
Darwin systems that don't need it, which leads to a bunch of
"unrecognized option '-pthread'" warnings. There is a proposed fix
for that in the upstream autoconf archive's bug tracker, see
https://savannah.gnu.org/patch/?8186. This commit updates our
version of ax_pthread.m4 to the "draft2" version proposed there by
Daniel Richard G. I'm using our buildfarm to help Daniel to test
this, before he commits this to the upstream repository.
http://git.postgresql.org/pg/commitdiff/a2932283c2a39efd2f713f647c9f7f3d34123bb3

- Fix calculation of latency of pgbench backslash commands. When we
loop back to the top of doCustom after processing a backslash
command, we must reset the "now" timestamp, because that's used to
calculate the time spent executing the previous command. Report and
fix by Fabien Coelho. Backpatch to 9.5, where this was broken.
http://git.postgresql.org/pg/commitdiff/5515ec0b3a9425a0c0e36e26b5774c1568f22b3a

- Fix race condition that lead to WALInsertLock deadlock with
commit_delay. If a call to WaitForXLogInsertionsToFinish() returned
a value in the middle of a page, and another backend then started to
insert a record to the same page, and then you called
WaitXLogInsertionsToFinish() again, the second call might return a
smaller value than the first call. The problem was in
GetXLogBuffer(), which always updated the insertingAt value to the
beginning of the requested page, not the actual requested location.
Because of that, the second call might return a xlog pointer to the
beginning of the page, while the first one returned a later position
on the same page. XLogFlush() performs two calls to
WaitXLogInsertionsToFinish() in succession, and holds WALWriteLock
on the second call, which can deadlock if the second call to
WaitXLogInsertionsToFinish() blocks. Reported by Spiros Ioannou.
Backpatch to 9.4, where the more scalable WALInsertLock mechanism,
and this bug, was introduced.
http://git.postgresql.org/pg/commitdiff/358cde320b17535011b0a912e5c9ce1453666ed1

- Fix output of ISBN-13 numbers beginning with 979. An EAN beginning
with 979 (but not 9790 - those are ISMN's) are accepted as ISBN
numbers, but they cannot be represented in the old, 10-digit ISBN
format. They must be output in the new 13-digit ISBN-13 format. We
printed out an incorrect value for those. Also add a regression
test, to test this and some other basic functionality of the module.
Patch by Fabien Coelho. This fixes bug #13442, reported by B.Z.
Backpatch to 9.1, where we started to recognize ISBN-13 numbers.
http://git.postgresql.org/pg/commitdiff/cb3384a0cb4cf900622b77865f60e31259923079

Tom Lane pushed:

- Fix pointer-arithmetic thinko in pg_stat_ssl patch. Nasty
memory-stomp bug in commit 9029f4b37406b21a. It's not apparent how
this survived even cursory testing :-(. Per report from Peter
Holzer.
http://git.postgresql.org/pg/commitdiff/7d791ed49be28f2dfdd31e262dc35c4595718267

- Further code review for pg_stat_ssl patch. Fix additional bogosity
in commit 9029f4b37406b21a. Include the BackendSslStatusBuffer in
the BackendStatusShmemSize calculation, avoid ugly and error-prone
casts to char* and back, put related code stanzas into a consistent
order (and fix a couple of previous instances of that sin). All
cosmetic except for the size oversight.
http://git.postgresql.org/pg/commitdiff/4c8f8ffaca42bb5678a0858ce56f9a384c21c583

- Remove an unsafe Assert, and explain join_clause_is_movable_into()
better. join_clause_is_movable_into() is approximate, in the sense
that it might sometimes return "false" when actually it would be
valid to push the given join clause down to the specified level.
This is okay ... but there was an Assert in
get_joinrel_parampathinfo() that's only safe if the answers are
always exact. Comment out the Assert, and add a bunch of commentary
to clarify what's going on. Per fuzz testing by Andreas
Seltenreich. The added regression test is a pretty silly query, but
it's based on his crasher example. Back-patch to 9.2 where the
faulty logic was introduced.
http://git.postgresql.org/pg/commitdiff/95f4e59c3286671656aff7db45b322f14a7bb8cc

- Reduce chatter from signaling of autovacuum workers. Don't print a
WARNING if we get ESRCH from a kill() that's attempting to cancel an
autovacuum worker. It's possible (and has been seen in the
buildfarm) that the worker is already gone by the time we are able
to execute the kill, in which case the failure is harmless. About
the only plausible reason for reporting such cases would be to help
debug corrupted lock table contents, but this is hardly likely to be
the most important symptom if that happens. Moreover issuing a
WARNING might scare users more than is warranted. Also, since
sending a signal to an autovacuum worker is now entirely a routine
thing, and the worker will log the query cancel on its end anyway,
reduce the message saying we're doing that from LOG to DEBUG1 level.
Very minor cosmetic cleanup as well. Since the main practical
reason for doing this is to avoid unnecessary buildfarm failures,
back-patch to all active branches.
http://git.postgresql.org/pg/commitdiff/d8f15c95bec50f552ad0b13ea6eafb6975126184

- Update our documentation concerning where to create data
directories. Although initdb has long discouraged use of a
filesystem mount-point directory as a PG data directory, this point
was covered nowhere in the user-facing documentation. Also, with
the popularity of pg_upgrade, we really need to recommend that the
PG user own not only the data directory but its parent directory
too. (Without a writable parent directory, operations such as "mv
data data.old" fail immediately. pg_upgrade itself doesn't do that,
but wrapper scripts for it often do.) Hence, adjust the "Creating a
Database Cluster" section to address these points. I also took the
liberty of wordsmithing the discussion of NFS a bit. These
considerations aren't by any means new, so back-patch to all
supported branches.
http://git.postgresql.org/pg/commitdiff/8c72a7fab47a7f501d211468d6e477e1f3a20599

- Suppress "variable may be used uninitialized" warning. Also
re-pgindent, just because I'm a neatnik.
http://git.postgresql.org/pg/commitdiff/2c698f438acc0b4ebc73b27da734e488a5a967cc

- Prevent platform-dependent output row ordering in a new test query.
Buildfarm indicates this is necessary.
http://git.postgresql.org/pg/commitdiff/5d0e8bc9e036bffb88b7598aa4d46b7368a10f0c

- Add some test coverage of EvalPlanQual with non-locked tables. A
Salesforce colleague of mine griped that the regression tests don't
exercise EvalPlanQualFetchRowMarks() and allied routines. Which is
a fair complaint. Add test cases that go through the REFERENCE and
COPY code paths. Unfortunately we don't have sufficient
infrastructure right now to exercise the FDW code path in the
isolation tests, but this is surely better than before.
http://git.postgresql.org/pg/commitdiff/342a1ffa2177c0b260b68d7891b0d04598a58432

- Avoid some zero-divide hazards in the planner. Although I think on
all modern machines floating division by zero results in Infinity
not SIGFPE, we still don't want infinities running around in the
planner's costing estimates; too much risk of that leading to insane
behavior. grouping_planner() failed to consider the possibility
that final_rel might be known dummy and hence have zero rowcount.
(I wonder if it would be better to set a rows estimate of 1 for
dummy relations? But at least in the back branches, changing this
convention seems like a bad idea, so I'll leave that for another
day.) Make certain that get_variable_numdistinct() produces a
nonzero result. The case that can be shown to be broken is with
stadistinct < 0.0 and small ntuples; we did not prevent the result
from rounding to zero. For good luck I applied clamp_row_est() to
all the nonconstant return values. In ExecChooseHashTableSize(),
Assert that we compute positive nbuckets and nbatch. I know of no
reason to think this isn't the case, but it seems like a good safety
check. Per reports from Piotr Stefaniak. Back-patch to all active
branches.
http://git.postgresql.org/pg/commitdiff/8693ebe37d65ec1bc1ebeab36f60b38f18fb25d6

- Fix an oversight in checking whether a join with LATERAL refs is
legal. In many cases, we can implement a semijoin as a plain
innerjoin by first passing the righthand-side relation through a
unique-ification step. However, one of the cases where this does
NOT work is where the RHS has a LATERAL reference to the LHS; that
makes the RHS dependent on the LHS so that unique-ification is
meaningless. joinpath.c understood this, and so would not generate
any join paths of this kind ... but join_is_legal neglected to check
for the case, so it would think that we could do it. The upshot
would be a "could not devise a query plan for the given query"
failure once we had failed to generate any join paths at all for the
bogus join pair. Back-patch to 9.3 where LATERAL was added.
http://git.postgresql.org/pg/commitdiff/a6492ff8970b06b9e27cc314c7d1aa574fcc7b04

- Teach predtest.c that "foo" implies "foo IS NOT NULL". Per
complaint from Peter Holzer. It's useful to cover this special
case, since for a boolean variable "foo", earlier parts of the
planner will have reduced variants like "foo = true" to just "foo",
and thus we may fail to recognize the applicability of a partial
index with predicate "foo IS NOT NULL". Back-patch to 9.5, but not
further; given the lack of previous complaints this doesn't seem
like behavior to change in stable branches.
http://git.postgresql.org/pg/commitdiff/dea1491ffb448d20764a5f2cec8ae33b64dd39f8

- Fix some planner issues with degenerate outer join clauses. An
outer join clause that didn't actually reference the RHS (perhaps
only after constant-folding) could confuse the join order
enforcement logic, leading to wrong query results. Also, nested
occurrences of such things could trigger an Assertion that on
reflection seems incorrect. Per fuzz testing by Andreas
Seltenreich. The practical use of such cases seems thin enough that
it's not too surprising we've not heard field reports about it.
This has been broken for a long time, so back-patch to all active
branches.
http://git.postgresql.org/pg/commitdiff/f69b4b9495269cc4957bac0f10aaada4d4cfa61e

- Fix incorrect order of lock file removal and failure to close()
sockets. Commit c9b0cbe98bd783e24a8c4d8d8ac472a494b81292
accidentally broke the order of operations during postmaster
shutdown: it resulted in removing the per-socket lockfiles after,
not before, postmaster.pid. This creates a race-condition hazard
for a new postmaster that's started immediately after observing that
postmaster.pid has disappeared; if it sees the socket lockfile still
present, it will quite properly refuse to start. This error appears
to be the explanation for at least some of the intermittent
buildfarm failures we've seen in the pg_upgrade test. Another
problem, which has been there all along, is that the postmaster has
never bothered to close() its listen sockets, but has just allowed
them to close at process death. This creates a different race
condition for an incoming postmaster: it might be unable to bind to
the desired listen address because the old postmaster is still
incumbent. This might explain some odd failures we've seen in the
past, too. (Note: this is not related to the fact that individual
backends don't close their client communication sockets. That
behavior is intentional and is not changed by this patch.) Fix by
adding an on_proc_exit function that closes the postmaster's ports
explicitly, and (in 9.3 and up) reshuffling the responsibility for
where to unlink the Unix socket files. Lock file unlinking can stay
where it is, but teach it to unlink the lock files in reverse order
of creation.
http://git.postgresql.org/pg/commitdiff/d73d14c271653dff10c349738df79ea03b85236c

- Avoid calling memcpy() with a NULL source pointer and count == 0.
As in commit 0a52d378b03b7d5a, avoid doing something that has
undefined results according to the C standard, even though in
practice there does not seem to be any problem with it. This fixes
two places in numeric.c that demonstrably could call memcpy() with
such arguments. I looked through that file and didn't see any other
places with similar hazards; this is not to claim that there are not
such places in other files. Per report from Piotr Stefaniak.
Back-patch to 9.5 which is where the previous commit was added.
We're more or less setting a precedent that we will not worry about
this type of issue in pre-9.5 branches unless someone demonstrates a
problem in the field.
http://git.postgresql.org/pg/commitdiff/13bba02271dce865cd20b6f49224889c73fed4e7

Stephen Frost pushed:

- Improve RLS handling in copy.c To avoid a race condition where the
relation being COPY'd could be changed into a view or otherwise
modified, keep the original lock on the relation. Further, fully
qualify the relation when building the query up. Also remove the
poorly thought-out Assert() and check the entire relationOids list
as, post-RLS, there can certainly be multiple relations involved and
the planner does not guarantee their ordering. Per discussion with
Noah and Andres. Back-patch to 9.5 where RLS was introduced.
http://git.postgresql.org/pg/commitdiff/3d5cb31c9a17bad8bbc587c09404b2aa1e6c3aac

Joe Conway pushed:

- Fix pg_dump output of policies. pg_dump neglected to wrap
parenthesis around USING and WITH CHECK expressions -- fixed.
Reported by Noah Misch.
http://git.postgresql.org/pg/commitdiff/e0d4a290f4a24b0a0a46af4d0f8551f8ee5e5513

- Plug RLS related information leak in pg_stats view. The pg_stats
view is supposed to be restricted to only show rows about tables the
user can read. However, it sometimes can leak information which
could not otherwise be seen when row level security is enabled. Fix
that by not showing pg_stats rows to users that would be subject to
RLS on the table the row is related to. This is done by
creating/using the newly introduced SQL visible function,
row_security_active(). Along the way, clean up three call sites of
check_enable_rls(). The second argument of that function should only
be specified as other than InvalidOid when we are checking as a
different user than the current one, as in when querying through a
view. These sites were passing GetUserId() instead of InvalidOid,
which can cause the function to return incorrect results if the
current user has the BYPASSRLS privilege and row_security has been
set to OFF. Additionally fix a bug causing RI Trigger error
messages to unintentionally leak information when RLS is enabled,
and other minor cleanup and improvements. Also add WITH
(security_barrier) to the definition of pg_stats. Bumped CATVERSION
due to new SQL functions and pg_stats view definition. Back-patch
to 9.5 where RLS was introduced. Reported by Yaroslav. Patch by Joe
Conway and Dean Rasheed with review and input by Michael Paquier and
Stephen Frost.
http://git.postgresql.org/pg/commitdiff/7b4bfc87d5e73c94ae1591c482f626a011498915

- Bump catversion so that HEAD is beyond 9.5. As pointed out by Tom,
since HEAD has progressed beyond 9.5 in terms of its catalog, we
need to be sure catversion of HEAD is advanced beyond that of 9.5.
Corrects my mistake in the pg_stats view commit cfa928ff.
http://git.postgresql.org/pg/commitdiff/1e2bd43b31cdb6d0425430f1568814bce588d8fd

- Create a pg_shdepend entry for each role in TO clause of policies.
CreatePolicy() and AlterPolicy() omit to create a pg_shdepend entry
for each role in the TO clause. Fix this by creating a new shared
dependency type called SHARED_DEPENDENCY_POLICY and assigning it to
each role. Reported by Noah Misch. Patch by me, reviewed by Alvaro
Herrera. Back-patch to 9.5 where RLS was introduced.
http://git.postgresql.org/pg/commitdiff/f781a0f1d88411978c9df5f05cbb4f46aabe3d24

- Disallow converting a table to a view if row security is present.
When DefineQueryRewrite() is about to convert a table to a view, it
checks the table for features unavailable to views. For example, it
rejects tables having triggers. It omits to reject tables having
relrowsecurity or a pg_policy record. Fix that. To faciliate the
repair, invent relation_has_policies() which indicates the presence
of policies on a relation even when row security is disabled for
that relation. Reported by Noah Misch. Patch by me, review by
Stephen Frost. Back-patch to 9.5 where RLS was introduced.
http://git.postgresql.org/pg/commitdiff/d824e2800f66f6180189d973c720611855c6f619

- Add missing post create and alter hooks to policy objects.
AlterPolicy() and CreatePolicy() lacked their respective hook
invocations. Noted by Noah Misch, review by Dean Rasheed.
Back-patch to 9.5 where RLS was introduced.
http://git.postgresql.org/pg/commitdiff/efe72a82aa5e0d6f907a5d21f1a99b677f44f817

- Create new ParseExprKind for use by policy expressions. Policy
USING and WITH CHECK expressions were using EXPR_KIND_WHERE for
parse analysis, which results in inappropriate ERROR messages when
the expression contains unsupported constructs such as aggregates.
Create a new ParseExprKind called EXPR_KIND_POLICY and tailor the
related messages to fit. Reported by Noah Misch. Reviewed by Dean
Rasheed, Alvaro Herrera, and Robert Haas. Back-patch to 9.5 where
RLS was introduced.
http://git.postgresql.org/pg/commitdiff/632cd9f892119858bc5b617bb60c0377a8a2ed13

- Use appropriate command type when retrieving relation's policies.
When retrieving policies, if not working on the root target
relation, we actually want the relation's SELECT policies,
regardless of the top level query command type. For example in
UPDATE t1...FROM t2 we need to apply t1's UPDATE policies and t2's
SELECT policies. Previously top level query command type was
applied to all relations, which was wrong. Add some regression
coverage to ensure we don't violate this principle in the future.
Report and patch by Dean Rasheed. Cherry picked from larger
refactoring patch and tweaked by me. Back-patched to 9.5 where RLS
was introduced.
http://git.postgresql.org/pg/commitdiff/1e15b212290bf6daff752f20e9e0356e4dac6e09

- Improve CREATE FUNCTION doc WRT to LEAKPROOF RLS interaction. Patch
by Dean Rasheed. Back-patched to 9.5 where RLS was introduced.
http://git.postgresql.org/pg/commitdiff/d6314b20cd872a542d71738df54a906d2962abb8

Robert Haas pushed:

- Centralize decision-making about where to get a backend's PGPROC.
This code was originally written as part of parallel query effort,
but it seems to have independent value, because if we make one
decision about where to get a PGPROC when we allocate and then put
it back on a different list at backend-exit time, bad things happen.
This isn't just a theoretical risk; we fixed an actual problem of
this type in commit e280c630a87e1b8325770c6073097d109d79a00f.
http://git.postgresql.org/pg/commitdiff/6f2871f12e9fba5deec4296cfe12e85c140261c4

- Document how to build the docs using the website style. Craig
Ringer
http://git.postgresql.org/pg/commitdiff/5f1066074cd85b829371a4123839d16f52553c3d

- Flesh out the background worker documentation. Make it more clear
that bgw_main is usually not what you want. Put the background
worker flags in a variablelist rather than having them as part of a
paragraph. Explain important limits on how bgw_main_arg can be
used. Craig Ringer, substantially revised by me.
http://git.postgresql.org/pg/commitdiff/38d4ce6b055ec7c26c421dcf1ced07e4d4220292

- Fix incorrect comment. Amit Langote
http://git.postgresql.org/pg/commitdiff/f04ce3147560de90d5250ae822d8d714b76f61cd

Andrew Dunstan pushed:

- Make tap tests store postmaster logs and handle vpaths correctly.
Given this it is possible that the buildfarm animals running these
tests will be able to capture adequate logging to allow diagnosis of
failures.
http://git.postgresql.org/pg/commitdiff/01f6bb4b2dcc571ad0cc6a404595de4b03157534

- Only adjust negative indexes in json_get up to the length of the
path. The previous code resulted in memory access beyond the path
bounds. The cure is to move it into a code branch that checks the
value of lex_level is within the correct bounds. Bug reported and
diagnosed by Piotr Stefaniak.
http://git.postgresql.org/pg/commitdiff/6d10f4e9d7f0051afb60c42409f2fe61cf4da348

- Add IF NOT EXISTS processing to ALTER TABLE ADD COLUMN. Fabrízio de
Royes Mello, reviewed by Payal Singh, Alvaro Herrera and Michael
Paquier.
http://git.postgresql.org/pg/commitdiff/2cd40adb85d1d5a7fd448721b326e73c1e318750

Andres Freund pushed:

- Remove ssl renegotiation support. While postgres' use of SSL
renegotiation is a good idea in theory, it turned out to not work
well in practice. The specification and openssl's implementation of
it have lead to several security issues. Postgres' use of
renegotiation also had its share of bugs. Additionally OpenSSL has
a bunch of bugs around renegotiation, reported and open for years,
that regularly lead to connections breaking with obscure error
messages. We tried increasingly complex workarounds to get around
these bugs, but we didn't find anything complete. Since these
connection breakages often lead to hard to debug problems, e.g.
spuriously failing base backups and significant latency spikes when
synchronous replication is used, we have decided to change the
default setting for ssl renegotiation to 0 (disabled) in the
released backbranches and remove it entirely in 9.5 and master.
Author: Andres Freund Discussion:
20150624144148(dot)GQ4797(at)alap3(dot)anarazel(dot)de Backpatch: 9.5 and master,
9.0-9.4 get a different patch
http://git.postgresql.org/pg/commitdiff/426746b93093a0fef53b23ce4b6421bed28e5c60

- Remove outdated comment in LWLockDequeueSelf's header. Noticed-By:
Robert Haas. Backpatch: 9.5, where the function was added.
http://git.postgresql.org/pg/commitdiff/3bc9356ddd52443e209fb60b94a2c29d66259d62

- Fix issues around the "variable" support in the lwlock
infrastructure. The lwlock scalability work introduced two race
conditions into the lwlock variable support provided for xlog.c.
First, and harmlessly on most platforms, it set/read the variable
without the spinlock in some places. Secondly, due to the removal of
the spinlock, it was possible that a backend missed changes to the
variable's state if it changed in the wrong moment because checking
the lock's state, the variable's state and the queuing are not
protected by a single spinlock acquisition anymore. To fix first
move resetting the variable's from LWLockAcquireWithVar to
WALInsertLockRelease, via a new function LWLockReleaseClearVar. That
prevents issues around waiting for a variable's value to change when
a new locker has acquired the lock, but not yet set the value.
Secondly re-check that the variable hasn't changed after enqueing,
that prevents the issue that the lock has been released and already
re-acquired by the time the woken up backend checks for the lock's
state. Reported-By: Jeff Janes. Analyzed-By: Heikki Linnakangas.
Reviewed-By: Heikki Linnakangas. Discussion:
5592DB35(dot)2060401(at)iki(dot)fi(dot) Backpatch: 9.5, where the lwlock
scalability went in.
http://git.postgresql.org/pg/commitdiff/7039760114da45552043f8fa229928e071650173

- Micro optimize LWLockAttemptLock() a bit. LWLockAttemptLock
pointlessly read the lock's state in every loop iteration, even
though pg_atomic_compare_exchange_u32() returns the old value.
Instead do that only once before the loop iteration. Additionally
there's no need to have the expected_state variable, old_state
mostly had the same value anyway. Noticed-By: Heikki Linnakangas.
Backpatch: 9.5, no reason to let the branches diverge at this point.
http://git.postgresql.org/pg/commitdiff/a4b09af3e93707351f1d4b8efafabfa135c0f70b

Peter Eisentraut pushed:

- pg_basebackup: Add tests for -R option. Reviewed-by: Michael
Paquier <michael(dot)paquier(at)gmail(dot)com>
http://git.postgresql.org/pg/commitdiff/36dc30aa7e0ab9628a5a975a921b8ff82e8f31bf

- pg_basebackup: Add --slot option. This option specifies a
replication slot for WAL streaming (-X stream), so that there can be
continuous replication slot use between WAL streaming during the
base backup and the start of regular streaming replication.
Reviewed-by: Michael Paquier <michael(dot)paquier(at)gmail(dot)com>
http://git.postgresql.org/pg/commitdiff/0dc848b0314d63188919f1ce943730eac684dccd

- pg_basebackup: Add tests for -X option. Reviewed-by: Michael
Paquier <michael(dot)paquier(at)gmail(dot)com>
http://git.postgresql.org/pg/commitdiff/90102bb538831fca734cc9fa1450b97c7d4e1190

Noah Misch pushed:

- MSVC: Future-proof installation file skip logic. This code relied
on knowing exactly where in the source tree temporary installations
might appear. A reasonable hacker may not think to update this code
when adding use of a temporary installation, making it fragile.
Observe that commit 9fa8b0ee90c44c0f97d16bf65e94322988c94864 broke
it unnoticed, and commit dcae5faccab64776376d354decda0017c648bb53
fixed it unnoticed. Back-patch to 9.5 only; use of temporary
installations is unlikely to change in released versions.
http://git.postgresql.org/pg/commitdiff/d69252285c618cf2e4bf660dbae7c6f4d15e9ce4

- Blacklist xlc 32-bit inlining. Per a suggestion from Tom Lane.
Back-patch to 9.0 (all supported versions). While only 9.4 and up
have code known to elicit this compiler bug, we were disabling
inlining by accident until commit
43d89a23d59c487bc9258fad7a6187864cb8c0c0.
http://git.postgresql.org/pg/commitdiff/c53f73879f552a3c7cc57d2d6a49d38cd80cf267

- MSVC: Revert most 9.5 changes to pre-9.5 vcregress.pl tests. The
reverted changes did not narrow the semantic gap between the MSVC
build system and the GNU make build system. For targets old and new
that run multiple suites (contribcheck, modulescheck, tapcheck),
restore vcregress.pl to mimicking "make -k" rather than the "make
-S" default. Lack of "-k" would be more burdensome than lack of
"-S". Keep changes reflecting contemporary changes to the GNU make
build system, and keep updates to Makefile parsing. Keep the loss
of --psqldir in "check" and "ecpgcheck" targets; it had been a no-op
when used alongside --temp-install. No log message mentioned any of
the reverted changes. Based on a germ by Michael Paquier.
Back-patch to 9.5.
http://git.postgresql.org/pg/commitdiff/e6ea46c30ec9d086d3317d293b1e249f21e43fbc

- Remove redundant "make install" from pg_upgrade test suite. A
top-level "make install" includes pg_upgrade since commit
9fa8b0ee90c44c0f97d16bf65e94322988c94864. Back-patch to 9.5, where
that commit first appeared.
http://git.postgresql.org/pg/commitdiff/021a56989aa5cbf10854ae3a0a17722e6ce1f9f9

- MSVC: Remove duplicate PATH entry in test harness. Back-patch to
9.5, where commit 4cb7d671fddc8855c8def2de51fb23df1c8ac0af
introduced it.
http://git.postgresql.org/pg/commitdiff/d6ab14674d636490cec40c9dfc16104f380ced70

- Clean up Makefile.win32 "-I" flag additions. The PGXS-case
directory does not exist in the non-PGXS case, and vice versa. Add
one or the other, not both. This is essentially cosmetic. It makes
Makefile.win32 more like the similar Makefile.global code.
http://git.postgresql.org/pg/commitdiff/16c4e6d8dc8fb98cd32d9af473fb06a730fb40d3

- Consolidate makefile code for setting top_srcdir, srcdir and VPATH.
Responsibility was formerly split between Makefile.global and
pgxs.mk. As a result of commit
b58233c71b93a32fcab7219585cafc25a27eb769, in the PGXS case, these
variables were unset while parsing Makefile.global and callees.
Inclusion of Makefile.custom did not work from PGXS, and the subtle
difference seemed like a recipe for future bugs. Back-patch to 9.4,
where that commit first appeared.
http://git.postgresql.org/pg/commitdiff/5da944fb4683762b8ccebbf7599f79db01e63132

Álvaro Herrera pushed:

- Fix broken assertion in BRIN code. The code was assuming that any
NULL value in scan keys was due to IS NULL or IS NOT NULL, but it
turns out to be possible to get them with other operators too, if
they are used in contrived-enough ways. Easiest way out of the
problem seems to check explicitely for the IS NOT NULL flag, instead
of assuming it must be set if the IS NULL flag is not set, when a
null scan key is found; if neither flag is set, follow the lead of
other index AMs and assume that all indexable operators must be
strict, and thus the query is never satisfiable. Also, add a
comment to try and lure some future hacker into improving analysis
of scan keys in brin. Per report from Andreas Seltenreich;
diagnosis by Tom Lane. Backpatch to 9.5. Discussion:
http://www.postgresql.org/message-id/20646.1437919632@sss.pgh.pa.us
http://git.postgresql.org/pg/commitdiff/c81276241b61d8759e024ed803e8f3f251d8e7c9

- Fix volatility marking of commit timestamp functions. They are
marked stable, but since they act on instantaneous state and it is
possible to consult state of transactions as they commit, the
results could change mid-query. They need to be marked volatile,
and this commit does so. There would normally be a catversion bump
here, but this is so much a niche feature and I don't believe
there's real damage from the incorrect marking, that I refrained.
Backpatch to 9.5, where commit timestamps where introduced. Per
note from Fujii Masao.
http://git.postgresql.org/pg/commitdiff/e8e86fbc8b3619da54c485cf05272ccf1dac1a7d

== Rejected Patches (for now) ==

No one was disappointed this week :-)

== Pending Patches ==

Thomas Munro sent in a doc patch to highlight the fact that the
special logic for "<row-valued-expression> IS [ NOT ] NULL" doesn't
apply anywhere else that we handle nulls or talk about [non]-null
values in the manual.

David Rowley and Heikki Linnakangas traded patches to allow multiple
aggregate functions in a query share state.

Ildus Kurbangaliev sent in another revision of a patch to replace
pg_stat_activity.waiting with something more descriptive.

Andres Freund and David Rowley traded patches to make timestamptz less
slow.

Fabien COELHO sent in a patch to fix some cases where pgbench could
produce negative latencies.

Qingqing Zhou sent in a patch to add some planner debugging
infrastructure.

Fabien COELHO sent in two more revisions of a patch to add pgbench
stats per script, etc.

SAWADA Masahiko sent in another revision of a patch to add a "frozen"
bit to the visibility map.

Pavel Stěhule sent in a patch to allow multiple commands to be
specified separately on the command line to psql.

Michael Paquier sent in another revision of a patch to add TAP tests
for MSVC builds.

Michael Paquier sent in two revisions of a patch to prevent starting
streaming after creating a slot in pg_receivexlog.

Michael Paquier sent in a patch to add a TAP test for pg_dump checking
data dump of extension tables.

Alexander Shulgin sent in a patch to add ddl_deparse core support.

Ashutosh Bapat sent in another revision of a patch to allow
transaction control across multiple nodes with FDWs.

Peter Geoghegan sent in two revisions of a patch to use quicksort and
a merge step to significantly improve on tuplesort's single run
"external sort".

Heikki Linnakangas sent in a patch to allow pgmemcache to create a
temporary installation.

Kaigai Kouhei sent in a patch to fix a bug where
customScan->custom_plans were not copied.

Tom Lane sent in a patch to add type-of-expression notation.

Kyotaro HORIGUCHI sent in another revision of a patch to add
multivariate statistics.

Anastasia Lubennikova sent in a patch to add microvacuum support for
the GiST access method.

Amit Kapila sent in another flock of patches intended to reduce
ProcArrayLock contention.

Michael Paquier sent in another revision of a patch to implement
CREATE EXTENSION ... CASCADE.

Josh Berkus sent in a patch to prevent users from setting wal_buffers
over 2GB bytes.

Fabrízio de Royes Mello sent in three more revisions of a patch to
change ALTER TABLE SET to reduce the lock level for autovac
reloptions.

Jesper Pedersen sent in two revisions of a patch to require extensions
to specify the correct number of LWLock's needed.

Álvaro Herrera sent in a patch to fix some infelicities between brin
index vacuum and transaction snapshots.

Tom Lane sent in a patch to clean up the missing ERRCODE assignments.

Peter Geoghegan sent in a patch to fix an incorrect comment about
abbreviated keys.

Nikolay Shaplov sent in a patch to for pageinspect that allows seeing
data stored in the tuple.

Noah Misch sent in a patch to fix some nodes/*funcs.c inconsistencies.