== PostgreSQL Weekly News - February 28 2016 ==

== PostgreSQL Weekly News - February 28 2016 ==

== PostgreSQL Product News ==

Ora2PG 17.0 a migration system for moving from Oracle and MySQL to
PostgreSQL, released.
http://ora2pg.darold.net/

pgBadger 8.0, a parallel PostgreSQL log analyzer written in Perl, released:
https://github.com/dalibo/pgbadger/blob/master/ChangeLog
Development:
https://github.com/dalibo/pgbadger/

pgbouncer 1.7.2, a light-weight connection pooler for PostgreSQL,
released.
https://pgbouncer.github.io/changelog.html

== PostgreSQL Jobs for February ==

http://archives.postgresql.org/pgsql-jobs/2016-02/

== PostgreSQL Local ==

The first pan-Asian PostgreSQL conference will be held March 17-19,
2016 in Singapore. Registration is open.
http://2016.pgday.asia/

Nordic PGDay 2016 is a one day one track conference which will be held in
Helsinki, Finland, on March 17, 2016. Registration is still open.
http://2016.nordicpgday.org/

PGDay Paris 2016, on March 31 in Paris, has opened registration.
http://www.pgday.paris/registration/ (scroll down for English)

The 8th PostgreSQL Session will be held on April 6th, 2016, in Lyon,
France. The CfP is open until February 29, 2016 at call-for-paper
<AT> postgresql-sessions <DOT> org.

PGConf US 2016 will take place April 18-20, 2016 in NYC. Registration
is open.
http://www.pgconf.us/2016/

LinuxFest Northwest will take place April 23-24, 2016 at Bellingham
Technical College in Bellingham, Washington, USA. The CfP is now
open.
http://www.linuxfestnorthwest.org/2016/present

FOSS4G NA, will be held May 2-5, 2016 in Raleigh, North Carolina.
The CfP is still open.
https://2016.foss4g-na.org/cfp

PGCon 2016 will be held May 17-21, 2016 in Ottawa.
http://www.pgcon.org/

This year's Swiss PGDay will be held on June 24, 2016 at the
University of Applied Sciences in Rapperswil (Switzerland).
The CfP is open.
http://www.pgday.ch/

"5432 ... Meet us!", will take place in Milan, Italy on June 28-29, 2016.
The CfP is open until February 28th, 2016.
http://5432meet.us/

== PostgreSQL in the News ==

Planet PostgreSQL: http://planet.postgresql.org/

PostgreSQL Weekly News is brought to you this week by David Fetter

Submit news and announcements by Sunday at 3:00pm Pacific time.
Please send English language ones to david(at)fetter(dot)org, German language
to pwn(at)pgug(dot)de, Italian language to pwn(at)itpug(dot)org(dot) Spanish language
to pwn(at)arpug(dot)com(dot)ar(dot)

== Applied Patches ==

Andres Freund pushed:

- Fix wrong keysize in PrivateRefCountHash creation. In 4b4b680c3 I
accidentally used sizeof(PrivateRefCountArray) instead of
sizeof(PrivateRefCountEntry) when creating the refcount overflow
hashtable. As the former is bigger than the latter, this luckily
only resulted in a slightly increased memory usage when many buffers
are pinned in a backend. Reported-By: Takashi Horikawa Discussion:
73FA3881462C614096F815F75628AFCD035A48C3(at)BPXM01GP(dot)gisp(dot)nec(dot)co(dot)jp
Backpatch: 9.5, where thew new ref count infrastructure was
introduced
http://git.postgresql.org/pg/commitdiff/ea56b06cf77a6932a74f9d4ec6c950a333d1527d

Tom Lane pushed:

- Remove redundant PGPROC.lockGroupLeaderIdentifier field. We don't
really need this field, because it's either zero or redundant with
PGPROC.pid. The use of zero to mark "not a group leader" is not
necessary since we can just as well test whether lockGroupLeader is
NULL. This does not save very much, either as to code or data, but
the simplification seems worthwhile anyway.
http://git.postgresql.org/pg/commitdiff/73bf8715aa7430bd003516bde448507fbe789c05

- Create a function to reliably identify which sessions block which
others. This patch introduces "pg_blocking_pids(int) returns
int[]", which returns the PIDs of any sessions that are blocking the
session with the given PID. Historically people have obtained such
information using a self-join on the pg_locks view, but it's
unreasonably tedious to do it that way with any modicum of
correctness, and the addition of parallel queries has pretty much
broken that approach altogether. (Given some more columns in the
view than there are today, you could imagine handling parallel-query
cases with a 4-way join; but ugh.) The new function has the
following behaviors that are painful or impossible to get right via
pg_locks: 1. Correctly understands which lock modes block which
other ones. 2. In soft-block situations (two processes both waiting
for conflicting lock modes), only the one that's in front in the
wait queue is reported to block the other. 3. In parallel-query
cases, reports all sessions blocking any member of the given PID's
lock group, and reports a session by naming its leader process's
PID, which will be the pg_backend_pid() value visible to clients.
The motivation for doing this right now is mostly to fix the
isolation tests. Commit 38f8bdcac4982215beb9f65a19debecaf22fd470
lobotomized isolationtester's is-it-waiting query by removing its
ability to recognize nonconflicting lock modes, as a crude
workaround for the inability to handle soft-block situations
properly. But even without the lock mode tests, the old query was
excessively slow, particularly in CLOBBER_CACHE_ALWAYS builds; some
of our buildfarm animals fail the new deadlock-hard test because the
deadlock timeout elapses before they can probe the waiting status of
all eight sessions. Replacing the pg_locks self-join with use of
pg_blocking_pids() is not only much more correct, but a lot faster:
I measure it at about 9X faster in a typical dev build with Asserts,
and 3X faster in CLOBBER_CACHE_ALWAYS builds. That should provide
enough headroom for the slower CLOBBER_CACHE_ALWAYS animals to pass
the test, without having to lengthen deadlock_timeout yet more and
thus slow down the test for everyone else.
http://git.postgresql.org/pg/commitdiff/52f5d578d6c29bf254e93c69043b817d4047ca67

Noah Misch pushed:

- MSVC: Clean tmp_check directory of pg_controldata test suite.
Back-patch to 9.4, where the suite was introduced.
http://git.postgresql.org/pg/commitdiff/41635887835d54715a2fd4745dbf0fdb99f57f4f

- Clean the last few TAP suite tmp_check directories. Back-patch to
9.5, where the suites were introduced.
http://git.postgresql.org/pg/commitdiff/25924ac47abde5330ca502e23796e9d37cd1ae68

Robert Haas pushed:

- Enable parallelism for prepared statements and extended query
protocol. Parallel query can't handle running a query only
partially rather than to completion. However, there seems to be no
way to run a statement prepared via SQL PREPARE other than to
completion, so we can enable it there without a problem. The
situation is more complicated for the extend query protocol. libpq
seems to provide no way to send an Execute message with a non-zero
rowcount, but some other client might. If that happens, and a
parallel plan was chosen, we'll execute the parallel plan without
using any workers, which may be somewhat inefficient but should
still work. Hopefully this won't be a problem; users can always set
max_parallel_degree=0 to avoid choosing parallel plans in the first
place. Amit Kapila, reviewed by me.
http://git.postgresql.org/pg/commitdiff/57a6a72b6bc98f3003e87bc31de4b9c2c89fe019

- Add new FDW API to test for parallel-safety. This is basically a
bug fix; the old code assumes that a ForeignScan is always
parallel-safe, but for postgres_fdw, for example, this is definitely
false. It should be true for file_fdw, though, since a worker can
read a file from the filesystem just as well as any other backend
process. Original patch by Thomas Munro. Documentation, and
changes to the comments, by me.
http://git.postgresql.org/pg/commitdiff/35746bc348b6bf1f690fe17f4f80cfb68e22f504

- On second thought, disable parallelism for prepared statements.
CREATE TABLE .. AS EXECUTE can turn an apparently read-only query
into a write operation, which parallel query can't handle. It's a
bit of a shame that requires us to avoid parallel query for queries
prepared via PREPARE in all cases, but for right now it does.
http://git.postgresql.org/pg/commitdiff/7bea19d0a9d3e6975418ffe685fb510bd31ab434

- Respect TEMP_CONFIG when running contrib regression tests. Thomas
Munro
http://git.postgresql.org/pg/commitdiff/9117985b6ba9beda4f280f596035649fc23b6233

Álvaro Herrera pushed:

- Fix typos Backpatch to: 9.4
http://git.postgresql.org/pg/commitdiff/343f709c0691abdc779bec91fa3587dda259808e

- Add README in src/test and src/test/modules. Author: Craig Ringer
Reviewed by: Michaël Paquier
http://git.postgresql.org/pg/commitdiff/bda0b081984011ba5347bf3eecc95b71833de082

- Add POD docs to PostgresNode. Also, the dump_info method got split
into another method that returns the stuff as a string instead of
just printing it to stdout. Add a new README in src/test/perl too.
Author: Craig Ringer Reviewed by: Michaël Paquier
http://git.postgresql.org/pg/commitdiff/e64009303d4e2434abafbdafe4d571cc4f279d39

- Move some code from RewindTest into PostgresNode. Some code in the
RewindTest test suite is more generally useful than just for that
suite, so put it where other test suites can reach it. Some
postgresql.conf parameters change their default values when a
cluster is initialized with 'allows_streaming' than the previous
behavior; most notably, autovacuum is no longer turned off. (Also,
we no longer call pg_ctl promote with -w, but that flag doesn't
actually do anything in promote so there's no behavior change.)
Author: Michael Paquier
http://git.postgresql.org/pg/commitdiff/89ac7004dadf4116d9b180bb5ff64b64bfce94b1

- Apply last revision of recovery patch. I applied the
previous-to-last revision of Michaël's submitted patch instead of
the last; these two tweaks pointed out by Craig were left out of the
previous commit by accident.
http://git.postgresql.org/pg/commitdiff/74d58425c70f9f7f1cb5d7f428ceaa1b56aa081d

- Add a test framework for recovery. This long-awaited framework is
an expansion of the existing PostgresNode stuff to support
additional features for recovery testing; the recovery tests
included in this commit are a starting point that cover some of the
recovery features we have. More scripts are expected to be added
later. Author: Michaël Paquier, a bit of help from Amir Rohan.
Reviewed by: Amir Rohan, Stas Kelvich, Kyotaro Horiguchi, Victor
Wagner, Craig Ringer, Álvaro Herrera. Discussion:
http://www.postgresql.org/message-id/CAB7nPqTf7V6rswrFa=q_rrWeETUWagP=h8LX8XAov2Jcxw0DRg@mail.gmail.com
Discussion:
http://www.postgresql.org/message-id/trinity-b4a8035d-59af-4c42-a37e-258f0f28e44a-1443795007012@3capp-mailcom-lxa08
http://git.postgresql.org/pg/commitdiff/49148645f7f30f461b09618b5342b37425f33b22

- Add isolationtester spec for old heapam.c bug. In 0e5680f4737a, I
fixed a bug in heapam that caused spurious deadlocks when multiple
updates concurrently attempted to modify the old version of an
updated tuple whose new version was key-share locked. I proposed an
isolationtester spec file that reproduced the bug, but back then
isolationtester wasn't mature enough to be able to run it. Now that
38f8bdcac498 is in the tree, we can have this spec file too.
Discussion:
https://www.postgresql.org/message-id/20141212205254.GC1768%40alvh.no-ip.org
http://git.postgresql.org/pg/commitdiff/c9578135f769072e2597b88402f256a398279c91

Andrew Dunstan pushed:

- Respect TEMP_CONFIG when pg_regress_check and friends are called.
This reverts commit 9117985b6ba9beda4f280f596035649fc23b6233 in
favor of a more general solution.
http://git.postgresql.org/pg/commitdiff/87cc6b57a911bf439ff9e53f1eddba58e9749ffe

- Allow multiple --temp-config arguments to pg_regress. This means that
if, for example, TEMP_CONFIG is set and a Makefile explicitly sets a
temp-config file, both will now be used. Patch from John Gorman.
http://git.postgresql.org/pg/commitdiff/26fdff1b8f76f3c47d4e19be7c4aef3cdcd3393c

== Rejected Patches (for now) ==

No one was disappointed this week :-)

== Pending Patches ==

Amit Kapila sent in another revision of a patch to extend
pg_stat_activity with wait_type_event.

Corey Huinker sent in another revision of a patch to add \gexec to
psql.

Rushabh Lathia sent in another revision of a patch to help fix some
slowness in FDW DML.

Michaël Paquier sent in another revision of a patch to add new
authentication methods with SCRAM as one example.

Vitaly Burovoy sent in a patch to fix handling of negative years.

Ashutosh Bapat sent in another revision of a patch to push sorted
joins down to FDWs.

Fujii Masao sent in a patch to add tab completion in psql to CREATE
USER MAPPING.

Michaël Paquier sent in a pair of patches, one which extends
XLogInsert() with an extra argument for flags, the other which
introduces XLogInsertExtended with this extra argument and leaves
XLogInsert() alone.

Michaël Paquier sent in another revision of a patch to fix a potential
data loss bug on ext4 filesystems.

Craig Ringer sent in another revision of a patch to implement failover
slots.

Tomas Vondra and Mark Dilger traded patches to improve GROUP BY
estimation.

Tomas Vondra and Kyotaro HORIGUCHI traded patches to allow index-only
scans with partial indexes.

Jim Nasby sent in another revision of a patch to convert PL/Tcl from
strings to objects.

Thomas Munro sent in two more revisions of a patch to make the
PostgreSQL in parallel mode safer on PPC.

Teodor Sigaev sent in two more revisions of a patch to fix a GIN
corruption bug.

Vitaly Burovoy sent in a patch to fix some overflows in timstamp[tz].

Iacob Catalin and Pavel Stěhule traded patches to add an ereport
function to PL/PythonU.

Mithun Cy sent in another revision of a patch to cache data in
GetSnapshotData().

Kyotaro HORIGUCHI and SAWADA Masahiko traded patches to support N>1
synchronous standby servers.

Petr Jelínek sent in another revision of a patch to add generic WAL
messages.

Julien Rouhaud sent in a patch to ensure that the CREATE OPERATOR
CLASS documentation mentions that BRIN indexes also support the
STORAGE parameter.

Kyotaro HORIGUCHI sent in a patch to fix wrong comments for PQmblen()
and PQdsplen().

Kyotaro HORIGUCHI sent in a patch to fix identifier completion with
multibyte characters.

Peter Eisentraut sent in a patch to add table qualifications to some
tags in pg_dump.

Kyotaro HORIGUCHI sent in another revision of a patch to add "IF [NOT]
EXISTS" support to psql's tab completion.

Michaël Paquier sent in four more revisions of a patch to add new
regression tests for recovery, etc.

Peter Eisentraut sent in a patch to introduce new configuration
parameters syslog_sequence_numbers and syslog_split_lines.

Amit Kapila sent in another revision of a patch to speed up CLOG
access.

Vitaly Burovoy sent in a patch to allow infinite values in
to_timestamp.

Pavel Stěhule sent in another revision of a patch to add a raw format
to COPY.

Vinayak Pokure sent in another revision of a patch to add a vacuum
progress checker.

Roma Sokolov sent in two revisions of a patch to fix DROP OPERATOR to
reset links to itself on commutator and negator.

Ivan Kartyshov sent in two revisions of a patch to add a
pg_oldest_xlog_location() function.

Simon Riggs sent in two revisions of a patch to fix an issue with
relcache invalidation on a physical replica.

Amit Langote sent in a patch to fix a typo in
src/backend/utils/init/postinit.c.

Joe Conway sent in two more revisions of a patch to add control data
functions.

Tom Lane sent in a WIP patch to path-ify the upper planner. And there
was much rejoicing.

Jim Nasby sent in a patch to improve error handling in PL/Tcl.