== PostgreSQL Weekly News - December 28 2014 ==

== PostgreSQL Weekly News - December 28 2014 ==

PGCon 2015 (June 16-20) call for papers is out. Talk submission ends
January 19, 2014. Details at
http://www.pgcon.org/2015/papers.php

== PostgreSQL Product News ==

Bucardo 5.3.0, a multiple source replication system for Postgres, released:
https://mail.endcrypt.com/pipermail/bucardo-announce/2014/000030.html

== PostgreSQL Jobs for December ==

http://archives.postgresql.org/pgsql-jobs/2014-12/threads.php

== PostgreSQL Local ==

FOSDEM PGDay (January 30, 2015) and the FOSDEM PostgreSQL
Dev Room (January 31-February 1, 2015)
http://fosdem2015.pgconf.eu/

Prague PostgreSQL Developer Day (P2D2) 2015 will be in Prague, Czech
Republic February 11-12, 2015.
http://www.p2d2.cz/

The Melbourne PostgreSQL meetup on February 18, 2015 will be hosting
Gabriele Bartolini on PostgreSQL 9.4 for devops. Details below, and
R, SVP.
http://www.meetup.com/melpug/events/219082475/

pgDaySF 2015 will be held March 10, 2015 in Burlingame, California.
http://sfpostgres.org/pgday-sf-2015-call-for-speakers-and-sponsors/

The CfP is open for Nordic PostgreSQL Day 2015, which will be held
March 11, 2015 in Copenhagen, Denmark.
http://2015.nordicpgday.org/cfp/

The CfP for PGConf US 2015 is open through December 17th, 2014
Notifications will go out on January 10, 2014. The event takes place
March 25-27, 2015 in NYC.
http://nyc.pgconf.us/2015/

== PostgreSQL in the News ==

Planet PostgreSQL: http://planet.postgresql.org/

PostgreSQL Weekly News is brought to you this week by David Fetter

Submit news and announcements by Sunday at 3:00pm Pacific time.
Please send English language ones to david(at)fetter(dot)org, German language
to pwn(at)pgug(dot)de, Italian language to pwn(at)itpug(dot)org(dot) Spanish language
to pwn(at)arpug(dot)com(dot)ar(dot)

== Applied Patches ==

Heikki Linnakangas pushed:

- Use a pairing heap for the priority queue in kNN-GiST searches.
This performs slightly better, uses less memory, and needs slightly
less code in GiST, than the Red-Black tree previously used.
Reviewed by Peter Geoghegan
http://git.postgresql.org/pg/commitdiff/e7032610f76e6c8345496ae7bbdf49d3c40df30f

- Turn much of the btree_gin macros into real functions. This makes
the functions much nicer to read and edit, and also makes debugging
easier.
http://git.postgresql.org/pg/commitdiff/7f0dccaed64a8ed6f5db8ad43e7612202fbeeeaf

- Move rbtree.c from src/backend/utils/misc to src/backend/lib. We
have other general-purpose data structures in src/backend/lib, so it
seems like a better home for the red-black tree as well.
http://git.postgresql.org/pg/commitdiff/955557ddccb4065831af80b0966cbd02937dfb72

Andrew Dunstan pushed:

- Further tidy up on json aggregate documentation
http://git.postgresql.org/pg/commitdiff/2a3f2743f200dbcdd412a00ff69f30d3e34b662b

- Fix documentation of argument type of json_agg and jsonb_agg.
json_agg was originally designed to aggregate records. However, it
soon became clear that it is useful for aggregating all kinds of
values and that's what we have on 9.3 and 9.4, and in head for it
and jsonb_agg. The documentation suggested otherwise, so this fixes
it.
http://git.postgresql.org/pg/commitdiff/b2d145bff02e5f278649aca0e6fb6142ea2f298d

- Fix installcheck case for tap tests
http://git.postgresql.org/pg/commitdiff/3f37b6c3166b2dc1956de8b0fb046ea6e7320d52

Peter Eisentraut pushed:

- Change local_preload_libraries to PGC_USERSET. This allows it to be
used with ALTER ROLE SET. Although the old setting of PGC_BACKEND
prevented changes after session start, after discussion it was more
useful to allow ALTER ROLE SET instead and just document that
changes during a session have no effect. This is similar to how
session_preload_libraries works already. An alternative would be to
change things to allow PGC_BACKEND and PGC_SU_BACKEND settings to be
changed by ALTER ROLE SET. But that might need further research
(e.g., log_connections would probably not work). based on patch by
Kyotaro Horiguchi
http://git.postgresql.org/pg/commitdiff/584e35d17c00b69d598522d3cf321fbd9b7dbd03

Álvaro Herrera pushed:

- get_object_address: separate domain constraints from table
constraints. Apart from enabling comments on domain constraints,
this enables a future project to replicate object dropping to remote
servers: with the current mechanism there's no way to distinguish
between the two types of constraints, so there's no way to know what
to drop. Also added support for the domain constraint comments in
psql's \dd and pg_dump. Catalog version bumped due to the change in
ObjectType enum.
http://git.postgresql.org/pg/commitdiff/7eca575d1c28f6eee2bf4564f3d458d10c4a8f47

- Use a bitmask to represent role attributes. The previous
representation using a boolean column for each attribute would not
scale as well as we want to add further attributes. Extra
auxilliary functions are added to go along with this change, to make
up for the lost convenience of access of the old representation.
Catalog version bumped due to change in catalogs and the new
functions. Author: Adam Brightwell, minor tweaks by Álvaro Reviewed
by: Stephen Frost, Andres Freund, Álvaro Herrera
http://git.postgresql.org/pg/commitdiff/1826987a46d079458007b7b6bbcbbd852353adbb

- Add SQL-callable pg_get_object_address. This allows access to
get_object_address from SQL, which is useful to obtain OID
addressing information from data equivalent to that emitted by the
parser. This is necessary infrastructure of a project to let
replication systems propagate object dropping events to remote
servers, where the schema might be different than the server
originating the DROP. This patch also adds support for
OBJECT_DEFAULT to get_object_address; that is, it is now possible to
refer to a column's default value. Catalog version bumped due to
the new function. Reviewed by Stephen Frost, Heikki Linnakangas,
Robert Haas, Andres Freund, Abhijit Menon-Sen, Adam Brightwell.
http://git.postgresql.org/pg/commitdiff/d7ee82e50f624221db76023c17137661fe69ec61

- Revert "Use a bitmask to represent role attributes". This reverts
commit 1826987a46d079458007b7b6bbcbbd852353adbb. The overall design
was deemed unacceptable, in discussion following the previous commit
message; we might find some parts of it still salvageable, but I
don't want to be on the hook for fixing it, so let's wait until we
have a new patch.
http://git.postgresql.org/pg/commitdiff/a609d96778c1714b9af916477b2c30891fbe578a

- Grab heavyweight tuple lock only before sleeping. We were trying to
acquire the lock even when we were subsequently not sleeping in some
other transaction, which opens us up unnecessarily to deadlocks. In
particular, this is troublesome if an update tries to lock an
updated version of a tuple and finds itself doing EvalPlanQual
update chain walking; more than two sessions doing this concurrently
will find themselves sleeping on each other because the HW tuple
lock acquisition in heap_lock_tuple called from EvalPlanQualFetch
races with the same tuple lock being acquired in heap_update -- one
of these sessions sleeps on the other one to finish while holding
the tuple lock, and the other one sleeps on the tuple lock. Per
trouble report from Andrew Sackville-West in
http://www.postgresql.org/message-id/20140731233051.GN17765@andrew-ThinkPad-X230
His scenario can be simplified down to a relatively simple
isolationtester spec file which I don't include in this commit; the
reason is that the current isolationtester is not able to deal with
more than one blocked session concurrently and it blocks instead of
raising the expected deadlock. In the future, if we improve
isolationtester, it would be good to include the spec file in the
isolation schedule. I posted it in
http://www.postgresql.org/message-id/20141212205254.GC1768@alvh.no-ip.org
Hat tip to Mark Kirkwood, who helped diagnose the trouble.
http://git.postgresql.org/pg/commitdiff/0e5680f4737a9c6aa94aa9e77543e5de60411322

- Remove event trigger from object_address test. It is causing
trouble when run in parallel mode, because dropping the function
other sessions are running concurrently causes them to fail due to
inability to find the function. Per buildfarm, as noted by Tom
Lane.
http://git.postgresql.org/pg/commitdiff/289121a452e3c255e4fd63dbb0252261605c2eaf

- Restrict name list len for domain constraints. This avoids an
ugly-looking "cache lookup failure" message. Ugliness pointed out
by Andres Freund.
http://git.postgresql.org/pg/commitdiff/6630420fc9139c3a1ee051bc2f2ae8fa32e71c62

Tom Lane pushed:

- Remove failing collation case from object_address regression test.
Per buildfarm, this test case does not yield consistent results. I
don't think it's useful enough to figure out a workaround, either.
http://git.postgresql.org/pg/commitdiff/3e2275355954cabf89ba044fe8557ef347bf3fb1

- Add CST (China Standard Time) to our lists of timezone
abbreviations. For some reason this seems to have been missed when
the lists in src/timezone/tznames/ were first constructed. We can't
put it in Default because of the conflict with US CST, but we should
certainly list it among the alternative entries in Asia.txt. (I
checked for other oversights, but all the other abbreviations that
are in current use according to the IANA files seem to be accounted
for.) Noted while responding to bug #12326.
http://git.postgresql.org/pg/commitdiff/5b89473d870dc2a9fec0926c5afccf53042dbb0a

- Temporarily revert "Move pg_lzcompress.c to src/common." This
reverts commit 60838df922345b26a616e49ac9fab808a35d1f85. That
change needs a bit more thought to be workable. In view of the
potentially machine-dependent stuff that went in today, we need all
of the buildfarm to be testing those other changes.
http://git.postgresql.org/pg/commitdiff/966115c305c297cf4a1ec5ebbd0a55be46ff33ad

- Remove duplicate assignment in new pg_get_object_address() function.
Noted by Coverity.
http://git.postgresql.org/pg/commitdiff/9a11df14498434bbb5e16dfa7aecbf71bf24d196

Bruce Momjian pushed:

- pgbench: remove odd trailing period in init progress output
http://git.postgresql.org/pg/commitdiff/83bcc70459c596ceb9a90e96f011d24b88d93a85

Fujii Masao pushed:

- Remove unused fields from ReindexStmt. fe263d1 changed the REINDEX
logic so that those fields are not used at all, but forgot to remove
them. Sawada Masahiko
http://git.postgresql.org/pg/commitdiff/3b6ca123b5665044557154385a6be5d1df76c5e5

- Remove duplicate include of slot.h. Back-patch to 9.4, where this
problem was added.
http://git.postgresql.org/pg/commitdiff/4a5593197b0ddec913fcd7758d61e782ab5c4d59

- Move pg_lzcompress.c to src/common. Exposing compression and
decompression APIs of pglz makes possible its use by extensions and
contrib modules. pglz_decompress contained a call to elog to emit an
error message in case of corrupted data. This function is changed to
return a status code to let its callers return an error instead.
This commit is required for upcoming WAL compression feature so that
the WAL reader facility can decompress the WAL data by using
pglz_decompress. Michael Paquier
http://git.postgresql.org/pg/commitdiff/60838df922345b26a616e49ac9fab808a35d1f85

Andres Freund pushed:

- Suppress MSVC warning in typeStringToTypeName function. MSVC
doesn't realize ereport(ERROR) doesn't return. David Rowley
http://git.postgresql.org/pg/commitdiff/cd5ebe1edd6d9c19caba406a90f626afec5d3f0a

- Convert the PGPROC->lwWaitLink list into a dlist instead of open
coding it. Besides being shorter and much easier to read it changes
the logic in LWLockRelease() to release all shared lockers when
waking up any. This can yield some significant performance
improvements - and the fairness isn't really much worse than before,
as we always allowed new shared lockers to jump the queue.
http://git.postgresql.org/pg/commitdiff/7882c3b0b95640e361f1533fe0f2d02e4e5d8610

- Add capability to suppress CONTEXT: messages to elog machinery.
Hiding context messages usually is not a good idea - except for
rather verbose debugging/development utensils like LOG_DEBUG. There
the amount of repeated context messages just bloat the log without
adding information.
http://git.postgresql.org/pg/commitdiff/570bd2b3fd20d3f8896f5e6a8133d6aed385ac86

- Lockless StrategyGetBuffer clock sweep hot path.
StrategyGetBuffer() has proven to be a bottleneck in a number of
buffer acquisition heavy workloads. To some degree this has already
been alleviated by 5d7962c6, but it still can be quite a heavy
bottleneck. The problem is that in unfortunate usage patterns a
single StrategyGetBuffer() call will have to look at a large number
of buffers - in turn making it likely that the process will be put
to sleep while still holding the spinlock. Replace most of the
usage of the buffer_strategy_lock spinlock for the clock sweep by a
atomic nextVictimBuffer variable. That variable, modulo NBuffers, is
the current hand of the clock sweep. The buffer clock-sweep then
only needs to acquire the spinlock after a wraparound. And even then
only in the process that did the wrapping around. That alleviates
nearly all the contention on the relevant spinlock, although
significant contention on the cacheline can still exist.
Reviewed-By: Robert Haas and Amit Kapila. Discussion:
20141010160020(dot)GG6670(at)alap3(dot)anarazel(dot)de,
20141027133218(dot)GA2639(at)awork2(dot)anarazel(dot)de
http://git.postgresql.org/pg/commitdiff/d72731a70450b5e7084991b9caa15cb58a2820df

- Blindly fix a dtrace probe in lwlock.c for a removed local variable.
Per buildfarm member locust.
http://git.postgresql.org/pg/commitdiff/740a4ec7f4adccd56eced452580ccec7157fc8d8

- Improve LWLock scalability. The old LWLock implementation had the
problem that concurrent lock acquisitions required exclusively
acquiring a spinlock. Often that could lead to acquirers waiting
behind the spinlock, even if the actual LWLock was free. The new
implementation doesn't acquire the spinlock when acquiring the lock
itself. Instead the new atomic operations are used to atomically
manipulate the state. Only the waitqueue, used solely in the slow
path, is still protected by the spinlock. Check lwlock.c's header
for an explanation about the used algorithm. For some common
workloads on larger machines this can yield significant performance
improvements. Particularly in read mostly workloads. Reviewed-By:
Amit Kapila and Robert Haas. Author: Andres Freund. Discussion:
20130926225545(dot)GB26663(at)awork2(dot)anarazel(dot)de
http://git.postgresql.org/pg/commitdiff/ab5194e6f617a9a9e7aadb3dd1cee948a42d0755

Noah Misch pushed:

- Have config_sspi_auth() permit IPv6 localhost connections. Windows
versions later than Windows Server 2003 map "localhost" to ::1.
Account for that in the generated pg_hba.conf, fixing another
oversight in commit f6dc6dd5ba54d52c0733aaafc50da2fbaeabb8b0.
Back-patch to 9.0, like that commit. David Rowley and Noah Misch
http://git.postgresql.org/pg/commitdiff/8d9cb0bc483431780aef483c7c110ab39e465958

== Rejected Patches (for now) ==

No one was disappointed this week :-)

== Pending Patches ==

Michael Paquier sent in another revision of a patch to clarify some
points about how numeric functions and operators work.

Etsuro Fujita sent in a patch to fix the fact that ExplainModifyTarget
does not always show the parent of the inheritance tree in
multi-target-table cases.

Heikki Linnakangas sent in another revision of a patch to advance
local xmin more aggressively using the new pairing heap code.

Robert Haas sent in another revision of a patch to implement parallel
mode and parallel contexts.

Petr (PJMODOS) Jelinek sent in another revision of a patch to
implement TABLESAMPLE.

Christoph Berg sent in a patch to correct the misapprehension that
pg_upgrade requires "postmaster," an entity not present in all live
versions of PostgreSQL, when "postgres" will do.

Michael Paquier sent in two more revisions of a patch to move contrib
binaries to bin.

Álvaro Herrera sent in a flock of five patches intended to complete
the feature which allows DROP to propagate via WAL.

Etsuro Fujita sent in two more revisions of a patch to allow foreign
tables to be part of table inheritance hierarchies.

Heikki Linnakangas and Teodor Sigaev traded patches to support ranges
with B-Tree GIN indexes.

Heikki Linnakangas sent in another revision of a patch to add a
compress method for SP-GiST.

Andres Freund sent in another revision of a patch to add the
capability to suppress CONTEXT: messages to elog machinery.

Andres Freund sent in another revision of a patch to convert the
PGPROC->lwWaitLink list into a dlist instead of open coding it, and
one to implement wait-free LW_SHARED LWLock acquisition.

Andres Freund sent in a patch to add a lockless StrategyGetBuffer
clock sweep hotpath.

Michael Paquier sent in a patch to install libpq.dll in bin directory
on Windows / Cygwin.

Andrew Dunstan sent in a patch to speed up the bin checks.

Teodor Sigaev sent in another revision of a patch to speed up
tidbitmap using a cache page.

Jim Nasby sent in a patch to add a hack to test out several hashing
strategies.

Andreas Karlsson sent in another revision of a patch to use 128-bit
integers for sum, avg and statistics aggregates.

Bruce Momjian sent in a patch to fix an issue where misaligned
BufferDescriptors is causing major performance problems on AMD.

Jeff Davis sent in another revision of a patch to implement
spill-to-disk for HashAggs.

Alexander Shulgin sent in another revision of a patch to turn
recovery.conf into GUCs.

Kaigai Kouhei sent in another revision of a patch to add a CTID scan
as an example of the custom scan capability.

David Rowley and Noah Misch traded patches to update Windows with the
changes that secured "make check."

Abhijit Menon-Sen sent in a patch to use CPUID at startup to determine
whether the SSE4.2 CRC instructions are available, and to use them
instead of the slice-by-8 implementation if they are.

Fujii Masao and Michael Paquier traded patches implementing
compression of full-page writes.

Abhijit Menon-Sen sent in a patch to implement a fast bloat
measurement tool.

Alexander Shulgin and Pavel Stehule traded patches to allow COPY to
handle wrong rows without rolling back entirely.

Alexey Vasiliev sent in a patch to add recovery_timeout option to
control timeout of restore_command nonzero status code.

Peter Geoghegan sent in another revision of a patch to implement
INSERT ... ON CONFLICT {UPDATE | IGNORE}.

Ian Lawrence Barwick sent in a patch to fix psql's tab completion of
COMMENT ON not to repeat IS more than needed.

Amit Kapila and Dilip Kumar traded patches to enable vacuumdb to use
parallel cores.