== PostgreSQL Weekly News - March 23 2014 ==

== PostgreSQL Weekly News - March 23 2014 ==

PostgreSQL 9.3.4, 9.2.8, 9.1.13, 9.0.17, and 8.4.21 bug fix releases
are out. Upgrade ASAP.
http://www.postgresql.org/about/news/1511/

The sixth PGDay Cubano be held on 13 and 14 October 2014 in Habana.
https://postgresql.uci.cu/?p=380

The CfPs for Char(13) and PGday UK, July 8 and 9, 2014,
respectively, are out and open until April 17, 2014.
speakers AT char14 DOT info, and speakers AT postgresqlusergroup DOT
org DOT uk, respectively.
http://www.char14.info

== PostgreSQL Product News ==

DataFiller version 2.0.0, a random data generator for PostgreSQL based
on the database schema + directives, released.
http://blog.coelho.net/database/2014/03/23/datafiller-2-0-0/

Npgsql2 2.1.0, a .NET connector for PostgreSQL, released.
https://www.nuget.org/packages/Npgsql/2.1.0
https://github.com/npgsql/Npgsql/releases/tag/v2.1.0

Repmgr 2.0, an HA and replication cluster management software for
PostgreSQL, released.
http://www.repmgr.org

== PostgreSQL Jobs for March ==

http://archives.postgresql.org/pgsql-jobs/2014-03/threads.php

== PostgreSQL Local ==

PGConf NYC 2014 will be held April 3-4, 2014 in New York, New York, USA.
http://nyc.pgconf.us/2014/

The Open Data Summit will be held Friday April 11, 2014 in Denver,
Colorado, USA.
http://www.opendatasummit.com

PGCon 2014, the world-wide developer conference for PostgreSQL, will
be in Ottawa, Ontario, Canada May 20-24, 2014.
http://www.pgcon.org/2014/

The sixth PGDay Cubano be held on 13 and 14 October 2014 in Habana.
https://postgresql.uci.cu/?p=380

== PostgreSQL in the News ==

Planet PostgreSQL: http://planet.postgresql.org/

PostgreSQL Weekly News is brought to you this week by David Fetter

Submit news and announcements by Sunday at 3:00pm Pacific time.
Please send English language ones to david(at)fetter(dot)org, German language
to pwn(at)pgug(dot)de, Italian language to pwn(at)itpug(dot)org(dot) Spanish language
to pwn(at)arpug(dot)com(dot)ar(dot)

== Applied Patches ==

Magnus Hagander pushed:

- Fix small typo in comment. Michael Paquier
http://git.postgresql.org/pg/commitdiff/02703ff2277791a7f1bccf61124bee830f2b5fcb

Fujii Masao pushed:

- Fix bug in clean shutdown of walsender that pg_receiving is
connecting to. On clean shutdown, walsender waits for all WAL to be
replicated to a standby, and exits. It determined whether that
replication had been completed by checking whether its sent location
had been equal to a standby's flush location. Unfortunately this
condition never becomes true when the standby such as pg_receivexlog
which always returns an invalid flush location is connecting to
walsender, and then walsender waits forever. This commit changes
walsender so that it just checks a standby's write location if a
flush location is invalid. Back-patch to 9.1 where enough
infrastructure for this exists.
http://git.postgresql.org/pg/commitdiff/5c6d9fc4b2b8b6688a482a4b4116d7642e36b9d9

- Fix typos in comments. Thom Brown
http://git.postgresql.org/pg/commitdiff/2bccced110025f48bf426a8a9d7f627ef3663fcd

- Fix help message and document in pg_receivexlog. Add SLOTNAME
placeholder to --slot option in help message and document.
http://git.postgresql.org/pg/commitdiff/e726e59dc4845fff1e0e5849ef8d0bae958002fd

- Some minor improvements to logical decoding document. Also improve
help message in pg_recvlogical.
http://git.postgresql.org/pg/commitdiff/fb1d92a9fa0226c435daf16d08e5d05d6986fc54

Heikki Linnakangas pushed:

- Fix thinko: have trueTriConsistentFn return GIN_TRUE. While we're
at it, also improve comments in ginlogic.c.
http://git.postgresql.org/pg/commitdiff/d663d4399e767223e454302ea90d04f78b2f9d29

- Make the handling of interrupted B-tree page splits more robust.
Splitting a page consists of two separate steps: splitting the child
page, and inserting the downlink for the new right page to the
parent. Previously, we handled the case that you crash in between
those steps with a cleanup routine after the WAL recovery had
finished, which finished the incomplete split. However, that doesn't
help if the page split is interrupted but the database doesn't
crash, so that you don't perform WAL recovery. That could happen for
example if you run out of disk space. Remove the end-of-recovery
cleanup step. Instead, when a page is split, the left page is marked
with a new INCOMPLETE_SPLIT flag, and when the downlink is inserted
to the parent, the flag is cleared again. If an insertion sees a
page with the flag set, it knows that the split was interrupted for
some reason, and inserts the missing downlink before proceeding. I
used the same approach to fix GIN and GiST split algorithms earlier.
This was the last WAL cleanup routine, so we could get rid of that
whole machinery now, but I'll leave that for a separate patch.
Reviewed by Peter Geoghegan.
http://git.postgresql.org/pg/commitdiff/40dae7ec537c5619fc93ad602c62f37be786d161

- Fix misc typos in comments.
http://git.postgresql.org/pg/commitdiff/1d3b258cbe4aedfb49c92c28b9cbd7c18d277e04

- Fix compilation of pg_xlogdump, now that rm_safe_restartpoint is no
more. Oops. Pointed out by Andres Freund.
http://git.postgresql.org/pg/commitdiff/033dc1c92cf018d396e983d425b821dda420cfff

- Remove rm_safe_restartpoint machinery. It is no longer used, none
of the resource managers have multi-record actions that would make
it unsafe to perform a restartpoint. Also don't allow rm_cleanup to
write WAL records, it's also no longer required. Move the call to
rm_cleanup routines to make it more symmetric with rm_startup.
http://git.postgresql.org/pg/commitdiff/59a5ab3f426e74e3f901dc2cf533726bcea08ed2

- Replace the XLogInsert slots with regular LWLocks. The special
feature the XLogInsert slots had over regular LWLocks is the
insertingAt value that was updated atomically with releasing
backends waiting on it. Add new functions to the LWLock API to do
that, and replace the slots with LWLocks. This reduces the amount of
duplicated code. (There's still some duplication, but at least it's
all in lwlock.c now.) Reviewed by Andres Freund.
http://git.postgresql.org/pg/commitdiff/68a2e52bbaf98f136a96b3a0d734ca52ca440a95

- Fix build with LWLOCK_STATS or dtrace. Also fix the name of the
dtrace probe for LWLockAcquireOrWait(). The function was renamed
from LWLockWaitUntilFree to LWLockAqcuireOrWait, but the dtrace
probe was neglected. Pointed out by Andres Freund and the
buildfarm.
http://git.postgresql.org/pg/commitdiff/dea6ed2c980286e89caf4166ad329f506abbff29

- Fix thinkos in GinLogicValue enum. It was incorrectly declared as
global variable, not an enum type, and the comments for GIN_FALSE
and GIN_TRUE were backwards.
http://git.postgresql.org/pg/commitdiff/4c0e97c2d58f1cec9fc24237342962811de3cfee

Tom Lane pushed:

- During index build, check and elog (not just Assert) for broken HOT
chain. The recently-fixed bug in WAL replay could result in not
finding a parent tuple for a heap-only tuple. The existing code
would either Assert or generate an invalid index entry, neither of
which is desirable. Throw a regular error instead.
http://git.postgresql.org/pg/commitdiff/d70cf811f7dd26c07dbb78df4a51b667e7a3489b

- Release notes for 9.3.4, 9.2.8, 9.1.13, 9.0.17, 8.4.21.
http://git.postgresql.org/pg/commitdiff/551fb5ac742eb7dbf92aa80743aa5a52b8a0189f

- Fix pg_dumpall option parsing: -i doesn't take an argument. This
used to work properly, but got fat-fingered in commit
3dee636e0404885d07885d41c0d70e50c784f324. Per bug #9620 from
Nicolas Payart.
http://git.postgresql.org/pg/commitdiff/19f2d6cdae2bfa97c2ce8a7f5ac453a91f40704a

- Fix relcache reference leak in refresh_by_match_merge(). One path
through the loop over indexes forgot to do index_close(). Rather
than adding a fourth call, restructure slightly so that there's only
one. In passing, get rid of an unnecessary syscache lookup: the
pg_index struct for the index is already available from its relcache
entry. Per report from YAMAMOTO Takashi, though this is a bit
different from his suggested patch. This is new code in HEAD, so no
need for back-patch.
http://git.postgresql.org/pg/commitdiff/f7271c44278352516ec66b2de311952ce330b6d5

- Fix some remaining int64 vestiges in contrib/test_shm_mq. Andres
Freund and Tom Lane
http://git.postgresql.org/pg/commitdiff/b6ec7c92ac7ab6223b3c45dc554efffd1953758f

- Fix memory leak during regular expression execution. For a regex
containing backrefs, pg_regexec() might fail to free all the
sub-DFAs that were created during execution, resulting in a
permanent (session lifespan) memory leak. Problem was introduced by
me in commit 587359479acbbdc95c8e37da40707e37097423f5. Per report
from Sandro Santilli; diagnosis by Greg Stark.
http://git.postgresql.org/pg/commitdiff/ea8c7e9054abf23fa3de2f8e4414f60ac8a8b620

- Again fix initialization of auto-tuned effective_cache_size. The
previous method was overly complex and underly correct; in
particular, by assigning the default value with PGC_S_OVERRIDE, it
prevented later attempts to change the setting in postgresql.conf,
as noted by Jeff Janes. We should just assign the default value
with source PGC_S_DYNAMIC_DEFAULT, which will have the desired
priority relative to the boot_val as well as user-set values. There
is still a gap in this method: if there's an explicit assignment of
effective_cache_size = -1 in the postgresql.conf file, and that
assignment appears before shared_buffers is assigned, the code will
substitute 4 times the bootstrap default for shared_buffers, and
that value will then persist (since it will have source PGC_S_FILE).
I don't see any very nice way to avoid that though, and it's not a
case to be expected in practice. The existing comments in
guc-file.l look forward to a redesign of the DYNAMIC_DEFAULT
mechanism; if that ever happens, we should consider this case as one
of the things we'd like to improve.
http://git.postgresql.org/pg/commitdiff/af930e606a3217db3909029c6c3f8d003ba70920

Robert Haas pushed:

- Make it easy to detach completely from shared memory. The new
function dsm_detach_all() can be used either by postmaster children
that don't wish to take any risk of accidentally corrupting shared
memory; or by forked children of regular backends with the same
need. This patch also updates the postmaster children that already
do PGSharedMemoryDetach() to do dsm_detach_all() as well. Per
discussion with Tom Lane.
http://git.postgresql.org/pg/commitdiff/79a4d24f31e09eb3c421deb34829eee0bf6acd67

- Improve shm_mq portability around MAXIMUM_ALIGNOF and sizeof(Size).
Revise the original decision to expose a uint64-based interface and
use Size everywhere possible. Avoid assuming that MAXIMUM_ALIGNOF
is 8, or making any assumption about the relationship between that
value and sizeof(Size). If MAXIMUM_ALIGNOF is bigger, we'll now
insert padding after the length word; if it's smaller, we are now
prepared to read and write the length word in chunks. Per
discussion with Tom Lane.
http://git.postgresql.org/pg/commitdiff/3bd261ca18c67eafe18088e58fab511e3b965418

- Rewrite comment for shm_mq_receive_bytes. The comment and the code
diverged at some point before the initial commit of this feature,
and I failed to notice. Noted by Tom Lane.
http://git.postgresql.org/pg/commitdiff/250f8a7bbe83a7dbc9618c0a506f7170ab610e4b

- test_shm_mq: Use Size rather than uint64. Commit
3bd261ca18c67eafe18088e58fab511e3b965418 updated the API but
neglected to make the corresponding edits here. Per Tom Lane and
the buildfarm.
http://git.postgresql.org/pg/commitdiff/c676ac0f3fc87911588ca8e5f92483d1671ba89f

- Add pg_recvlogical, a tool to receive data logical decoding data.
This is fairly basic at the moment, but it's at least useful for
testing and debugging, and possibly more. Andres Freund
http://git.postgresql.org/pg/commitdiff/8bdd12bbf093c846c9383c205e363d6ae35f11a6

- Fix uninitialized variable. Report from Andres Freund, but not his
fix.
http://git.postgresql.org/pg/commitdiff/a3b30d4cfe5f3ae718fd2cc6aa05b7707cac88dd

- Logical decoding documentation corrections. Thom Brown
http://git.postgresql.org/pg/commitdiff/3ee4fcfc63452a1ab5586b2d4cb4b6908d1badff

- Documentation for logical decoding. Craig Ringer, Andres Freund,
Christian Kruse, with edits by me.
http://git.postgresql.org/pg/commitdiff/49c0864d7ef5227faa24f903902db90e5c9d5d69

- test_shm_mq: Improve regression tests. Per discussion with Tom
Lane.
http://git.postgresql.org/pg/commitdiff/d1bdab2fa39f9a29fc806e1f2b6b5428b88d7cc5

Alvaro Herrera pushed:

- Setup error context callback for transaction lock waits With this in
place, a session blocking behind another one because of tuple locks
will get a context line mentioning the relation name, tuple TID, and
operation being done on tuple. For example: LOG: process 11367
still waiting for ShareLock on transaction 717 after 1000.108 ms
DETAIL: Process holding the lock: 11366. Wait queue: 11367.
CONTEXT: while updating tuple (0,2) in relation "foo" STATEMENT:
UPDATE foo SET value = 3; Most usefully, the new line is displayed
by log entries due to log_lock_waits, although of course it will be
printed by any other log message as well. Author: Christian Kruse,
some tweaks by Álvaro Herrera Reviewed-by: Amit Kapila, Andres
Freund, Tom Lane, Robert Haas
http://git.postgresql.org/pg/commitdiff/f88d4cfc9d417dac2ee41a8f5e593898e56fd2bd

Bruce Momjian pushed:

- libpq: pass a memory allocation failure error up to
PQconndefaults(). Previously user name memory allocation failures
were ignored and the default user name set to NULL.
http://git.postgresql.org/pg/commitdiff/a4c8f14364c27508233f8a31ac4b10a4c90235a9

- Properly check for readdir/closedir() failures. Clear errno before
calling readdir() and handle old MinGW errno bug while adding full
test coverage for readdir/closedir failures. Backpatch through 8.4.
http://git.postgresql.org/pg/commitdiff/6f03927fce038096f53ca67eeab9adb24938f8a6

- Remove MinGW readdir/errno bug workaround fixed on 2003-10-10
http://git.postgresql.org/pg/commitdiff/1494931d7375ccdc6afd34f135bc708f8954eecc

Noah Misch pushed:

- Address ccvalid/ccnoinherit in TupleDesc support functions.
equalTupleDescs() neglected both of these ConstrCheck fields, and
CreateTupleDescCopyConstr() neglected ccnoinherit. At this time,
the only known behavior defect resulting from these omissions is
constraint exclusion disregarding a CHECK constraint validated by an
ALTER TABLE VALIDATE CONSTRAINT statement issued earlier in the same
transaction. Back-patch to 9.2, where these fields were introduced.
http://git.postgresql.org/pg/commitdiff/c31305de5f5a4880b0ba2f5983025ef0210a3b2a

- Offer triggers on foreign tables. This covers all the SQL-standard
trigger types supported for regular tables; it does not cover
constraint triggers. The approach for acquiring the old row mirrors
that for view INSTEAD OF triggers. For AFTER ROW triggers, we spool
the foreign tuples to a tuplestore. This changes the FDW API
contract; when deciding which columns to populate in the slot
returned from data modification callbacks, writable FDWs will need
to check for AFTER ROW triggers in addition to checking for a
RETURNING clause. In support of the feature addition, refactor the
TriggerFlags bits and the assembly of old tuples in ModifyTable.
Ronan Dunklau, reviewed by KaiGai Kohei; some additional hacking by
me.
http://git.postgresql.org/pg/commitdiff/7cbe57c34dec4860243e6d0f81738cfbb6e5d069

- Improve comments about AfterTriggerBeginQuery() query level usage.
http://git.postgresql.org/pg/commitdiff/6115480c543c0141011a99db78987ad13540be59

- Don't test xmin/xmax columns of a postgres_fdw foreign table. Their
values are unspecified and system-dependent. Per buildfarm member
kouprey.
http://git.postgresql.org/pg/commitdiff/b2b2491b06074e68fc7c96148cb0fdf0c8eb0469

Andrew Dunstan pushed:

- Introduce jsonb, a structured format for storing json. The new
format accepts exactly the same data as the json type. However, it
is stored in a format that does not require reparsing the orgiginal
text in order to process it, making it much more suitable for
indexing and other operations. Insignificant whitespace is
discarded, and the order of object keys is not preserved. Neither
are duplicate object keys kept - the later value for a given key is
the only one stored. The new type has all the functions and
operators that the json type has, with the exception of the json
generation functions (to_json, json_agg etc.) and with identical
semantics. In addition, there are operator classes for hash and
btree indexing, and two classes for GIN indexing, that have no
equivalent in the json type. This feature grew out of previous work
by Oleg Bartunov and Teodor Sigaev, which was intended to provide
similar facilities to a nested hstore type, but which in the end
proved to have some significant compatibility issues. Authors: Oleg
Bartunov, Teodor Sigaev, Peter Geoghegan and Andrew Dunstan.
Review: Andres Freund
http://git.postgresql.org/pg/commitdiff/d9134d0a355cfa447adc80db4505d5931084278a

- Fix mis-spelling in jsonb docs. Per Thom Brown.
http://git.postgresql.org/pg/commitdiff/ca07cd59b24e00e428ed26716754244cec7f56b7

- Do jsonb regression test input in the conventional way. This should
make the buildfarm happier.
http://git.postgresql.org/pg/commitdiff/ab22b149c60a10b842e3ec7fe3eb3b0b66c6611a

== Rejected Patches (for now) ==

No one was disappointed this week :-)

== Pending Patches ==

Mitsumasa KONDO sent in another revision of a patch to allow using a
Gaussian distribution in pgbench.

Jürgen Strobel sent in a patch to add a command line option
--no-table-lock to pg_dump.

Fabrízio de Royes Mello sent in another revision of a patch to fix
some wrong behavior in ALTER ... RESET.

Petr (PJMODOS) Jelinek sent in two more revisions of a patch to add
plpgsql.warn_shadow.

Vaishnavi Prabakaran sent in another revision of a patch to add a
catalog view to pg_hba.conf.

Heikki Linnakangas sent in a patch to change the way locks are
acquired on B-trees.

Bruce Momjian sent in two revisions of a patch to fix a bug in
pg_archivecleanup.

Jing Wang sent in another revision of a patch to issue a log message
to suggest VACUUM FULL if a table is nearly empty.

Gurjeet Singh sent in a patch to send transaction commit/rollback
stats to the stats collector unconditionally.

Kyotaro HORIGUCHI sent in another revision of a patch to fix an
infelicity in some situations in archive recovery.

Petr (PJMODOS) Jelinek sent in two more revisions of a patch to add
plpgsql.extra_warnings and plpgsql.extra_errors.

Fujii Masao sent in a patch to fix an issue where effective_cache_size
cannot be changed by a reload (HUP) of the backend.

Vik Fearing sent in a patch to fix some typos in the patch to reduce
lock strengths needed for ALTER TABLE.

Etsuro Fujita sent in another revision of a patch to allow foreign
tables in table inheritance hierarchies.

Dilip Kumar sent in another revision of a patch to add parallelism to
the vacuumdb program.

MauMau sent in another revision of a patch to fix an issue where
PostgreSQL fails to start on Windows if it crashes after tablespace
creation.

Andres Freund sent in a PoC patch not to require a NBuffer-sized
PrivateRefCount array of local buffer pins.

Emanuel Calvo sent in a patch to clarify the documentation of what
events rewrite RULEs can apply to.

Bruce Momjian sent in another revision of a patch to fix some psql
output for the Replica type displayed.

Noah Misch sent in two more revisions of a patch to warn about some
escalation attacks possible during "make check".