== PostgreSQL Weekly News - July 21 2013 ==

== PostgreSQL Weekly News - July 21 2013 ==

The Call for papers for PGConf.EU 2013 has been extended until July
29, 2013.
http://2013.pgconf.eu/

The CfP for Italian PostgreSQL day ends July 28, 2013.
http://2013.pgday.it/index.php/programma/call-for-papers-english/

== PostgreSQL Product News ==

PG Partition Manager 1.3.0, an extension to manage time- or
serial-based table partitioning, released.
https://github.com/keithf4/pg_partman

== PostgreSQL Jobs for July ==

http://archives.postgresql.org/pgsql-jobs/2013-07/threads.php

== PostgreSQL Local ==

PostgreSQL Brazil will be held August 15-17, 2013 in Porto Velho, RO,
Brazil.
http://pgbr.postgresql.org.br/2013/chamada.en.php

Postgres Open 2013 will be in Chicago, IL, USA, September 16-18.
The list of talks has been posted on the site.
http://postgresopen.org/

The Italian PGDay (PGDay.IT) will be held on October the 25th in
Prato, Tuscany, Italy, at Monash University Center.
The International Call For Papers is open:
http://2013.pgday.it/index.php/programma/call-for-papers-english/

pgconf.EU 2013 will be held on Oct 29-Nov 1, 2013 at the Conrad Hotel
in downtown Dublin, Ireland. The CfP is open.
http://2013.pgconf.eu/

PGConf.DE 2013 will be held November 8th, 2013, at the Rhineland
Industrial Museum in Oberhausen. The CfP is open through September
15, 2013.
http://2013.pgconf.de/

== PostgreSQL in the News ==

Planet PostgreSQL: http://planet.postgresql.org/

PostgreSQL Weekly News is brought to you this week by David Fetter

Submit news and announcements by Sunday at 3:00pm Pacific time.
Please send English language ones to david(at)fetter(dot)org, German language
to pwn(at)pgug(dot)de, Italian language to pwn(at)itpug(dot)org(dot) Spanish language
to pwn(at)arpug(dot)com(dot)ar(dot)

== Applied Patches ==

Stephen Frost pushed:

- Correct off-by-one when reading from pipe. In
pg_basebackup.c:reached_end_position(), we're reading from an
internal pipe with our own background process but we're possibly
reading more bytes than will actually fit into our buffer due to an
off-by-one error. As we're reading from an internal pipe there's no
real risk here, but it's good form to not depend on such convenient
arrangements. Bug spotted by the Coverity scanner. Back-patch to
9.2 where this showed up.
http://git.postgresql.org/pg/commitdiff/03010366b6fb61aac0998f234478cc745ff97b0c

- Check get_tle_by_resno() result before deref. When creating a sort
to support a group by, we need to look up the target entry in the
target list by the resno using get_tle_by_resno(). This particular
code-path didn't check the result prior to attempting to dereference
it, while all other callers did. While I can't see a way for this
usage of get_tle_by_resno() to fail (you can't ask for a column to
be sorted on which isn't included in the group by), it's probably
best to check that we didn't end up with a NULL somehow anyway than
risk the segfault. I'm willing to back-patch this if others feel
it's necessary, but my guess is new features are what might tickle
this rather than anything existing. Missing check spotted by the
Coverity scanner.
http://git.postgresql.org/pg/commitdiff/4ed22e891f9915b02b753ee8763a8f2438234fc6

- Use correct parameter name for view_option_value. The documentation
for ALTER VIEW had a minor copy-and-paste error in defining the
parameters. Noticed when reviewing the WITH CHECK OPTION patch.
Backpatch to 9.2 where this was first introduced.
http://git.postgresql.org/pg/commitdiff/f2719f6975398727ed492ecbcb2028bd134e5f6b

- WITH CHECK OPTION support for auto-updatable VIEWs. For simple
views which are automatically updatable, this patch allows the user
to specify what level of checking should be done on records being
inserted or updated. For 'LOCAL CHECK', new tuples are validated
against the conditionals of the view they are being inserted into,
while for 'CASCADED CHECK' the new tuples are validated against the
conditionals for all views involved (from the top down). This
option is part of the SQL specification. Dean Rasheed, reviewed by
Pavel Stehule
http://git.postgresql.org/pg/commitdiff/4cbe3ac3e86790d05c569de4585e5075a62a9b41

Robert Haas pushed:

- vacuumlo: Use a cursor to limit client-side memory usage. This
prevents the client from gobbling up too much memory when the number
of large objects to be removed is very large. Andrew Dunstan,
reviewed by Josh Kupershmidt
http://git.postgresql.org/pg/commitdiff/67ccbb080d87a5379dffc1cdff046c4caf534a6c

- Regression tests for LOCK TABLE. Robins Tharakan, reviewed by
Szymon Guz, substantially revised by me.
http://git.postgresql.org/pg/commitdiff/ac33c7e2c1304dffcf107065d7fc04232a8a94fa

- Assert that syscache lookups don't happen outside transactions.
Andres Freund
http://git.postgresql.org/pg/commitdiff/42c80c696e9c8323841180029cc62741c21bd356

- Tab completion for \lo_import. Josh Kupershmidt
http://git.postgresql.org/pg/commitdiff/4403a9d7913043d99de9e13a9b54c57ed0c4501c

- Allow background workers to be started dynamically. There is a new
API, RegisterDynamicBackgroundWorker, which allows an ordinary user
backend to register a new background writer during normal running.
This means that it's no longer necessary for all background workers
to be registered during processing of shared_preload_libraries,
although the option of registering workers at that time remains
available. When a background worker exits and will not be
restarted, the slot previously used by that background worker is
automatically released and becomes available for reuse. Slots used
by background workers that are configured for automatic restart
can't (yet) be released without shutting down the system. This
commit adds a new source file, bgworker.c, and moves some of the
existing control logic for background workers there. Previously,
there was little enough logic that it made sense to keep everything
in postmaster.c, but not any more. This commit also makes the
worker_spi contrib module into an extension and adds a new function,
worker_spi_launch, which can be used to demonstrate the new
facility.
http://git.postgresql.org/pg/commitdiff/7f7485a0cde92aa4ba235a1ffe4dda0ca0b6cc9a

- doc: Fix typos in conversion names. David Christensen
http://git.postgresql.org/pg/commitdiff/c8fb821afc2ce37b48b678251480b52e6b116868

Peter Eisentraut pushed:

- Fix PQconninfoParse error message handling. The returned error
message already includes a newline, but the callers were adding
their own when printing it out.
http://git.postgresql.org/pg/commitdiff/233bfe06735411f08d231764dfd3a6fcf7aef9a3

- doc: Remove tab from SGML file
http://git.postgresql.org/pg/commitdiff/e8b1d1befeab17453eb2977e83a45649e6ed8790

- Clean up new JSON API typedefs. The new JSON API uses a bit of an
unusual typedef scheme, where for example OkeysState is a pointer to
okeysState. And that's not applied consistently either. Change
that to the more usual PostgreSQL style where struct typedefs are
upper case, and use pointers explicitly.
http://git.postgresql.org/pg/commitdiff/ff41a5de0995f01a98b6fbc3a0c774b9a53e609a

Kevin Grittner pushed:

- Add support for REFRESH MATERIALIZED VIEW CONCURRENTLY. This allows
reads to continue without any blocking while a REFRESH runs. The
new data appears atomically as part of transaction commit. Review
questioned the Assert that a matview was not a system relation.
This will be addressed separately. Reviewed by Hitoshi Harada,
Robert Haas, Andres Freund. Merged after review with security patch
f3ab5d4.
http://git.postgresql.org/pg/commitdiff/cc1965a99bf87005f431804bbda0f723887a04d6

Tatsuo Ishii pushed:

- Add --progress option to show progress report. Patch contributed by
Fabien COELHO, reviewed by KONDO Mitsumasa.
http://git.postgresql.org/pg/commitdiff/4a87f308b33457670f9ab4bc622678e5d285b9c2

Noah Misch pushed:

- Fix systable_recheck_tuple() for MVCC scan snapshots. Since this
function assumed non-MVCC snapshots, it broke when commit
568d4138c646cd7cd8a837ac244ef2caf27c6bb8 switched its one caller
from SnapshotNow scans to MVCC-snapshot scans. Reviewed by Robert
Haas, Tom Lane and Andres Freund.
http://git.postgresql.org/pg/commitdiff/ffcf654547ef38555203e6d716f47b7065a0a87d

- Implement the FILTER clause for aggregate function calls. This is
SQL-standard with a few extensions, namely support for subqueries
and outer references in clause expressions. catversion bump due to
change in Aggref and WindowFunc. David Fetter, reviewed by Dean
Rasheed.
http://git.postgresql.org/pg/commitdiff/b560ec1b0d7b910ce13edc51ffaafaca72136e3b

- Comment on why planagg.c punts "MIN(x ORDER BY y)".
http://git.postgresql.org/pg/commitdiff/7a8e9f298e7b8158296e1ea72ca8987323c10edf

Fujii Masao pushed:

- Fix typo in previous pgbench --progress patch.
http://git.postgresql.org/pg/commitdiff/750f43685b1b88d4281074940cdcea0e328593aa

- Fix pgstattuple functions to use regclass-type as the argument.
This allows us to specify the target relation with several
expressions, 'relname', 'schemaname.relname' and OID in all
pgstattuple functions. pgstatindex() and pg_relpages() could not
accept OID as the argument so far. Per discussion on -hackers, we
decided to keep two types of interfaces, with regclass-type and
TEXT-type argument, for each pgstattuple function because of the
backward-compatibility issue. The functions which have TEXT-type
argument will be deprecated in the future release. Patch by Satoshi
Nagayasu, reviewed by Rushabh Lathia and Fujii Masao.
http://git.postgresql.org/pg/commitdiff/1dc118660b8f12d3bdec94b6f6e1068966ca62e9

- Fix typo in update scripts for some contrib modules.
http://git.postgresql.org/pg/commitdiff/6f9e39bc9993c18686f0950f9b9657c7c97c7450

Heikki Linnakangas pushed:

- Fix end-of-loop optimization in pglz_find_match() function. After
the recent pglz optimization patch, the next/prev pointers in the
hash table are never NULL, INVALID_ENTRY_PTR is used to represent
invalid entries instead. The end-of-loop check in pglz_find_match()
function didn't get the memo. The result was the same from a
correctness point of view, but because the NULL-check would never
fail, the tiny optimization turned into a pessimization. Reported
by Stephen Frost, using Coverity scanner.
http://git.postgresql.org/pg/commitdiff/3f2adace1ec025908b5189f0773b4eaab3d228d5

- Fix assert failure at end of recovery, broken by XLogInsert scaling
patch. Initialization of the first XLOG buffer at end-of-recovery
was broken for the case that the last read WAL record ended at a
page boundary. Instead of trying to copy the last full xlog page to
the buffer cache in that case, just set shared state so that the
next page is initialized when the first WAL record after startup is
inserted. (that's what we did in earlier version, too) To make the
shared state required for that case less surprising, replace the
XLogCtl->curridx variable, which was the index of the latest
initialized buffer, with an XLogRecPtr of how far the buffers have
been initialized. That also allows us to get rid of the
XLogRecEndPtrToBufIdx macro. While we're at it, make a similar
change for XLogCtl->Write.curridx, getting rid of that variable and
calculating the next buffer to write from XLogCtl->LogwrtResult
instead.
http://git.postgresql.org/pg/commitdiff/59c02a36f0ef317958b2d14313b5c8e41cfd9be6

- Fix variable names mentioned in comment to match the code. Also, in
another comment, explain why holding an insertion slot is a critical
section. Per review by Amit Kapila.
http://git.postgresql.org/pg/commitdiff/107cbc90a7564b6cd773c56ad3da8b66c7377df5

Tom Lane pushed:

- Fix a few problems in barrier.h. On HPPA, implement
pg_memory_barrier() as pg_compiler_barrier(), which should be
correct since this arch doesn't do memory access reordering, and is
anyway better than the completely-nonfunctional-on-this-arch
dummy_spinlock code. (But note this patch only fixes things for
gcc, not for builds with HP's compiler.) Also, fix incorrect default
definition of pg_memory_barrier as a macro requiring an argument.
Also, fix incorrect spelling of "#elif" as "#else if" in icc code
path (spotted by pgindent). This doesn't come close to fixing all
of the functional and stylistic deficiencies in barrier.h, but at
least it un-breaks my personal build. Now that we're actually using
barriers in the code, this file is going to need some serious
attention.
http://git.postgresql.org/pg/commitdiff/89779bf2c8f9aa480e0ceb8883f93e9d65c43a6e

- Fix direct access to Relation->rd_indpred. Should use
RelationGetIndexPredicate(), since rd_indpred is just a cache that
is not computed until/unless demanded. Per buildfarm failure on
CLOBBER_CACHE_ALWAYS animals; diagnosis and fix by Hitoshi Harada.
http://git.postgresql.org/pg/commitdiff/405a468b02dd80a069b95c4eb9838cb6a562eddd

- Fix regex match failures for backrefs combined with non-greedy
quantifiers. An ancient logic error in cfindloop() could cause the
regex engine to fail to find matches that begin later than the start
of the string. This function is only used when the regex pattern
contains a back reference, and so far as we can tell the error is
only reachable if the pattern is non-greedy (i.e. its first
quantifier uses the ? modifier). Furthermore, the actual match must
begin after some potential match that satisfies the DFA but then
fails the back-reference's match test. Reported and fixed by Jeevan
Chalke, with cosmetic adjustments by me.
http://git.postgresql.org/pg/commitdiff/e2bd904955e2221eddf01110b1f25002de2aaa83

- Add checks for valid multibyte character length in UtfToLocal,
LocalToUtf. This is mainly to suppress "uninitialized variable"
warnings from very recent versions of gcc. But it seems like a good
robustness thing anyway, not to mention that we might someday decide
to support 6-byte UTF8. Per report from Karol Trzcionka. No
back-patch since there's no reason at the moment to think this is
more than cosmetic.
http://git.postgresql.org/pg/commitdiff/d9f37e6661ec654f7252bfc23e084c9e2ca5db14

- Fix error handling in PLy_spi_execute_fetch_result(). If an error
is thrown out of the datatype I/O functions called by this function,
we need to do subtransaction cleanup, which the previous coding
entirely failed to do. Fortunately, both existing callers of this
function already have proper cleanup logic, so re-throwing the
exception is enough. Also, postpone creation of the resultset
tupdesc until after the I/O conversions are complete, so that we
won't leak memory in TopMemoryContext when such an error happens.
http://git.postgresql.org/pg/commitdiff/b3b10c39038c20457ef058c7f4e5589c28a84f1c

Andrew Dunstan pushed:

- Move checking an explicit VARIADIC "any" argument into the parser.
This is more efficient and simpler. It does mean that an untyped
NULL can no longer be used in such cases, which should be mentioned
in Release Notes, but doesn't seem a terrible loss. The workaround
is to cast the NULL to some array type. Pavel Stehule, reviewed by
Jeevan Chalke.
http://git.postgresql.org/pg/commitdiff/d26888bc4d1e539a82f21382b0000fe5bbf889d9

Michael Meskes pushed:

- Initialize day of year value. There are cases where the day of year
value in struct tm is used, but it never got calculated. Problem
found by Coverity scan.
http://git.postgresql.org/pg/commitdiff/a2c1c3308f33a293152e65bbc0d25e7976dde23d

Alvaro Herrera pushed:

- Fix HeapTupleSatisfiesVacuum on aborted updater xacts. By using
only the macro that checks infomask bits HEAP_XMAX_IS_LOCKED_ONLY to
verify whether a multixact is not an updater, and not the full
HeapTupleHeaderIsOnlyLocked, it would come to the wrong result in
case of a multixact containing an aborted update; therefore
returning the wrong result code. This would cause predicate.c to
break completely (as in bug report #8273 from David Leverton), and
certain index builds would misbehave. As far as I can tell, other
callers of the bogus routine would make harmless mistakes or not be
affected by the difference at all; so this was a pretty narrow case.
Also, no other user of the HEAP_XMAX_IS_LOCKED_ONLY macro is as
careless; they all check specifically for the HEAP_XMAX_IS_MULTI
case, and they all verify whether the updater is InvalidXid before
concluding that it's a valid updater. So there doesn't seem to be
any similar bug.
http://git.postgresql.org/pg/commitdiff/6737aa72ba7621d4db8e09210c65eecafc42b616

== Rejected Patches (for now) ==

No one was disappointed this week :-)

== Pending Patches ==

Michael Paquier sent in another revision of a patch to change
recovery.conf parameters to GUCs.

Gurjeet Singh sent in another revision of a patch to speed up
processing of long chains of AND and OR conditions.

Robins Tharakan sent in another revision of a patch to add regression
tests for ROLE (USER).

Robins Tharakan sent in another revision of a patch to add regression
tests for SCHEMA.

Robins Tharakan sent in another revision of a patch to add regression
tests for SEQUENCE.

Satoshi Nagayasu sent in two more revisions of a patch to change
pgstattuple/pgstatindex to use regclass-type as the argument.

Amit Kapila sent in another revision of a patch to implement ALTER
SYSTEM.

Satoshi Nagayasu and Kyotaro HORIGUCHI traded patches to add
visibility map information to pg_freespace.

Fabien COELHO, and Tatsuo Ishii traded patches to implement pgbench --throttle.

Hari Babu sent in another revision of a patch to improve performance
by reducing WAL for update operations.

David Fetter sent in a patch to allow foreign tables to be partitions
of local tables.

Robert Haas sent in a patch to get rid of SnapshotNow.

Greg Smith sent in another revision of a patch to implement row-level
accesss control.

Atri Sharma sent in a PoC patch to implement WITHIN GROUP, one
application of which is median.

Karol Trzcionka sent in two more revisions of a patch to implement
UPDATE ... RETURNING BEFORE/AFTER.

Tom Lane sent in a patch to prevent tuple-table leakage in plpgsql.

Heikki Linnakangas sent in a WIP patch to do segment-at-a-time writes
of dirty buffers in checkpoints.

Noah Misch sent in a patch to make all REINDEXes re-validate
constraints, as this behavior had accidentally been removed by an
earlier patch.

Quan Zongliang sent in a patch to improve performance for
Chinese-language databases.