== PostgreSQL Weekly News - August 23 2015 ==

== PostgreSQL Weekly News - August 23 2015 ==

PgCUBA will take place October 19-23, 2015 in Havana. Spanish
language information below:
http://www.postgresql.org/about/event/1813/

== PostgreSQL Product News ==

Further enhancements to 2UDA (Alpha 2 release), a platform for data
analytics, available now.
http://2ndquadrant.com/2uda

2ndQuadrant now offers production support on IBM z Series mainframe
http://2ndquadrant.com/en/support/support-ibm-z-systems/

== PostgreSQL Jobs for August ==

http://archives.postgresql.org/pgsql-jobs/2015-08/

== PostgreSQL Local ==

PostgresOpen 2015 will being held in Dallas, Texas September 16-18.
http://2015.postgresopen.org/

PostgreSQL Session #7, will be held September 24th, 2015 in Paris,
France,
http://www.postgresql-sessions.org/7/about

PGDay.IT 2015 will take place in Prato on October 23, 2015.
http://pgday.it

PostgreSQL Conference Europe 2015 will be held on October 27-30 in the
Vienna Marriott Hotel, in Vienna, Austria.
http://2015.pgconf.eu/

PGConf Silicon Valley 2015 is November 17-18 at the South San
Francisco Convention Center.
http://www.pgconfsv.com

PGBR2015 (The Brazilian PostgreSQL Conference) will take place in Porto
Alegre, Rio Grande do Sul, on November 18, 19 and 20. The CfP is open
until August 31.
http://pgbr.postgresql.org.br/2015/en/#call-for-papers

== PostgreSQL in the News ==

Planet PostgreSQL: http://planet.postgresql.org/

PostgreSQL Weekly News is brought to you this week by David Fetter

Submit news and announcements by Sunday at 3:00pm Pacific time.
Please send English language ones to david(at)fetter(dot)org, German language
to pwn(at)pgug(dot)de, Italian language to pwn(at)itpug(dot)org(dot) Spanish language
to pwn(at)arpug(dot)com(dot)ar(dot)

== Applied Patches ==

Heikki Linnakangas pushed:

- Fix reporting of skipped transactions in pgbench. Broken by commit
1bc90f7a. Fabien Coelho.
http://git.postgresql.org/pg/commitdiff/0e8efed59e7f98f237e90fba20364165f7cc9ba9

- Add hint to run "pgbench -i", if test tables don't exist. Fabien
Coelho, reviewed by Julien Rouhaud
http://git.postgresql.org/pg/commitdiff/e7b90c52b3ee440f63445ba8f83419fd713069cf

Andres Freund pushed:

- Improve configure test for the sse4.2 crc instruction. With
optimizations enabled at least one compiler, clang 3.7, optimized
away the crc intrinsics knowing that the result went on unused and
has no side effects. That can trigger errors in code generation when
the intrinsic is used, as we chose to use the intrinsics without any
additional compiler flag. Return the computed value to prevent that.
With some more pedantic warning flags (-Wold-style-definition) the
configure test failed to recognize the existence of _mm_crc32_u*
intrinsics due to an independent warning in the test because the
test turned on -Werror, but that's not actually needed here.
Discussion: 20150814092039(dot)GH4955(at)awork2(dot)anarazel(dot)de Backpatch: 9.5,
where the use of crc intrinsics was integrated.
http://git.postgresql.org/pg/commitdiff/6cf72879e944f32b5b8232541cccd575de797fa4

- docs: Fix "typo" introduced in 3f811c2d. Reported-By: Michael
Paquier Discussion:
CAB7nPqSco+RFw9C-VgbCpyurQB3OocS-fuTOa_gFnUy1EE-pyQ(at)mail(dot)gmail(dot)com
http://git.postgresql.org/pg/commitdiff/47ebbdcee7bc4e8dd1b88750ed778c61c4c5ec1b

Tom Lane pushed:

- Fix performance bug from conflict between two previous improvements.
My expanded-objects patch (commit 1dc5ebc9077ab742) included code to
make plpgsql pass expanded-object variables as R/W pointers to
certain functions that are trusted for modifying such variables
in-place. However, that optimization got broken by commit
6c82d8d1fdb1f126, which arranged to share a single ParamListInfo
across most expressions evaluated by a plpgsql function. We don't
want a R/W pointer to be passed to other functions just because we
decided one function was safe! Fortunately, the breakage was in the
other direction, of never passing a R/W pointer at all, because we'd
always have pre-initialized the shared array slot with a R/O
pointer. So it was still functionally correct, but we were back to
O(N^2) performance for repeated use of "a := a || x". To fix, force
an unshared param array to be used when the R/W param optimization
is active. Commit 6c82d8d1fdb1f126 is in HEAD only, so no need for
a back-patch.
http://git.postgresql.org/pg/commitdiff/d3eaab3ef0d552a2f6555b0424a32dc9e77fb17c

- Fix a few bogus statement type names in plpgsql error messages.
plpgsql's error location context messages ("PL/pgSQL function
fn-name line line-no at stmt-type") would misreport a CONTINUE
statement as being an EXIT, and misreport a MOVE statement as being
a FETCH. These are clear bugs that have been there a long time, so
back-patch to all supported branches. In addition, in 9.5 and HEAD,
change the description of EXECUTE from "EXECUTE statement" to just
plain EXECUTE; there seems no good reason why this statement type
should be described differently from others that have a well-defined
head keyword. And distinguish GET STACKED DIAGNOSTICS from plain
GET DIAGNOSTICS. These are a bit more of a judgment call, and also
affect existing regression-test outputs, so I did not back-patch
into stable branches. Pavel Stehule and Tom Lane
http://git.postgresql.org/pg/commitdiff/2edb9491155ad70e57b5d18f2aa0d8d5a09386cd

- Remove xpath namespace-handling change from 9.5 release notes.
Although commit 79af9a1d2 was initially applied to HEAD only, we
later back-patched the change into all branches (commits 6bbf75192
et al). So it's not a new behavior in 9.5 and should not be
release-noted here.
http://git.postgresql.org/pg/commitdiff/a93545e13f832d457e00420d44ccce1f88f899d4

- Allow record_in() and record_recv() to work for transient record
types. If we have the typmod that identifies a registered record
type, there's no reason that record_in() should refuse to perform
input conversion for it. Now, in direct SQL usage, record_in() will
always be passed typmod = -1 with type OID RECORDOID, because no
typmodin exists for type RECORD, so the case can't arise. However,
some InputFunctionCall users such as PLs may be able to supply the
right typmod, so we should allow this to support them. Note: the
previous coding and comment here predate commit 59c016aa9f490b53.
There has been no case since 8.1 in which the passed type OID
wouldn't be valid; and if it weren't, this error message wouldn't be
apropos anyway. Better to let lookup_rowtype_tupdesc complain about
it. Back-patch to 9.1, as this is necessary for my upcoming
plpython fix. I'm committing it separately just to make it a bit
more visible in the commit history.
http://git.postgresql.org/pg/commitdiff/09b3d27256d26e258c7802cfd8fea06d81b0a62c

- Fix plpython crash when returning string representation of a RECORD
result. PLyString_ToComposite() blithely overwrote
proc->result.out.d, even though for a composite result type the
other union variant proc->result.out.r is the one that should be
valid. This could result in a crash if out.r had in fact been
filled in (proc->result.is_rowtype == 1) and then somebody later
attempted to use that data; as per bug #13579 from Paweł Michalak.
Just to add insult to injury, it didn't work for RECORD results
anyway, because record_in() would refuse the case. Fix by doing the
I/O function lookup in a local PLyTypeInfo variable, as we were
doing already in PLyObject_ToComposite(). This is not a great
technique because any fn_extra data allocated by the input function
will be leaked permanently (thanks to using TopMemoryContext as
fn_mcxt). But that's a pre-existing issue that is much less serious
than a crash, so leave it to be fixed separately. This bug would be
a potential security issue, except that plpython is only available
to superusers and the crash requires coding the function in a way
that didn't work before today's patches. Add regression test cases
covering all the supported methods of converting composite results.
Back-patch to 9.1 where the faulty coding was introduced.
http://git.postgresql.org/pg/commitdiff/f469f634ad7338b7eab046238354f07930fb6bca

- Detect mismatched CONTINUE and EXIT statements at plpgsql compile
time. With a bit of tweaking of the compile namestack data
structure, we can verify at compile time whether a CONTINUE or EXIT
is legal. This is surely better than leaving it to runtime, both
because earlier is better and because we can issue a proper error
pointer. Also, we can get rid of the ad-hoc old way of detecting
the problem, which only took care of CONTINUE not EXIT. Jim Nasby,
adjusted a bit by me
http://git.postgresql.org/pg/commitdiff/fcdfce6820373422bcdb5630f9eb63df14fd0764

- Avoid O(N^2) behavior when enlarging SPI tuple table in
spi_printtup(). For no obvious reason, spi_printtup() was coded to
enlarge the tuple pointer table by just 256 slots at a time, rather
than doubling the size at each reallocation, as is our usual habit.
For very large SPI results, this makes for O(N^2) time spent in
repalloc(), which of course soon comes to dominate the runtime. Use
the standard doubling approach instead. This is a longstanding
performance bug, so back-patch to all active branches. Neil Conway
http://git.postgresql.org/pg/commitdiff/6e5d9f278c1209936d973930996857f55e119cd8

- Avoid use of float arithmetic in bipartite_match.c. Since the
distances used in this algorithm are small integers (not more than
the size of the U set, in fact), there is no good reason to use
float arithmetic for them. Use short ints instead: they're smaller,
faster, and require no special portability assumptions. Per testing
by Greg Stark, which disclosed that the code got into an infinite
loop on VAX for lack of IEEE-style float infinities. We don't
really care all that much whether Postgres can run on a VAX anymore,
but there seems sufficient reason to change this code anyway. In
passing, make a few other small adjustments to make the code match
usual Postgres coding style a bit better.
http://git.postgresql.org/pg/commitdiff/44ed65a545970829322098e22d10947e6d545d9a

- Reduce number of bytes examined by convert_one_string_to_scalar().
Previously, convert_one_string_to_scalar() would examine up to 20
bytes of the input string, producing a scalar conversion with
theoretical precision far greater than is of any possible use
considering the other limitations on the accuracy of the resulting
selectivity estimate. (I think this choice might pre-date the
caller-level logic that strips any common prefix of the strings;
before that, there could have been value in scanning the strings far
enough to use all the precision available in a double.) Aside from
wasting cycles to little purpose, this choice meant that the "denom"
variable could grow to as much as 256^21 = 3.74e50, which could
overflow in some non-IEEE float arithmetics. While we don't really
support any machines with non-IEEE arithmetic anymore, this still
seems like quite an unnecessary platform dependency. Limit the scan
to 12 bytes instead, thus limiting "denom" to 256^13 = 2.03e31, a
value more likely to be computable everywhere. Per testing by Greg
Stark, which showed overflow failures in our standard regression
tests on VAX.
http://git.postgresql.org/pg/commitdiff/aad663a0b4af785d0b245bbded27537f23932839

Robert Haas pushed:

- psql: Make EXECUTE PROCEDURE tab completion a bit narrower. If the
user has typed GRANT EXECUTE, the correct completion is "ON", not
"PROCEDURE". Daniel Verite
http://git.postgresql.org/pg/commitdiff/db5a703bf69d6302d0d515a9068baebffd2d227e

Kevin Grittner pushed:

- Fix bug in calculations of hash join buckets. Commit
8cce08f168481c5fc5be4e7e29b968e314f1b41e used a left-shift on a
literal of 1 that could (in large allocations) be shifted by 31 or
more bits. This was assigned to a local variable that was already
declared to be a long to protect against overruns of int, but the
literal in this shift needs to be declared long to allow it to work
correctly in some compilers. Backpatch to 9.5, where the bug was
introduced. Report and patch by KaiGai Kohei, slighly modified
based on discussion.
http://git.postgresql.org/pg/commitdiff/1cac8c98201cb0ed1a3592cf355a2670d3771689

- Fix typo in C comment. Merlin Moncure Backpatch to 9.5, where the
misspelling was introduced
http://git.postgresql.org/pg/commitdiff/5956b7f9e858ac5613dd0214ac7fb2476f900771

Peter Eisentraut pushed:

- Update config.guess and config.sub
http://git.postgresql.org/pg/commitdiff/960ea971e66bcd621ba88841b4cb85c7f0e7c383

- doc: Whitespace and formatting fixes
http://git.postgresql.org/pg/commitdiff/90a1d0aa76a7a95ff36f72cadd0d7aa0c3cb9b31

- Improve whitespace
http://git.postgresql.org/pg/commitdiff/b386271594687205becc9518cff2d2a4dca2c933

- Improve spelling
http://git.postgresql.org/pg/commitdiff/6103b3f368e1a379705f866c74a9be1e28cb1c77

Stephen Frost pushed:

- Rename 'cmd' to 'cmd_name' in CreatePolicyStmt. To avoid confusion,
rename CreatePolicyStmt's 'cmd' to 'cmd_name',
parse_policy_command's 'cmd' to 'polcmd', and AlterPolicy's
'cmd_datum' to 'polcmd_datum', per discussion with Noah and as a
follow-up to his correction of copynodes/equalnodes handling of the
CreatePolicyStmt 'cmd' field. Back-patch to 9.5 where the
CreatePolicyStmt was introduced, as we are still only in alpha.
http://git.postgresql.org/pg/commitdiff/3c99788797e8269ac19c7c8e3fb99dd9613646ea

- In AlterRole, make bypassrls an int. When reworking bypassrls in
AlterRole to operate the same way the other attribute handling is
done, I missed that the variable was incorrectly a bool rather than
an int. This meant that on platforms with an unsigned char, we
could end up with incorrect behavior during ALTER ROLE. Pointed out
by Andres thanks to tests he did changing our bool to be the one
from stdbool.h which showed this and a number of other issues. Add
regression tests to test CREATE/ALTER role for the various role
attributes. Arrange to leave roles behind for testing pg_dumpall,
but none which have the LOGIN attribute. Back-patch to 9.5 where
the AlterRole bug exists.
http://git.postgresql.org/pg/commitdiff/7ec8296e70f0f03cbdb3e0eb4f05ad5be0f810c8

- Clean up roles from roleattributes test. Having the roles remain
after the test ends up causing repeated 'make installcheck' runs to
fail and may be risky from a security perspective also, so remove
them at the end of the test.
http://git.postgresql.org/pg/commitdiff/072710dff3eef4540f1c64d07890eb128535e212

Álvaro Herrera pushed:

- Remove ExecGetScanType function. This became unused in
a191a169d6d0b9558da4519e66510c4540204a51.
http://git.postgresql.org/pg/commitdiff/8c3d63c521a28c2224aefbdc28ae7e24e81a0156

- Do not allow *timestamp to be passed as NULL. The code had bugs
that would cause crashes if NULL was passed as that argument
(originally intended to mean not to bother returning its value), and
after inspection it turns out that nothing seems interested in the
case that *ts is NULL anyway. Therefore, remove the partial checks
intended to support that case. Author: Michael Paquier though I
didn't include a proposed Assert. Backpatch to 9.5.
http://git.postgresql.org/pg/commitdiff/e68be16b0d0e857bc05949b8ee78c5aaf7d8954a

== Rejected Patches (for now) ==

No one was disappointed this week :-)

== Pending Patches ==

Mark Johnston sent in a patch to fix a bug in the make rules when
DTrace is enabled.

Fabien COELHO sent in another revision of a patch to add pgbench
statistics per script, etc.

Robert Haas sent in a flock of patches to allow multiple isolation
tester sessions to wait, specify permutations for isolation tests with
"invalid" permutations, and aAdd simple isolation tests for deadlock
detection.

Ewan Higgs sent in a patch to add stdatomic.h and support for same to
the thread_test.c.

Michael Paquier sent in another revision of a patch to document the
fact that partial and incomplete WALs can be archived upon promotion
for debugging purposes.

Michael Paquier sent in another revision of a patch to implement SCRAM
authentication.

Dmitry Dolgov sent in a patch to add jsonb array-style subscripting
and auto-vivification.

Jeff Janes sent in a patch which takes a ShareUpdateExclusiveLock lock
on the index in order to clean the GIN pending list.

Fabien COELHO sent in two more revisions of a patch to implement
checkpointer continuous flushing.

Marko Tiikkaja sent in another revision of a patch to add support for
RADIUS passwords longer than 16 octets.

Kyotaro HORIGUCHI sent in a patch to export the psql parser for use in
pgbench.

Paul A Jungwirth sent in a patch to add GiST support for UUIDs.

Kaigai Kouhei sent in a patch to fix an issue where the number of hash
buckets can overflow its type.

Qingqing Zhou sent in a patch to substitute CTEs with subqueries,
resulting in a significant performance boost.

SAWADA Masahiko sent in another revision of a patch to add a "frozen"
bit to the visibility map.

Jeff Janes and Tom Lane traded patches to make HeapTupleSatisfiesMVCC
more concurrent.

John Naylor and Pavel Stěhule traded patches for CONTINUE in PL/pgsql.

Tomas Vondra sent in another revision of a patch to use foreign keys
to improve join estimates.

Peter Geoghegan sent in a patch to use quicksort for every external
sort run.

Tomas Vondra sent in two revisions of a patch to fix hash bucket
allocation.

Alexander Shulgin sent in two more revisions of a patch to add
deparsing utility commands.

Tom Lane sent in a patch to summarize memory context stats.

Joe Conway sent in another revision of a patch to add pg_controldata
and pg_config as functions.

Tomas Vondra and Fabien COELHO traded patches to allow numeric
timestamp in log_line_prefix.

Fabien COELHO sent in a patch to add pgbench progress with timestamps.

Michael Paquier sent in two more revisions of a patch to add a
function for SSL extension information in sslinfo and some sanity
checks in sslinfo.

Pavel Stěhule sent in another revision of a patch to add a
--strict-names mode to pg_dump.

Pavel Stěhule sent in a patch to add a parse_ident() function.