Skip site navigation (1) Skip section navigation (2)

== PostgreSQL Weekly News - August 14 2017 ==

From: David Fetter <david(at)fetter(dot)org>
To: PostgreSQL Announce <pgsql-announce(at)postgresql(dot)org>
Subject: == PostgreSQL Weekly News - August 14 2017 ==
Date: 2017-08-14 01:51:44
Message-ID: (view raw, whole thread or download thread mbox)
Lists: pgsql-announce
== PostgreSQL Weekly News - August 14 2017 ==

PostgreSQL 10 Beta 3 released.  Test!

Security releases 9.6.4, 9.5.8, 9.4.13, 9.3.18 and 9.2.22 are out.  Please read
the announcement below and upgrade at the next available downtime.

2ndQuadrant PostgreSQL Conference 2017 (2Q PGConf, for short) will be hosted on
November 6th & 7th in New York City, and November 9th in Chicago.

PGConf Local: Seattle Has been rescheduled to November 13 - 14, 2017.
Registration at

PGConf Local: Austin will be held December 4 - 5, 2017. Call for Papers is
now open at

== PostgreSQL Product News ==

PGroonga 1.2.3, a full text search platform for all languages, released.

JDBC 42.1.4 released

psqlODBC 09.06.0410 released.

psycopg2 2.7.3, a Python connector for PostgreSQL, released.

pg_repack 1.4.1, an extension for removing bloat, released.

== PostgreSQL Jobs for August ==

== PostgreSQL Local ==

PostgresOpen will occur September 6-8, 2017 in San Francisco.

PGBR2017 will take place in Porto Alegre, Rio Grande do Sul,
Brazil on September 14-16 2017.

PGDay.IT 2017 will take place in October 13th, in Milan, Italy.

PostgreSQL Conference Europe 2017 will be held on October 24-27, 2017 in the
Warsaw Marriott Hotel, in Warsaw, Poland.

PGConf Local: Seattle will be held November 13 - 14, 2017.

PGDay Austin 2017 will be held on December 4 - 5, 2017.

PGConf.ASIA 2017 will take place on December 4-6 2017 in Akihabara, Tokyo,

== PostgreSQL in the News ==

Planet PostgreSQL:

PostgreSQL Weekly News is brought to you this week by David Fetter

Submit news and announcements by Sunday at 3:00pm Pacific time.
Please send English language ones to david(at)fetter(dot)org, German language
to pwn(at)pgug(dot)de, Italian language to pwn(at)itpug(dot)org(dot)

== Applied Patches ==

Noah Misch pushed:

- Disclaim xmltable() support for non-UTF8 databases.  The xmltable()
  implementation mirrors xpath(), including its lack of character encoding

- Fix vertical spanning in table "wait_event Description".  Michael Paquier

Álvaro Herrera pushed:

- Fix replication origin-related race conditions.  Similar to what was fixed in
  commit 9915de6c1cb2 for replication slots, but this time it's related to
  replication origins: DROP SUBSCRIPTION attempts to drop the replication
  origin, but that fails if the replication worker process hasn't yet marked it
  unused.  This causes failures in the buildfarm: ERROR:  could not drop
  replication origin with OID 1, in use by PID 34069 Like the aforementioned
  commit, fix by having the process running DROP SUBSCRIPTION sleep until the
  worker marks the the replication origin struct as free.  This uses a condition
  variable on each replication origin shmem state struct, so that the session
  trying to drop can sleep and expect to be awakened by the process keeping the
  origin open.  Also fix a SGML markup in the previous commit.  Discussion:

- Fix inadequacies in recently added wait events In commit 9915de6c1cb2, we
  introduced a new wait point for replication slots and incorrectly labelled it
  as wait event PG_WAIT_LOCK.  That's wrong, so invent an appropriate new wait
  event instead, and document it properly.  While at it, fix numerous other
  problems in the vicinity: - two different walreceiver wait events were being
  mixed up in a single wait event (which wasn't documented either); split it out
  so that they can be distinguished, and document the new events properly.  -
  ParallelBitmapPopulate was documented but didn't exist.  - ParallelBitmapScan
  was not documented (I think this should be called "ParallelBitmapScanInit"
  instead.) - Logical replication wait events weren't documented - various
  symbols had been added in dartboard order in various places.  Put them in
  alphabetical order instead, as was originally intended.  Discussion:

- Fix typo in comment.

- Reword some unclear comments.

- Reword comment for clarity.  Reported by Masahiko Sawada Discussion:

Tom Lane pushed:

- Fix yet another race condition in recovery/t/  In commit
  5c77690f6, we added polling in front of most of the get_slot_xmins calls in, but today's results from buildfarm member nightjar show
  that at least one more poll loop is needed.  Proactively add a poll loop
  before the next-to-last get_slot_xmins call as well.  It may be that there is
  no race condition there because the standby_2 server is shut down at that
  point, but I'm quite tired of fighting with this test script.  The empirical
  evidence that it's safe, from the buildfarm, is no stronger than the evidence
  for the other call that nightjar just proved unsafe.  The only remaining
  get_slot_xmins calls without wait_slot_xmins protection are the first two,
  which should be OK since nothing has happened at that point.  It's tempting to
  ignore that special case and merge get_slot_xmins and wait_slot_xmins into a
  single function.  I didn't go that far though.  Discussion:

- Fix datumSerialize infrastructure to not crash on non-varlena data.  Commit
  1efc7e538 did a poor job of emulating existing logic for touching Datums that
  might be expanded-object pointers.  It didn't check for typlen being -1 first,
  which meant it could crash on fixed-length pass-by-ref values, and probably on
  cstring values as well.  It also didn't use DatumGetPointer before
  VARATT_IS_EXTERNAL_EXPANDED, which while currently harmless is not according
  to documentation nor prevailing style.  I also think the lack of any
  explanation as to why datumSerialize makes these particular nonobvious choices
  is pretty awful, so fix that.  Per report from Jarred Ward.  Back-patch to 9.6
  where this code came in.  Discussion:

- Prevent passing down MAKELEVEL/MAKEFLAGS from non-GNU make to GNU make.
  FreeBSD's make, for one, sets the MAKELEVEL environment variable when invoking
  commands.  In the special Makefile we provide to hand off control from a
  non-GNU make to GNU make, this causes GNU make to think it is a child make
  invocation rather than top-level.  That interferes with the hack added in
  commit dcae5facc to cause the temp-install tree to be made only by the
  top-level invocation of gmake.  Unset the variable to prevent that.  Likewise
  unset MAKEFLAGS, which FreeBSD's make also sets, and which could easily
  confuse gmake.  There are no reports of actual trouble from that, but it seems
  better to be proactive.  Back-patch to 9.5 where dcae5facc came in.  Thomas
  Munro, hacked a bit more by me Discussion:

- Fix handling of container types in find_composite_type_dependencies.
  find_composite_type_dependencies correctly found columns that are of the
  specified type, and columns that are of arrays of that type, but not columns
  that are domains or ranges over the given type, its array type, etc.  The most
  general way to handle this seems to be to assume that any type that is
  directly dependent on the specified type can be treated as a container type,
  and processed recursively (allowing us to handle nested cases such as ranges
  over domains over arrays ...).  Since a type's array type already has such a
  dependency, we can drop the existing special case for the array type.  The
  very similar logic in get_rels_with_domain was likewise a few bricks shy of a
  load, as it supposed that a directly dependent type could *only* be a
  sub-domain.  This is already wrong for ranges over domains, and it'll someday
  be wrong for arrays over domains.  Add test cases illustrating the problems,
  and back-patch to all supported branches.  Discussion:

- Remove pgbench's restriction on placement of -M switch.  Previously the -M
  switch had to appear before any switch that directly or indirectly specified a
  benchmarking script.  This was both confusing and inadequately documented, as
  per gripe from Tatsuo Ishii.  We can remove the restriction at the cost of
  making an extra pass over the lists of SQL commands, which seems like a cheap
  price (the string scans themselves likely cost much more).  The change is just
  to not extract parameters from the SQL commands until we have finished parsing
  the switches and know the final value of -M.  Per discussion, we'll treat this
  as a low-grade bug fix and sneak it into v10, rather than holding it for v11.
  Tom Lane, reviewed by Tatsuo Ishii and Fabien Coelho Discussion:

- Add regression tests exercising more code paths in nodeLimit.c.  Perusal of
  the code coverage report shows that the existing regression test cases for
  LIMIT/OFFSET don't exercise the nodeLimit code paths involving backwards scan,
  empty results, or null values of LIMIT/OFFSET.  Improve the coverage.

- Add regression tests exercising the non-hashed code paths in nodeSetop.c.
  Perusal of the code coverage report shows that the existing regression test
  cases for INTERSECT and EXCEPT seemingly all prefer the SETOP_HASHED
  implementation.  Add some test cases in which we force use of the SETOP_SORTED

- Be more thorough about cleaning out gcov litter.  At least on my machine, a
  run with code coverage enabled produces some ".gcov" files whose names begin
  with ".".  "rm -f *.gcov" fails to match those, so they don't get cleaned up
  by "make clean".  Fix it.

- Simplify fetch-slot-xmins logic in recovery TAP tests.  Merge
  wait_slot_xmins() into get_slot_xmins().  At this point the only place that
  wasn't doing a wait was the initial-state test, and a wait there seems pretty
  harmless.  Michael Paquier Discussion:

- Remove AtEOXact_CatCache().  The sole useful effect of this function, to check
  that no catcache entries have positive refcounts at transaction end, has
  really been obsolete since we introduced ResourceOwners in PG 8.1.  We reduced
  the checks to assertions years ago, so that the function was a complete no-op
  in production builds.  There have been previous discussions about removing it
  entirely, but consensus up to now was that it had some small value as a
  cross-check for bugs in the ResourceOwner logic.  However, it now emerges that
  it's possible to trigger these assertions if you hit an assert-enabled backend
  with SIGTERM during a call to SearchCatCacheList, because that function
  temporarily increases the refcounts of entries it's intending to add to a
  catcache list construct.  In a normal ERROR scenario, the extra refcounts are
  cleaned up by SearchCatCacheList's PG_CATCH block; but in a FATAL exit we do a
  transaction abort and exit without ever executing PG_CATCH handlers.  There's
  a case to be made that this is a generic hazard and we should consider
  restructuring elog(FATAL) handling so that pending PG_CATCH handlers do get
  run.  That's pretty scary though: it could easily create more problems than it
  solves.  Preliminary stress testing by Andreas Seltenreich suggests that there
  are not many live problems of this ilk, so we rejected that idea.  There are
  more-localized ways to fix the problem; the most principled one would be to
  use PG_ENSURE_ERROR_CLEANUP instead of plain PG_TRY.  But adding cycles to
  SearchCatCacheList isn't very appealing.  We could also weaken the assertions
  in AtEOXact_CatCache in some more or less ad-hoc way, but that just makes its
  raison d'etre even less compelling.  In the end, the most reasonable solution
  seems to be to just remove AtEOXact_CatCache altogether, on the grounds that
  it's not worth trying to fix it.  It hasn't found any bugs for us in many
  years.  Per report from Jeevan Chalke.  Back-patch to all supported branches.

Peter Eisentraut pushed:

- doc: Add missing pieces to logical replication protocol doc.  Reported-by:
  Kyle Conroy <kyle(at)kyleconroy(dot)com>

- Fix order of ICU_CFLAGS.  It must be before CPPFLAGS so that an ICU
  installation in a nonstandard path can take precedence over one in the system

- Reject use of ucol_strcollUTF8() before ICU 53.  Various bugs can cause
  crashes, so don't use that function before ICU 53.  It will fall back to the
  code path used for other encodings.  Since we now tie the function
  availability to an ICU version, we don't need the configure test anymore.
  That also resolves the issue that the test result was previously hardcoded for
  Windows.  researched by Daniel Verite <daniel(at)manitou-mail(dot)org>, Peter
  Geoghegan <pg(at)bowt(dot)ie>, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> Discussion:

- Remove uses of "slave" in replication contexts.  This affects mostly code
  comments, some documentation, and tests.  Official APIs already used

- pg_upgrade: Clarify one message.  Reported-by: Dennis Björklund

- doc: Small wording improvement.  Author: Jeff Janes <jeff(dot)janes(at)gmail(dot)com>

- doc: Update description of rolreplication column.  Since PostgreSQL 9.6,
  rolreplication no longer determines whether a role can run pg_start_backup()
  and pg_stop_backup(), so remove that.  Add that this attribute determines
  whether a role can create and drop replication slots.  Reported-by: Fujii
  Masao <masao(dot)fujii(at)gmail(dot)com>

- doc: Add example for inet vs cidr difference.  Reported-by: kes-kes(at)yandex(dot)ru

Robert Haas pushed:

- Remove incorrect assertion in clog.c.  We must advance the oldest XID that can
  be safely looked up in clog *before* truncating CLOG, and the oldest XID that
  can't be reused *after* truncating CLOG.  This assertion, and the accompanying
  comment, are confused; remove them.  Reported by Neha Sharma.  Discussion:

- pgstatindex: Insert some casts to prevent overflow.  This could cause hash
  indexes to report greater than 100% free space.  Ashutosh Sharma, reviewed by
  Amit Kapila Discussion:

- Fix typo in comment.  Etsuro Fujita Discussion:

- Make some more improvements to parallel query documentation.  Many places that
  mentioned only Gather should also mention Gather Merge, or should be phrased
  in a more neutral way.  Be more clear about the fact that
  max_parallel_workers_per_gather affects the number of workers the planner may
  want to use.  Fix a typo.  Explain how Gather Merge works.  Adjust wording
  around parallel scans to be a bit more clear.  Adjust wording around
  parallel-restricted operations for the fact that uncorrelated subplans are no
  longer restricted.  Patch by me, reviewed by Erik Rijkers Discussion:

- Improve the error message when creating an empty range partition.  The
  previous message didn't mention the name of the table or the bounds.  Put the
  table name in the primary error message and the bounds in the detail message.
  Amit Langote, changed slightly by me.  Suggestions on the exac phrasing from
  Tom Lane, David G. Johnston, and Dean Rasheed.  Discussion:

== Pending Patches ==

Piotr Stefaniak sent in a PoC patch to clarify the "sorry, too many clients
already" error.

Mark Rofail sent in another revision of a patch to implement Foreign Key Arrays.

Masahiko Sawada sent in a patch to enable pgbench to skip creating primary keys
after initialization.

Ashutosh Bapat and Thomas Munro traded patches to enable partition-wise joins
between declaratively partitioned tables.

Shubham Barai sent in another revision of a patch to add predicate locking in
GIN indexes.

Masahiko Sawada sent in another revision of a patch to improve the B-tree code.

Beena Emerson sent in two more revisions of a patch to add default partition
support for range partitions.

Amit Kapila sent in two more revisions of a patch to parallelize queries
containing initplans.

Masahiko Sawada sent in a patch to fix a typo in the login replication message.

Thomas Munro sent in another revision of a patch to log LDAP "diagnostic

Thomas Munro sent in another revision of a patch to implement causal reads.

Vaishnavi Prabakaran sent in another revision of a patch to add batch/pipelining
support for libpq.

Amit Khandekar sent in another revision of a patch to implement parallel append.

Michaël Paquier sent in a patch to enable creating backup history files for
backups taken from standbys.

Amit Langote sent in a patch to expand inheritance in partition bound order.

Oliver Ford sent in a patch to fix number-skipping in to_number.

Tom Lane sent in a patch to fix an issue where the PL/perl extension failed to
build on Windows.

Nicolas Thauvin and Ashutosh Bapat traded patches to fix an issue where foreign
tables privileges were not shown in information_schema.table_privileges.

Rushabh Lathia and Robert Haas traded patches to enable loading partitioned data
through the root.

Yura Sokolov sent in another revision of a patch to make a hash table for xip in

Michaël Paquier sent in a patch to simplify the TAP test around replication

Peter Eisentraut sent in a patch to rework and adjust how the coverage analysis
tools are run.

Amit Khandekar sent in another revision of a patch to enable UPDATEs of a
partition key.

Ashutosh Sharma sent in another revision of a patch to rewrite hash index scan
to work page at a time, remove redundant hash function _hash_step and do some
code cleanup, and improve locking startegy during VACUUM in Hash Index for
regular tables.

Thomas Munro sent in another revision of a patch to enable sharing record
typmods among backends.

Peter Eisentraut sent in a patch to add new contrib test suites.

Petr Jelínek sent in a patch to allow pg_dump -j on standby when dumping from

Dmitry Dolgov sent in another revision of a patch to enable generic type

Alik Khilazhev sent in another revision of a patch to enable generating zipfian
distributions in pgbench.

Fabien COELHO sent in another revision of a patch to enable pgbench to store
select results into variables.

Marko Tiikkaja sent in another revision of a patch to enable INSTEAD OF DELETE
triggers to modify the tuple for RETURNING.

Tom Lane sent in another revision of a patch to fix a bug that broke replication
to Windows in PostgreSQL 10.

Tomas Vondra sent in a patch to add multivariate histograms and MCV lists.

Tomas Vondra sent in a patch to create a generational memory allocator.

pgsql-announce by date

Next:From: D'Arcy CainDate: 2017-08-15 12:21:32
Subject: Release of PyGreSQL 5.0.4
Previous:From: Daniele VarrazzoDate: 2017-08-12 12:22:29
Subject: pg_repack 1.4.1 released

Privacy Policy | About PostgreSQL
Copyright © 1996-2018 The PostgreSQL Global Development Group