Skip site navigation (1) Skip section navigation (2)

== PostgreSQL Weekly News - July 16 2017 ==

From: David Fetter <david(at)fetter(dot)org>
To: PostgreSQL Announce <pgsql-announce(at)postgresql(dot)org>
Subject: == PostgreSQL Weekly News - July 16 2017 ==
Date: 2017-07-16 18:35:40
Message-ID: (view raw, whole thread or download thread mbox)
Lists: pgsql-announce
== PostgreSQL Weekly News - July 16 2017 ==

PostgreSQL 10 Beta 2 released.  Test!

== PostgreSQL Product News ==

Pgpool-II 3.6.5, 3.5.9, 3.4.12, 3.3.16 and 3.2.21 released.

== PostgreSQL Jobs for July ==

== PostgreSQL Local ==

PGBR2017 will take place in Porto Alegre, Rio Grande do Sul,
Brazil on September 14-16 2017.

PostgresOpen will occur September 6-8, 2017 in San Francisco.

PGDay Austin 2017 will be held on Saturday, August 26 2017.

PGConf Local: Seattle will be held August 11 - 12, 2017.

PGDay.IT 2017 will take place in October 13th, in Milan, Italy.  The CfP is open
until July 9th, 2017 at

PostgreSQL Conference Europe 2017 will be held on October 24-27, 2017 in the
Warsaw Marriott Hotel, in Warsaw, Poland.

PGConf.ASIA 2017 will take place on December 4-6 2017 in Akihabara, Tokyo,
Japan.  The CfP is open until July 31, 2017.  Send submissions to
pgconf-asia-2017-submission AT pgconf DOT asia

== PostgreSQL in the News ==

Planet PostgreSQL:

PostgreSQL Weekly News is brought to you this week by David Fetter

Submit news and announcements by Sunday at 3:00pm Pacific time.
Please send English language ones to david(at)fetter(dot)org, German language
to pwn(at)pgug(dot)de, Italian language to pwn(at)itpug(dot)org(dot)

== Applied Patches ==

Tom Lane pushed:

- Doc: clarify wording about tool requirements in sourcerepo.sgml.  Original
  wording had confusingly vague antecedent for "they", so replace that with a
  more repetitive but clearer formulation.  In passing, make the link to the
  installation requirements section more specific.  Per gripe from Martin Mai,
  though this is not the fix he initially proposed.  Discussion:

- Doc: desultory copy-editing for v10 release notes.  Improve many item
  descriptions, improve markup, relocate some items that seemed to be in the
  wrong section.

- Doc: remove claim that PROVE_FLAGS defaults to '--verbose'.  Commit e9c81b601
  changed this, but missed updating the documentation.  The adjacent claim that
  we use TAP tests only in src/bin seems pretty obsolete as well.  Minor other

- On Windows, retry process creation if we fail to reserve shared memory.  We've
  heard occasional reports of backend launch failing because
  pgwin32_ReserveSharedMemoryRegion() fails, indicating that something has
  already used that address space in the child process.  It's not very clear
  what, given that we disable ASLR in Windows builds, but suspicion falls on
  antivirus products.  It'd be better if we didn't have to disable ASLR, anyway.
  So let's try to ameliorate the problem by retrying the process launch after
  such a failure, up to 100 times.  Patch by me, based on previous work by Amit
  Kapila and others.  This is a longstanding issue, so back-patch to all
  supported branches.  Discussion:

- Stamp 10beta2.

- Fix multiple assignments to a column of a domain type.  We allow INSERT and
  UPDATE commands to assign to the same column more than once, as long as the
  assignments are to subfields or elements rather than the whole column.
  However, this failed when the target column was a domain over array rather
  than plain array.  Fix by teaching process_matched_tle() to look through
  CoerceToDomain nodes, and add relevant test cases.  Also add a group of test
  cases exercising domains over array of composite.  It's doubtless accidental
  that CREATE DOMAIN allows this case while not allowing straight domain over
  composite; but it does, so we'd better make sure we don't break it.  (I could
  not find any documentation mentioning either side of that, so no doc changes.)
  It's been like this for a long time, so back-patch to all supported branches.

- Avoid integer overflow while sifting-up a heap in tuplesort.c.  If the number
  of tuples in the heap exceeds approximately INT_MAX/2, this loop's calculation
  "2*i+1" could overflow, resulting in a crash.  Fix it by using unsigned int
  rather than int for the relevant local variables; that shouldn't cost anything
  extra on any popular hardware.  Per bug #14722 from Sergey Koposov.  Original
  patch by Sergey Koposov, modified by me per a suggestion from Heikki
  Linnakangas to use unsigned int not int64.  Back-patch to 9.4, where
  tuplesort.c grew the ability to sort as many as INT_MAX tuples in-memory
  (commit 263865a48).  Discussion:

- Fix ruleutils.c for domain-over-array cases, too.  Further investigation shows
  that ruleutils isn't quite up to speed either for cases where we have a
  domain-over-array: it needs to be prepared to look past a CoerceToDomain at
  the top level of field and element assignments, else it decompiles them
  incorrectly.  Potentially this would result in failure to dump/reload a rule,
  if it looked like the one in the new test case.  (I also added a test for
  EXPLAIN; that output isn't broken, but clearly we need more test coverage
  here.) Like commit b1cb32fb6, this bug is reachable in cases we already
  support, so back-patch all the way.

- Fix dumping of FUNCTION RTEs that contain non-function-call expressions.  The
  grammar will only accept something syntactically similar to a function call in
  a function-in-FROM expression.  However, there are various ways to input
  something that ruleutils.c won't deparse that way, potentially leading to a
  view or rule that fails dump/reload.  Fix by inserting a dummy CAST around
  anything that isn't going to deparse as a function (which is one of the ways
  to get something like that in there in the first place).  In HEAD, also make
  use of the infrastructure added by this to avoid emitting unnecessary
  parentheses in CREATE INDEX deparsing.  I did not change that in back
  branches, thinking that people might find it to be unexpected/unnecessary
  behavioral change.  In HEAD, also fix incorrect logic for when to add extra
  parens to partition key expressions.  Somebody apparently thought they could
  get away with simpler logic than pg_get_indexdef_worker has, but they were
  wrong --- a counterexample is PARTITION BY LIST ((a[1])).  Ignoring the
  prettyprint flag for partition expressions isn't exactly a nice solution
  anyway.  This has been broken all along, so back-patch to all supported
  branches.  Discussion:

- Fix broken link-command-line ordering for libpgfeutils.  In the frontend
  Makefiles that pull in libpgfeutils, we'd generally done it like this: LDFLAGS
  += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) That method is
  badly broken, as seen in bug #14742 from Chris Ruprecht.  The -L flag for
  src/fe_utils ends up being placed after whatever random -L flags are in
  LDFLAGS already.  That puts us at risk of pulling in libpgfeutils.a from some
  previous installation rather than the freshly built one in src/fe_utils.
  Also, the lack of an "override" is hazardous if someone tries to specify some
  LDFLAGS on the make command line.  The correct way to do it is like this:
  override LDFLAGS := -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport)
  $(LDFLAGS) so that libpgfeutils, along with libpq, libpgport, and libpgcommon,
  are guaranteed to be pulled in from the build tree and not from any referenced
  system directory, because their -L flags will appear first.  In some places
  we'd been even lazier and done it like this: LDFLAGS +=
  -L$(top_builddir)/src/fe_utils -lpgfeutils -lpq which is subtly wrong in an
  additional way: on platforms where we can't restrict the symbols exported by, it allows libpgfeutils to latch onto libpgport and libpgcommon
  symbols from, rather than directly from those static libraries as
  intended.  This carries hazards like those explained in the comments for the
  libpq_pgport macro.  In addition to fixing the broken libpgfeutils usages, I
  tried to standardize on using $(libpq_pgport) like so: override LDFLAGS :=
  $(libpq_pgport) $(LDFLAGS) even where libpgfeutils is not in the picture.
  This makes no difference right now but will hopefully discourage future
  mistakes of the same ilk.  And it's more like the way we handle CPPFLAGS in
  libpq-using Makefiles.  In passing, just for consistency, make pgbench include
  PTHREAD_LIBS the same way everyplace else does, ie just after LIBS rather than
  in some random place in the command line.  This might have practical effect if
  there are -L switches in that macro on some platform.  It looks to me like the
  MSVC build scripts are not affected by this error, but someone more familiar
  with them than I might want to double check.  Back-patch to 9.6 where
  libpgfeutils was introduced.  In 9.6, the hazard this error creates is that a
  reinstallation might link to the prior installation's copy of libpgfeutils.a
  and thereby fail to absorb a minor-version bug fix.  Discussion:

- Code review for NextValueExpr expression node type.  Add missing
  infrastructure for this node type, notably in ruleutils.c where its lack could
  demonstrably cause EXPLAIN to fail.  Add outfuncs/readfuncs support.
  (outfuncs support is useful today for debugging purposes.  The readfuncs
  support may never be needed, since at present it would only matter for
  parallel query and NextValueExpr should never appear in a parallelizable
  query; but it seems like a bad idea to have a primnode type that isn't fully
  supported here.)  Teach planner infrastructure that NextValueExpr is a
  volatile, parallel-unsafe, non-leaky expression node with cost
  cpu_operator_cost.  Given its limited scope of usage, there *might* be no live
  bug today from the lack of that knowledge, but it's certainly going to bite us
  on the rear someday.  Teach pg_stat_statements about the new node type, too.
  While at it, also teach cost_qual_eval() that MinMaxExpr, SQLValueFunction,
  XmlExpr, and CoerceToDomain should be charged as cpu_operator_cost.  Failing
  to do this for SQLValueFunction was an oversight in my commit 0bb51aa96.  The
  others are longer-standing oversights, but no time like the present to fix
  them.  (In principle, CoerceToDomain could have cost much higher than this,
  but it doesn't presently seem worth trying to examine the domain's constraints
  here.) Modify execExprInterp.c to execute NextValueExpr as an out-of-line
  function; it seems quite unlikely to me that it's worth insisting that it be
  inlined in all expression eval methods.  Besides, providing the out-of-line
  function doesn't stop anyone from inlining if they want to.  Adjust some
  places where NextValueExpr support had been inserted with the aid of a
  dartboard rather than keeping it in the same order as elsewhere.  Discussion:

- Improve comments for execExpr.c's isAssignmentIndirectionExpr().  I got
  confused about why this function doesn't need to recursively search the
  expression tree for a CaseTestExpr node.  After figuring that out, add a
  comment to save the next person some time.

- Improve comments for execExpr.c's handling of FieldStore subexpressions.
  Given this code's general eagerness to use subexpressions' output variables as
  temporary workspace, it's not exactly clear that it is safe for FieldStore to
  tell a newval subexpression that it can write into the same variable that is
  being supplied as a potential input.  Document the chain of assumptions needed
  for that to be safe.

Andrew Gierth pushed:

- Fix COPY's handling of transition tables with indexes.  Commit
  c46c0e5202e8cfe750c6629db7852fdb15d528f3 failed to pass the
  TransitionCaptureState object to ExecARInsertTriggers() in the case where it's
  using heap_multi_insert and there are indexes.  Repair.  Thomas Munro, from a
  report by David Fetter Discussion:

Heikki Linnakangas pushed:

- Fix check for empty hostname.  As reported by Arthur Zakirov, Gcc 7.1
  complained about this with -Wpointer-compare.  Discussion:

- Fix missing tag in the docs.  Masahiko Sawada Discussion:

- Allow multiple hostaddrs to go with multiple hostnames.  Also fix two other
  issues, while we're at it: * In error message on connection failure, if
  multiple network addresses were given as the host option, as in
  "host=,", the error message printed the address twice.  * If
  there were many more ports than hostnames, the error message would always
  claim that there was one port too many, even if there was more than one. For
  example, if you gave 2 hostnames and 5 ports, the error message claimed that
  you gave 2 hostnames and 3 ports.  Discussion:

- Allow multiple hostaddrs to go with multiple hostnames.  Also fix two other
  issues, while we're at it: * In error message on connection failure, if
  multiple network addresses were given as the host option, as in
  "host=,", the error message printed the address twice.  * If
  there were many more ports than hostnames, the error message would always
  claim that there was one port too many, even if there was more than one. For
  example, if you gave 2 hostnames and 5 ports, the error message claimed that
  you gave 2 hostnames and 3 ports.  Discussion:

- Remove unnecessary braces, to match the surrounding style.  Mostly in the new
  subscription-related commands. Backport the few that were also present in
  older versions.  Thomas Munro Discussion:

- Fix ordering of operations in SyncRepWakeQueue to avoid assertion failure.
  Commit 14e8803f1 removed the locking in SyncRepWaitForLSN, but that introduced
  a race condition, where SyncRepWaitForLSN might see syncRepState already set
  to SYNC_REP_WAIT_COMPLETE, but the process was not yet removed from the queue.
  That tripped the assertion, that the process should no longer be in the uqeue.
  Reorder the operations in SyncRepWakeQueue to remove the process from the
  queue first, and update syncRepState only after that, and add a memory barrier
  in between to make sure the operations are made visible to other processes in
  that order.  Fixes bug #14721 reported by Const Zhang. Analysis and fix by
  Thomas Munro.  Backpatch down to 9.5, where the locking was removed.

- Fix variable and type name in comment.  Kyotaro Horiguchi Discussion:

- Reduce memory usage of tsvector type analyze function.
  compute_tsvector_stats() detoasted and kept in memory every tsvector value in
  the sample, but that can be a lot of memory. The original bug report described
  a case using over 10 gigabytes, with statistics target of 10000 (the maximum).
  To fix, allocate a separate copy of just the lexemes that we keep around, and
  free the detoasted tsvector values as we go. This adds some palloc/pfree
  overhead, when you have a lot of distinct lexemes in the sample, but it's
  better than running out of memory.  Fixes bug #14654 reported by James C.
  Reviewed by Tom Lane. Backport to all supported versions.  Discussion:

- Fix race between GetNewTransactionId and GetOldestActiveTransactionId.  The
  race condition goes like this: 1. GetNewTransactionId advances nextXid e.g.
  from 100 to 101 2. GetOldestActiveTransactionId reads the new nextXid, 101 3.
  GetOldestActiveTransactionId loops through the proc array. There are no active
  XIDs there, so it returns 101 as the oldest active XID.  4.
  GetNewTransactionid stores XID 100 to MyPgXact->xid So,
  GetOldestActiveTransactionId returned XID 101, even though 100 only just
  started and is surely still running.  This would be hard to hit in practice,
  and even harder to spot any ill effect if it happens.
  GetOldestActiveTransactionId is only used when creating a checkpoint in a
  master server, and the race condition can only happen on an online checkpoint,
  as there are no backends running during a shutdown checkpoint. The
  oldestActiveXid value of an online checkpoint is only used when starting up a
  hot standby server, to determine the starting point where pg_subtrans is
  initialized from. For the race condition to happen, there must be no other
  XIDs in the proc array that would hold back the oldest-active XID value, which
  means that the missed XID must be a top transaction's XID. However,
  pg_subtrans is not used for top XIDs, so I believe an off-by-one error is in
  fact inconsequential. Nevertheless, let's fix it, as it's clearly wrong and
  the fix is simple.  This has been wrong ever since hot standby was introduced,
  so backport to all supported versions.  Discussion:

- Fix pg_basebackup output to stdout on Windows.  When writing a backup to
  stdout with pg_basebackup on Windows, put stdout to binary mode. Any CR bytes
  in the output will otherwise be output incorrectly as CR+LF.  In the passing,
  standardize on using "_setmode" instead of "setmode", for the sake of
  consistency. They both do the same thing, but according to MSDN documentation,
  setmode is deprecated.  Fixes bug #14634, reported by Henry Boehlert. Patch by
  Haribabu Kommi.  Backpatch to all supported versions.  Discussion:

Álvaro Herrera pushed:

- Translation updates.  Source-Git-URL:
  git:// Source-Git-Hash:

- commit_ts test: Set node name in test.  Otherwise, the script output has a lot
  of pointless warnings.  This was forgotten in

- Fix typo in v10 release notes.  The new functions return a list of files in
  the corresponding directory, not the name of the directory itself.  Pointed
  out by Gianni Ciolli.

- pg_upgrade i18n: Fix "%s server/cluster" wording.  The original wording was
  impossible to translate correctly.  Discussion:

Andrew Dunstan pushed:

- Fix PROVE_FLAGS bug in commit 93b7d9731f.  This change didn't
  adjust the publicly visible taptest function, causing buildfarm failures on
  bowerbird.  Backpatch to 9.4 like previous change.

- fix typo.

== Pending Patches ==

Michaël Paquier sent in another revision of a patch to fix an issue where
pg_receivewal and messages were printed in non-verbose mode.

Mithun Cy sent in another revision of a patch to cache data in

Thomas Munro sent in another revision of a patch to share record typmods among

Ashutosh Bapat sent in another revision of a patch to skip partitions which
can't be part of the path.

Amit Langote sent in a patch to properly set the ri_RangeTableIndex of partition
result rels and correctly format the row shown in the WITH CHECK OPTION error

Alexander Korotkov sent in two revisions of a patch to double the shared memory
allocation for SLRU LWLocks.

Ashutosh Bapat sent in two more revisions of a patch to enable partition-wise
joins for declaratively partitioned tables.

Marina Polyakova sent in another revision of a patch to correct some pgbench
serialization and deadlock errors.

Alik Khilazhev and Fabien COELHO traded patches to add a Zipfian distribution to

Kyotaro HORIGUCHI and Amit Kapila traded patches to fix an issue where hash
index on unlogged tables doesn't behave as expected.

Claudio Freire sent in three more revisions of a patch to enable VACUUM to use
over 1GB of work_mem.

Masahiko Sawada sent in two more revisions of a patch to fix
pg_stop_backup(wait_for_archive := true) on standby servers.

Amit Langote sent in another revision of a patch to cope with differing attnos
in ATExecAttachPartition code and teach ATExecAttachPartition to skip validation
in more cases.

Thomas Munro sent in a patch to tell DSA that atomic u64 is always available.

Amit Kapila sent in another revision of a patch to adjust the cost based on
whether the below node is projection-capable.

Jeevan Ladhe sent in another revision of a patch to add support for default
partitions in declaratively partitioned tables.

Mark Rofail sent in another revision of a patch to allow foreign keys to arrays
using GIN indexes.

Masahiko Sawada sent in a patch to fix a typo in pg_upgrade/info.c.

Etsuro Fujita sent in a patch to fix a bug in ExecModifyTable() and some trigger
issues for foreign tables.

Amit Khandekar sent in a patch to generate the result rels in canonical order.

Heikki Linnakangas sent in a patch to always use 2048 bit DH parameters for
OpenSSL ephemeral DH ciphers.

Amit Langote sent in two revisions of a patch to update the description of
\d[S+] that \? emits in psql.

Kyotaro HORIGUCHI sent in two revisions of a patch to ensure that PgFDW
connection gets invalidated correctly by ALTER SERVER/ALTER USER MAPPING.

Ashutosh Bapat sent in a patch to replace lfirst() with lfirst_node()
appropriately in planner.c.

Álvaro Herrera sent in a patch to fix some more race conditions in logical

Thomas Munro sent in a patch to add more flexible LDAP auth search filters.

Thomas Munro sent in a patch to shore up inadequate infrastructure for

Ashutosh Sharma sent in a patch to fix an issue where PL/Perl didn't work on

Beena Emerson sent in another revision of a patch to implement default
partitions for declarative partitions on RANGE.

Mithun Cy sent in another revision of a patch to implement auto_prewarm.

Haribabu Kommi sent in a patch to fix pg_basebackup Windows tar mode to stdout
and replace setmode with _setmode function.

Álvaro Herrera sent in a patch to fix a bug in locking an update tuple chain.

Haribabu Kommi sent in a WIP patch to replace tuple with slot.  This is
infrastructure for pluggable storage engines.

Michaël Paquier sent in a patch to add an -E (encoding) switch to pg_dumpall.

Julien Rouhaud sent in a patch to fix an issue which manifested as a segfault in
HEAD when too many nested functions were called.

pgsql-announce by date

Next:From: Daniel GustafssonDate: 2017-07-17 19:16:19
Subject: Reminder: Call for Papers - PostgreSQL Conference Europe 2017
Previous:From: Devrim GündüzDate: 2017-07-13 16:44:14
Subject: Fedora 26 support added to PostgreSQL yum repository

Privacy Policy | About PostgreSQL
Copyright © 1996-2017 The PostgreSQL Global Development Group