Skip site navigation (1) Skip section navigation (2)

== PostgreSQL Weekly News - February 26 2017 ==

From: David Fetter <david(at)fetter(dot)org>
To: PostgreSQL Announce <pgsql-announce(at)postgresql(dot)org>
Subject: == PostgreSQL Weekly News - February 26 2017 ==
Date: 2017-02-26 23:01:50
Message-ID: (view raw, whole thread or download thread mbox)
Lists: pgsql-announce
== PostgreSQL Weekly News - February 26 2017 ==

== PostgreSQL Jobs for February ==

== PostgreSQL Local ==

The first pgDay Israel 2017 will take place on March 2, 2017.  Registration is

PGConf India 2017 will be on March 2-3, 2017 in Bengaluru, Karnataka.

PostgreSQL(at)SCaLE will take place on March 2-3, 2017, at Pasadena Convention
Center, as part of SCaLE 15X.

PgConf.Russia 2017 will take place on 15-17 March 2017 in Moscow.

PGDay Asia 2017 will be held March 17-18 in Singapore.

Nordic PGDay 2017 will be held in Stockholm, Sweden, at the Sheraton
Hotel, on March 21, 2017.

pgDay Paris 2017 will be held in Paris, France on March 23, 2017.

PGConf US 2017 will be on March 28-31 in Jersey City, New Jersey.

PGCon 2017 will take place in Ottawa on 23-26 May.

Postgres Vision will take place in Boston, June 26 - 28, 2017.

Swiss PGDay in Rapperswil will take place June 30, 2017.
The CfP is open through April 14, 2017.

== PostgreSQL in the News ==

Planet PostgreSQL:

PostgreSQL Weekly News is brought to you this week by David Fetter

Submit news and announcements by Sunday at 3:00pm Pacific time.
Please send English language ones to david(at)fetter(dot)org, German language
to pwn(at)pgug(dot)de, Italian language to pwn(at)itpug(dot)org(dot)

== Applied Patches ==

Tom Lane pushed:

- Fix documentation of to_char/to_timestamp TZ, tz, OF formatting patterns.
  These are only supported in to_char, not in the other direction, but the
  documentation failed to mention that.  Also, describe TZ/tz as printing the
  time zone "abbreviation", not "name", because what they print is elsewhere
  referred to that way.  Per bug #14558.

- Improve error message for misuse of TZ, tz, OF formatting patterns.  Be
  specific about which pattern is being complained of, and avoid saying "it's
  not supported in to_date", which is just confusing if the error is actually
  coming out of to_timestamp.  We can phrase it as "is only supported in
  to_char", instead.  Also, use the term "formatting field" not "format
  pattern", because other error messages in the same file prefer that
  terminology.  (This isn't terribly consistent with the documentation, so maybe
  we should change all these error messages?)

- Reject too-old Python versions a bit sooner.  Commit 04aad4018 added this
  check after the search for a Python shared library, which seems to me to be a
  pretty unfriendly ordering.  The search might fail for what are basically
  version-related reasons, and in such a case it'd be better to say "your Python
  is too old" than "could not find shared library for Python".

- Use less-generic table name in new regression test case.  Creating global
  objects named "foo" isn't an especially wise thing, but especially not in a
  test script that has already used that name for something else, and most
  especially not in a script that runs in parallel with other scripts that use
  that name :-( Per buildfarm.

- Fix sloppy handling of corner-case errors in fd.c.  Several places in fd.c had
  badly-thought-through handling of error returns from lseek() and close().  The
  fact that those would seldom fail on valid FDs is probably the reason we've
  not noticed this up to now; but if they did fail, we'd get quite confused.
  LruDelete and LruInsert actually just Assert'd that lseek never fails, which
  is pretty awful on its face.  In LruDelete, we indeed can't throw an error,
  because that's likely to get called during error abort and so throwing an
  error would probably just lead to an infinite loop.  But by the same token,
  throwing an error from the close() right after that was ill-advised, not to
  mention that it would've left the LRU state corrupted since we'd already
  unlinked the VFD from the list.  I also noticed that really, most of the time,
  we should know the current seek position and it shouldn't be necessary to do
  an lseek here at all.  As patched, if we don't have a seek position and an
  lseek attempt doesn't give us one, we'll close the file but then subsequent
  re-open attempts will fail (except in the somewhat-unlikely case that a
  FileSeek(SEEK_SET) call comes between and allows us to re-establish a known
  target seek position).  This isn't great but it won't result in any state
  corruption.  Meanwhile, having an Assert instead of an honest test in
  LruInsert is really dangerous: if that lseek failed, a subsequent read or
  write would read or write from the start of the file, not where the caller
  expected, leading to data corruption.  In both LruDelete and FileClose, if
  close() fails, just LOG that and mark the VFD closed anyway.  Possibly leaking
  an FD is preferable to getting into an infinite loop or corrupting the VFD
  list.  Besides, as far as I can tell from the POSIX spec, it's unspecified
  whether or not the file has been closed, so treating it as still open could be
  the wrong thing anyhow.  I also fixed a number of other places that were being
  sloppy about behaving correctly when the seekPos is unknown.  Also, I changed
  FileSeek to return -1 with EINVAL for the cases where it detects a bad offset,
  rather than throwing a hard elog(ERROR).  It seemed pretty inconsistent that
  some bad-offset cases would get a failure return while others got elog(ERROR).
  It was missing an offset validity check for the SEEK_CUR case on a closed
  file, too.  Back-patch to all supported branches, since all this code is
  fundamentally identical in all of them.  Discussion:

- Suppress unused-variable warning.  Rearrange so we don't have an unused
  variable in disable-cassert case.  Discussion:

- Fix contrib/pg_trgm's extraction of trigrams from regular expressions.  The
  logic for removing excess trigrams from the result was faulty.  It intends to
  avoid merging the initial and final states of the NFA, which is necessary, but
  in testing whether removal of a specific trigram would cause that, it failed
  to consider the combined effects of all the state merges that that trigram's
  removal would cause.  This could result in a broken final graph that would
  never match anything, leading to GIN or GiST indexscans not finding anything.
  To fix, add a "tentParent" field that is used only within this loop, and set
  it to show state merges that we are tentatively going to do.  While examining
  a particular arc, we must chase up through tentParent links as well as regular
  parent links (the former can only appear atop the latter), and we must account
  for state init/fin flag merges that haven't actually been done yet.  To
  simplify the latter, combine the separate init and fin bool fields into a
  bitmap flags field.  I also chose to get rid of the "children" state list,
  which seems entirely inessential.  Per bug #14563 from Alexey Isayko, which
  the added test cases are based on.  Back-patch to 9.3 where this code was
  added.  Report:

- De-support floating-point timestamps.  Per discussion, the time has come to do
  this.  The handwriting has been on the wall at least since 9.0 that this would
  happen someday, whenever it got to be too much of a burden to support the
  float-timestamp option.  The triggering factor now is the discovery that there
  are multiple bugs in the code that attempts to implement use of integer
  timestamps in the replication protocol even when the server is built for float
  timestamps.  The internal float timestamps leak into the protocol fields in
  places.  While we could fix the identified bugs, there's a very high risk of
  introducing more.  Trying to build a wall that would positively prevent mixing
  integer and float timestamps is more complexity than we want to undertake to
  maintain a long-deprecated option.  The fact that these bugs weren't found
  through testing also indicates a lack of interest in float timestamps.  This
  commit disables configure's --disable-integer-datetimes switch (it'll still
  accept --enable-integer-datetimes, though), removes direct references to
  USE_INTEGER_DATETIMES, and removes discussion of float timestamps from the
  user documentation.  A considerable amount of code is rendered dead by this,
  but removing that will occur as separate mop-up.  Discussion:

- Remove pg_control's enableIntTimes field.  We don't need it any more.
  pg_controldata continues to report that date/time type storage is "64-bit
  integers", but that's now a hard-wired behavior not something it sees in the
  data.  This avoids breaking pg_upgrade, and perhaps other utilities that
  inspect pg_control this way.  Ditto for pg_resetwal.  I chose to remove the
  "bigint_timestamps" output column of pg_control_init(), though, as that
  function hasn't been around long and probably doesn't have ossified users.

- Remove now-dead code for !HAVE_INT64_TIMESTAMP.  This is a basically
  mechanical removal of #ifdef HAVE_INT64_TIMESTAMP tests and the negative-case
  controlled code.  Discussion:

- Consistently declare timestamp variables as TimestampTz.  Twiddle the
  replication-related code so that its timestamp variables are declared
  TimestampTz, rather than the uninformative "int64" that was previously used
  for meant-to-be-always-integer timestamps.  This resolves the
  int64-vs-TimestampTz declaration inconsistencies introduced by commit
  7c030783a, though in the opposite direction to what was originally suggested.
  This required including datatype/timestamp.h in a couple more places than
  before.  I decided it would be a good idea to slim down that header by not
  having it pull in <float.h> etc, as those headers are no longer at all
  relevant to its purpose.  Unsurprisingly, a small number of .c files turn out
  to have been depending on those inclusions, so add them back in the .c files
  as needed.  Discussion:

- Add an Assert that enum_cmp_internal() gets passed an FmgrInfo pointer.  If
  someone were to try to call one of the enum comparison functions using
  DirectFunctionCallN, it would very likely seem to work, because only in
  unusual cases does enum_cmp_internal() need to access the typcache.  But once
  such a case occurred, code like that would crash with a null pointer
  dereference.  To make an oversight of that sort less likely to escape
  detection, add a non-bypassable Assert that fcinfo->flinfo isn't NULL.

- Suppress compiler warnings in ecpg test on newer Windows toolchains.
  nan_test.pgc supposed that it could unconditionally #define isnan() and
  isinf() on WIN32.  This was evidently copied at some point from
  src/include/port/win32.h, but nowadays there's a test on _MSC_VER there.  Make
  nan_test.pgc look the same.  Per buildfarm warnings.  There's no evidence this
  produces anything worse than a warning, and besides it's only a test case, so
  I don't feel a need to back-patch.

- Fix unportable definition of BSWAP64() macro.  We have a portable way of
  writing uint64 constants, but whoever wrote this macro didn't know about it.
  While at it, fix unsafe under-parenthesization of arguments.  That might be
  moot, because there are already good reasons not to use the macro on anything
  more complicated than a simple variable, but it's still poor practice.  Per
  buildfarm warnings.

- Remove useless duplicate inclusions of system header files.  c.h #includes a
  number of core libc header files, such as <stdio.h>.  There's no point in
  re-including these after having read postgres.h, postgres_fe.h, or c.h; so
  remove code that did so.  While at it, also fix some places that were ignoring
  our standard pattern of "include postgres[_fe].h, then system header files,
  then other Postgres header files".  While there's not any great magic in doing
  it that way rather than system headers last, it's silly to have just a few
  files deviating from the general pattern.  (But I didn't attempt to enforce
  this globally, only in files I was touching anyway.) I'd be the first to say
  that this is mostly compulsive neatnik-ism, but over time it might save enough
  compile cycles to be useful.

- Remove some configure header-file checks that we weren't really using.  We had
  some AC_CHECK_HEADER tests that were really wastes of cycles, because the code
  proceeded to #include those headers unconditionally anyway, in all or a large
  majority of cases.  The lack of complaints shows that those headers are
  available on every platform of interest, so we might as well let configure run
  a bit faster by not probing those headers at all.  I suspect that some of the
  tests I left alone are equally useless, but since all the existing #includes
  of the remaining headers are properly guarded, I didn't touch them.

- Put back #include <windows.h> in dirmod.c.  I removed this in commit
  9e3755ecb, reasoning that the win32.h port-specific header file included by
  c.h would have provided it.  However, that's only true on native win32 builds,
  not Cygwin builds.  It may be that some of the other <windows.h> inclusions
  also need to be put back on the same grounds; but this is the only one that is
  clearly meant to be included #ifdef __CYGWIN__, so maybe this is the extent of
  the problem.  Awaiting further buildfarm results.

Simon Riggs pushed:

- Small correction to BRIN docs.  Replace incorrect word "index" with "heap"
  Takayuki Tsunakawa

Peter Eisentraut pushed:

- Drop support for Python 2.3.  There is no specific reason for this right now,
  but keeping support for old Python versions around indefinitely increases the
  maintenance burden.  The oldest supported Python version is now Python 2.4,
  which is still shipped in RHEL/CentOS 5 by default.  In configure, add a check
  for the required Python version and give a friendly error message for an old
  version, instead of relying on an obscure build error later on.

- Make more use of castNode().

- doc: Update URL for plr.

- Fix whitespace.

- Remove deprecated COMMENT ON RULE syntax.  This was only used for allowing
  upgrades from pre-7.3 instances, which was a long time ago.

- Fix logical replication with different encodings.  reported by Shinoda,
  Noriyoshi <noriyoshi(dot)shinoda(at)hpe(dot)com>; partial patch by Kyotaro HORIGUCHI

Fujii Masao pushed:

- Remove confusing comment about unsupported feature.  The initial table
  synchronization feature has not been supported yet, but there was the
  confusing header comment about it in logical/worker.c.

- Fix typo in comment.  neha khatri

- Make walsender always initialize the buffers.  Walsender uses the local
  buffers for each outgoing and incoming message.  Previously when creating
  replication slot, walsender forgot to initialize one of them and which can
  cause the segmentation fault error. To fix this issue, this commit changes
  walsender so that it always initialize them before it executes the requested
  replication command.  Back-patch to 9.4 where replication slot was introduced.
  Problem report and initial patch by Stas Kelvich, modified by me.  Report:

- Fix connection leak in DROP SUBSCRIPTION command.  Previously the command
  forgot to close the connection to the publisher when it failed to drop the
  replication slot.

Álvaro Herrera pushed:

- Add tests for two-phase commit.  There's some ongoing performance work on this
  area, so let's make sure we don't break things.  Extracted from a larger patch
  originally by Stas Kelvich.  Authors: Stas Kelvich, Nikhil Sontakke, Michael
  Paquier Discussion:

Robert Haas pushed:

- doc: Add missing comma.  Yugo Nagata

- Shut down Gather's children before shutting down Gather itself.  It turns out
  that the original shutdown order here does not work well.  Multiple people
  attempting to develop further parallel query patches have discovered that they
  need to do cleanup before the DSM goes away, and you can't do that if the
  parent node gets cleaned up first.  Patch by me, reviewed by KaiGai Kohei and
  Dilip Kumar.  Discussion:

- Fix incorrect typecast.  Ashutosh Sharma, per a report from Mithun Cy.

- Pass the source text for a parallel query to the workers.  With this change,
  you can see the query that a parallel worker is executing in pg_stat_activity,
  and if the worker crashes you can see what query it was executing when it
  crashed.  Rafia Sabih, reviewed by Kuntal Ghosh and Amit Kapila and slightly
  revised by me.

- Make tablesample work with partitioned tables.  This was an oversight in the
  original partitioning commit.  Amit Langote, reviewed by David Fetter

- Allow custom and foreign scans to have shutdown callbacks.  This is expected
  to be useful mostly when performing such scans in parallel, because in that
  case it allows (in combination with commit
  acf555bc53acb589b5a2827e65d655fa8c9adee0) nodes below a Gather to get control
  just before the DSM segment goes away.  KaiGai Kohei, except that I rewrote
  the documentation.  Reviewed by Claudio Freire.  Discussion:

- Basic tab completion for partitioning.  Amit Langote Discussion:

- TAP tests for target_session_attrs connection parameter.  Michael Paquier

Andrew Dunstan pushed:

- Correctly handle array pseudotypes in to_json and to_jsonb.  Columns with
  array pseudotypes have not been identified as arrays, so they have been
  rendered as strings in the json and jsonb conversion routines. This change
  allows them to be rendered as json arrays, making it possible to deal
  correctly with the anyarray columns in pg_stats.

Bruce Momjian pushed:

- pg_upgrade docs:  clarify instructions on standby extensions.  Previously the
  pg_upgrade standby upgrade instructions said not to execute pgcrypto.sql, but
  it should have referenced the extension command "CREATE EXTENSION pgcrypto".
  This patch makes that doc change.  Reported-by: a private bug report
  Backpatch-through: 9.4, where standby instructions were added

Magnus Hagander pushed:

- Clarify the role of checkpoint at the begininng of base backups.  Output a
  message about checkpoint starting in verbose mode of pg_basebackup, and make
  the documentation state more clearly that this happens.  Author: Michael Banck

- Add missing progname prefix to some messages.  Author: Michael Banck

== Pending Patches ==

Rushabh Lathia sent in another revision of a patch to implement Gather Merge.

Michaël Paquier sent in another revision of a patch to make hba configuration
for SASL more extensible and implement SASLprep aka NFKC for SCRAM

Amit Langote and Ashutosh Bapat traded patches to allow dropping partitioned
table without CASCADE.

Rushabh Lathia sent in another revision of a patch to implement wait events for
disk I/O.

Dave Page sent in a patch to implement pg_ls_wal_dir() and pg_ls_logdir().

Tomas Vondra sent in a patch to add page_checksum and bt_page_items(bytea) to
the pageinspect supplied module.

Pavel Stěhule sent in another revision of a patch to implement xmltable().

Takeshi Ideriha sent in another revision of a patch to allow DECLARE STATEMENT
to set up a connection in ECPG.

Michaël Paquier sent in another revision of a patch to add tab completion in
psql for the new SUBSCRIPTION commands.

Thomas Munro sent in another revision of a patch to make SERIALIZABLE work with
parallel query.

Etsuro Fujita and Rushabh Lathia traded patches to push down more
UPDATEs/DELETEs in postgres_fdw.

Petr Jelínek sent in a patch to smooth the transition from floating point

Amit Langote and Ashutosh Bapat traded patches to take note of the fact that
partitioned tables are empty themselves by preventing the from trying to do
things to them that need to access files, not allocating storage for partitioned
tables, and avoiding creating scan nodes for partitioned tables.

Simon Riggs sent in two revisions of a patch to ensure that SnapshotResetXmin()
is not issued at the end of xact and reduce the calls to SnapshotResetXmin()
using a simple heuristic to reduce the effects.

Beena Emerson sent in two more revisions of a patch to allow increasing the
default WAL segment size.

Peter Moser sent in another revision of a patch to add ALIGN and NORMALIZE
operators for temporal queries.

Tomas Vondra and Andres Freund traded patches to add two slab-like memory

Dilip Kumar sent in another revision of a patch to implement parallel bitmap
heap scan.

Jim Mlodgenski sent in a patch to add system views for monitoring materialized

Amit Langote sent in a patch to show only the partition key upon failing to find
a partition.

Mithun Cy sent in another revision of a patch to expand hash indexes

Tatsuo Ishii sent in a patch to improve the calculation of statement_timeout for
the extended query protocol.

Amit Langote sent in a patch to add regression tests foreign partition DDL.

Vaishnavi Prabakaran sent in another revision of a patch to add batch/pipelining
support to libpq.

Masahiko Sawada sent in two more revisions of a patch to fix an infelicity

Thomas Munro sent in two more revisions of a patch to measure replication lag.

Petr Jelínek sent in three more revisions of a patch to fix some snapbuild woes.

Pavan Deolasee sent in another revision of a patch to implement WARM.

Robins Tharakan and Simon Riggs traded patches to make pg_dumpall work without
access to pg_authid.

Rafia Sabih sent in two revisions of a patch to make parallelism work for
queries coming from functions (SQL and several PLs).

Michael Banck sent in two revisions of a patch to reorder tablespaces in
basebackup tar stream for backup_label.

Petr Jelínek sent in two more revisions of a patch to use asynchronous connect
API in libpqwalreceiver, fix after trigger execution in logical replication, and

Pavan Deolasee sent in a patch to remove all direct references to ip_posid and
ip_blkid members of ItemPointerData struct and instead use
ItemPointerGetOffsetNumber and ItemPointerGetBlockNumber macros respectively to
access these members.

Peter Eisentraut sent in a patch to silence compiler warnings from gcc -O3.

Nikolay Shaplov sent in another revision of a patch to move all am-related
reloption code into src/backend/access/[am-name] and get rid of relopt_kind for
custom AM.

Jim Nasby sent in a patch to move refreshes of materialized views to last in

Jim Nasby sent in a patch to standardize on one of objsubid, subobjid.

Takayuki Tsunakawa sent in another revision of a patch to support huge pages on

Kyotaro HORIGUCHI sent in two more revisions of a patch to implement
asynchronous execution.

Yugo Nagata sent in two more revisions of a patch to fix a comment on the
JunkFilter struct.

Bernd Helmle sent in a patch to make subquery alias optional in FROM clause.

Chhatoi Pritam Baral sent in a patch to make the planner expand "foocol <@
'x,y'::foorange" into "foocol between x and y".

Petr Jelínek sent in two more revisions of a patch to add logical replication
support for initial data copy.

Mithun Cy sent in another revision of a patch to implement auto_prewarm.

Corey Huinker sent in four more revisions of a patch to add \if and friends to

Stephen Frost sent in another revision of a patch to fix pg_upgrade blob

Masahiko Sawada sent in a patch to report the number of skipped frozen pages by
manual VACUUM.

Craig Ringer sent in a patch to better detect timeouts in PostgresNode::psql
using regex.

Eiji Seki sent in another revision of a patch to add a GetOldestXminExtend for
ignoring arbitrary vacuum flags.

Haribabu Kommi sent in a patch to make it possible to have utility commands
benefit from a parallel plan.

Tom Lane sent in a patch to turn AllocBlockData from a linked list into a
doubly-linked list.

Andrew Dunstan sent in a patch to make enums indexable in the btree_gin and
btree_gist extensions.

Tom Lane sent in a patch to assert that an enum comparator can use cache.

Dave Page sent in a patch to add a default role called pg_monitor, complete with
a lot of things such a role should have.

Naoki Okano sent in a patch to add an --enable-parse-comment option to ECPG.

Pavel Stěhule sent in a patch to add a VERBOSE_SORT variable to psql.

Andrew Gierth sent in another revision of a patch to add hash support for
grouping sets.

Dilip Kumar sent in two more revisions of a patch to implement parallel merge

David Rowley sent in a patch to add recognition of "range queries" like "x > 34
AND x < 42" and IS [NOT] NULL to clausesel.

Simon Riggs sent in another revision of a patch to make some changes to the
recovery.conf API.

Peter Eisentraut sent in another revision of a patch to enable DROP FUNCTION to
drop multiple functions at once.

Peter Eisentraut sent in a patch to add cursor and execute methods to plan
object in PL/PythonU.

Simon Riggs sent in another revision of a patch to reduce lock levels for

Andrew Dunstan sent in two revisions of a patch to turn the
DirectFunctionCall{n]Coll functions into macros calling these functions and
passing NULL as the flinfo param.

Petr Jelínek sent in a patch to prevent pg_dump -t from dumping publications.

pgsql-announce by date

Next:From: Bo PengDate: 2017-02-28 03:12:45
Subject: pgpoolAdmin 3.6.0 released
Previous:From: Christoph BergDate: 2017-02-23 11:01:41
Subject: Development snapshot packages for postgresql-10 available on

Privacy Policy | About PostgreSQL
Copyright © 1996-2017 The PostgreSQL Global Development Group