Skip site navigation (1) Skip section navigation (2)

== PostgreSQL Weekly News - January 29 2017 ==

From: David Fetter <david(at)fetter(dot)org>
To: PostgreSQL Announce <pgsql-announce(at)postgresql(dot)org>
Subject: == PostgreSQL Weekly News - January 29 2017 ==
Date: 2017-01-29 22:52:58
Message-ID: (view raw or whole thread)
Lists: pgsql-announce
== PostgreSQL Weekly News - January 29 2017 ==

You can register for SCALE 15x at 50% off with code PGDAY.

== PostgreSQL Product News ==

pgFormatter 1.6, a formatter/beautifier for SQL code, released.

PostgreSQL 10 devel RPMs and Dockerfiles have been released.;a=blob;f=docker/README

pgBadger v9.1, a PostgreSQL log analyzer and graph tool written in
Perl, released.

== PostgreSQL Jobs for January ==

== PostgreSQL Local ==

PGConf India 2017 will be on March 2-3, 2017 in Bengaluru, Karnataka.

PostgreSQL(at)SCaLE will take place on March 2-3, 2017, at Pasadena Convention
Center, as part of SCaLE 15X.

PgConf.Russia 2017 will take place on 15-17 March 2017 in Moscow.

PGDay Asia 2017 will be held March 17-18 in Singapore.

Nordic PGDay 2017 will be held in Stockholm, Sweden, at the Sheraton
Hotel, on March 21, 2017.

pgDay Paris 2017 will be held in Paris, France on March 23, 2017.

PGConf US 2017 CfP will be on March 28-31 in Jersey City, New Jersey.

PGCon 2017 will take place in Ottawa on 23-26 May.

== PostgreSQL in the News ==

Planet PostgreSQL:

PostgreSQL Weekly News is brought to you this week by David Fetter

Submit news and announcements by Sunday at 3:00pm Pacific time.
Please send English language ones to david(at)fetter(dot)org, German language
to pwn(at)pgug(dot)de, Italian language to pwn(at)itpug(dot)org(dot)

== Applied Patches ==

Peter Eisentraut pushed:

- Fix typo.

- pg_dump: Fix minor memory leak.  Missing a destroyPQExpBuffer() in the early
  exit branch.  The early exits aren't really necessary.  Most similar functions
  just proceed running the rest of the code zero times and clean up at the end.

- Fix memory leaks in libpqwalreceiver.  The results of the libpq functions
  PQescapeIdentifier() and PQescapeLiteral() must be freed explicitly.  Also
  handle errors in these functions better.

- Fix pointer confusion.  get_object_address_publication_rel() needed to check
  *relation, not relation.  Rename the variables to match style used nearby to
  avoid the confusion.

- Fix NULL pointer access in logical replication workers.  From: Petr Jelinek

- Don't error when no system locales were found.  initdb used to warn about
  that, but it was changed to an error in pg_import_system_locales, but some
  build farm members failed because of that.  Change it back to a warning.

- Fix default minimum value for descending sequences.  For some reason that is
  lost in history, a descending sequence would default its minimum value to
  -2^63+1 (-PG_INT64_MAX) instead of -2^63 (PG_INT64_MIN), even though
  explicitly specifying a minimum value of -2^63 would work.  Fix this
  inconsistency by using the full range by default.  Reported-by: Daniel Verite
  <daniel(at)manitou-mail(dot)org> Reviewed-by: Michael Paquier

- Fix parallel make issue with new fmgrprotos.h.  The make rules needed further
  refinement so that we don't run multiple generations per build.  reported by
  Tom Lane

- pg_dump: Fix some schema issues when dumping sequences.  In the new code for
  selecting sequence data from pg_sequence, set the schema to pg_catalog instead
  of the sequences own schema, and refer to the sequence by OID instead of name,
  which was missing a schema qualification.  Reported-by: Stephen Frost

- doc: Update ALTER SEQUENCE documentation to match.  Update documentation to
  match change in 0bc1207aeb3de951bf95a9e9899b1256216d65f5.

- doc: Fix typo.

- Close replication connection when slot creation errors.  From: Petr Jelinek

- Update copyright years in some recently added files.

- doc: Logical replication documentation improvements.  From: Erik Rijkers

- doc: Fix typo.

- Fix typo: pg_statistics -> pg_statistic.

- Add object_address tests for publications and subscriptions.  Add test cases
  to object_address.sql to test the new logical replication related object
  classes, and fix some small bugs discovered by that.

- Simplify sequence test.  We maintained two separate expected files because
  log_cnt could be one of two values.  Rewrite the test so that we only need one
  file.  Reviewed-by: Petr Jelinek <petr(dot)jelinek(at)2ndquadrant(dot)com>

- doc: Update privileges documentation.  The CREATE privilege on databases now
  also enables creating publications.

- Fill in no_priv_msg for publications and subscriptions.  Even though these
  messages are not used yet, we should keep the list complete.

Tom Lane pushed:

- Volatile-ize some plperl variables that must survive into PG_CATCH blocks.
  This appears to be necessary to fix a failure seen on buildfarm member
  sittella.  It shouldn't be necessary according to the letter of the C
  standard, because we don't change the values of these variables within the
  PG_TRY blocks; but somehow gcc 4.7.2 is dropping the ball.  Discussion:

- Fix example plan in optimizer/README.  Joining three tables only takes two
  join nodes.  I think when I (tgl) wrote this, I was envisioning possible
  additional joins; but since the example doesn't show any fourth table, it's
  just confusing to write a third join node.  Etsuro Fujita Discussion:

- Allow password file name to be specified as a libpq connection parameter.
  Formerly an alternate password file could only be selected via the environment
  variable PGPASSFILE; now it can also be selected via a new connection
  parameter "passfile", corresponding to the conventions for most other
  connection parameters.  There was some concern about this creating a security
  weakness, but it was agreed that that argument was pretty thin, and there are
  clear use-cases for handling password files this way.  Julian Markwort,
  reviewed by Fabien Coelho, some adjustments by me Discussion:

- Use non-conflicting table names in new regression test case.  Commit 587cda35c
  added a test to updatable_views.sql that created tables named the same as
  tables used by the concurrent inherit.sql script.  Unsurprisingly, this
  results in random failures.  Pick different names.  Per buildfarm.

- Improve speed of contrib/postgres_fdw regression tests.  Commit 7012b132d
  added some tests that consumed an excessive amount of time, more than tripling
  the time needed for "make installcheck" for this module.  Add filter
  conditions to reduce the number of rows scanned, bringing the runtime down to
  within hailing distance of what it was before.  Jeevan Chalke and Ashutosh
  Bapat, per a gripe from me Discussion:

- Change unknown-type literals to type text in SELECT and RETURNING lists.
  Previously, we left such literals alone if the query or subquery had no
  properties forcing a type decision to be made (such as an ORDER BY or DISTINCT
  clause using that output column).  This meant that "unknown" could be an
  exposed output column type, which has never been a great idea because it could
  result in strange failures later on.  For example, an outer query that tried
  to do any operations on an unknown-type subquery output would generally fail
  with some weird error like "failed to find conversion function from unknown to
  text" or "could not determine which collation to use for string comparison".
  Also, if the case occurred in a CREATE VIEW's query then the view would have
  an unknown-type column, causing similar failures in queries trying to use the
  view.  To fix, at the tail end of parse analysis of a query, forcibly convert
  any remaining "unknown" literals in its SELECT or RETURNING list to type text.
  However, provide a switch to suppress that, and use it in the cases of SELECT
  inside a set operation or INSERT command.  In those cases we already had type
  resolution rules that make use of context information from outside the
  subquery proper, and we don't want to change that behavior.  Also, change
  creation of an unknown-type column in a relation from a warning to a hard
  error.  The error should be unreachable now in CREATE VIEW or CREATE MATVIEW,
  but it's still possible to explicitly say "unknown" in CREATE TABLE or CREATE
  (composite) TYPE.  We want to forbid that because it's nothing but a foot-gun.
  This change creates a pg_upgrade failure case: a matview that contains an
  unknown-type column can't be pg_upgraded, because reparsing the matview's
  defining query will now decide that the column is of type text, which doesn't
  match the cstring-like storage that the old materialized column would actually
  have.  Add a checking pass to detect that.  While at it, we can detect tables
  or composite types that would fail, essentially for free.  Those would fail
  safely anyway later on, but we might as well fail earlier.  This patch is by
  me, but it owes something to previous investigations by Rahila Syed.  Also
  thanks to Ashutosh Bapat and Michael Paquier for review.  Discussion:

- Make UNKNOWN into an actual pseudo-type.  Previously, type "unknown" was
  labeled as a base type in pg_type, which perhaps had some sense to it because
  you were allowed to create tables with unknown-type columns.  But now that we
  don't allow that, it makes more sense to label it a pseudo-type.  This has the
  additional effects of forbidding use of "unknown" as a domain base type, cast
  source or target type, PL function argument or result type, or plpgsql local
  variable type; all of which seem like good holes to plug.  Discussion:

- Remove vestigial resolveUnknown arguments from transformSortClause etc.
  There's really no situation where we don't want these unknown-to-text
  conversions to happen.  The alternative is failure anyway, and the one caller
  that was passing "false" did so only because it expected the case could not
  arise.  Might as well simplify the code.  Discussion:

- Introduce convenience macros to hide JsonbContainer header accesses better.
  This improves readability a bit and may make future improvements easier.  In
  passing, make sure that the JB_ROOT_IS_XXX macros deliver boolean (0/1)
  results; the previous coding was a bug hazard, though no actual bugs are
  known.  Nikita Glukhov, extended a bit by me Discussion:

- Ensure that a tsquery like '!foo' matches empty tsvectors.  !foo means "the
  tsvector does not contain foo", and therefore it should match an empty
  tsvector.  ts_match_vq() overenthusiastically supposed that an empty tsvector
  could never match any query, so it forcibly returned FALSE, the wrong answer.
  Remove the premature optimization.  Our behavior on this point was
  inconsistent, because while seqscans and GIST index searches both failed to
  match empty tsvectors, GIN index searches would find them, since GIN scans
  don't rely on ts_match_vq().  That makes this certainly a bug, not a debatable
  definition disagreement, so back-patch to all supported branches.  Report and
  diagnosis by Tom Dunstan (bug #14515); added test cases by me.  Discussion:

- Use castNode() in a bunch of statement-list-related code.  When I wrote commit
  ab1f0c822, I really missed the castNode() macro that Peter E. had proposed
  shortly before.  This back-fills the uses I would have put it to.  It's
  probably not all that significant, but there are more assertions here than
  there were before, and conceivably they will help catch any bugs associated
  with those representation changes.  I left behind a number of usages like
  "(Query *) copyObject(query_var)".  Those could have been converted as well,
  but Peter has proposed another notational improvement that would handle
  copyObject cases automatically, so I let that be for now.

- Orthography fixes for new castNode() macro.  Clean up hastily-composed
  comment.  Normalize whitespace.  Erik Rijkers and myself

- Improve comments about ProcessUtility's queryString parameter.  Per discussion
  with Craig Ringer.

- Restructure hba.c to replace 3 parallel lists with single list of structs.
  tokenize_file() now returns a single list of TokenizedLine structs, carrying
  the same information as before.  We were otherwise going to grow a fourth list
  to deal with error messages, and that was getting a bit silly.  Haribabu
  Kommi, revised a bit by me Discussion:

Álvaro Herrera pushed:

- Prefetch blocks during lazy vacuum's truncation scan.  Vacuum truncation scan
  can be sped up on rotating media by prefetching blocks in forward direction.
  That makes the blocks already present in memory by the time they are needed,
  while also letting OS read-ahead kick in.  The truncate scan has been measured
  to be five times faster than without this patch (that was on a slow disk, but
  it shouldn't hurt on fast disks.) Author: Álvaro Herrera, loosely based on a
  submission by Claudio Freire Discussion:

- Remove test for COMMENT ON DATABASE.  Our current DDL only allows a database
  name to be specified in COMMENT ON DATABASE, which Andrew Dunstan reports to
  make this test fail on the buildfarm.  Remove the line until we gain a DDL
  command that allows the current database to be operated on without having the
  specify it by name.  Backpatch to 9.5, where these tests appeared.

Tatsuo Ishii pushed:

- Fix comments in StrategyNotifyBgWriter().  The interface for the function was
  changed in d72731a70450b5e7084991b9caa15cb58a2820df but the comments of the
  function was not updated.  Patch by Yugo Nagata.

Fujii Masao pushed:

- Be sure to release the lock on failure to launch logical replication worker.
  Petr Jelinek

- Mention logical replication tests in src/test/README.  Craig Ringer

- Fix typo in description for pg_replication_origin_advance function.

Robert Haas pushed:

- Fix interaction of partitioned tables with BulkInsertState.  When copying into
  a partitioned table, the target heap may change from one tuple to next.  We
  must ask ReadBufferBI() to get a new buffer every time such change occurs.  To
  do that, use new function ReleaseBulkInsertStatePin().  This fixes the bug
  that tuples ended up being inserted into the wrong partition, which occurred
  exactly because the wrong buffer was used.  Amit Langote, per a suggestion
  from Robert Haas.  Some cosmetic adjustments by me.  Reports by 高增琦 (Gao
  Zengqi), Venkata B Nagothi, and Ragnar Ouchterlony.  Discussion:

- Don't invoke arbitrary code inside a possibly-aborted transaction.  The code
  here previously tried to call the partitioning operator, but really the right
  thing to do (and the safe thing to do) is use datumIsEqual().  Amit Langote,
  but I expanded the comment and fixed a compiler warning.

- Remove unused variable.  This was intended to be included in the previous
  commit, but I goofed.

- Fix incorrect comment: pgtime's tm_mon is 1-based, not 0-based.  The comments
  in formatting.c already said that tm_mon was 1-based not 0-based, but the
  comments here disagreed.  Dmitry Fedin

- Reindent table partitioning code.  We've accumulated quite a bit of stuff with
  which pgindent is not quite happy in this code; clean it up to provide a
  less-annoying base for future pgindent runs.

- Set ecxt_scantuple correctly for tuple routing.  In
  2ac3ef7a01df859c62d0a02333b646d65eaec5ff, we changed things so that it's
  possible for a different TupleTableSlot to be used for partitioned tables at
  successively lower levels.  If we do end up changing the slot from the
  original, we must update ecxt_scantuple to point to the new one for partition
  key of the tuple to be computed correctly.  Reported by Rajkumar Raghuwanshi.
  Patch by Amit Langote.  Discussion:

- Fix things so that updatable views work with partitioned tables.  Previously,
  ExecInitModifyTable was missing handling for WITH CHECK OPTION, and
  view_query_is_auto_updatable was missing handling for
  RELKIND_PARTITIONED_TABLE.  Amit Langote, reviewed by me.

- Extend index AM API for parallel index scans.  This patch doesn't actually
  make any index AM parallel-aware, but it provides the necessary functions at
  the AM layer to do so.  Rahila Syed, Amit Kapila, Robert Haas

- Add a new DestReceiver for printing tuples without catalog access.  If you
  create a DestReciver of type DestRemote and try to use it from a replication
  connection that is not bound to a specific daabase, or any other hypothetical
  type of backend that is not bound to a specific database, it will fail because
  it doesn't have a pg_proc catalog to look up properties of the types being
  printed.  In general, that's an unavoidable problem, but we can hardwire the
  properties of a few builtin types in order to support utility commands.  This
  new DestReceiver of type DestRemoteSimple does just that.  Patch by me,
  reviewed by Michael Paquier.  Discussion:

- Add a SHOW command to the replication command language.  This is useful
  infrastructure for an upcoming proposed patch to allow the WAL segment size to
  be changed at initdb time; tools like pg_basebackup need the ability to
  interrogate the server setting.  But it also doesn't seem like a bad thing to
  have independently of that; it may find other uses in the future.  Robert Haas
  and Beena Emerson.  (The original patch here was by Beena, but I rewrote it to
  such a degree that most of the code being committed here is mine.) Discussion:

- Be more aggressive in avoiding tuple conversion.  According to the comments in
  tupconvert.c, it's necessary to perform tuple conversion when either table has
  OIDs, and this was previously checked by ensuring that the tdtypeid value
  matched between the tables in question.  However, that's overly stringent: we
  have access to tdhasoid and can test directly whether OIDs are present, which
  lets us avoid conversion in cases where the type OIDs are different but the
  tuple descriptors are entirely the same (and neither has OIDs).  This is
  useful to the partitioning code, which can thereby avoid converting tuples
  when inserting into a partition whose columns appear in the same order as the
  parent columns, the normal case.  It's possible for the tuple routing code to
  avoid some additional overhead in this case as well, so do that, too.  It's
  not clear whether it would be OK to skip this when both tables have OIDs: do
  callers count on this to build a new tuple (losing the previous OID) in such
  instances?  Until we figure it out, leave the behavior in that case alone.
  Amit Langote, reviewed by me.

- Refactor bitmap heap scan estimation of heap pages fetched.  Currently, we
  only need this logic in order to cost a Bitmap Heap Scan.  But a pending patch
  for Parallel Bitmap Heap Scan also uses it to help figure out how many workers
  to use for the scan, which has to be determined prior to costing.  So, move
  the logic to a separate function to make that easier.  Dilip Kumar.  The patch
  series of which this is a part has been reviewed by Andres Freund, Amit
  Khendekar, Tushar Ahuja, Rafia Sabih, Haribabu Kommi, and me; it is not clear
  from the email discussion which of those people have looked specifically at
  this part.  Discussion:

- Fix typo in comment.  Etsuro Fujita

Simon Riggs pushed:

- Reset hot standby xmin on master after restart.  Hot_standby_feedback could be
  reset by reload and worked correctly, but if the server was restarted rather
  than reloaded the xmin was not reset.  Force reset always if
  hot_standby_feedback is enabled at startup.  Ants Aasma, Craig Ringer
  Reported-by: Ants Aasma

- Check interrupts during hot standby waits.

Andres Freund pushed:

- Add castNode(type, ptr) for safe casting between NodeTag based types.  The new
  function allows to cast from one NodeTag based type to another, while
  asserting that the conversion is valid.  This replaces the common pattern of
  doing a cast and a Assert(IsA(ptr, type)) close-by.  As this seems likely to
  be used pervasively, we decided to backpatch this change the addition of this
  macro. Otherwise backpatched fixes are more likely not to work on
  back-branches.  On branches before 9.6, where we do not yet rely on inline
  functions being available, the type assertion is only performed if
  PG_USE_INLINE support is detected. The cast obviously is performed regardless.
  For the benefit of verifying the macro compiles in the back-branches, this
  commit contains a single use of the new macro. On master, a somewhat larger
  conversion will be committed separately.  Author: Peter Eisentraut and Andres
  Freund.  Reviewed-By: Tom Lane.  Discussion:
  Backpatch: 9.2-

- Use the new castNode() macro in a number of places.  This is far from a
  pervasive conversion, but it's a good starting point.  Author: Peter
  Eisentraut, with some minor changes by me Reviewed-By: Tom Lane Discussion:

== Pending Patches ==

Michaël Paquier sent in a patch to enable replication connections by default in

Michaël Paquier sent in another revision of a patch to ensure that pg_dump and
pg_dumpall sync their output to disk by default.

Michaël Paquier sent in a patch to ensure that launching replication workers is
a NOOP when wal_level < logical, and to set the default number of such workers
to 0.

Craig Ringer sent in three more revisions of a patch to fix a race between clog
truncation and lookup, and introduce txid_status(bigint) to get status of an

Petr Jelínek sent in two more revisions of a patch to use the asynchronous
connect API in libpqwalreceiver, close the replication connection when slot
creation gets canceled, ensure that stringinfo buffers in walsender are always
initialized, fix after trigger execution in logical replication, and add RENAME

Amit Kapila sent in another revision of a patch to implement parallel index

Dilip Kumar sent in three more revisions of a patch to implement parallel bitmap
heap scans.

Beena Emerson sent in another revision of a patch to enable increasing the
default WAL segment size.

Corey Huinker sent in three more revisions of a patch to add \if and friends to

Etsuro Fujita sent in another revision of a patch to fix a bug in the PostgreSQL

Ivan Kartyshov sent in another revision of a patch to make async slave to wait
for lsn to be replayed.

Dagfinn Ilmari Mannsåker sent in another revision of a patch to add GUCs for
predicate lock promotion thresholds.

Craig Ringer sent in another revision of a patch to implement logical decoding
on standby.

Nico Williams sent in a patch to implement an expanded version of materialized
views and a contrib extension.

Nico Williams sent in a patch to implement pqasyncnotifier.c, a shell command
client for LISTEN.

Amit Langote sent in a patch to add relkind checks to certain contrib modules.

Kyotaro HORIGUCHI sent in another revision of a patch to clean up the negative
cache of pg_statistic when dropping a relation and of pg_class when dropping a

Amit Kapila sent in another revision of a patch to parallelize queries
containing subplans.

Ashutosh Sharma sent in two more revisions of a patch to add pgstathashindex()
to pgstattuple extension.

Jim Nasby sent in another revision of a patch to add faster methods for getting
SPI results.

Claudio Freire sent in two more revisions of a patch to allow usage of more than
1GB of work mem in VACUUM.

Pavel Stěhule sent in two revisions of a patch to enable forcing a custom or
generic plan in PL/pgsql.

Stas Kelvich and Nikhil Sontakke traded patches to speed up two-phase

Haribabu Kommi sent in three more revisions of a patch to implement a 64-bit
(EUI-64) macaddr data type.

Haribabu Kommi and Tom Lane traded patches to implement a pg_hba_file_settings

Peter Eisentraut sent in another revision of a patch to add ICU support.

Beena Emerson sent in a patch to add tab completion to ALTER in psql.

Daniel Vérité sent in another revision of a patch to improve psql hooks for

Nikita Glukhov sent in another revision of a patch to make a recursive version
of json_populate_record().

Álvaro Herrera sent in another revision of a patch to implement xmltable().

Fabien COELHO sent in two more revisions of a patch to add more functions and
operators to pgbench.

Pavan Deolasee sent in another revision of a patch to implement WARM.

Michaël Paquier sent in three revisions of a patch to refactor the replication
commands output.

Vladimir Rusinov sent in a patch to rename the sql-callable functions with xlog
to have wal instead.

Etsuro Fujita sent in another revision of a patch to push down more
UPDATEs/DELETEs in postgres_fdw.

Ashutosh Bapat sent in another revision of a patch to speed up aggregate
pushdown tests.

Julian Markwort sent in a patch to extended the functionality of
pg_stat_statements so it can track worst and best case execution plans.

Fabien COELHO sent in a patch to fix an infelicity between pgbench's --connect
and --rate options.

Kyotaro HORIGUCHI sent in another revision of a patch to use a radix tree for
character conversion.

Peter Eisentraut sent in two more revisions of a patch to add test coverage for

David Fetter and Corey Huinker traded patches to add copy_srf(), a set-returning
function corresponding to COPY IN.

Masahiko Sawada sent in two more revisions of a patch to support 2PC in FDWs.

Michaël Paquier sent in two revisions of a patch to remove race conditions
between the checkpointer and the init fork creations by making index init forks
go through the shared buffers instead of having their empty() routines handle
the flush of the page created.

Robert Haas sent in a patch to remove some hard-coded superuser checks.

Robert Haas sent in another revision of a patch to rename things *xlog* to the
corresponding *wal*.

Mithun Cy sent in two more revisions of a patch to cache hash index meta pages.

David Rowley sent in three more revisions of a patch to improve performance
where for joins where the outer side is unique.

Venkata B Nagothi sent in a patch to generate an error by aborting the recovery
process instead of starting up the cluster if the intended recovery target point
is not reached, and give an option to DBA to resume the recovery process from
exactly where it stopped.

Ashutosh Sharma sent in another revision of a patch to add microvacuum support
for hash indexes.

Noah Misch sent in another revision of a patch to remove link-time cross-module
refs in contrib.

Christoph Berg sent in a patch to use \G to use expanded output for a query or
current query buffer.

Robert Haas and Álvaro Herrera traded patches to add hash index support to
to the pageinspect contrib extension.

Thomas Munro sent in another revision of a patch to implement parallel shared

David Rowley sent in another revision of a patch to fix an infelicity between
CONNECTION LIMIT and Parallel Query.

Tom Lane sent in a patch to create a separate test file for exercising system

pgsql-announce by date

Next:From: Regina ObeDate: 2017-01-30 04:33:07
Subject: PostGIS 2.2.5 is released
Previous:From: Michael GoldbergDate: 2017-01-29 17:52:04
Subject: Registration for pgDay Israel 2017 is now open

Privacy Policy | About PostgreSQL
Copyright © 1996-2017 The PostgreSQL Global Development Group