Skip site navigation (1) Skip section navigation (2)

== PostgreSQL Weekly News - September 24 2017 ==

From: David Fetter <david(at)fetter(dot)org>
To: PostgreSQL Announce <pgsql-announce(at)postgresql(dot)org>
Subject: == PostgreSQL Weekly News - September 24 2017 ==
Date: 2017-09-25 03:08:20
Message-ID: 20170925030820.GA18035@fetter.org (view raw, whole thread or download thread mbox)
Thread:
Lists: pgsql-announce
== PostgreSQL Weekly News - September 24 2017 ==

PostgreSQL 10 Release Candidate 1 released.  Test!
https://www.postgresql.org/about/news/1783/

== PostgreSQL Product News ==

pg-wrapper 0.2.2, a wrapper for PHP's pgsql extension, released.
https://github.com/sad-spirit/pg-wrapper

brimbox 2.3.4, a web database interface and framework built atop PostgreSQL, released.
https://www.brimbox.com/brimbox-version-2-3-4-released/

pg_dirtyread 1.2, an extension that provides the ability to read dead but
unvacuumed rows from a relation, released.
https://github.com/ChristophBerg/pg_dirtyread

pg_builder 0.2.1, a PHP query builder for PostgreSQL, released.
https://github.com/sad-spirit/pg-builder

== PostgreSQL Jobs for September ==

http://archives.postgresql.org/pgsql-jobs/2017-09/

== PostgreSQL Local ==

PGDay.IT 2017 will take place October 13th, in Milan, Italy.
http://pgday.it

PostgreSQL Conference Europe 2017 will be held on October 24-27, 2017 in the
Warsaw Marriott Hotel, in Warsaw, Poland.
https://2017.pgconf.eu/

PGConf Local: Seattle will be held November 13 - 14, 2017.
https://www.pgconf.us/#Seattle2017

PGDay Australia 2017 will be held on November 17 in Melbourne.
http://2017.pgday.com.au/

2ndQuadrant PostgreSQL Conference 2017 (2Q PGConf, for short) will be hosted on
November 6th & 7th in New York City, and November 9th in Chicago.
http://www.2qpgconf.com/

PostgreSQL Session November 17th, 2017, in Paris, France.  The CfP is open until
September 30, 2017 at call-for-paper AT postgresql-sessions DOT org.  Details at
http://blog.dalibo.com/2017/08/29/cfp_pgsession9.html

PGConf Local: Austin will be held December 4 - 5, 2017. Call for Papers is
now open at https://www.pgconf.us/conferences/Austin2017

PGConf.ASIA 2017 will take place on December 4-6 2017 in Akihabara, Tokyo,
Japan.
http://www.pgconf.asia/EN/2017/

PGConf India 2018 will be on February 22-23, 2018 in Bengaluru, Karnataka.
Proposals are due via https://goo.gl/forms/F9hRjOIsaNasVOAz2 by October 31st, 2017.
http://pgconf.in/

== PostgreSQL in the News ==

Planet PostgreSQL: http://planet.postgresql.org/

PostgreSQL Weekly News is brought to you this week by David Fetter

Submit news and announcements by Sunday at 3:00pm EST5EDT.  Please send English
language ones to david(at)fetter(dot)org, German language to pwn(at)pgug(dot)de, Italian
language to pwn(at)itpug(dot)org(dot)

== Applied Patches ==

Peter Eisentraut pushed:

- Remove dead external links from documentation.
  https://git.postgresql.org/pg/commitdiff/d31892e2105cf48d8430807d74d5fdf1434af541

- Update some dead external links in the documentation.
  https://git.postgresql.org/pg/commitdiff/4b17c894293d0c3ed944da76aeb9bc2bb02a6db6

- Sync process names between ps and pg_stat_activity.  Remove gratuitous
  differences in the process names shown in pg_stat_activity.backend_type and
  the ps output.  Reviewed-by: Takayuki Tsunakawa
  <tsunakawa(dot)takay(at)jp(dot)fujitsu(dot)com>
  https://git.postgresql.org/pg/commitdiff/be87b70b6117609b7db0cd5e0cd96e7f569bdddb

- Fix compiler warning.  from gcc-7 -Wformat-truncation (via -Wall)
  https://git.postgresql.org/pg/commitdiff/d42294fc00da4b97d04ddb4401b76295e8d86816

- Add basic TAP test setup for pg_upgrade.  The plan is to convert the current
  pg_upgrade test to the TAP framework.  This commit just puts a basic TAP test
  in place so that we can see how the build farm behaves, since the build farm
  client has some special knowledge of the pg_upgrade tests.  Author: Michael
  Paquier <michael(dot)paquier(at)gmail(dot)com>
  https://git.postgresql.org/pg/commitdiff/f41e56c76e39f02bef7ba002c9de03d62b76de4d

- Fix build with !USE_WIDE_UPPER_LOWER.  The placement of the ifdef blocks in
  formatting.c was pretty bogus, so the code failed to compile if
  USE_WIDE_UPPER_LOWER was not defined.  Reported-by: Peter Geoghegan
  <pg(at)bowt(dot)ie> Reported-by: Noah Misch <noah(at)leadboat(dot)com>
  https://git.postgresql.org/pg/commitdiff/e6023ee7fa73a2d9a2d7524f63584844b2291def

- Remove contrib/chkpass.  The recent addition of a test suite for this module
  revealed a few problems.  It uses a crypt() method that is no longer
  considered secure and doesn't work anymore on some platforms.  Using a
  volatile input function violates internal sanity check assumptions and leads
  to failures on the build farm.  So this module is neither a usable security
  tool nor a good example for an extension.  No one wanted to argue for keeping
  or improving it, so remove it.  Discussion:
  https://www.postgresql.org/message-id/5645b0d7-cc40-6ab5-c553-292a91091ee7%402ndquadrant.com
  https://git.postgresql.org/pg/commitdiff/5d3cad564729f64d972c5c803ff34f0eb40bfd0c

- doc: Document commands that cannot be run in a transaction block.  Mainly
  covering the new CREATE SUBSCRIPTION and DROP SUBSCRIPTION, but ALTER DATABASE
  SET TABLESPACE was also missing.
  https://git.postgresql.org/pg/commitdiff/91ad8b416cee753eaa6f520ee2d21c2d41853381

- Revert "Add basic TAP test setup for pg_upgrade".  This reverts commit
  f41e56c76e39f02bef7ba002c9de03d62b76de4d.  The build farm client would run the
  pg_upgrade tests twice, once as part of the existing pg_upgrade check run and
  once as part of picking up all TAP tests by looking for "t" directories.
  Since the pg_upgrade tests are pretty slow, we will need a better solution or
  possibly a build farm client change before we can proceed with this.
  https://git.postgresql.org/pg/commitdiff/58ffe141eb37c3f027acd25c1fc6b36513bf9380

- Fix saving and restoring umask.  In two cases, we set a different umask for
  some piece of code and restore it afterwards.  But if the contained code
  errors out, the umask is not restored.  So add TRY/CATCH blocks to fix that.
  https://git.postgresql.org/pg/commitdiff/aa6b7b72d9bcf967cbccd378de4bc5cef33d02f9

- Refactor new file permission handling.  The file handling functions from fd.c
  were called with a diverse mix of notations for the file permissions when they
  were opening new files.  Almost all files created by the server should have
  the same permissions set.  So change the API so that e.g. OpenTransientFile()
  automatically uses the standard permissions set, and OpenTransientFilePerm()
  is a new function that takes an explicit permissions set for the few cases
  where it is needed.  This also saves an unnecessary argument for call sites
  that are just opening an existing file.  While we're reviewing these APIs, get
  rid of the FileName typedef and use the standard const char * for the file
  name and mode_t for the file mode.  This makes these functions match other
  file handling functions and removes an unnecessary layer of mysteriousness.
  We can also get rid of a few casts that way.  Author: David Steele
  <david(at)pgmasters(dot)net>
  https://git.postgresql.org/pg/commitdiff/0c5803b450e0cc29b3527df3f352e6f18a038cc6

- Fix pg_basebackup test to original intent.  One test case was meant to check
  that pg_basebackup does not succeed when a slot is specified with -S but WAL
  streaming is not selected, which used to require specifying -X stream.  Since
  -X stream is the default in PostgreSQL 10, this test case no longer covers
  that meaning, but the pg_basebackup invocation happened to fail anyway for the
  unrelated reason that the specified replication slot does not exist.  To fix,
  move the test case to later in the file where the slot does exist, and add -X
  none to the invocation so that it covers the originally meant behavior.
  extracted from a patch by Michael Banck <michael(dot)banck(at)credativ(dot)de>
  https://git.postgresql.org/pg/commitdiff/74ca8f9b9077017529fe658e445a11da296ac6ab

- doc: Expand user documentation on SCRAM.  Explain more about how the different
  password authentication methods and the password_encryption settings relate to
  each other, give some upgrading advice, and set a better link from the release
  notes.  Reviewed-by: Jeff Janes <jeff(dot)janes(at)gmail(dot)com>
  https://git.postgresql.org/pg/commitdiff/9b31c72a9492880e657b68b1ed971dec3c361c95

- Allow ICU to use SortSupport on Windows with UTF-8.  There is no reason to
  ever prevent the use of SortSupport on Windows when ICU locales are used.  We
  previously avoided SortSupport on Windows with UTF-8 server encoding and a non
  C-locale due to restrictions in Windows' libc functionality.  This is now
  considered to be a restriction in one platform's libc collation provider, and
  not a more general platform restriction.  Reported-by: Peter Geoghegan
  <pg(at)bowt(dot)ie>
  https://git.postgresql.org/pg/commitdiff/6dda0998afc7d449145b9ba216844bdba7a817d6

Tom Lane pushed:

- Fix, or at least ameliorate, bugs in logicalrep_worker_launch().  If we failed
  to get a background worker slot, the code just walked away from the
  logicalrep-worker slot it already had, leaving that looking like the worker is
  still starting up.  This led to an indefinite hang in subscription startup, as
  reported by Thomas Munro.  We must release the slot on failure.  Also fix a
  thinko: we must capture the worker slot's generation before releasing
  LogicalRepWorkerLock the first time, else testing to see if it's changed is
  pretty meaningless.  BTW, the CHECK_FOR_INTERRUPTS() in
  WaitForReplicationWorkerAttach is a ticking time bomb, even without
  considering the possibility of elog(ERROR) in one of the other functions it
  calls.  Really, this entire business needs a redesign with some actual thought
  about error recovery.  But for now I'm just band-aiding the case observed in
  testing.  Back-patch to v10 where this code was added.  Discussion:
  https://postgr.es/m/CAEepm=2bP3TBMFBArP6o20AZaRduWjMnjCjt22hSdnA-EvrtCw@mail.gmail.com
  https://git.postgresql.org/pg/commitdiff/3e1683d37e1d751eb2df9a5cb0507bebc6cf7d05

- Make DatumGetFoo/PG_GETARG_FOO/PG_RETURN_FOO macro names more consistent.  By
  project convention, these names should include "P" when dealing with a pointer
  type; that is, if the result of a GETARG macro is of type FOO *, it should be
  called PG_GETARG_FOO_P not just PG_GETARG_FOO.  Some newer types such as JSONB
  and ranges had not followed the convention, and a number of contrib modules
  hadn't gotten that memo either.  Rename the offending macros to improve
  consistency.  In passing, fix a few places that thought PG_DETOAST_DATUM()
  returns a Datum; it does not, it returns "struct varlena *".  Applying
  DatumGetPointer to that happens not to cause any bad effects today, but it's
  formally wrong.  Also, adjust an ltree macro that was designed without any
  thought for what pgindent would do with it.  This is all cosmetic and
  shouldn't have any impact on generated code.  Mark Dilger, some further tweaks
  by me Discussion:
  https://postgr.es/m/EA5676F4-766F-4F38-8348-ECC7DB427C6A@gmail.com
  https://git.postgresql.org/pg/commitdiff/4bd1994650fddf49e717e35f1930d62208845974

- Make ExplainOpenGroup and ExplainCloseGroup public.  Extensions with custom
  plan nodes might like to use these in their EXPLAIN output.  Hadi Moshayedi
  Discussion:
  https://postgr.es/m/CA+_kT_dU-rHCN0u6pjA6bN5CZniMfD=-wVqPY4QLrKUY_uJq5w@mail.gmail.com
  https://git.postgresql.org/pg/commitdiff/66917bfaa7bb0b6bae52a5fe631a8b6443203f55

- Minor code-cleanliness improvements for btree.  Make the btree page-flags test
  macros (P_ISLEAF and friends) return clean boolean values, rather than values
  that might not fit in a bool.  Use them in a few places that were randomly
  referencing the flag bits directly.  In passing, change access/nbtree/'s only
  direct use of BUFFER_LOCK_SHARE to BT_READ.  (Some think we should go the
  other way, but as long as we have BT_READ/BT_WRITE, let's use them
  consistently.) Masahiko Sawada, reviewed by Doug Doole Discussion:
  https://postgr.es/m/CAD21AoBmWPeN=WBB5Jvyz_Nt3rmW1ebUyAnk3ZbJP3RMXALJog@mail.gmail.com
  https://git.postgresql.org/pg/commitdiff/eb5c404b17752ca566947f12cb702438dcccdcb1

- Cache datatype-output-function lookup info across calls of concat().  Testing
  indicates this can save a third to a half of the runtime of the function.
  Pavel Stehule, reviewed by Alexander Kuzmenkov Discussion:
  https://postgr.es/m/CAFj8pRAT62pRgjoHbgTfJUc2uLmeQ4saUj+yVJAEZUiMwNCmdg@mail.gmail.com
  https://git.postgresql.org/pg/commitdiff/ed22fb8b0091deea23747310fa7609079a96cf82

- Remove no-op GiST support functions in the core GiST opclasses.  The preceding
  patch allowed us to remove useless GiST support functions.  This patch
  actually does that for all the no-op cases in the core GiST code.  This buys
  us whatever performance gain is to be had, and more importantly exercises the
  preceding patch.  There remain no-op functions in the contrib GiST opclasses,
  but those will take more work to remove.  Discussion:
  https://postgr.es/m/CAJEAwVELVx9gYscpE=Be6iJxvdW5unZ_LkcAaVNSeOwvdwtD=A@mail.gmail.com
  https://git.postgresql.org/pg/commitdiff/2d484f9b058035d41204f2eb8a0a8d2e8ee57b44

- Allow no-op GiST support functions to be omitted.  There are common use-cases
  in which the compress and/or decompress functions can be omitted, with the
  result being that we make no data transformation when storing or retrieving
  index values.  Previously, you had to provide a no-op function anyway, but
  this patch allows such opclass support functions to be omitted.  Furthermore,
  if the compress function is omitted, then the core code knows that the stored
  representation is the same as the original data.  This means we can allow
  index-only scans without requiring a fetch function to be provided either.
  Previously you had to provide a no-op fetch function if you wanted IOS to
  work.  This reportedly provides a small performance benefit in such cases, but
  IMO the real reason for doing it is just to reduce the amount of useless
  boilerplate code that has to be written for GiST opclasses.  Andrey Borodin,
  reviewed by Dmitriy Sarafannikov Discussion:
  https://postgr.es/m/CAJEAwVELVx9gYscpE=Be6iJxvdW5unZ_LkcAaVNSeOwvdwtD=A@mail.gmail.com
  https://git.postgresql.org/pg/commitdiff/d3a4f89d8a3e500bd7c0b7a8a8a5ce1b47859128

- Fix erroneous documentation about noise word GROUP.  GRANT, REVOKE, and some
  allied commands allow the noise word GROUP before a role name (cf. grantee
  production in gram.y).  This option does not exist elsewhere, but it had
  nonetheless snuck into the documentation for ALTER ROLE, ALTER USER, and
  CREATE SCHEMA.  Seems to be a copy-and-pasteo in commit 31eae6028, which did
  expand the syntax choices here, but not in that way.  Back-patch to 9.5 where
  that came in.  Discussion:
  https://postgr.es/m/20170916123750.8885.66941@wrigleys.postgresql.org
  https://git.postgresql.org/pg/commitdiff/36b564c648a044e42ca461466ae14d8588e6c5e2

- Fix instability in subscription regression test.  005_encoding.pl neglected to
  wait for the subscriber's initial synchronization to happen.  While we have
  not seen this fail in the buildfarm, it's pretty easy to demonstrate there's
  an issue by hacking logicalrep_worker_launch() to fail most of the time.
  Michael Paquier Discussion: https://postgr.es/m/27032.1505749806@sss.pgh.pa.us
  https://git.postgresql.org/pg/commitdiff/4939488af9b86edfff9b981773cd388d361c5830

- Improve dubious memory management in pg_newlocale_from_collation().
  pg_newlocale_from_collation() used malloc() and strdup() directly, which is
  generally not per backend coding style, and it didn't bother to check for
  failure results, but would just SIGSEGV instead.  Also, if one of the numerous
  error checks in the middle of the function failed, the already-allocated
  memory would be leaked permanently.  Admittedly, it's not a lot of memory, but
  it could build up if this function were called repeatedly for a bad collation.
  The first two problems are easily cured by palloc'ing in TopMemoryContext
  instead of calling libc directly.  We can fairly easily dodge the leakage
  problem for the struct pg_locale_struct by filling in a temporary variable and
  allocating permanent storage only once we reach the bottom of the function.
  It's harder to get rid of the potential leakage for ICU's copy of the
  collcollate string, but at least that's only allocated after most of the error
  checks; so live with that aspect.  Back-patch to v10 where this code came in,
  with one or another of the ICU patches.
  https://git.postgresql.org/pg/commitdiff/7b86c2ac9563ffd9b870cfd73a769431b7922e81

- Give a better error for duplicate entries in VACUUM/ANALYZE column list.
  Previously, the code didn't think about this case and would just try to
  analyze such a column twice.  That would fail at the point of inserting the
  second version of the pg_statistic row, with obscure error messsages like
  "duplicate key value violates unique constraint" or "tuple already updated by
  self", depending on context and PG version.  We could allow the case by
  ignoring duplicate column specifications, but it seems better to reject it
  explicitly.  The bogus error messages seem like arguably a bug, so back-patch
  to all supported versions.  Nathan Bossart, per a report from Michael Paquier,
  and whacked around a bit by me.  Discussion:
  https://postgr.es/m/E061A8E3-5E3D-494D-94F0-E8A9B312BBFC@amazon.com
  https://git.postgresql.org/pg/commitdiff/71480501057fee9fa3649b072173ff10e2b842d0

- Revert "Fix bool/int type confusion".  This reverts commit
  0ec2e908babfbfde83a3925680f06b16408739ff.  We'll use the upstream (IANA) fix
  instead.
  https://git.postgresql.org/pg/commitdiff/a890432a872afc9ca2327573f3313fd994d17384

- Sync our copy of the timezone library with IANA tzcode master.  This patch
  absorbs a few unreleased fixes in the IANA code.  It corresponds to commit
  2d8b944c1cec0808ac4f7a9ee1a463c28f9cd00a in https://github.com/eggert/tz.
  Non-cosmetic changes include: TZDEFRULESTRING is updated to match current US
  DST practice, rather than what it was over ten years ago.  This only matters
  for interpretation of POSIX-style zone names (e.g., "EST5EDT"), and only if
  the timezone database doesn't include either an exact match for the zone name
  or a "posixrules" entry.  The latter should not be true in any current
  Postgres installation, but this could possibly matter when using
  --with-system-tzdata.  Get rid of a nonportable use of "++var" on a bool var.
  This is part of a larger fix that eliminates some vestigial support for
  consecutive leap seconds, and adds checks to the "zic" compiler that the data
  files do not specify that.  Remove a couple of ancient compatibility hacks.
  The IANA crew think these are obsolete, and I tend to agree.  But perhaps our
  buildfarm will think different.  Back-patch to all supported branches, in line
  with our policy that all branches should be using current IANA code.  Before
  v10, this includes application of current pgindent rules, to avoid whitespace
  problems in future back-patches.  Discussion:
  https://postgr.es/m/E1dsWhf-0000pT-F9@gemulon.postgresql.org
  https://git.postgresql.org/pg/commitdiff/47f849a3c9005852926dca551d70ad8111f09f3a

- Assume wcstombs(), towlower(), and sibling functions are always present.
  These functions are required by SUS v2, which is our minimum baseline for Unix
  platforms, and are present on all interesting Windows versions as well.  Even
  our oldest buildfarm members have them.  Thus, we were not testing the
  "!USE_WIDE_UPPER_LOWER" code paths, which explains why the bug fixed in commit
  e6023ee7f escaped detection.  Per discussion, there seems to be no more
  real-world value in maintaining this option.  Hence, remove the configure-time
  tests for wcstombs() and towlower(), remove the USE_WIDE_UPPER_LOWER symbol,
  and remove all the !USE_WIDE_UPPER_LOWER code.  There's not actually all that
  much of the latter, but simplifying the #if nests is a win in itself.
  Discussion: https://postgr.es/m/20170921052928.GA188913@rfd.leadboat.com
  https://git.postgresql.org/pg/commitdiff/85feb77aa09cda9ff3e12cf95c757c499dc25343

- Mop-up for commit 85feb77aa09cda9ff3e12cf95c757c499dc25343.  Adjust commentary
  in regc_pg_locale.c to remove mention of the possibility of not having
  <wctype.h> functions, since we no longer consider that.  Eliminate duplicate
  code in wparser_def.c by generalizing the p_iswhat macro to take a parameter
  saying what to return for non-ASCII chars in C locale.  (That's not really a
  consequence of the USE_WIDE_UPPER_LOWER-ectomy, but I noticed it while doing
  that.)
  https://git.postgresql.org/pg/commitdiff/ed87e1980706975e7aa412bee200087774c5ff22

- Allow up to 3 "-P 1" reports per thread in pgbench run of 2 seconds.  There
  seems to be some considerable imprecision in the timing of -P progress
  reports.  Nominally each thread ought to produce 2 reports in this test, but
  about 10% of the time we only get one, and 1% of the time we get three, as per
  buildfarm results so far.  Pending further investigation, treat the last case
  as a "pass".  (I, tgl, am suspicious that this still might not be lax enough,
  now that it's obvious that the behavior is load-dependent; but there's not yet
  buildfarm evidence to confirm that suspicion.) Fabien Coelho Discussion:
  https://postgr.es/m/26654.1505232433@sss.pgh.pa.us
  https://git.postgresql.org/pg/commitdiff/0f574a7afb5c998d19dc3d981e45cb10267286ed

- Ten-second timeout in 013_crash_restart.pl is not enough, let's try 60.  Per
  buildfarm member topminnow.
  https://git.postgresql.org/pg/commitdiff/01c7d3ef85d4b0e1c52cc1a3542864f95f386f76

- Improve memory management in autovacuum.c.  Invoke vacuum(), as well as "work
  item" processing, in the PortalContext that do_autovacuum() has manufactured,
  which will be reset before each such invocation.  This ensures cleanup of any
  memory leaked by these operations.  It also avoids the rather dangerous
  practice of calling vacuum() in a context that vacuum() itself will destroy
  while it runs.  There's no known live bug there, but it's not hard to imagine
  introducing one if we leave it like this.  Tom Lane, reviewed by Michael
  Paquier and Alvaro Herrera Discussion:
  https://postgr.es/m/13849.1506114543@sss.pgh.pa.us
  https://git.postgresql.org/pg/commitdiff/335f3d04e4c8dd495c4dd30ab1049b6fe8f25052

- Fix bogus size calculation in strlist_to_textarray().  It's making an array of
  Datum, not an array of text *.  The mistake is harmless since those are
  currently the same size, but it's still wrong.
  https://git.postgresql.org/pg/commitdiff/737639017c87d5a0a466e8676f1eadc61d775c78

- ... and the very same bug in publicationListToArray().  Sigh.
  https://git.postgresql.org/pg/commitdiff/24541ffd788d56009126fff52b2341ada6c84245

- Remove pgbench "progress" test pending solution of its timing issues.
  Buildfarm member skink shows that this is even more flaky than I thought.
  There are probably some actual pgbench bugs here as well as a timing
  dependency.  But we can't have stuff this unstable in the buildfarm, it
  obscures other issues.
  https://git.postgresql.org/pg/commitdiff/ad51c6fb5708342e603d12a730bbc4e663bd637e

- Fix assorted infelicities in new SetWALSegSize() function.  * Failure to check
  for malloc failure (ok, pretty unlikely here, but that's not an excuse).  *
  Leakage of open fd on read error, and of malloc'd buffer always.  * Incorrect
  assumption that a short read would set errno to zero.  * Failure to adhere to
  message style conventions (in particular, not reporting errno where relevant;
  using "couldn't open" rather than "could not open" is not really in line with
  project style either).  * Missing newlines on some messages.  Coverity spotted
  the leak problems; I noticed the rest while fixing the leaks.
  https://git.postgresql.org/pg/commitdiff/8485a25a8c9a419ff3e0d30e43e4abd5e680cc65

Andres Freund pushed:

- Fix crash restart bug introduced in 8356753c212.  The bug was caused by not
  re-reading the control file during crash recovery restarts, which lead to an
  attempt to pfree() shared memory contents. The fix is to re-read the control
  file, which seems good anyway.  It's unclear as of this moment, whether we
  want to keep the refactoring introduced in the commit referenced above, or
  come up with an alternative approach. But fixing the bug in the mean time
  seems like a good idea regardless.  A followup commit will introduce
  regression test coverage for crash restarts.  Reported-By: Tom Lane
  Discussion: https://postgr.es/m/14134.1505572349@sss.pgh.pa.us
  https://git.postgresql.org/pg/commitdiff/ec9e05b3c392ba9587f283507459737684539574

- Fix uninitialized variable in dshash.c.  A bugfix for commit
  8c0d7bafad36434cb08ac2c78e69ae72c194ca20.  The code would have crashed if
  hashtable->size_log2 ever had the same value as hashtable->control->size_log2
  by coincidence.  Per Valgrind.  Author: Thomas Munro Reported-By: Tomas Vondra
  Discussion:
  https://postgr.es/m/e72fb33c-4f31-f276-e972-263d9b59554d%402ndquadrant.com
  https://git.postgresql.org/pg/commitdiff/0fb9e4ace5ce4d479d839a720f32b99fdc87f455

- Rearm statement_timeout after each executed query.  Previously
  statement_timeout, in the extended protocol, affected all messages till a Sync
  message.  For clients that pipeline/batch query execution that's problematic.
  Instead disable timeout after each Execute message, and enable, if necessary,
  the timer in start_xact_command(). As that's done only for Execute and not
  Parse / Bind, pipelining the latter two could still cause undesirable
  timeouts. But a survey of protocol implementations shows that all drivers
  issue Sync messages when preparing, and adding timeout rearming to both is
  fairly expensive for the common parse / bind / execute sequence.  Author:
  Tatsuo Ishii, editorialized by Andres Freund Reviewed-By: Takayuki Tsunakawa,
  Andres Freund Discussion:
  https://postgr.es/m/20170222.115044.1665674502985097185.t-ishii@sraoss.co.jp
  https://git.postgresql.org/pg/commitdiff/f8e5f156b30efee5d0038b03e38735773abcb7ed

- Add test for postmaster crash restarts.  Given that I managed to break this...
  We probably should extend the tests to also cover other sub-processes dying,
  but that's something for later.  Author: Andres Freund Discussion:
  https://postgr.es/m/20170917080752.rcmihzfmgbeuqjk2@alap3.anarazel.de
  https://git.postgresql.org/pg/commitdiff/a1924a4ea29399111e5155532ca24c9c51d3c82d

- Speedup pgstat_report_activity by moving mb-aware truncation to read side.
  Previously multi-byte aware truncation was done on every
  pgstat_report_activity() call - proving to be a bottleneck for workloads with
  long query strings that execute quickly.  Instead move the truncation to the
  read side, which commonly is executed far less frequently. That's possible
  because all server encodings allow to determine the length of a multi-byte
  string from the first byte.  Rename PgBackendStatus.st_activity to
  st_activity_raw so existing extension users of the field break - their code
  has to be adjusted to use pgstat_clip_activity().  Author: Andres Freund
  Tested-By: Khuntal Ghosh Reviewed-By: Robert Haas, Tom Lane Discussion:
  https://postgr.es/m/20170912071948.pa7igbpkkkviecpz@alap3.anarazel.de
  https://git.postgresql.org/pg/commitdiff/54b6cd589ac2f5635a42511236a5eb7299e2dcaf

- Avoid use of non-portable strnlen() in pgstat_clip_activity().  The use of
  strnlen rather than strlen was just paranoia. Instead of giving up on the
  paranoia, just implement the safeguard differently. And add a comment
  explaining why we're careful.  Author: Andres Freund Discussion:
  https://postgr.es/m/E1duOkJ-0001Mc-U5@gemulon.postgresql.org
  https://git.postgresql.org/pg/commitdiff/71edbb6f66f7139d6209334ef8734a122ba06b56

- s/NULL byte/NUL byte/ in comment refering to C string terminator.
  Reported-By: Robert Haas Discussion:
  https://postgr.es/m/CA+Tgmoa+YBvWgFST2NVoeXjVSohEpK=vqnVCsoCkhTVVxfLcVQ@mail.gmail.com
  https://git.postgresql.org/pg/commitdiff/896537f078ba4d709ce754ecaff8350fd55bdfd8

- Make WAL segment size configurable at initdb time.  For performance reasons a
  larger segment size than the default 16MB can be useful. A larger segment size
  has two main benefits: Firstly, in setups using archiving, it makes it easier
  to write scripts that can keep up with higher amounts of WAL, secondly, the
  WAL has to be written and synced to disk less frequently.  But at the same
  time large segment size are disadvantageous for smaller databases. So far the
  segment size had to be configured at compile time, often making it unrealistic
  to choose one fitting to a particularly load. Therefore change it to a initdb
  time setting.  This includes a breaking changes to the xlogreader.h API, which
  now requires the current segment size to be configured.  For that and similar
  reasons a number of binaries had to be taught how to recognize the current
  segment size.  Author: Beena Emerson, editorialized by Andres Freund
  Reviewed-By: Andres Freund, David Steele, Kuntal Ghosh, Michael Paquier, Peter
  Eisentraut, Robert Hass, Tushar Ahuja Discussion:
  https://postgr.es/m/CAOG9ApEAcQ--1ieKbhFzXSQPw_YLmepaa4hNdnY5+ZULpt81Mw@mail.gmail.com
  https://git.postgresql.org/pg/commitdiff/fc49e24fa69a15efacd5b8958115ed9c43c48f9a

- Make new crash restart test a bit more robust.  Add timeouts in case psql
  doesn't deliver the expected output, and try to cause the monitoring psql to
  be fully connected to a backend.  This isn't necessarily everything needed,
  but at least the timeouts should reduce the pain for buildfarm owners.
  Author: Andres Freund Reported-By: Tom Lane, BF animals prairiedog and
  calliphoridae Discussion:
  https://postgr.es/m/E1du6ZT-00043I-91@gemulon.postgresql.org
  https://git.postgresql.org/pg/commitdiff/1910353675bd149e1020b29c0fae02538fc358cd

- Accept that server might not be able to send error in crash recovery test.  As
  it turns out we can't rely that the script's monitoring session is terminated
  with a proper error by the server, because the session might be terminated
  while already trying to send data.  Also improve robustness and error
  reporting facilities of the test, developed while debugging this issue.
  Discussion:
  https://postgr.es/m/20170920020038.kllxgilo7xzwmtto@alap3.anarazel.de
  https://git.postgresql.org/pg/commitdiff/5ada1fcd0c30be1b0b793a802cf6da386a6c1925

- Fix s/intidb/initdb/ typo.  Reported-By: Michael Paquier Discussion:
  https://postgr.es/m/CAB7nPqTfaKAYZ4wuUM-W8kc4VnXrxX1=5-a9i==VoUPTMFpsgg@mail.gmail.com
  https://git.postgresql.org/pg/commitdiff/f9583e86b4bfa8c4e4d83ab33e5dcdaeab5c45a1

- Expand expected output for recovery test even further.  I'd assumed that the
  backend being killed should be able to get out an error message - but it turns
  out it's not guaranteed that it's not still sending a ready-for-query.  Really
  need to do something about getting these error message to the client.
  Reported-By: Thomas Munro, Tom Lane Discussion:
  https://postgr.es/m/CAEepm=0TE90nded+bNthP45_PEvGAAr=3gxhHJObL4xmOLtX0w@mail.gmail.com
  https://postgr.es/m/14968.1506101414@sss.pgh.pa.us
  https://git.postgresql.org/pg/commitdiff/8d926029e817d280b2376433e3aaa3895e1a7128

- Add inline murmurhash32(uint32) function.  The function already existed in
  tidbitmap.c but more users requiring fast hashing of 32bit ints are coming up.
  Author: Andres Freund Discussion:
  https://postgr.es/m/20170914061207.zxotvyopetm7lrrp@alap3.anarazel.de
  https://git.postgresql.org/pg/commitdiff/791961f59b792fbd4f0a992d3ccab47298e79103

Andrew Dunstan pushed:

- Add citext_pattern_ops for citext contrib module.  This is similar to
  text_pattern_ops.  Alexey Chernyshov, reviewed by Jacob Champion.
  https://git.postgresql.org/pg/commitdiff/f2464997644c64b5dec93ab3c08305f48bfe14f1

- Set client encoding to UTF8 for the citext regression script.  Problem
  introduced with non-ascii characters in commit f2464997644c and discovered on
  various buildfarm animals.
  https://git.postgresql.org/pg/commitdiff/890faaf1957759c6e17fbcbfd16f7cabc4a59d07

- Disable multi-byte citext tests.  This reverts commit 890faaf1 which attempted
  unsuccessfully to deal with the problem, and instead just comments out these
  tests like other similar tests elsewhere in the script.
  https://git.postgresql.org/pg/commitdiff/d1687c6926819f023c78b353458950a303796aba

- Quieten warnings about unused variables.  These variables are only ever
  written to in assertion-enabled builds, and the latest Microsoft compilers
  complain about such variables in non-assertion-enabled builds.  Apparently
  they don't worry so much about variables that are written to but not read
  from, so most of our PG_USED_FOR_ASSERTS_ONLY variables don't cause the
  problem.  Discussion: https://postgr.es/m/7800.1505950322@sss.pgh.pa.us
  https://git.postgresql.org/pg/commitdiff/28ae524bbf865d23eb10f6ae1b996d59dcc30e4e

- Provide a test for variable existence in psql.  "\if :{?variable_name}" will
  be translated to "\if TRUE" if the variable exists and "\if FALSE" otherwise.
  Thus it will be possible to execute code conditionally on the existence of the
  variable, regardless of its value.  Fabien Coelho, with some review by Robins
  Tharakan and some light text editing by me.  Discussion:
  https://postgr.es/m/alpine.DEB.2.20.1708260835520.3627@lancre
  https://git.postgresql.org/pg/commitdiff/d57c7a7c506276597af619bdb8c62fa5b592745a

Bruce Momjian pushed:

- doc:  add example of % substitution for connection URIs.  Reported-by: Zhou
  Digoal Discussion:
  https://postgr.es/m/20170912133722.25637.91@wrigleys.postgresql.org
  Backpatch-through: 10
  https://git.postgresql.org/pg/commitdiff/d61f5bb7c444255b064a60df782907f7dddad61a

- docs:   re-add instructions on setting wal_level for rsync use.  This step was
  erroneously removed four days ago by me.  Reported-by: Magnus via IM
  Backpatch-through: 9.5
  https://git.postgresql.org/pg/commitdiff/00210e3fb974ff2b9affc4d8f3b29f9cb3645a60

Magnus Hagander pushed:

- Mention need for --no-inc-recursive in rsync command.  Since rsync 3.0.0
  (released in 2008), the default way to enumerate changes was changed in a way
  that makes it less likely that the hardlink sync mode works. Since the whole
  point of the documented procedure is for the hardlinks to work, change our
  docs to suggest using the backwards compatibility switch.
  https://git.postgresql.org/pg/commitdiff/2c74e6c1dcc5002fa8b822e5757f6c95d899fb7a

Robert Haas pushed:

- Fix typo.  Thomas Munro Discussion:
  http://postgr.es/m/CAEepm=2j-HAgnBUrAazwS0ry7Z_ihk+d7g+Ye3u99+6WbiGt_Q@mail.gmail.com
  https://git.postgresql.org/pg/commitdiff/7f3a3312abf34ea7e899046e326775612802764b

- Fix create_lateral_join_info to handle dead relations properly.  Commit
  0a480502b092195a9b25a2f0f199a21d592a9c57 broke it.  Report by Andreas
  Seltenreich.  Fix by Ashutosh Bapat.  Discussion:
  http://postgr.es/m/874ls2vrnx.fsf@ansel.ydns.eu
  https://git.postgresql.org/pg/commitdiff/57eebca03a9eb61eb18f8ea9db94775653f797d1

- Associate partitioning information with each RelOptInfo.  This is not used for
  anything yet, but it is necessary infrastructure for partition-wise join and
  for partition pruning without constraint exclusion.  Ashutosh Bapat, reviewed
  by Amit Langote and with quite a few changes, mostly cosmetic, by me.
  Additional review and testing of this patch series by Antonin Houska, Amit
  Khandekar, Rafia Sabih, Rajkumar Raghuwanshi, Thomas Munro, and Dilip Kumar.
  Discussion:
  http://postgr.es/m/CAFjFpRfneFG3H+F6BaiXemMrKF+FY-POpx3Ocy+RiH3yBmXSNw@mail.gmail.com
  https://git.postgresql.org/pg/commitdiff/9140cf8269b0c4ae002b2748d93979d535891311

- hash: Implement page-at-a-time scan.  Commit
  09cb5c0e7d6fbc9dee26dc429e4fc0f2a88e5272 added a similar optimization to btree
  back in 2006, but nobody bothered to implement the same thing for hash
  indexes, probably because they weren't WAL-logged and had lots of other
  performance problems as well.  As with the corresponding btree case, this
  eliminates the problem of potentially needing to refind our position within
  the page, and cuts down on pin/unpin traffic as well.  Ashutosh Sharma,
  reviewed by Alexander Korotkov, Jesper Pedersen, Amit Kapila, and me.  Some
  final edits to comments and README by me.  Discussion:
  http://postgr.es/m/CAE9k0Pm3KTx93K8_5j6VMzG4h5F+SyknxUwXrN-zqSZ9X8ZS3w@mail.gmail.com
  https://git.postgresql.org/pg/commitdiff/7c75ef571579a3ad7a1d3ee909f11dba5e0b9440

- For wal_consistency_checking, mask page checksum as well as page LSN.  If the
  LSN is different, the checksum will be different, too.  Ashwin Agrawal,
  reviewed by Michael Paquier and Kuntal Ghosh Discussion:
  http://postgr.es/m/CALfoeis5iqrAU-+JAN+ZzXkpPr7+-0OAGv7QUHwFn=-wDy4o4Q@mail.gmail.com
  https://git.postgresql.org/pg/commitdiff/6a2fa09c0cba0e5a11854d733872ac18511f4c83

Álvaro Herrera pushed:

- Document further existing locks as wait events.  Reported-by: Jeremy Schneider
  Author: Michael Paquier Discussion:
  https://postgr.es/m/CA+fnDAZaPCwfY8Lp-pfLnUGFAXRu1VfLyRgdup-L-kwcBj8MqQ@mail.gmail.com
  https://git.postgresql.org/pg/commitdiff/885cab58115a5af9484926ddee8dca3dc0106c1e

- Test BRIN autosummarization.  There was no coverage for this code.
  Reported-by: Nikolay Shaplov, Tom Lane Discussion:
  https://postgr.es/m/2700647.XEouBYNZic@x200m
  https://postgr.es/m/13849.1506114543@sss.pgh.pa.us
  https://git.postgresql.org/pg/commitdiff/404ba54e8fd3036eee0f9241f68b17092ce734ee

== Pending Patches ==

Rushabh Lathia sent in another revision of a patch to fix an error that
manifested as GatherMerge failing to push a target list down.

Jesper Pedersen sent in another revision of a patch to implement hash
partitioning.

Oliver Ford sent in another revision of a patch to add Roman numeral conversion
to to_number.

Dilip Kumar sent in another revision of a patch to prune partitions more
efficiently in queries.

Ashutosh Bapat sent in a patch to skip "dead" relations in
create_lateral_join_info().

Jeevan Chalke sent in another revision of a patch to implement partition-wise
aggregation/grouping for declaratively partitioned tables.

Kyotaro HORIGUCHI sent in another revision of a patch to fix a race condition
between SELECT and ALTER TABLE NO INHERIT.

Noah Misch sent in a patch to change the release notes to do much less invasive
surgery on the information_schema.table_privileges view, replacing it rather
than re-creating the information schema whole cloth.

Michaël Paquier sent in another revision of a patch to remove
ALLOW_DANGEROUS_LO_FUNCTIONS for LO-related superuser checks, replace superuser
checks of large object import/export by ACL checks, remove ACL checks for large
objects when opening them.

Haribabu Kommi sent in another revision of a patch to add the support of storage
handler as an access method, add storage AM API hooks and related functions, add
storageam handler to the relation structure, add a tuple visibility function,
slot hooks, a tuple insert API, and scan functions to the storage AM, and remove
HeapScanDesc uses outside heap-specific code.

Rushabh Lathia sent in another revision of a patch to add parallel B-tree index
build sorting.

Jeff Janes sent in another revision of a patch to ensure that parallel paths
include target list cost.

Thomas Munro sent in another revision of a patch to make SERIALIZABLE isolation
work with parallel operations.

Masahiko Sawada sent in three more revisions of a patch to enable custom
initialization for pgbench.

Ashutosh Bapat sent in two more revisions of a patch to implement partition-wise
join for joins between (declaratively) partitioned tables.

Michaël Paquier sent in another revision of a patch to rewrite tests of
pg_upgrade as TAP tests.

Craig Ringer sent in a patch to add a utility class for value and identifier
quoting for use in TAP tests.

Vaishnavi Prabakaran sent in a patch to correct the debug log to reflect
the change of function name from lo_open to be_lo_open.

Amit Khandekar sent in another revision of a patch to implement parallel append.

Andres Freund sent in a patch to add the ability to force libpq to negotiate a
specific version of the protocol and add minimal v2 protocol regression tests
based on same.

Alexander Korotkov sent in two more revisions of a patch to add SP-GiST compress
methods, and GiST support for circles and polygons.

Masahiko Sawada sent in three revisions of a patch to fix an assertion failure
when the non-exclusive pg_stop_backup aborted.

Amul Sul sent in a patch to only handle oid sysattr, to reduce overhead in a
patch set intended to improve syscache/catcache performance.

Rajkumar Raghuwanshi sent in a patch to add some tests for partition-wise join
between declaratively partitioned tables.

Thomas Munro sent in a shell program which shows which blocks of SQL hit which
lines of C.

Peter Eisentraut sent in another revision of a patch to run only top-level
recursive lcov, have lcov exclude external files, add lcov --initial, add the
PostgreSQL version to coverage output, remove the coverage details view, run
coverage commands quietly, improve vpath support in the plperl build, support
coverage on vpath builds, and exclude flex-generated code from coverage testing.

Ashutosh Sharma sent in three more revisions of a patch to rewrite hash index
scan to work page at a time, remove redundant hash function _hash_step and do
some code cleanup, and improve the locking startegy during VACUUM in hash index
for regular tables.

Lucas Fittl sent in a patch to only skip query cancel itself when query cancel
holdoff count is positive.

Rafia Sabih sent in another revision of a patch to make gather faster.

Tom Lane sent in another revision of a patch to support arrays of domains.

Kyotaro HORIGUCHI sent in a patch to fix a misbehavior of hash indexes on
unlogged tables.

Julien Rouhaud sent in two more revisions of a patch to implement sorted append.

Nathan Bossart sent in three more revisions of a patch to enable specifying
multiple tables in VACUUM commands.

Nikita Glukhov sent in a patch to fix a circle bounding box inconsistency in the
GiST compress method and fix GiST ordering by distance for NULLs and NaNs.

Kyotaro HORIGUCHI sent in a patch to fix an issue where analyzeCTE is too strict
about typmods.

Amit Khandekar sent in another revision of a patch to enable UPDATEs of a
partition key which move the row to another partition.

Michaël Paquier sent in a patch to document new wait events.

Andres Freund sent in another revision of a patch to improve sys/catcache
performance.

Yura Sokolov sent in a patch to skip scanning a btree index if no pending
deleted pages exists.

Chen Huajun sent in another revision of a patch to make pg_rewind to not copy
useless WAL files.

Robert Haas sent in a patch to skip LSN checks during hash index scans.

Peter Eisentraut sent in a patch to expand the collation documentation and add
test cases for the new advanced ICU features.

Fabien COELHO sent in a patch to pgbench to use enum to store its meta commands.

Yura Sokolov sent in another revision of a patch to improve compactify_tuples.

Fabien COELHO sent in another revision of a patch to add server version
information to psql.

Tom Lane sent in a patch to make construct_[md_]array() handle empty arrays
correctly.

Andrew Dunstan sent in two more revisions of a patch to fix an infelicity in the
enum type.

Amit Kapila sent in another revision of a patch to change metapage usage for
B-tree and hash indexes.

Fabien COELHO sent in another revision of a patch to fix a pgbench regression
test failure.

Thomas Munro sent in a patch to improve the LDAP cleanup code in error paths,
log diagnostic messages if errors occur during LDAP auth, and add a regression
test to log an LDAP diagnostic message.

Mark Rofail sent in another revision of a patch to add foreign key arrays.

Tomas Vondra sent in another revision of a patch to make VACUUM and ANALYZE
agree on what reltuples means.

Erik Rijkers sent in a patch to improve some comments in prepunion.c.

Tomas Vondra sent in another revision of a patch to implement a generational
memory allocator.

Thomas Munro sent in another revision of a patch to use fallocate() to
preallocate "posix" DSM segments on Linux.


pgsql-announce by date

Next:From: DevartDate: 2017-09-28 09:57:12
Subject: Support for PostgreSQL 9.6.5 in dbForge Data Compare for PostgreSQL
Previous:From: Christoph BergDate: 2017-09-23 21:10:05
Subject: pg_dirtyread 1.2 released

Privacy Policy | About PostgreSQL
Copyright © 1996-2017 The PostgreSQL Global Development Group