Skip site navigation (1) Skip section navigation (2)

== PostgreSQL Weekly News - April 09 2017 ==

From: David Fetter <david(at)fetter(dot)org>
To: PostgreSQL Announce <pgsql-announce(at)postgresql(dot)org>
Subject: == PostgreSQL Weekly News - April 09 2017 ==
Date: 2017-04-09 23:14:42
Message-ID: (view raw, whole thread or download thread mbox)
Lists: pgsql-announce
== PostgreSQL Weekly News - April 09 2017 ==

== PostgreSQL Jobs for April ==

== PostgreSQL Local ==

PGCon 2017 will take place in Ottawa on 23-26 May.

Postgres Vision will take place in Boston, June 26 - 28, 2017.

Swiss PGDay in Rapperswil will take place June 30, 2017.
The CfP is open through April 14, 2017.

PostgresOpen will occur September 6-8, 2017 in San Francisco.
The CfP is open until May 30, 2017.

PGDay UK 2017 will take place July 4, 2017 in London, England, UK.
The CfP closes April 20th 2017.

== PostgreSQL in the News ==

Planet PostgreSQL:

PostgreSQL Weekly News is brought to you this week by David Fetter

Submit news and announcements by Sunday at 3:00pm Pacific time.
Please send English language ones to david(at)fetter(dot)org, German language
to pwn(at)pgug(dot)de, Italian language to pwn(at)itpug(dot)org(dot)

== Applied Patches ==

Tom Lane pushed:

- Document psql's behavior of recalling the previously executed query.  Various
  psql slash commands that normally act on the current query buffer will
  automatically recall and re-use the most recently executed SQL command
  instead, if the current query buffer is empty.  Although this behavior is
  ancient (dating apparently to commit 77a472993), it was documented nowhere in
  the psql reference page.  For that matter, we'd never bothered to define the
  concept of "current query buffer" explicitly.  Fix that.  Do some wordsmithing
  on relevant command descriptions to improve clarity and consistency.

- Doc: clarify behavior of OT_WHOLE_LINE and OT_FILEPIPE psql slash commands.
  This is another bit of ancient behavior that was documented poorly (in a
  couple of cases) or not at all (in several others).  Discussion:

- Remove reinvention of stringify macro.  We already have CppAsString2, there's
  no need for the MSVC support to re-invent a macro to do that (and especially
  not to inject it in as ugly a way as this).  Discussion:

- Fix integer-overflow problems in interval comparison.  When using integer
  timestamps, the interval-comparison functions tried to compute the overall
  magnitude of an interval as an int64 number of microseconds.  As reported by
  Frazer McLean, this overflows for intervals exceeding about 296000 years,
  which is bad since we nominally allow intervals many times larger than that.
  That results in wrong comparison results, and possibly in corrupted btree
  indexes for columns containing such large interval values.  To fix, compute
  the magnitude as int128 instead.  Although some compilers have native support
  for int128 calculations, many don't, so create our own support functions that
  can do 128-bit addition and multiplication if the compiler support isn't
  there.  These support functions are designed with an eye to allowing the
  int128 code paths in numeric.c to be rewritten for use on all platforms,
  although this patch doesn't do that, or even provide all the int128 primitives
  that will be needed for it.  Back-patch as far as 9.4.  Earlier releases did
  not guard against overflow of interval values at all (commit 146604ec4 fixed
  that), so it seems not very exciting to worry about overly-large intervals for
  them.  Before 9.6, we did not assume that unreferenced "static inline"
  functions would not draw compiler warnings, so omit functions not directly
  referenced by timestamp.c, the only present consumer of int128.h.  (We could
  have omitted these functions in HEAD too, but since they were written and
  debugged on the way to the present patch, and they look likely to be needed by
  numeric.c, let's keep them in HEAD.)  I did not bother to try to prevent such
  warnings in a --disable-integer-datetimes build, though.  Before 9.5,
  configure will never define HAVE_INT128, so the part of int128.h that exploits
  a native int128 implementation is dead code in the 9.4 branch.  I didn't
  bother to remove it, thinking that keeping the file looking similar in
  different branches is more useful.  In HEAD only, add a simple test harness
  for int128.h in src/tools/.  In back branches, this does not change the
  float-timestamps code path.  That's not subject to the same kind of overflow
  risk, since it computes the interval magnitude as float8.  (No doubt, when
  this code was originally written, overflow was disregarded for exactly that
  reason.)  There is a precision hazard instead :-(, but we'll avert our eyes
  from that question, since no complaints have been reported and that code's
  deprecated anyway.  Kyotaro Horiguchi and Tom Lane Discussion:

- Clean up psql/describe.c's messy query for extended stats.  Remove unnecessary
  casts, safely schema-qualify the ones that remain, lose an unnecessary level
  of sub-SELECT, reformat for tidiness.

- Clean up after insufficiently-researched optimization of tuple conversions.
  tupconvert.c's functions formerly considered that an explicit tuple conversion
  was necessary if the input and output tupdescs contained different type OIDs.
  The point of that was to make sure that a composite datum resulting from the
  conversion would contain the destination rowtype OID in its composite-datum
  header.  However, commit 3838074f8 entirely misunderstood what that check was
  for, thinking that it had something to do with presence or absence of an OID
  column within the tuple.  Removal of the check broke the no-op conversion path
  in ExecEvalConvertRowtype, as reported by Ashutosh Bapat.  It turns out that
  of the dozen or so call sites for tupconvert.c functions,
  ExecEvalConvertRowtype is the only one that cares about the composite-datum
  header fields in the output tuple.  In all the rest, we'd much rather avoid an
  unnecessary conversion whenever the tuples are physically compatible.
  Moreover, the comments in tupconvert.c only promise physical compatibility not
  a metadata match.  So, let's accept the removal of the guarantee about the
  output tuple's rowtype marking, recognizing that this is a API change that
  could conceivably break third-party callers of tupconvert.c.  (So, let's
  remember to mention it in the v10 release notes.) However, commit 3838074f8
  did have a bit of a point here, in that two tuples mustn't be considered
  physically compatible if one has HEAP_HASOID set and the other doesn't.  (Some
  of the callers of tupconvert.c might not really care about that, but we can't
  assume it in general.)  The previous check accidentally covered that issue,
  because no RECORD types ever have OIDs, while if two tupdescs have the same
  named composite type OID then, a fortiori, they have the same tdhasoid
  setting.  If we're removing the type OID match check then we'd better include
  tdhasoid match as part of the physical compatibility check.  Without that hack
  in tupconvert.c, we need ExecEvalConvertRowtype to take responsibility for
  inserting the correct rowtype OID label whenever tupconvert.c decides it need
  not do anything.  This is easily done with heap_copy_tuple_as_datum, which
  will be considerably faster than a tuple disassembly and reassembly anyway; so
  from a performance standpoint this change is a win all around compared to what
  happened in earlier branches.  It just means a couple more lines of code in
  ExecEvalConvertRowtype.  Ashutosh Bapat and Tom Lane Discussion:

- Fix planner error (or assert trap) with nested set operations.  As reported by
  Sean Johnston in bug #14614, since 9.6 the planner can fail due to trying to
  look up the referent of a Var with varno 0.  This happens because we generate
  such Vars in generate_append_tlist, for lack of any better way to describe the
  output of a SetOp node.  In typical situations nothing really cares about
  that, but given nested set-operation queries we will call estimate_num_groups
  on the output of the subquery, and that wants to know what a Var actually
  refers to.  That logic used to look at subquery->targetList, but in commit
  3fc6e2d7f I'd switched it to look at subroot->processed_tlist, ie the actual
  output of the subquery plan not the parser's idea of the result.  It seemed
  like a good idea at the time :-(.  As a band-aid fix, change it back.  Really
  we ought to have an honest way of naming the outputs of SetOp steps, which
  suggests that it'd be a good idea for the parser to emit an RTE corresponding
  to each one.  But that's a task for another day, and it certainly wouldn't
  yield a back-patchable fix.  Report:

- Ensure that ExecPrepareExprList's result is all in one memory context.  Noted
  by Amit Langote.  Discussion:

- Optimize joins when the inner relation can be proven unique.  If there can
  certainly be no more than one matching inner row for a given outer row, then
  the executor can move on to the next outer row as soon as it's found one
  match; there's no need to continue scanning the inner relation for this outer
  row.  This saves useless scanning in nestloop and hash joins.  In merge joins,
  it offers the opportunity to skip mark/restore processing, because we know we
  have not advanced past the first possible match for the next outer row.  Of
  course, the devil is in the details: the proof of uniqueness must depend only
  on joinquals (not otherquals), and if we want to skip mergejoin mark/restore
  then it must depend only on merge clauses.  To avoid adding more planning
  overhead than absolutely necessary, the present patch errs in the conservative
  direction: there are cases where inner_unique or skip_mark_restore processing
  could be used, but it will not do so because it's not sure that the uniqueness
  proof depended only on "safe" clauses.  This could be improved later.  David
  Rowley, reviewed and rather heavily editorialized on by me Discussion:

- Add newly-symlinked files to "make clean" target.  Oversight in 60f11b87a.

- Clean up bugs in clause_selectivity() cleanup.  Commit ac2b09508 was not
  terribly carefully reviewed.  Band-aid it to not fail on non-RestrictInfo
  input, per report from Andreas Seltenreich.  Also make it do something more
  reasonable with variable-free clauses, and improve nearby comments.

Peter Eisentraut pushed:

- doc: Change xref style to number only.  Change the style of links generated by
  xrefs to section number only, as it was with DSSSL, instead of number and
  title, as is the default of the XSLT stylesheets.  Our documentation is mostly
  written expecting the old style, so keep that for the time being, per

- Handle change of slot name in logical replication apply.  Since change of slot
  name is a supported operation, handle it more gracefully, instead of in the
  this-should-not-happen way.  Author: Petr Jelinek

- Fix whitespace.

- Make header self-contained.  Add necessary include files for things used in
  the header.

- Fix remote position tracking in logical replication.  We need to set the
  origin remote position to end_lsn, not commit_lsn, as commit_lsn is the start
  of commit record, and we use the origin remote position as start position when
  restarting replication stream.  If we'd use commit_lsn, we could request data
  that we already received from the remote server after a crash of a downstream
  server.  Author: Petr Jelinek <petr(dot)jelinek(at)2ndquadrant(dot)com>

- psql: Add some missing tab completion.  Add tab completion for
  Frost <sfrost(at)snowman(dot)net>

- Fix typo.  Author: Masahiko Sawada <sawada(dot)mshk(at)gmail(dot)com>

- Adjust min/max values when changing sequence type.  When changing the type of
  a sequence, adjust the min/max values of the sequence if it looks like the
  previous values were the default values.  Previously, it would leave the old
  values in place, requiring manual adjustments even in the usual/default cases.
  Reviewed-by: Michael Paquier <michael(dot)paquier(at)gmail(dot)com> Reviewed-by: Vitaly
  Burovoy <vitaly(dot)burovoy(at)gmail(dot)com>

- pageinspect: Add bt_page_items function with bytea argument.  Author: Tomas
  Vondra <tomas(dot)vondra(at)2ndquadrant(dot)com> Reviewed-by: Ashutosh Sharma

- Capitalize names of PLs consistently.  Author: Daniel Gustafsson

- dblink: Small code rearrangement for clarity.  suggested by Tom Lane

- doc: Remove remark elements.  They were not rendered with DSSSL, but now they
  show up by default using XSLT.  Just remove them, since they are not useful.

- Use American English in error message.  All error messages use the American
  English spelling of recognize, apply to the single one not doing so to be
  consistent.  Author: Daniel Gustafsson <daniel(at)yesql(dot)se>

- Fix compiler warning and add some more comments.

- Fix AclResult vs bool type mix-up.  Using AclResult as a bool or vice versa
  works by accident, but it's unusual and possibly confusing style, so write it
  out more explicitly.

- doc: Formatting fix for XSL-FO PDF build.

- Fix mixup of bool and ternary value.  Not currently a problem, but could be
  with stricter bool behavior under stdbool or C++.  Reviewed-by: Andres Freund

- pg_dump: Rename some typedefs to avoid name conflicts.  In struct
  _archiveHandle, some of the fields have the same name as a typedef.  This is
  kind of confusing, so rename the types so they have names distinct from the
  struct fields.  In C++, the previous coding changes the meaning of the typedef
  in the scope of the struct, causing warnings and possibly other problems.
  Reviewed-by: Andres Freund <andres(at)anarazel(dot)de>

- Mark immutable functions in information schema as parallel safe.  Also add
  opr_sanity check that all preloaded immutable functions are parallel safe.
  (Per discussion, this does not necessarily have to be true for all possible
  such functions, but deviations would be unlikely enough that maintaining such
  a test is reasonable.) Reported-by: David Rowley
  <david(dot)rowley(at)2ndquadrant(dot)com> Reviewed-by: Robert Haas
  <robertmhaas(at)gmail(dot)com> Reviewed-by: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>

- Fix logical replication between different encodings.  When sending a tuple
  attribute, the previous coding erroneously sent the length byte before
  encoding conversion, which would lead to protocol failures on the receiving
  side if the length did not match the following string.  To fix that, use
  pq_sendcountedtext() for sending tuple attributes, which takes care of all of
  that internally.  To match the API of pq_sendcountedtext(), send even text
  values without a trailing zero byte and have the receiving end put it in place
  instead.  This matches how the standard FE/BE protocol behaves.  Reported-by:
  Kyotaro HORIGUCHI <horiguchi(dot)kyotaro(at)lab(dot)ntt(dot)co(dot)jp>

- Remove use of Jade and DSSSL.  All documentation is now built using XSLT.
  Remove all references to Jade, DSSSL, also JadeTex and some other outdated
  tooling.  For chunked HTML builds, this changes nothing, but removes the
  transitional "oldhtml" target.  The single-page HTML build is ported over to
  XSLT.  For PDF builds, this removes the JadeTex builds and moves the FOP
  builds in their place.

- Avoid using a C++ keyword in header file.  per cpluspluscheck

- Fix compiler warning on MSVC 2010.  Author: Michael Paquier

- Identity columns.  This is the SQL standard-conforming variant of PostgreSQL's
  serial columns.  It fixes a few usability issues that serial columns have:
  * CREATE TABLE / LIKE copies default but refers to same sequence * cannot
  add/drop serialness with ALTER TABLE * dropping default does not drop sequence
  * need to grant separate privileges to sequence * other slight weirdnesses
  because serial is some kind of special macro.  Reviewed-by: Vitaly Burovoy

- doc: Add SPFLAGS to osx calls.  This enables the same OpenSP warnings on osx
  calls that we get from onsgmls (make check) and formerly from openjade.  Older
  tool chains apparently have some of these warnings on by default (see comment
  at SPFLAGS assignment).  So users of such tool chains would complain about
  warnings or errors that users of newer tool chains would not see, unless they
  used "make check".

- doc: Add missing closing tag.  Author: Erik Rijkers <er(at)xs4all(dot)nl>

- doc: Add some markup.

Andrew Gierth pushed:

- Try and silence spurious Coverity warning.  gset_data (aka gd) in planner.c is
  always non-null if and only if parse->groupingSets is non-null, but Coverity
  doesn't know that and complains.  Feed it an assertion to see if that keeps it

Robert Haas pushed:

- Properly acquire buffer lock for page-at-a-time hash vacuum.  In a couple of
  places, _hash_kill_items was mistakenly called with the buffer lock not held.
  Repair.  Ashutosh Sharma, per a report from Andreas Seltenreich Discussion:

- Abstract logic to allow for multiple kinds of child rels.  Currently, the only
  type of child relation is an "other member rel", which is the child of a
  baserel, but in the future joins and even upper relations may have child rels.
  To facilitate that, introduce macros that test to test for particular
  RelOptKind values, and use them in various places where they help to clarify
  the sense of a test.  (For example, a test may allow RELOPT_OTHER_MEMBER_REL
  either because it intends to allow child rels, or because it intends to allow
  simple rels.) Also, remove find_childrel_top_parent, which will not work for a
  child rel that is not a baserel.  Instead, add a new RelOptInfo member
  top_parent_relids to track the same kind of information in a more generic
  manner.  Ashutosh Bapat, slightly tweaked by me.  Review and testing of the
  patch set from which this was taken by Rajkumar Raghuwanshi and Rafia Sabih.

- Update comment.  Craig Ringer, reviewed by me.

- Print new RelOptInfo field top_parent_relids in outfuncs.c.  I intended to
  include this adjustment in the previous commit
  (7a39b5e4d11229ece930a51fd7cb29e535db44940) but messed up.

- Further corrections and improvements to partitioning documentation.  Amit
  Langote Discussion:

- Expand hash indexes more gradually.  Since hash indexes typically have very
  few overflow pages, adding a new splitpoint essentially doubles the on-disk
  size of the index, which can lead to large and abrupt increases in disk usage
  (and perhaps long delays on occasion).  To mitigate this problem to some
  degree, divide larger splitpoints into four equal phases.  This means that,
  for example, instead of growing from 4GB to 8GB all at once, a hash index will
  now grow from 4GB to 5GB to 6GB to 7GB to 8GB, which is perhaps still not as
  smooth as we'd like but certainly an improvement.  This changes the on-disk
  format of the metapage, so bump HASH_VERSION from 2 to 3.  This will force a
  REINDEX of all existing hash indexes, but that's probably a good idea anyway.
  First, hash indexes from pre-10 versions of PostgreSQL could easily be
  corrupted, and we don't want to confuse corruption carried over from an older
  release with any corruption caused despite the new write-ahead logging in v10.
  Second, it will let us remove some backward-compatibility code added by commit
  293e24e507838733aba4748b514536af2d39d7f2.  Mithun Cy, reviewed by Amit Kapila,
  Jesper Pedersen and me.  Regression test outputs updated by me.  Discussion:

- Fix formula in _hash_spareindex.  This was correct in earlier versions of the
  patch that lead to commit ea69a0dead5128c421140dc53fac165ba4af8520, but
  somehow got broken in the last version which I actually committed.  Mithun Cy,
  per an off-list report from Ashutosh Sharma Discussion:

- Fix thinko in BitmapAdjustPrefetchIterator.  Dilip Kumar Discussion:

- hash: Fix write-ahead logging bug.  The size of the data is not the same thing
  as the size of the size of the data.  Reported off-list by Tushar Ahuja.  Fix
  by Ashutosh Sharma, reviewed by Amit Kapila.  Discussion:

- Fix pageinspect failures on hash indexes.  Make every page in a hash index
  which isn't all-zeroes have a valid special space, so that tools like
  pageinspect don't error out.  Also, make pageinspect cope with all-zeroes
  pages, because _hash_alloc_buckets can leave behind large numbers of those
  until they're consumed by splits.  Ashutosh Sharma and Robert Haas, reviewed
  by Amit Kapila.  Original trouble report from Jeff Janes.  Discussion:

- Add ProcArrayGroupUpdate wait event.  Discussion:

- Use English, instead of internal names, for translatable messages.

Stephen Frost pushed:

- Remove --verbose from PROVE_FLAGS.  Per discussion, the TAP tests are really
  more verbose than necessary, so remove the --verbose flag from PROVE_FLAGS.
  Also add comments to let folks know how they can enable it if they really wish
  to, as suggested by Craig Ringer.  Author: Michael Paquier, additional
  comments by me.  Discussion:

Simon Riggs pushed:

- Speedup 2PC recovery by skipping two phase state files in normal path.  2PC
  state info held in shmem at PREPARE, then cleaned at COMMIT PREPARED/ABORT
  PREPARED, avoiding writing/fsyncing any state information to disk in the
  normal path, greatly enhancing replay speed.  Prepared transactions that live
  past one checkpoint redo horizon will be written to disk as now.  Similar
  conceptually to 978b2f65aa1262eb4ecbf8b3785cb1b9cf4db78e and building upon the
  infrastructure created by that commit.  Authors, in equal measure: Stas
  Kelvich, Nikhil Sontakke and Michael Paquier Discussion:

- Fix uninitialized variables in twophase.c.

- Make min_wal_size/max_wal_size use MB internally.  Previously they were
  defined using multiples of XLogSegSize.  Remove GUC_UNIT_XSEGS. Introduce
  GUC_UNIT_MB Extracted from patch series on XLogSegSize infrastructure.  Beena

- Spelling mistake in comment in utility.c.

- Allow --with-wal-segsize=n up to n=1024MB.  Other part of Beena Emerson's
  patch to allow testing

- Collect and use multi-column dependency stats.  Follow on patch in the
  multi-variate statistics patch series.  CREATE STATISTICS s1 WITH
  (dependencies) ON (a, b) FROM t; ANALYZE; will collect dependency stats on (a,
  b) and then use the measured dependency in subsequent query planning.  Commit
  7b504eb282ca2f5104b5c00b4f05a3ef6bb1385b added CREATE STATISTICS with
  n-distinct coefficients. These are now specified using the mutually exclusive
  option WITH (ndistinct).  Author: Tomas Vondra, David Rowley Reviewed-by:
  Kyotaro HORIGUCHI, Álvaro Herrera, Dean Rasheed, Robert Haas and many other
  comments and contributions Discussion:

- Reduce lock level for CREATE STATISTICS.  In line with other lock reductions
  related to planning.  Simon Riggs

- Avoid SnapshotResetXmin() during AtEOXact_Snapshot().  For normal commits and
  aborts we already reset PgXact->xmin, so we can simply avoid running
  SnapshotResetXmin() twice.  During performance tests by Alexander Korotkov,
  diagnosis by Andres Freund showed PgXact array as a bottleneck. After manual
  analysis by me of the code paths that touch those memory locations, I was able
  to identify extraneous code in the main transaction commit path.  Avoiding
  touching highly contented shmem improves concurrent performance slightly on
  all workloads, confirmed by tests run by Ashutosh Sharma and Alexander
  Korotkov.  Simon Riggs Discussion:

- Always SnapshotResetXmin() during ClearTransaction().  Avoid corner cases
  during 2PC with 6bad580d9e678a0b604883e14d8401d469b06566

- Reset API of clause_selectivity().  Discussion:

Andrew Dunstan pushed:

- Clarify documentation of to_tsvector(json(b)).  Per gripe from Sven R. Kunze

- Make json_populate_record and friends operate recursively.  With this change
  array fields are populated from json(b) arrays, and composite fields are
  populated from json(b) objects.  Along the way, some significant code
  refactoring is done to remove redundancy in the way to populate_record[_set]
  and to_record[_set] functions operate, and some significant efficiency gains
  are made by caching tuple descriptors.  Nikita Glukhov, edited some by me.
  Reviewed by Aleksander Alekseev and Tom Lane.

- Remove extraneous comma to satisfy picky compiler.  per buildfarm

Andres Freund pushed:

- Force synchronous commit in new-ish test_decoding test.  This was missed in
  a924c327 ff.

- Fix two valgrind issues in slab allocator.  During allocation
  VALGRIND_MAKE_MEM_DEFINED was called with a pointer as size. That kind of
  works, but makes valgrind exceedingly slow for workloads involving the slab
  allocator.  Secondly there was an access to memory marked as unreachable
  within SlabCheck(). Fix that too.  Author: Tomas Vondra Discussion:

- Code review for recent slot.c changes.

- Increase parallel bitmap scan test coverage.  Author: Dilip Kumar Discussion:

- Add minimal test for EXPLAIN ANALYZE of parallel query.  This displays the
  number of workers launched, thus the test is dependant on configuration to
  some degree.  We'll see whether that turns out ot be a problem.  Author: Rafia
  Sabih Discussion:

- Fix parallel bitmapscan tests on builds without USE_PREFETCH.  This was broken
  in 5a5931533edd2.

- Allow avoiding tuple copy within tuplesort_gettupleslot().  Add a "copy"
  argument to make it optional to receive a copy of caller tuple that is safe to
  use following a subsequent manipulating of tuplesort's state.  This is a
  performance optimization.  Most existing tuplesort_gettupleslot() callers are
  made to opt out of copying.  Existing callers that happen to rely on the
  validity of tuple memory beyond subsequent manipulations of the tuplesort
  request their own copy.  This brings tuplesort_gettupleslot() in line with
  tuplestore_gettupleslot().  In the future, a "copy" tuplesort_getdatum()
  argument may be added, that similarly allows callers to opt out of receiving
  their own copy of tuple.  In passing, clarify assumptions that callers of
  other tuplesort fetch routines may make about tuple memory validity, per gripe
  from Tom Lane.  Author: Peter Geoghegan Discussion:

- Fix issues in e8fdbd58fe.  When the 64bit atomics simulation is in use, we
  can't necessarily guarantee the correct alignment of the atomics due to lack
  of compiler support for doing so- that's fine from a safety perspective,
  because everything is protected by a lock, but we asserted the alignment in
  all cases.  Weaken them.  Per complaint from Alvaro Herrera.  My #ifdefery for
  PG_HAVE_8BYTE_SINGLE_COPY_ATOMICITY wasn't sufficient. Fix that.  Per
  complaint from Alexander Korotkov.

- Improve 64bit atomics support.  When adding atomics back in b64d92f1a, I added
  64bit support as optional; there wasn't yet a direct user in sight.  That
  turned out to be a bit short-sighted, it'd already have been useful a number
  of times.  Add a fallback implementation of 64bit atomics, just like the one
  we have for 32bit atomics.  Additionally optimize reads/writes to 64bit on a
  number of platforms where aligned writes of that size are atomic. This can now
  be tested with PG_HAVE_8BYTE_SINGLE_COPY_ATOMICITY.  Author: Andres Freund
  Reviewed-By: Amit Kapila Discussion:

Kevin Grittner pushed:

- Follow-on cleanup for the transition table patch.  Commit 59702716 added
  transition table support to PL/pgsql so that SQL queries in trigger functions
  could access those transient tables.  In order to provide the same level of
  support for PL/perl, PL/python and PL/tcl, refactor the relevant code into a
  new function SPI_register_trigger_data.  Call the new function in the trigger
  handler of all four PLs, and document it as a public SPI function so that
  authors of out-of-tree PLs can do the same.  Also get rid of a second
  QueryEnvironment object that was maintained by PL/pgsql.  That was previously
  used to deal with cursors, but the same approach wasn't appropriate for PLs
  that are less tangled up with core code.  Instead, have SPI_cursor_open
  install the connection's current QueryEnvironment, as already happens for
  SPI_execute_plan.  While in the docs, remove the note that transition tables
  were only supported in C and PL/pgSQL triggers, and correct some ommissions.
  Thomas Munro with some work by Kevin Grittner (mostly docs)

- Add isolation test for SERIALIZABLE READ ONLY DEFERRABLE.  This improves code
  coverage and lays a foundation for testing similar issues in a distributed
  environment.  Author: Thomas Munro <thomas(dot)munro(at)enterprisedb(dot)com>
  Reviewed-by: Michael Paquier <michael(dot)paquier(at)gmail(dot)com>

- Fix the RTE_NAMEDTUPLESTORE case in get_rte_attribute_is_dropped().  Problems
  pointed out by Andres Freund and Thomas Munro.

- Add GUCs for predicate lock promotion thresholds.  Defaults match the fixed
  behavior of prior releases, but now DBAs have better options to tune
  serializable workloads.  It might be nice to be able to set this per relation,
  but that part will need to wait for another release.  Author: Dagfinn Ilmari

Heikki Linnakangas pushed:

- Remove dead code and fix comments in fast-path function handling.
  HandleFunctionRequest() is no longer responsible for reading the protocol
  message from the client, since commit 2b3a8b20c2. Fix the outdated comments.
  HandleFunctionRequest() now always returns 0, because the code that used to
  return EOF was moved in 2b3a8b20c2. Therefore, the caller no longer needs to
  check the return value.  Reported by Andres Freund. Backpatch to all supported
  versions, even though this doesn't have any user-visible effect, to make
  backporting future patches in this area easier.  Discussion:

- Remove bogus SCRAM_ITERATION_LEN constant.  It was not used for what the
  comment claimed, at all. It was actually used as the 'base' argument to
  strtol(), when reading the iteration count. We don't need a constant for
  base-10, so remove it.

- Remove duplicate assignment.  Harmless, but clearly wrong.  Kyotaro Horiguchi

- Use SASLprep to normalize passwords for SCRAM authentication.  An important
  step of SASLprep normalization, is to convert the string to Unicode
  normalization form NFKC. Unicode normalization requires a fairly large table
  of character decompositions, which is generated from data published by the
  Unicode consortium. The script to generate the table is put in
  src/common/unicode, as well test code for the normalization.  A pre-generated
  version of the tables is included in src/include/common, so you don't need the
  code in src/common/unicode to build PostgreSQL, only if you wish to modify the
  normalization tables.  The SASLprep implementation depends on the UTF-8
  functions from src/backend/utils/mb/wchar.c. So to use it, you must also
  compile and link that. That doesn't change anything for the current users of
  these functions, the backend and libpq, as they both already link with
  wchar.o.  It would be good to move those functions into a separate file in
  src/commmon, but I'll leave that for another day.  No documentation changes
  included, because there is no details on the SCRAM mechanism in the docs
  anyway. An overview on that in the protocol specification would probably be
  good, even though SCRAM is documented in detail in RFC5802. I'll write that as
  a separate patch. An important thing to mention there is that we apply
  SASLprep even on invalid UTF-8 strings, to support other encodings.  Patch by
  Michael Paquier and me.  Discussion:

- Fix the new SASLprep tests to work with non-UTF-8 locales.  Fix by forcing
  database encoding to UTF-8, regardless of the current locale.  Pointed out by
  Tom Lane.  Discussion:

Álvaro Herrera pushed:

- Comment fixes for extended statistics.  Clean up some code comments in new
  extended statistics code, from 7b504eb282.

- Fix BRIN cost estimation.  The original code was overly optimistic about the
  cost of scanning a BRIN index, leading to BRIN indexes being selected when
  they'd be a worse choice than some other index.  This complete rewrite should
  be more accurate.  Author: David Rowley, based on an earlier patch by Emre
  Hasegeli Reviewed-by: Emre Hasegeli Discussion:

- Fix new BRIN desummarize WAL record.  The WAL-writing piece was forgetting to
  set the pages-per-range value.  Also, fix the declared type of struct member
  heapBlk, which I mistakenly set as OffsetNumber rather than BlockNumber.
  Problem was introduced by commit c655899ba9ae (April 1st).  Any system that
  tries to replay the new WAL record written before this fix is likely to die on
  replay and require pg_resetwal.  Reported by Tom Lane.  Discussion:

- Reduce the number of pallocs() in BRIN.  Instead of allocating memory in
  brin_deform_tuple and brin_copy_tuple over and over during a scan, allow reuse
  of previously allocated memory.  This is said to make for a measurable
  performance improvement.  Author: Jinyu Zhang, Álvaro Herrera Reviewed by:
  Tomas Vondra Discussion:

- Fix printf format to use %zd when printing sizes.  Using %ld as we were doing
  raises compiler warnings on 32 bit platforms.  Reported by Andres Freund.

Joe Conway pushed:

- Silence compiler warning in sepgsql.  <selinux/label.h> includes <stdbool.h>,
  which creates an incompatible We don't care if <stdbool.h> redefines
  "true"/"false"; those are close enough.  Complaint and initial patch by Mike
  Palmiotto. Final approach per Tom Lane's suggestion, as discussed on hackers.
  Backpatching to all supported branches.  Discussion:

- Silence uninitialized variable compiler warning in sepgsql.  At -Og
  optimization gcc warns that variable tclass may be used uninitialized when
  relkind == RELKIND_INDEX. Actually that can't happen due to an early return,
  but quiet the compiler by initializing tclass to 0.  In passing, use uint16_t
  consistently for the declaration of tclass.  Complaint and initial patch by
  Mike Palmiotto. Editorializing by me.  Probably not worth backpatching given
  that it is cosmetic, so apply to development head only.  Discussion:

Magnus Hagander pushed:

- Fix typo in comment.  Masahiko Sawada

== Pending Patches ==

Vaishnavi Prabakaran sent in two more revisions of a patch to add
batch/pipelining support to libpq.

Craig Ringer sent in five more revisions of a patch to add logical decoding on

Ashutosh Bapat and Amit Langote traded patches to document some intricacies of
the relationship between declarative partitions and foreign tables.

Takayuki Tsunakawa sent in three more revisions of a patch to support huge pages
on Windows.

Amit Khandekar sent in two more revisions of a patch to enable UPDATEs on
partitioned tables which would cause rows to move among partitions to work.

Rushabh Lathia sent in a patch to add coverage tests for gather merge.

Amit Langote sent in a patch to enable ON CONFLICT DO NOTHING on partitioned

Haribabu Kommi sent in another revision of a patch to refactor handling of
database attributes between pg_dump and pg_dumpall.

Alexander Korotkov sent in another revision of a patch to implement incremental

Daniel Gustafsson sent in a patch to use strcmp() instead of pg_strcasecmp() for
identifier matching.

Kyotaro HORIGUCHI sent in a patch to distinguish "aggressive" VACUUMs from non-
in logs.

Vinayak Pokale sent in another revision of a patch to add an ANALYZE progress

Tatsuo Ishii and Andres Freund traded patches to rearm statement_timeout after
each executed query.

Michaël Paquier sent in two revisions of a patch to rewrite the test of
pg_upgrade as a TAP test.

Kuntal Ghosh sent in two revisions of a patch to fix parallel worker counts
after a crash.

Amit Langote sent in a patch to remove pg_stat_progress_vacuum from Table 28.2.

David Rowley sent in three more revisions of a patch to make clausesel smarter.

Antonin Houska sent in another revision of a patch to implement aggregation

Kyotaro HORIGUCHI sent in another revision of a patch to implement asynchronous

Maksim Milyutin sent in a patch to create a "local" index for partitioned

Fabien COELHO sent in a patch to add a special variable in psql to reflect the
last query status.

Ashutosh Bapat sent in two more revisions of a patch to implement partition-wise
joins for declaratively partitioned tables.

Rahila Syed sent in three more revisions of a patch to add support for a default
partition in declarative partitions.

Craig Ringer and Jim Nasby traded patches to add SPI_execute_callback and
documentation for same, then use it in PL/PythonU to speed up SPI results.

Dmitry Dolgov sent in two more revisions of a patch to implement generic type

Amit Khandekar sent in another revision of a patch to implement parallel append.

Etsuro Fujita sent in another revision of a patch to fix a bug in the PostgreSQL

Pavan Deolasee sent in two more revisions of a patch to implement WARM.

Mark Dilger sent in a patch to create a generic PG_GETARG_*_P infrastructure.

Robert Haas sent in another revision of a patch to add pgstathashindex() to get
hash index table statistics.

Etsuro Fujita sent in another revision of a patch to support parameterized
foreign joins.

Fabien COELHO sent in three more revisions of a patch to enable pgbench to store
select results into variables.

Jeff Davis sent in a patch to implement range merge join.

Ashutosh Bapat sent in a patch to implement constraint exclusion for partitioned

Kyotaro HORIGUCHI sent in a patch to ensure that enabling subscription starts a

Michaël Paquier sent in a patch to use base64-based encoding for stored and
server keys in SCRAM verifiers, move the routine to build the SCRAM verifier
into src/common/, refactor frontend-side random number generation, extend
PQencryptPassword with a hashing method, and extend psql's \password and
createuser to handle SCRAM verifier creation.

Tatsuro Yamada sent in a patch to make postgresGetForeignPlan use foreignrel
instead of baserel.

Thomas Munro sent in a patch to implement hash tables in dynamic shared memory.

Thomas Munro sent in a PoC patch to enable sharing record typmods between

Vitaly Burovoy sent in a patch to implement SET IDENTITY ... IF NOT EXISTS.

Kyotaro HORIGUCHI sent in a patch to fix distclean of

Yorick Peterse sent in two revisions of a patch to update the hot-standby
documentation (in the high availability section) so it explicitly mentions that
certain settings need to be applied to servers in a particular order.

Beena Emerson sent in a patch to update the initdb regression tests to include
increasing the default WAL segment size.

Aleksander Alekseev sent in another revision of a patch to remove an unused
argument in btree_xlog_split.

Tatsuo Ishii sent in a patch to document the fact that pg_export_snapshot()
cannot be used during recovery (i.e. on standby servers).

Aleksander Alekseev sent in a patch to warn users about duplicate configuration
parameters in postgresql.conf.

Álvaro Herrera sent in another revision of a patch to fix wal_level_minimal.

Thomas Munro sent in a patch to initialise a freed segment counter in dsa.c.

Claudio Freire sent in another revision of a patch to allow VACUUM to use more
than 1GB work mem in, and make it free dead tuples array as early as possible.

Thomas Munro sent in a patch to fix and document some snapshot issues.

Thomas Munro sent in a patch to add a pg_waiting_for_safe_snapshot() function.

Tom Lane and Mark Dilger traded patches to use IsA checks for bitmapsets.

Michael Banck sent in another revision of a patch to add an option to create a
replication slot in pg_basebackup if not yet present.

pgsql-announce by date

Next:From: Martín MarquésDate: 2017-04-10 12:39:59
Subject: PgDay Argentina 2017 in Santa Fe - Call For Papers is Open
Previous:From: David FetterDate: 2017-04-02 22:50:15
Subject: == PostgreSQL Weekly News - April 02 2017 ==

Privacy Policy | About PostgreSQL
Copyright © 1996-2017 The PostgreSQL Global Development Group