Skip site navigation (1) Skip section navigation (2)

== PostgreSQL Weekly News - April 02 2017 ==

From: David Fetter <david(at)fetter(dot)org>
To: PostgreSQL Announce <pgsql-announce(at)postgresql(dot)org>
Subject: == PostgreSQL Weekly News - April 02 2017 ==
Date: 2017-04-02 22:50:15
Message-ID: (view raw, whole thread or download thread mbox)
Lists: pgsql-announce
== PostgreSQL Weekly News - April 02 2017 ==

PGDay UK 2017 - Registration and CFP are open

== PostgreSQL Product News ==

postgresql-unit 3.0, an extension which implements a PostgreSQL datatype for SI
units, some non-SI units and bytes, released.

== PostgreSQL Jobs for April ==

== PostgreSQL Local ==

PGCon 2017 will take place in Ottawa on 23-26 May.

Postgres Vision will take place in Boston, June 26 - 28, 2017.

Swiss PGDay in Rapperswil will take place June 30, 2017.
The CfP is open through April 14, 2017.

PostgresOpen will occur September 6-8, 2017 in San Francisco.
The CfP is open until May 30, 2017.

== PostgreSQL in the News ==

Planet PostgreSQL:

PostgreSQL Weekly News is brought to you this week by David Fetter

Submit news and announcements by Sunday at 3:00pm Pacific time.
Please send English language ones to david(at)fetter(dot)org, German language
to pwn(at)pgug(dot)de, Italian language to pwn(at)itpug(dot)org(dot)

== Applied Patches ==

Tom Lane pushed:

- Fix unportable disregard of alignment requirements in RADIUS code.  The
  compiler is entitled to store a char[] local variable with no particular
  alignment requirement.  Our RADIUS code cavalierly took such a local variable
  and cast its address to a struct type that does have alignment requirements.
  On an alignment-picky machine this would lead to bus errors.  To fix, declare
  the local variable honestly, and then cast its address to char * for use in
  the I/O calls.  Given the lack of field complaints, there must be very few if
  any people affected; but nonetheless this is a clear portability issue, so
  back-patch to all supported branches.  Noted while looking at a Coverity
  complaint in the same code.

- Fix some minor resource leaks in PerformRadiusTransaction().  Failure to free
  serveraddrs pointed out by Coverity, failure to close socket noted by
  code-reading.  These bugs seem to be quite old, but given the low probability
  of taking these error-exit paths and the minimal consequences of the leaks
  (since the process would presumably exit shortly anyway), it doesn't seem
  worth back-patching.  Michael Paquier and Tom Lane

- Fix typos in logical replication support for initial data copy.  Fix an
  incorrect assert condition (noted by Coverity), and spell the new name of the
  function correctly.  Typos introduced in commit 7c4f52409.  Michael Paquier

- Use ExecPrepareExpr in place of ExecPrepareCheck where appropriate.  Change
  one more place where ExecInitCheck/ExecPrepareCheck's insistence on getting
  implicit-AND-format quals wasn't really helpful, because the caller had to do
  make_ands_implicit() for no reason that it cared about.  Using ExecPrepareExpr
  directly simplifies the code and saves cycles.  The only remaining use of
  these functions is to process resultRelInfo->ri_PartitionCheck quals.
  However, implicit-AND format does seem to be what we want for that, so leave
  it alone.

- Improve performance of ExecEvalWholeRowVar.  In commit b8d7f053c, we needed to
  fix ExecEvalWholeRowVar to not change the state of the slot it's copying.  The
  initial quick hack at that required two rounds of tuple construction, which is
  not very nice.  To fix, add another primitive to tuptoaster.c that does
  precisely what we need.  (I initially tried to do this by refactoring one of
  the existing functions into two pieces; but it looked like that might hurt
  performance for the existing case, and the amount of code that could be shared
  is not very large, so I gave up on that.) Discussion:

- Show ignored constants as "$N" rather than "?" in pg_stat_statements.  The
  trouble with the original choice here is that "?" is a valid (and indeed used)
  operator name, so that you could end up with ambiguous statement texts like
  "SELECT ? ? ?".  With this patch, you instead see "SELECT $1 ? $2", which
  seems significantly more readable.  The numbers used for this purpose begin
  after the last actual $N parameter in the particular query.  The conflict with
  external parameters has its own potential for confusion of course, but it was
  agreed to be an improvement over the previous behavior.  Lukas Fittl

- Suppress implicit-conversion warnings seen with newer clang versions.  We were
  assigning values near 255 through "char *" pointers.  On machines where char
  is signed, that's not entirely kosher, and it's reasonable for compilers to
  warn about it.  A better solution would be to change the pointer type to
  "unsigned char *", but that would be vastly more invasive.  For the moment,
  let's just apply this simple backpatchable solution.  Aleksander Alekseev

- Make new expression eval code reject references to dropped columns.  Formerly,
  a Var referencing an already-dropped column was allowed and would always
  produce a NULL value.  However, that behavior was implemented in slot_getattr
  which the new expression code doesn't use; thus there is now a risk of
  returning theoretically-deleted data.  We had regression test cases that
  purported to exercise this, but they failed to expose any problem, apparently
  because plpgsql filters the dropped column and produces an output tuple that
  has a NULL there already.  Ideally the DROP or ALTER attempt in these test
  cases would get rejected due to dependency checks; but until that happens,
  let's modify the behavior so that we fail the query during executor start.
  This was already true for the related case of a column having changed type
  underneath us, and there's no obvious reason why we need to be laxer for
  dropped columns.  In passing, adjust the error messages in
  CheckVarSlotCompatibility to include the composite type name.  In the cases
  shown in the regression tests this is always just "record", but it should be
  more useful in actual stale-plan cases, where the slot tupdesc would be a
  table's tupdesc directly.  Discussion:

- Support \if ... \elif ... \else ... \endif in psql scripting.  This patch adds
  nestable conditional blocks to psql.  The control structure feature per se is
  complete, but the boolean expressions understood by \if and \elif are pretty
  primitive; basically, after variable substitution and backtick expansion, the
  result has to be "true" or "false" or one of the other standard spellings of a
  boolean value.  But that's enough for many purposes, since you can always do
  the heavy lifting on the server side; and we can extend it later.  Along the
  way, pay down some of the technical debt that had built up around
  psql/command.c: * Refactor exec_command() into a function per command, instead
  of being a 1500-line monstrosity.  This makes the file noticeably longer
  because of repetitive function header/trailer overhead, but it seems much more
  readable.  * Teach psql_get_variable() and psqlscanslash.l to suppress
  variable substitution and backtick expansion on the basis of the conditional
  stack state, thereby allowing removal of the OT_NO_EVAL kluge.  * Fix the
  no-doubt-once-expedient hack of sometimes silently substituting mainloop.c's
  previous_buf for query_buf when calling HandleSlashCmds.  (It's a bit
  remarkable that commands like \r worked at all with that.) Recall of a
  previous query is now done explicitly in the slash commands where that should
  happen.  Corey Huinker, reviewed by Fabien Coelho, further hacking by me

- Fix broken markup.  Per buildfarm.

- For foreign keys, check REFERENCES privilege only on the referenced table.  We
  were requiring that the user have REFERENCES permission on both the referenced
  and referencing tables --- but this doesn't seem to have any support in the
  SQL standard, which says only that you need REFERENCES permission on the
  referenced table.  And ALTER TABLE ADD FOREIGN KEY has already checked that
  you own the referencing table, so the check could only fail if a table owner
  has revoked his own REFERENCES permission.  Moreover, the symmetric
  interpretation of this permission is unintuitive and confusing, as per
  complaint from Paul Jungwirth.  So let's drop the referencing-side check.  In
  passing, do a bit of wordsmithing on the GRANT reference page so that all the
  privilege types are described in similar fashion.  Discussion:

- Fix unstable regression test result.  Commit e306df7f9 added a test case that
  depends on "the" being a stop word, which it is not in non-English locales.
  Since the point of the test is to check stopword behavior, fix by forcibly
  selecting the 'english' configuration.  Per buildfarm.

- Fix unstable regression test result.  Whoops, missed that same test was made
  for json as well as jsonb.

- Allow psql variable substitution to occur in backtick command strings.
  Previously, text between backquotes in a psql metacommand's arguments was
  always passed to the shell literally.  That considerably hobbles the
  usefulness of the feature for scripting, so we'd foreseen for a long time that
  we'd someday want to allow substitution of psql variables into the shell
  command.  IMO the addition of \if metacommands has brought us to that point,
  since \if can greatly benefit from some sort of client-side expression
  evaluation capability, and psql itself is not going to grow any such thing in
  time for v10.  Hence, this patch.  It allows :VARIABLE to be replaced by the
  exact contents of the named variable, while :'VARIABLE' is replaced by the
  variable's contents suitably quoted to become a single shell-command argument.
  (The quoting rules for that are different from those for SQL literals, so this
  is a bit of an abuse of the :'VARIABLE' notation, but I doubt anyone will be
  confused.) As with other situations in psql, no substitution occurs if the
  word following a colon is not a known variable name.  That limits the risk of
  compatibility problems for existing psql scripts; but the risk isn't zero, so
  this needs to be called out in the v10 release notes.  Discussion:

- Fix behavior of psql's \p to agree with \g, \w, etc.  In commit e984ef586 I
  (tgl) simplified the behavior of \p to just print the current query buffer;
  but Daniel Vérité points out that this made it inconsistent with the behavior
  of \g and \w.  It should print the same thing \g would execute.  Fix that, and
  improve related comments.  Daniel Vérité Discussion:

Peter Eisentraut pushed:

- Fix cpluspluscheck warning.  Structure tag cannot be the same as a typedef
  that is a pointer to that struct.

- Change default of log_directory to 'log'.  The previous default 'pg_log' might
  have indicated by its "pg_" prefix that it is an internal system directory.
  The new default is more in line with the typical naming of directories with
  user-facing log files.  Together with the renaming of pg_clog and pg_xlog,
  this should clear up that difference.  Author: Andreas Karlsson

- PL/Python: Add cursor and execute methods to plan object.  Instead of plan =
  plpy.prepare(...) res = plpy.execute(plan, ...) you can now write plan =
  plpy.prepare(...) res = plan.execute(...) or even res =
  plpy.prepare(...).execute(...) and similarly for the cursor() method.  This is
  more in object oriented style, and makes the hybrid nature of the existing
  execute() function less confusing.  Reviewed-by: Andrew Dunstan

- doc: Fix oldhtml/old PDF build again.  Commit
  e259e1f748c7a6d67e307a90d6c27b8ab8b90df8 was faulty and created some broken
  output.  This one fixes it better.

- doc: Improve rendering of notes/cautions using XSL-FO.  Center title and put a
  border around it, like the output that the DSSSL version gave.

- Clean up Perl code according to perlcritic.  Fix all perlcritic warnings of
  severity level 5, except in src/backend/utils/, which is
  automatically generated.  Reviewed-by: Dagfinn Ilmari Mannsåker
  <ilmari(at)ilmari(dot)org> Reviewed-by: Daniel Gustafsson <daniel(at)yesql(dot)se>

- Fix Perl code which had broken the Windows build.  The previous change wanted
  to avoid modifying $_ in grep, but the code just made the change in a local
  variable and then lost it.  Rewrite the code using a separate map and grep,
  which is clearer anyway.  Author: Dagfinn Ilmari Mannsåker <ilmari(at)ilmari(dot)org>

- dblink: Fix error reporting.  The conname variable was not initialized in some
  code paths, resulting in error reports referring to the "unnamed" connection
  rather than the correct connection name.  Author: Rushabh Lathia

- Fix ssl tests.  facde2a98f0b5f7689b4e30a9e7376e926e733b8 introduced a typo
  during rebasing.

- Change 'diag' to 'note' in TAP tests.  Reduce noise from TAP tests by changing
  'diag' to 'note', so output only goes to the test's log file not stdout,
  unless in verbose mode.  This also removes the junk on screen when running the
  TAP tests in parallel.  Author: Craig Ringer <craig(at)2ndquadrant(dot)com>

- doc: Mention --enable-tap-tests in regression test chapter.  Reported-by: Jeff
  Janes <jeff(dot)janes(at)gmail(dot)com>

- Cast result of copyObject() to correct type.  copyObject() is declared to
  return void *, which allows easily assigning the result independent of the
  input, but it loses all type checking.  If the compiler supports typeof or
  something similar, cast the result to the input type.  This creates a greater
  amount of type safety.  In some cases, where the result is assigned to a
  generic type such as Node * or Expr *, new casts are now necessary, but in
  general casts are now unnecessary in the normal case and indicate that
  something unusual is happening.  Reviewed-by: Mark Dilger

- Improve Node vs Expr use a bit.  Author: Mark Dilger <hornschnorter(at)gmail(dot)com>

- Fix configure check for typeof.

- Fix hardcoded typeof check result for Windows.  The test result that I had
  blindly stipulated didn't work out on the build farm, so disable the feature
  in Windows MSVC for now.

- Update copyright year in recently added files.  Author: Masahiko Sawada

- pg_dump: Remove query truncation in error messages.  Remove the behavior that
  a query mentioned in an error message would be truncated to 128 characters.
  The queries that pg_dump runs are often longer than that, and this behavior
  makes analyzing failures harder unnecessarily.  Discussion:

Robert Haas pushed:

- Show more processes in pg_stat_activity.  Previously, auxiliary processes and
  background workers not connected to a database (such as the logical
  replication launcher) weren't shown.  Include them, so that we can see the
  associated wait state information.  Add a new column to identify the processes
  type, so that people can filter them out easily using SQL if they wish.
  Before this patch was written, there was discussion about whether we should
  expose this information in a separate view, so as to avoid contaminating
  pg_stat_activity with things people might not want to see.  But putting
  everything in pg_stat_activity was a more popular choice, so that's what the
  patch does.  Kuntal Ghosh, reviewed by Amit Langote and Michael Paquier.  Some
  revisions and bug fixes by me.  Discussion:

- Fix comment.  Cut-and-paste led to something silly.  Ashutosh Sharma, reviewed
  by Amit Kapila and by me Discussion:

- Pass DSA_ALLOC_HUGE when allocating a shared TIDBitmap.  When creating an
  unshared TIDBitmap, we pass MCXT_ALLOC_HUGE to allow allocations >1GB, so by
  analogy we pass DSA_ALLOC_HUGE for a shared TIDBitmap.  Bug introduced by
  commit 98e6e89040a0534ca26914c66cae9dd49ef62ad9.  Report by Rafia Sabih, fix
  by Dilip Kumar, adjusted by me.  Discussion:

- Allow ON CONFLICT .. DO NOTHING on a partitioned table.  ON CONFLICT .. DO
  UPDATE still doesn't work, for lack of a way of enforcing uniqueness across
  partitions, but we can still allow this case.  Amit Langote, per discussion
  with Peter Geoghegan.  Additional wordsmithing by me.  Discussion:

- Fix improper NULL handling in list partitioning code.  The previous logic was
  wrong when the value was NULL but there was no partition for NULL.  Amit
  Langote, reviewed by Jeevan Ladhe Discussion:

- Tidy up the CREATE TABLE documentation for partitioning.  Remove some <note>
  tags that make this too "loud".  Fix some typos.  Amit Langote, with a few
  minor corrections by me Discussion:

- Improve documentation of how NOT NULL works with partitioning.  Amit Langote

- Still more code review for single-page hash vacuuming.  Most seriously, fix
  use of incorrect block ID, per a report from Jeff Janes that it causes a crash
  and a diagnosis from Amit Kapila.  Improve consistency between the hash and
  btree versions of this code by adding back a PANIC that btree has, and by
  registering data in the xlog record in the same way, per complaints from Jeff
  Janes and Amit Kapila.  Tidy up some minor cosmetic points, per complaints
  from Amit Kapila.  Patch by Ashutosh Sharma, reviewed by Amit Kapila, and
  tested by Jeff Janes.  Discussion:

- Plug race in dsa_attach.  With sufficiently bad luck, it was possible for a
  parallel worker to attempt attach to a DSA area after all other backends have
  detached from it, which is not legal.  If the worker had waited a little
  longer to get started, the DSM itself would have been destroyed, which is why
  this wasn't noticed before.  Thomas Munro, per a report from Andreas
  Seltenreich Discussion:

- Mark more functions parallel-restricted.  Commit
  61c2e1a95f94bb904953a6281ce17a18ac38ee6d allowed parallel query to be used in
  more places, revealing via buildfarm member mandrill that several functions
  intended to be called from triggers were incorrectly marked parallel-safe
  rather than parallel-restricted.  Report by Tom Lane.  Patch by Rafia Sabih.
  Reviewed by me.  Discussion:

- postgres_fdw: Teach IMPORT FOREIGN SCHEMA about partitioning.  Don't import
  partitions.  Do import partitioned tables which are not themselves partitions.
  Report by Stephen Frost.  Design and patch by Michael Paquier, reviewed by
  Amit Langote.  Documentation revised by me.  Discussion:

- Don't allocate storage for partitioned tables.  Also, don't allow setting
  reloptions on them, since that would have no effect given the lack of storage.
  The patch does this by introducing a new reloption kind for which there are
  currently no reloptions -- we might have some in the future -- so it adjusts
  parseRelOptions to handle that case correctly.  Bumped catversion.  System
  catalogs that contained reloptions for partitioned tables are no longer valid;
  plus, there are now fewer physical files on disk, which is not technically a
  catalog change but still a good reason to re-initdb.  Amit Langote, reviewed
  by Maksim Milyutin and Kyotaro Horiguchi and revised a bit by me.  Discussion:

- Revert "Allow ON CONFLICT .. DO NOTHING on a partitioned table.".  This
  reverts commit 8355a011a0124bdf7ccbada206a967d427039553, which turns out to
  have been a misguided effort.  We can't really support this in a partitioning
  hierarchy after all for exactly the reasons stated in the documentation
  removed by that commit.  It's still possible to use ON CONFLICT .. DO NOTHING
  (or for that matter ON CONFLICT .. DO UPDATE) on individual partitions if
  desired, but but to allow this on a partitioned table implies that we have
  some way of evaluating uniqueness across the whole partitioning hierarchy,
  which is false.  Shinoda Noriyoshi noticed that the old code was crashing
  (which we could fix, though not in a nice way) and Amit Langote realized that
  this was indicative of a fundamental problem with the commit being reverted
  here.  Discussion:

- Improve documentation for table partitioning.  Emphasize the new declarative
  partitioning more, and compare and contrast it more clearly with
  inheritance-based partitioning.  Amit Langote, reviewed and somewhat revised
  by me Discussion:

- Fix typos.  Brandur Leach

- Don't use bgw_main even to specify in-core bgworker entrypoints.  On
  EXEC_BACKEND builds, this can fail if ASLR is in use.  Backpatch to 9.5.  On
  master, completely remove the bgw_main field completely, since there is no
  situation in which it is safe for an EXEC_BACKEND build.  On 9.6 and 9.5,
  leave the field intact to avoid breaking things for third-party code that
  doesn't care about working under EXEC_BACKEND.  Prior to 9.5, there are no
  in-core bgworker entrypoints.  Petr Jelinek, reviewed by me.  Discussion:

- Fix parallel query so it doesn't spoil row estimates above Gather.  Commit
  45be99f8cd5d606086e0a458c9c72910ba8a613d removed GatherPath's num_workers
  field, but this is entirely bogus.  Normally, a path's parallel_workers flag
  is supposed to indicate the number of workers that it wants, and should be 0
  for a non-partial path.  In that commit, I mistakenly thought that GatherPath
  could also use that field to indicate the number of workers that it would try
  to start, but that's disastrous, because then it can propagate up to higher
  nodes in the plan tree, which will then get incorrect rowcounts because the
  parallel_workers flag is involved in computing those values.  Repair by
  putting the separate field back.  Report by Tomas Vondra.  Patch by me,
  reviewed by Amit Kapila.  Discussion:

- Avoid GatherMerge crash when there are no workers.  It's unnecessary to return
  an actual slot when we have no tuple.  We can just return NULL, which avoids
  the risk of indexing into an array that might not contain any elements.
  Rushabh Lathia, per a report from Tomas Vondra Discussion:

Andrew Gierth pushed:

- Support hashed aggregation with grouping sets.  This extends the Aggregate
  node with two new features: HashAggregate can now run multiple hashtables
  concurrently, and a new strategy MixedAggregate populates hashtables while
  doing sorted grouping.  The planner will now attempt to save as many sorts as
  possible when planning grouping sets queries, while not exceeding work_mem for
  the estimated combined sizes of all hashtables used.  No SQL-level changes are
  required.  There should be no user-visible impact other than the new EXPLAIN
  output and possible changes to result ordering when ORDER BY was not used
  (which affected a few regression tests).  The enable_hashagg option is
  respected.  Author: Andrew Gierth Reviewers: Mark Dilger, Andres Freund

- Attempt to stabilize grouping sets regression test plans.  Per buildfarm
  members dromedary and arapaima.

Teodor Sigaev pushed:

- Improve performance of find_tabstat_entry()/get_tabstat_entry().  Patch
  introduces a hash map reloid -> PgStat_TableStatus which improves performance
  in case of large number of tables/partitions.  Author: Aleksander Alekseev
  Reviewed-by: Andres Freund, Anastasia Lubennikova, Tels, me

- Improve performance of find_all_inheritors().  Previous coding uses three
  nested loops which obviously were a pain for large number of table's children.
  Patch replaces inner loop with a hashmap.  Author: Aleksander Alekseev
  Reviewed-by: me

- Fsync directory after creating or unlinking file.  If file was created/deleted
  just before powerloss it's possible that file system will miss that. To
  prevent it, call fsync() where creating/ unlinkg file is critical.  Author:
  Michael Paquier Reviewed-by: Ashutosh Bapat, Takayuki Tsunakawa, me

- Altering default privileges on schemas.  Extend ALTER DEFAULT PRIVILEGES
  command to schemas.  Author: Matheus Oliveira Reviewed-by: Petr Jelínek,
  Ashutosh Sharma

- Implement SortSupport for macaddr data type.  Introduces a scheme to produce
  abbreviated keys for the macaddr type.  Bump catalog version.  Author: Brandur
  Leach Reviewed-by: Julien Rouhaud, Peter Geoghegan

Álvaro Herrera pushed:

- Rework the stats_ext test.  As suggested by Tom Lane, avoid printing specific
  estimated cost values, because they vary across architectures; instead, verify
  plan shapes (in this case, HashAggregate vs. GroupAggregate), as we do in
  other planner tests.  We can now remove expected/stats_ext_1.out.  Author:
  Tomas Vondra

- Fix a couple of problems in pg_get_statisticsextdef.  There was a thinko
  whereby we tested the wrong tuple after fetching it from cache; avoid that by
  using generate_relation_name instead, which is simpler.  Also, the statistics
  name was not qualified, so add that.  (It could be argued that qualification
  should be conditional on the schema not being on search path.  We can add that
  later, but at least this form is correct.) Author: David Rowley, Álvaro
  Herrera Discussion:

- Fix thinko in estimate_num_groups.  The code for the reworked n-distinct
  estimation on commit 7b504eb282 was written differently in a previous version
  of the patch, prior to commit; on rewriting it, we missed updating an
  initializer.  This caused the code to (mistakenly) apply a fudge factor even
  in the case where a single value is applied, leading to incorrect results.
  This means that the 'relvarcount' variable name is now wrong.  Add a comment
  to try and make the situation clearer, and remove an incorrect comment I
  added.  Problem noticed, and code patch, by Tomas Vondra.  Additional
  commentary by Álvaro.

- Fix uninitialized memory propagation mistakes.  Valgrind complains that some
  uninitialized bytes are being passed around by the extended statistics code
  since commit 7b504eb282ca2f, as reported by Andres Freund.  Silence it.  Tomas
  Vondra submitted a patch which he verified to fix the complaints in his
  machine; however I messed with it a bit before pushing, so any remaining
  problems are likely my (Álvaro's) fault.  Author: Tomas Vondra Discussion:

- Remove direct uses of ItemPointer.{ip_blkid,ip_posid}.  There are no
  functional changes here; this simply encapsulates knowledge of the
  ItemPointerData struct so that a future patch can change things without more
  breakage.  All direct users of ip_blkid and ip_posid are changed to use
  existing macros ItemPointerGetBlockNumber and ItemPointerGetOffsetNumber
  respectively.  For callers where that's inappropriate (because they Assert
  that the itempointer is is valid-looking), add
  ItemPointerGetBlockNumberNoCheck and ItemPointerGetOffsetNumberNoCheck, which
  lack the assertion but are otherwise identical.  Author: Pavan Deolasee

- Allow DSM segments to be created as pinned.  dsm_create and dsm_attach assumed
  that a current resource owner was always in place.  Exploration with the API
  show that this is inconvenient: sometimes one must create a dummy resowner,
  create/attach the DSM, only to pin the mapping later, which is wasteful.
  Change create/attach so that if there is no current resowner, the dsm is
  effectively pinned right from the start.  Discussion: Reviewed by
  Thomas Munro.

- Simplify check of modified attributes in heap_update.  The old coding was
  getting more complicated as new things were added, and it would be barely
  tolerable with upcoming WARM updates and other future features such as
  indirect indexes.  The new coding incurs a small performance cost in synthetic
  benchmark cases, and is barely measurable in normal cases.  A much larger
  benefit is expected from WARM, which could actually bolt its needs on top of
  the existing coding, but it is much uglier and bug-prone than doing it on this
  new code.  Additional optimization can be applied on top of this, if need be.
  Reviewed-by: Pavan Deolasee, Amit Kapila, Mithun CY Discussion:

- Fix expected output.  Previous commit had a thinko in the expected output for
  new tests.  Per buildfarm

- BRIN de-summarization.  When the BRIN summary tuple for a page range becomes
  too "wide" for the values actually stored in the table (because the tuples
  that were present originally are no longer present due to updates or deletes),
  it can be useful to remove the outdated summary tuple, so that a future
  summarization can install a tighter summary.  This commit introduces a
  SQL-callable interface to do so.  Author: Álvaro Herrera Reviewed-by: Eiji
  Seki Discussion:

- BRIN auto-summarization.  Previously, only VACUUM would cause a page range to
  get initially summarized by BRIN indexes, which for some use cases takes too
  much time since the inserts occur.  To avoid the delay, have brininsert
  request a summarization run for the previous range as soon as the first tuple
  is inserted into the first page of the next range.  Autovacuum is in charge of
  processing these requests, after doing all the regular vacuuming/ analyzing
  work on tables.  This doesn't impose any new tasks on autovacuum, because
  autovacuum was already in charge of doing summarizations.  The only actual
  effect is to change the timing, i.e. that it occurs earlier.  For this reason,
  we don't go any great lengths to record these requests very robustly; if they
  are lost because of a server crash or restart, they will happen at a later
  time anyway.  Most of the new code here is in autovacuum, which can now be
  told about "work items" to process.  This can be used for other things such as
  GIN pending list cleaning, perhaps visibility map bit setting, both of which
  are currently invoked during vacuum, but do not really depend on vacuum taking
  place.  The requests are at the page range level, a granularity for which we
  did not have SQL-level access; we only had index-level summarization requests
  via brin_summarize_new_values().  It seems reasonable to add SQL-level access
  to range-level summarization too, so add a function brin_summarize_range() to
  do that.  Authors: Álvaro Herrera, based on sketch from Simon Riggs.
  Reviewed-by: Thomas Munro.  Discussion:

Simon Riggs pushed:

- Cleanup slots during drop database.  Automatically drop all logical
  replication slots associated with a database when the database is dropped.
  Previously we threw an ERROR if a slot existed. Now we throw ERROR only if a
  slot is active in the database being dropped.  Craig Ringer

- Correct grammar in error message.  "could not generate" rather than "could not
  generation" from commit 818fd4a67d610991757b610755e3065fb99d80a5

- Fix pgrowlocks minor coding oversight.

- Default monitoring roles.  Three nologin roles with non-overlapping privs are
  created by default * pg_read_all_settings - read all GUCs.  *
  pg_read_all_stats - pg_stat_*, pg_database_size(), pg_tablespace_size() *
  pg_stat_scan_tables - may lock/scan tables Top level role - pg_monitor
  includes all of the above by default, plus others Author: Dave Page
  Reviewed-by: Stephen Frost, Robert Haas, Peter Eisentraut, Simon Riggs

Andres Freund pushed:

- Remove support for version-0 calling conventions.  The V0 convention is
  failure prone because we've so far assumed that a function is V0 if
  PG_FUNCTION_INFO_V1 is missing, leading to crashes if a function was coded
  against the V1 interface.  V0 doesn't allow proper NULL, SRF and toast
  handling.  V0 doesn't offer features that V1 doesn't.  Thus remove V0 support
  and obsolete fmgr README contents relating to it.  Author: Andres Freund, with
  contributions by Peter Eisentraut & Craig Ringer Reviewed-By: Peter
  Eisentraut, Craig Ringer Discussion:

- Move contrib/seg to only use V1 calling conventions.  A later commit will
  remove V0 support.  Author: Andres Freund, with contributions by Craig Ringer
  Reviewed-By: Peter Eisentraut, Craig Ringer Discussion:

- Try to fix xml docs build broken in 5ded4bd.  Apparently the sgml to xml
  conversion treats non-closed <para>s differently than jade does.

- Based on the infrastructure added in b8d7f053c5c2b, this allows.  expressions
  in expensive queries to be compiled to native code.  In TPC-H this yields up
  to 40% of speedups.  Whenever a query has a total cost over
  jit_expression_minimum, all expressions in it will be JIT compiled.  That
  allows to perform JIT compilation of all expressions in one go, which is a lot
  more efficient.  Originally slated for v11, discussions at convinced
  me to add this to v10.  Author: Andres Freund Reviewed-By: Robert Haas, Heikki
  Linnakangas Discussion:

Fujii Masao pushed:

- Simplify the example of VACUUM in documentation.  Previously a detailed
  activity report by VACUUM VERBOSE ANALYZE was described as an example of
  VACUUM in docs. But it had been obsolete for a long time. For example, commit
  feb4f44d296b88b7f0723f4a4f3945a371276e0b updated the content of that activity
  report in 2003, but we had forgotten to update the example.  So basically we
  need to update the example. But since no one cared about the details of VACUUM
  output and complained about that mistake for such long time, per discussion on
  hackers, we decided to get rid of the detailed activity report from the
  example and simplify it.  Back-patch to all supported versions.  Reported by
  Masahiko Sawada, patch by me.  Discussion:

Magnus Hagander pushed:

- Fix typo in comment.  Daniel Gustafsson

- Write "waiting for checkpoint" on regular progress row.  When reporting
  progress, make the "waiting for checkpoint" test be overwritten by the
  file-based progress once it's completed. This is more consistent with how we
  report the rest of the progress.  Suggested by Jeff Janes

- Fix some typos and spelling errors in comments.  Author: Erik Rijkers

Andrew Dunstan pushed:

- Transform or iterate over json(b) string values.  Dmitry Dolgov, reviewed and
  lightly edited by me.

- Full Text Search support for json and jsonb.  The new functions are
  ts_headline() and to_tsvector.  Dmitry Dolgov, edited and documented by me.

Kevin Grittner pushed:

- Add transition table support to plpgsql.  Kevin Grittner and Thomas Munro
  Reviewed by Heikki Linnakangas, David Fetter, and Thomas Munro with valuable
  comments and suggestions from many others

- Add infrastructure to support EphemeralNamedRelation references.  A
  QueryEnvironment concept is added, which allows new types of objects to be
  passed into queries from parsing on through execution.  At this point, the
  only thing implemented is a collection of EphemeralNamedRelation objects --
  relations which can be referenced by name in queries, but do not exist in the
  catalogs.  The only type of ENR implemented is NamedTuplestore, but provision
  is made to add more types fairly easily.  An ENR can carry its own TupleDesc
  or reference a relation in the catalogs by relid.  Although these features can
  be used without SPI, convenience functions are added to SPI so that ENRs can
  easily be used by code run through SPI.  The initial use of all this is going
  to be transition tables in AFTER triggers, but that will be added to each PL
  as a separate commit.  An incidental effect of this patch is to produce a more
  informative error message if an attempt is made to modify the contents of a
  CTE from a referencing DML statement.  No tests previously covered that
  possibility, so one is added.  Kevin Grittner and Thomas Munro Reviewed by
  Heikki Linnakangas, David Fetter, and Thomas Munro with valuable comments and
  suggestions from many others

- Try to fix breakage of sepgsql hooks by ENR patch.  Turned up by buildfarm
  animal rhinoceros.  Fixing blind.  Will have to wait for next run by
  rhinoceros to know whether it worked.

- Fix two undocumented parameters to functions from ENR patch.  On
  ProcessUtility document the parameter, to match others.  On CreateCachedPlan
  drop the queryEnv parameter.  It was not referenced within the function, and
  had been added on the assumption that with some unknown future usage of
  QueryEnvironment it might be useful to do something there.  We have avoided
  other "just in case" implementation of unused paramters, so drop it here.  Per
  gripe from Tom Lane

== Pending Patches ==

Kyotaro HORIGUCHI sent in a patch to add encoding_dumper and map_dumper.

Michaël Paquier sent in two more revisions of a patch to change detection of
corrupted 2PC files as FATAL and minimize window between history file and

Nikhil Sontakke sent in another revision of a patch to speed up two-phase

Stas Kelvich and Craig Ringer traded patches to implement logical decoding of
two-phase transactions.

Haribabu Kommi sent in three more revisions of a patch to implement a
pg_stat_wal_write statistics view.

Ashutosh Bapat sent in two more revisions of a patch to implement partition-wise
join for declaratively partitioned tables.

Kyotaro HORIGUCHI sent in a patch to fix a bug in physical replication slots.

Beena Emerson sent in another revision of a patch to allow increasing the
default WAL segment size.

Amit Khandekar sent in two more revisions of a patch to enable UPDATE of
partition key for declaratively partitioned tables.

Vaishnavi Prabakaran sent in two more revisions of a patch to implement
batch/pipelining support for libpq.

Pavan Deolasee sent in six more revisions of a patch to implement WARM.

Mithun Cy sent in eight more revisions of a patch to expand hash indexes more

Ashutosh Sharma sent in two more revisions of a patch to fix a failed assertion
found by sqlsmith in _hash_kill_items/MarkBufferDirtyHint.

Pavel Stěhule and Surafel Temesgen traded patches to add CORRESPONDING.

Dmitry Dolgov sent in three more revisions of a patch to implement generic type

Ashutosh Sharma sent in three more revisions of a patch to rewrite hash index
scans to work a page at a time, remove redundant function _hash_step() and some
of the unused members of HashScanOpaqueData, and improve the locking strategy
during VACUUM in hash index.

Jan Michálek sent in four more revisions of a patch to add markdown and rst to
psql's output formats.

Pavel Stěhule sent in a patch to add an xmltable doc fix and example for

Pavel Stěhule and Petr Jelínek traded patches to add a plan_cache control PRAGMA
to pl/pgsql.

Jeff Janes sent in a patch to fix an infelicity between free space map and
visibility map.

Claudio Freire sent in another revision of a patch for VACUUM which frees the
dead tuples array as early as possible.

Jim Nasby sent in a patch to fix a missing increment of

Ashutosh Bapat and Amit Langote traded patches to fix a comment in

Haribabu Kommi sent in another revision of a patch to refactor handling of
database attributes between pg_dump and pg_dumpall.

Kyotaro HORIGUCHI sent in a patch to show vacuum aggressiveness in autovacuum

Alexander Law sent in a patch to remove trailing spaces in strings in the source

Daniel Gustafsson sent in a patch to use American English spelling in pg_waldump
error message.

Nikhil Sontakke sent in another revision of a patch to move all am-related
reloption code into src/backend/access/[am-name] and get rid of relopt_kind for
custom AM.

Alexander Korotkov sent in another revision of a patch to implement incremental

Masahiko Sawada sent in another revision of a patch to implement support for
transactions involving multiple postgres foreign servers.

Daniel Gustafsson sent in a patch to error out more informatively when multiple
TO VERSION arguments are supplied to ALTER EXTENSION.

Takeshi Ideriha sent in another revision of a patch to implement DECLARE
STATEMENT setting up a connection in ECPG.

Peter Eisentraut sent in a patch to adjust min/max values when changing sequence

Peter Eisentraut sent in another revision of a patch to implement IDENTITY

Michaël Paquier sent in three more revisions of a patch to allow interrupts on
waiting standby.

Vinayak Pokale sent in another revision of a patch to implement an ANALYZE
command progress checker.

Tomas Vondra sent in two more revisions of a patch to add page_checksum and
bt_page_items(bytea) to page_inspect.

Ashutosh Sharma sent in a patch to fix an issue that manifested as an
inconsistent page found on STANDBY server.

Takayuki Tsunakawa sent in another revision of a patch to implement huge pages
on Windows.

Kuntal Ghosh sent in a patch to fix some strange parallel query behavior after
OOM crashes.

David Rowley sent in two more revisions of a patch to fix functional
dependencies in materialized views in light of the extended statistics recently

Thomas Munro sent in another revision of a patch to implement [[Parallel]
Shared] Hash.

Anastasia Lubennikova and Teodor Sigaev traded patches to add covering + unique

Michaël Paquier sent in another revision of a patch to implement SASLprep aka
NFKC for SCRAM authentication.

Aleksander Alekseev sent in a patch to remove an unused argument in

Etsuro Fujita sent in another revision of a patch to add epqpath support for
foreign joins.

Masahiko Sawada sent in a patch to remove some dead code from the table sync

Fabien COELHO sent in a patch to refactor psql's ef/ev and sf/sv handling
functions so they're not copypasta.

Feike Steenbergen sent in a patch to fix an issue where passwords in user
mappings were leaked by psql's \deu+ command.

Alexander Korotkov sent in another revision of a patch to implement a LWLock
optimization for multicore Power machines.

Takayuki Tsunakawa sent in a patch to fix an issue where savepoint-related
statements in a multi-command query terminates the connection unexpectedly.

Mike Palmiotto sent in two revisions of a patch to silence some sepgsql compiler
warnings and add partitioned table support to sepgsql.

Petr Jelínek sent in a patch to use weaker locks when updating subscription
relation state.

Dilip Kumar sent in a patch to ensure that parallel bitmapscan is exercised in
regression tests.

David Rowley sent in another revision of a patch to improve performance for
outer joins where the outer side is unique.

pgsql-announce by date

Next:From: David FetterDate: 2017-04-09 23:14:42
Subject: == PostgreSQL Weekly News - April 09 2017 ==
Previous:From: Stephen FrostDate: 2017-03-24 13:38:33
Subject: PostgresOpen SV 2017 in San Francisco - Call For Papers!

Privacy Policy | About PostgreSQL
Copyright © 1996-2017 The PostgreSQL Global Development Group