== PostgreSQL Weekly News - January 20, 2019 ==

From: David Fetter <david(at)fetter(dot)org>
To: PostgreSQL Announce <pgsql-announce(at)postgresql(dot)org>
Subject: == PostgreSQL Weekly News - January 20, 2019 ==
Date: 2019-01-20 22:38:57
Message-ID: 20190120223857.GA5507@fetter.org
Views: Raw Message | Whole Thread | Download mbox
Lists: pgsql-announce

== PostgreSQL Weekly News - January 20, 2019 ==

PostgreSQL(at)SCaLE is a two day, two track event which takes place on
March 7-8, 2019, at Pasadena Convention Center, as part of SCaLE 17X.

== PostgreSQL Product News ==

Ora2Pg 20.0, a tool for migrating Oracle databases to PostgreSQL, released.

pg_probackup 2.0.26, a utility to manage backup and recovery of PostgreSQL
database clusters, released.

== PostgreSQL Jobs for January ==


== PostgreSQL Local ==

The Turkish PostgreSQL conference will take place January 24,
2019 in Ankara.

FOSDEM PGDay 2019, a one day conference held before the main FOSDEM event will
be held in Brussels, Belgium, on Feb 1st, 2019.

Prague PostgreSQL Developer Day 2019 (P2D2 2019) is a two-day
conference that will be held on February 13-14, 2019 in Prague, Czech Republic.

PGConf India 2019 will be on February 13-15, 2019 in Bengaluru, Karnataka.

pgDay Paris 2019 will be held in Paris, France on March 12, 2019
at 199bis rue Saint-Martin.

PGConf APAC 2019 will be held in Singapore March 19-21, 2019.

The German-speaking PostgreSQL Conference 2019 will take place on May 10, 2019
in Leipzig. The CfP is open until February 26, 2019 at http://2019.pgconf.de/cfp

PGDay.IT 2019 will take place May 16th and May 17th in Bologna, Italy. Both the
CfP https://2019.pgday.it/en/blog/cfp and the Call for Workshops
https://2019.pgday.it/en/blog/cfw are open until January 15, 2019.

PGCon 2019 will take place in Ottawa on May 28-31, 2019. The CfP is open
through January 19, 2019 at http://www.pgcon.org/2019/papers.php

Swiss PGDay 2019 will take place in Rapperswil (near Zurich) on June 28, 2019.
The CfP is open January 17, 2019 through April 18, 2019, and registration will
open January 17, 2019.

== PostgreSQL in the News ==

Planet PostgreSQL: http://planet.postgresql.org/

PostgreSQL Weekly News is brought to you this week by David Fetter

Submit news and announcements by Sunday at 3:00pm PST8PDT to david(at)fetter(dot)org(dot)

== Applied Patches ==

Andrew Dunstan pushed:

- fix typo.

Heikki Linnakangas pushed:

- Detach postmaster process from pg_ctl's session at server startup. pg_ctl is
supposed to daemonize the postmaster process, so that it's not affected by
signals to the launching process group. Before this patch, if you had a shell
script that used "pg_ctl start", and you interrupted the shell script after
postmaster had been launched, postmaster was also killed. To fix, call
setsid() after forking the postmaster process. Long time ago, we had a
'silent_mode' option, which daemonized the postmaster process by calling
setsid(), but that was removed back in 2011 (commit f7ea6beaf4). We discussed
bringing that back in some form, but pg_ctl is the documented way of launching
postmaster to the background, so putting the setsid() call in pg_ctl itself
seems appropriate. Just putting postmaster in a separate session would change
the behavior when you interrupt "pg_ctl -w start", e.g. with CTRL-C, while
it's waiting for postmaster to start. The historical behavior has been that
interrupting pg_ctl aborts the server launch, which is handy if the server is
stuck in recovery, for example, and won't fully start up. To keep that
behavior, install a signal handler in pg_ctl, to explicitly kill postmaster,
if pg_ctl is interrupted while it's waiting for the server to start up. This
isn't 100% watertight, there is a small window after forking the postmaster
process, where the signal handler doesn't know the postmaster's PID yet, but
seems good enough. Arguably this is a long-standing bug, but I refrained from
back-batching, out of fear of breaking someone's scripts that depended on the
old behavior. Reviewed by Tom Lane. Report and original patch by Paul Guo,
with feedback from Michael Paquier. Discussion:

Álvaro Herrera pushed:

- Fix unique INCLUDE indexes on partitioned tables. We were considering the
INCLUDE columns as part of the key, allowing unicity-violating rows to be
inserted in different partitions. Concurrent development conflict in
eb7ed3f30634 and 8224de4f42cc. Reported-by: Justin Pryzby Discussion:

- Reorganize planner code moved in b60c39759908. It seems modules are better
defined like this instead of the original split. Per complaints from David
Rowley as well as Amit Langote's self review. Discussion:

- Refactor duplicate code into DeconstructFkConstraintRow. My commit
3de241dba86f introduced some code (in tablecmds.c) to obtain data from a
pg_constraint row for a foreign key, that already existed in ri_triggers.c.
Split it out into its own routine in pg_constraint.c, where it naturally
belongs. No functional code changes, only code movement. Backpatch to pg11,
because a future bugfix is simpler after this.

- Move CloneForeignKeyConstraints to tablecmds.c. My commit 3de241dba86f
introduced some code to create a clone of a foreign key to a partition, but I
put it in pg_constraint.c because it was too close to the contents of the
pg_constraint row. With the previous commit that split out the constraint
tuple deconstruction into its own routine, it makes more sense to have the
FK-cloning function in tablecmds.c, mostly because its static subroutine can
then be used by a future bugfix. My initial posting of this patch had this
routine as static in tablecmds.c, but sadly this function is already part of
the Postgres 11 ABI as exported from pg_constraint.c, so keep it as exported
also just to avoid breaking any possible users of it.

- Fix creation of duplicate foreign keys on partitions. When creating a foreign
key in a partitioned table, if some partitions already have equivalent
constraints, we wastefully create duplicates of the constraints instead of
attaching to the existing ones. That's inconsistent with the de-duplication
that is applied when a table is attached as a partition. To fix, reuse the
FK-cloning code instead of having a separate code path. Backpatch to Postgres
11. This is a subtle behavior change, but surely a welcome one since there's
no use in having duplicate foreign keys. Discovered by Álvaro Herrera while
thinking about a different problem reported by Jesper Pedersen (bug #15587).
Author: Álvaro Herrera Discussion:

Andres Freund pushed:

- Re-add default_with_oids GUC to avoid breaking old dump files. After
578b229718 / the removal of WITH OIDS support, older dump files containing
SET default_with_oids = false; either report unnecessary errors (as the
subsequent tables have no oids) or even fail to restore entirely (when using
transaction mode). To avoid that, re-add the GUC, but don't allow setting it
to true. Per complaint from Tom Lane. Author: Amit Khandekar, editorialized
by me Discussion:

- Don't include heapam.h from others headers. heapam.h previously was included
in a number of widely used headers (e.g. execnodes.h, indirectly in
executor.h, ...). That's problematic on its own, as heapam.h contains a lot of
low-level details that don't need to be exposed that widely, but becomes more
problematic with the upcoming introduction of pluggable table storage - it
seems inappropriate for heapam.h to be included that widely afterwards.
heapam.h was largely only included in other headers to get the HeapScanDesc
typedef (which was defined in heapam.h, even though HeapScanDescData is
defined in relscan.h). The better solution here seems to be to just use the
underlying struct (forward declared where necessary). Similar for
BulkInsertState. Another problem was that LockTupleMode was used in
executor.h - parts of the file tried to cope without heapam.h, but due to the
fact that it indirectly included it, several subsequent violations of that
goal were not not noticed. We could just reuse the approach of declaring
parameters as int, but it seems nicer to move LockTupleMode to lockoptions.h -
that's not a perfect location, but also doesn't seem bad. As a number of
files relied on implicitly included heapam.h, a significant number of files
grew an explicit include. It's quite probably that a few external projects
will need to do the same. Author: Andres Freund Reviewed-By: Alvaro Herrera

- Remove too generically named MissingPtr typedef. As there's only a single user
of the typedef in the entire codebase, just use the underlying struct
directly. Per complaint from Alvaro Herrera Author: Andres Freund
Discussion: https://postgr.es/m/201901141836.oxtm4uzc63j3@alvherre.pgsql

- Make naming of tupdesc related structs more consistent with the rest of PG. We
usually don't change the name of structs between the struct name itself and
the name of the typedef. Additionally, structs that are usually used via a
typedef that hides being a pointer, are commonly suffixed Data. Change
tupdesc code to follow those convention. This is triggered by a future patch
that intends to forward declare TupleDescData in another header - keeping with
the naming scheme makes that easier to understand. Author: Andres Freund

- Don't include genam.h from execnodes.h and relscan.h anymore. This is the
genam.h equivalent of 4c850ecec649c (which removed heapam.h from a lot of
other headers). There's still a few header includes of genam.h, but not from
central headers anymore. As a few headers are not indirectly included
anymore, execnodes.h and relscan.h need a few additional includes. Some of the
depended on types were replacable by using the underlying structs, but e.g.
for Snapshot in execnodes.h that'd have gotten more invasive than reasonable
in this commit. Like the aforementioned commit 4c850ecec649c, this requires
adding new genam.h includes to a number of backend files, which likely is also
required in a few external projects. Author: Andres Freund Discussion:

- Fix parent of WCO qual. The parent of some WCO expressions was, apparently by
accident, set to the the source of DML queries, rather than the target table.
This causes problems for the upcoming pluggable storage work, because the
target and source table might be of different storage types. It's possible
that this is already problematic, but neither experimenting nor inquiries on
-hackers have found them. So don't backpatch for now. Author: Andres Freund

- Move vacuumlazy.c into access/heap. It's heap table storage specific code that
can't realistically be generalized into table AM agnostic code. Author:
Andres Freund Discussion:

- Don't duplicate parallel seqscan shmem sizing logic in nbtree. This is
architecturally mildly problematic, which becomes more pronounced with the
upcoming introduction of pluggable storage. To fix, teach
heap_parallelscan_estimate() to deal with SnapshotAny snapshots, and then use
it from _bt_parallel_estimate_shared(). Author: Andres Freund Discussion:

Michaël Paquier pushed:

- Fix typos in documentation and for one wait event. These have been found while
cross-checking for the use of unique words in the documentation, and a wait
event was not getting generated in a way consistent to what the documentation
provided. Author: Alexander Lakhin Discussion:

- Restrict the use of temporary namespace in two-phase transactions. Attempting
to use a temporary table within a two-phase transaction is forbidden for ages.
However, there have been uncovered grounds for a couple of other object types
and commands which work on temporary objects with two-phase commit. In short,
trying to create, lock or drop an object on a temporary schema should not be
authorized within a two-phase transaction, as it would cause its state to
create dependencies with other sessions, causing all sorts of side effects
with the existing session or other sessions spawned later on trying to use the
same temporary schema name. Regression tests are added to cover all the
grounds found, the original report mentioned function creation, but monitoring
closer there are many other patterns with LOCK, DROP or CREATE EXTENSION which
are involved. One of the symptoms resulting in combining both is that the
session which used the temporary schema is not able to shut down completely,
waiting for being able to drop the temporary schema, something that it cannot
complete because of the two-phase transaction involved with temporary objects.
In this case the client is able to disconnect but the session remains alive on
the backend-side, potentially blocking connection backend slots from being
used. Other problems reported could also involve server crashes. This is
back-patched down to v10, which is where 9b013dc has introduced MyXactFlags,
something that this patch relies on. Reported-by: Alexey Bashtanov Author:
Michael Paquier Reviewed-by: Masahiko Sawada Discussion:
Backpatch-through: 10

- Enforce non-parallel plan when calling current_schema() in newly-added test.
current_schema() gets called in the recently-added regression test from
c5660e0, and can be used in a parallel context, causing its call to fail when
creating a temporary schema. Per buildfarm members crake and lapwing.
Discussion: https://postgr.es/m/20190118005949.GD1883@paquier.xyz

- Fix incorrect relation name in comment of vacuumlazy.c. Author: Masahiko
Sawada Discussion:

Tom Lane pushed:

- Finish reverting "recheck_on_update" patch. This reverts commit c203d6cf8 and
some follow-on fixes, completing the task begun in commit 5d28c9bd7. If that
feature is ever resurrected, the code will look quite a bit different from
this, so it seems best to start from a clean slate. The v11 branch is not
touched; in that branch, the recheck_on_update storage option remains present,
but nonfunctional and undocumented. Discussion:

- Avoid assuming that we know the spelling of getopt_long's error messages. I've
had enough of "fixing" this test case. Whatever value it has is limited to
verifying that pgbench fails for an unrecognized switch, and we don't need to
assume anything about what getopt_long prints in order to do that.
Discussion: https://postgr.es/m/9427.1547701450@sss.pgh.pa.us

- Avoid sometimes printing both tables and their columns in DROP CASCADE. A
cascaded drop might find independent reasons to drop both a table and some
column of the table (for instance, a schema drop might include dropping a data
type used in some table in the schema). Depending on the order of visitation
of pg_depend entries, we might report the table column and the whole table as
separate objects-to-be-dropped, or we might only report the table. This is
confusing and leads to unstable regression test output, so fix it to report
only the table regardless of visitation order. Per gripe from Peter
Geoghegan. This is just cosmetic from a user's standpoint, and we haven't
actually seen regression test problems in practice (yet), so I'll refrain from
back-patching. Discussion: https://postgr.es/m/15908.1547762076@sss.pgh.pa.us

- Use our own getopt() on OpenBSD. Recent OpenBSD (at least 5.9 and up) has a
version of getopt(3) that will not cope with the "-:" spec we use to accept
double-dash options in postgres.c and postmaster.c. Admittedly, that's a hack
because POSIX only requires getopt() to allow alphanumeric option characters.
I have no desire to find another way, however, so let's just do what we were
already doing on Solaris: force use of our own src/port/getopt.c
implementation. In passing, improve some of the comments around said
implementation. Per buildfarm and local testing. Back-patch to all supported
branches. Discussion: https://postgr.es/m/30197.1547835700@sss.pgh.pa.us

- Fix under-quoted filename pattern in pgbench TAP test. Avoids issues if build
directory's pathname contains regex metacharacters. Raúl Marín Rodríguez

Peter Eisentraut pushed:

- Increase test coverage in RI_FKey_pk_upd_check_required(). This checks the
case where the primary key has at least one null column. Reviewed-by: Alvaro
Herrera <alvherre(at)2ndquadrant(dot)com> Reviewed-by: Mi Tar <mmitar(at)gmail(dot)com>

- Add test case for ON DELETE NO ACTION/RESTRICT. This was previously not
covered at all; function RI_FKey_restrict_del() was not exercised in the
tests. Reviewed-by: Alvaro Herrera <alvherre(at)2ndquadrant(dot)com> Reviewed-by: Mi
Tar <mmitar(at)gmail(dot)com> Discussion:

- Increase test coverage in RI_Initial_Check(). This covers the special error
handling of FKCONSTR_MATCH_FULL. Reviewed-by: Alvaro Herrera
<alvherre(at)2ndquadrant(dot)com> Reviewed-by: Mi Tar <mmitar(at)gmail(dot)com> Discussion:

- Increase test coverage in RI_FKey_fk_upd_check_required(). This checks the
code path of FKCONSTR_MATCH_FULL and RI_KEYS_SOME_NULL. Reviewed-by: Alvaro
Herrera <alvherre(at)2ndquadrant(dot)com> Reviewed-by: Mi Tar <mmitar(at)gmail(dot)com>

- configure: More use of AC_ARG_VAR. AC_ARG_VAR is necessary if an environment
variable influences a configure result that is then used by other tests that
are cached. With AC_ARG_VAR, a change in the variable is detected on
subsequent configure runs and the user is then advised to remove the cache.
Reviewed-by: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> Discussion:

- Remove obsolete comment.

- Fix outdated comment. The issue the comment is referring to was fixed by

Etsuro Fujita pushed:

- postgres_fdw: Remove duplicate code in DML execution callback functions.
postgresExecForeignInsert(), postgresExecForeignUpdate(), and
postgresExecForeignDelete() are coded almost identically, except that
postgresExecForeignInsert() does not need CTID. Extract that code into a
separate function and use it in all the three function implementations.
Author: Ashutosh Bapat Reviewed-By: Michael Paquier Discussion:

Tatsuo Ishii pushed:

- Doc: enhance pgbench manual. Clarify the difference between "prepared mode"
and other query modes. Discussion:
Reviewed by: Fabien Coelh and Alvaro Herrera.

Andrew Gierth pushed:

- Postpone aggregate checks until after collation is assigned. Previously,
parseCheckAggregates was run before assign_query_collations, but this causes
problems if any expression has already had a collation assigned by some
transform function (e.g. transformCaseExpr) before parseCheckAggregates runs.
The differing collations would cause expressions not to be recognized as equal
to the ones in the GROUP BY clause, leading to spurious errors about
unaggregated column references. The result was that CASE expr WHEN val ...
would fail when "expr" contained a GROUPING() expression or matched one of the
group by expressions, and where collatable types were involved; whereas the
supposedly identical CASE WHEN expr = val ... would succeed. Backpatch all
the way; this appears to have been wrong ever since collations were
introduced. Per report from Guillaume Lelarge, analysis and patch by me.
Discussion: https://postgr.es/m/87muo0k0c7.fsf@news-spur.riddles.org.uk

Magnus Hagander pushed:

- Remove references to Majordomo. Lists are not handled by Majordomo anymore and
haven't been for a while, so remove the reference and instead direct people to
the list server.

- Replace references to mailinglists with @lists.postgresql.org. The namespace
for all lists have changed a while ago, so all references should use the
correct address.

- Replace @postgresql.org with @lists.postgresql.org for mailinglists. Commit
c0d0e54084 replaced the ones in the documentation, but missed out on the ones
in the code. Replace those as well, but unlike c0d0e54084, don't backpatch the
code changes to avoid breaking translations.

Tomáš Vondra pushed:

- Revert "Add valgrind suppressions for wcsrtombs optimizations". This reverts
commit d3bbc4b96a5b4d055cf636596c6865913a099929. Per discussion, it's not
desirable to add valgrind suppressions for outside our own code base (e.g.
glibc in this case), especially when the suppressions may be
platform-specific. There are better ways to deal with that, e.g. by providing
local suppressions. Discussion:

- Allow COPY FROM to filter data using WHERE conditions. Extends the COPY FROM
command with a WHERE condition, which allows doing various types of filtering
while importing the data (random sampling, condition on a data column, etc.).
Until now such filtering required either preprocessing of the input data, or
importing all data and then filtering in the database. COPY FROM ... WHERE is
an easy-to-use and low-overhead alternative for most simple cases. Author:
Surafel Temesgen Reviewed-by: Tomas Vondra, Masahiko Sawada, Lim Myungkyu

== Pending Patches ==

Tomáš Vondra sent in another revision of a patch to fix an infelicity between
logical decoding and long-running transactions.

Tom Lane sent in another revision of a patch to get rid of empty jointrees.

Andres Freund and Álvaro Herrera traded patches to reduce header
interdependencies around heapam.h et al.

Tomáš Vondra sent in two more revisions of a patch to implement explain plans
with information about (modified) GUCs.

James Coleman sent in four more revisions of a patch to prove IS NOT NULL
inference for ScalarArrayOpExpr's.

Tomáš Vondra sent in two more revisions of a patch to implement multivariate
histograms and MCV lists.

Tomáš Vondra sent in another revision of a patch to fix a bug in xmltable xpath
result processing.

Masahiko Sawada sent in another revision of a patch to implement a copy function
for logical and physical replication slots.

Antonin Houska sent in another revision of a patch to implement aggregate

Shay Rojansky sent in two more revisions of a patch to allow UNLISTEN during

Peter Eisentraut sent in another revision of a patch to implement collations
with nondeterministic comparison per

Amit Langote sent in two more revisions of a patch to allow generalized
expression syntax for partition bounds.

Masahiko Sawada sent in two more revisions of a patch to add a parallel option
both to VACUUM and vacuumdb.

Adrien Nayrat sent in two more revisions of a patch to log a sample of

Andres Freund sent in another revision of a patch to introduce access/table.h
access/relation.h, replace heapam.h includes with relation.h / table.h where
applicable, and replace uses of heap_open et al with table_open et al.

Haribabu Kommi sent in two revisions of a patch to make the default value of
log_file_mode 0640 to allow reading of log files by the members of the same

Dmitry Dolgov sent in three more revisions of a patch to add access methods to
pg_dump and psql.

Surafel Temesgen sent in another revision of a patch to add the WITH TIES option

Alexander Kuzmenkov sent in a patch to recognize that an expanded bool index
clause is equivalent to the original one.

Peter Eisentraut sent in a patch to start syslogger earlier in the postmaster

Tomáš Vondra and Artur Zakirov traded patches to store ispell dictionaries in a
shared location.

Andrew Gierth sent in four revisions of a patch to replace strtod(), which has a
double-rounding issue, with strtof() for float4 output, at least on platforms
where strtof() is available.

Amit Langote sent in two more revisions of a patch to speed up planning with

Peter Eisentraut sent in a patch to configure to use AC_ARG_VAR for MSGFMT,

Amit Kapila and John Naylor traded patches to Avoid creation of the free space
map for small heap relations and skip transfer of same in pg_upgrade.

Chris Travers sent in another revision of a patch to add some source comments to
elucidate the way signals are detected.

Surafel Temesgen sent in two more revisions of a patch to add a multiple-values
INSERT option to pg_dump.

Tom Lane sent in a patch to add a new DEPFLAG_IS_SUBOBJECT flag to the column
object's flags, denoting that we know the whole table will be dropped later.
The only effect of this flag is to suppress reporting of the column object in

Michaël Paquier sent in another revision of a patch to implement REINDEX

Michaël Paquier sent in a patch to simplify 2PC restriction handling for
temporary objects.

Pavel Stěhule sent in another revision of a patch to add a unique statement
identifier to PL/pgsql.

David Rowley sent in another revision of a patch to forgo generating
single-subpath Append and MergeAppend paths.

Mitar sent in another revision of a patch to implement temporary materialized

Raúl Marín Rodríguez sent in a patch to fix an issue where pgbench tap tests
fail if the path contains a Perl special character.

Kyotaro HORIGUCHI sent in another revision of a patch to remove catcache entries
that haven't been used for a certain time, collect syscache usage statictics and
show them using a view: pg_stat_syscache, enable setting GUCs
non-transactionally, and enable setting GUCs remotely.

Álvaro Herrera sent in a patch to add a missing relcache reset and fix action
triggers for FK constraints on detached partitions.

Amit Langote sent in another revision of a patch to ensure PK-side action
triggers for partitions after being detached and cease tracking foreign key
inheritance by dependencies.

Etsuro Fujita sent in a patch to postpone generating tlists and EC members for
inheritance dummy children.

Andrew Gierth sent in two more revisions of a patch to use the Ryu system to
round-trips in floating point work.

Nishant Fnu sent in a patch to change heap page lock acquisition from bufferId
order to block number order.

Daniel Vérité sent in a patch to clarify the \copy documentation for psql's new
suffix and make same more efficient.

Michaël Paquier sent in a patch to mark current_schema[s]() parallel unsafe.

Tom Lane sent in a patch to force a consistent deletion order.

Tom Lane sent patches implementing two approaches improve the logic around

Alexander Korotkov sent in another revision of a patch to implement JSONPATH.

Vik Fearing sent in another revision of a patch to jumble rowmarks in

Nikolay Shaplov sent in another revision of a patch to replace StdRdOptions with
individual binary reloptions representation for each relation kind.

Tom Lane sent in a patch to fix some thread-unsafe code in ECPG.

Fabien COELHO sent in a patch to define an internal interface for a PRNG, and
use it within pgbench, with several possible implementations which default to

Kim Rose Carlsen sent in a patch to remove useless joins when using partial
unique index.

Browse pgsql-announce by date

  From Date Subject
Next Message Joe Conway 2019-01-21 18:19:52 PostgreSQL@SCaLE17x - schedule is posted
Previous Message Grigory Smolkin 2019-01-19 13:40:21 [ANNOUNCE] pg_probackup 2.0.26 released