== PostgreSQL Weekly News - February 18 2018 ==

From: David Fetter <david(at)fetter(dot)org>
To: PostgreSQL Announce <pgsql-announce(at)postgresql(dot)org>
Subject: == PostgreSQL Weekly News - February 18 2018 ==
Date: 2018-02-20 21:24:16
Message-ID: 20180220212416.GA29131@fetter.org
Views: Raw Message | Whole Thread | Download mbox
Lists: pgsql-announce

== PostgreSQL Weekly News - February 18 2018 ==

== PostgreSQL Product News ==

Pgpool-II 3.7.2, 4.6.9, 3.5.13, 3.4.16 and 3.3.20 released.

OmniDB 2.5, a browser-based database management tool, released.

DBD::Pg 3.7.4, a Perl driver for PostgreSQL, released.

== PostgreSQL Jobs for February ==


== PostgreSQL Local ==

PGConf India 2018 will be on February 22-23, 2018 in Bengaluru, Karnataka.

PostgreSQL(at)SCaLE is a two day, two track event which takes place on
March 8-9, 2018, at Pasadena Convention Center, as part of SCaLE 16X.

Nordic PGDay 2018 will be held in Oslo, Norway, at the Radisson Blu Hotel
Nydalen, on March 13, 2018. Registration is open and the schedule is posted.

pgDay Paris 2018 will be held in Paris, France at the Espace Saint-Martin, on
March 15 2018. Registration is open.

PGConf APAC 2018 will be held in Singapore March 22-23, 2018.

The German-speaking PostgreSQL Conference 2018 will take place on April 13th,
2018 in Berlin.

PGConfNepal 2018 will be held May 4-5, 2018 at Kathmandu University, Dhulikhel,

PGCon 2018 will take place in Ottawa on May 29 - June 1, 2018.

Swiss PGDay 2018 will take place in Rapperswil (near Zurich) on June 29, 2018.
The CfP is open February 6, 2018 through April 14, 2018, and registration is
open February 6, 2018 through June 28, 2018.

PGConf.Brazil 2018 will take place in São Paulo, Brazil on August 3-4 2018. The
CfP is open until February 28, 2018.

== PostgreSQL in the News ==

Planet PostgreSQL: http://planet.postgresql.org/

PostgreSQL Weekly News is brought to you this week by David Fetter

Submit news and announcements by Sunday at 3:00pm EST5EDT. Please send English
language ones to david(at)fetter(dot)org, German language to pwn(at)pgug(dot)de, Italian
language to pwn(at)itpug(dot)org(dot)

== Applied Patches ==

Bruce Momjian pushed:

- psql: give ^D hint for \q in place where \q is ignored. Also add comment on
why exit/quit are not documented. Discussion:

Álvaro Herrera pushed:

- Add missing article. Noticed while reviewing nearby text

- get_relid_attribute_name is dead, long live get_attname. The modern way is to
use a missing_ok argument instead of two separate almost-identical routines,
so do that. Author: Michaël Paquier Reviewed-by: Álvaro Herrera Discussion:

- Mention trigger name in trigger test. This makes it more explicit exactly
what is going on, for further proposed behavior changes. Discussion:

- Refactor format_type APIs to be more modular. Introduce a new
format_type_extended, with a flags bitmask argument that can modify the
default behavior. A few compatibility and readability wrappers remain:
format_type_be format_type_be_qualified format_type_with_typemod while
format_type_with_typemod_qualified, which had a single caller, is removed.
Author: Michael Paquier, some revisions by me Discussion:

Robert Haas pushed:

- Fix parallel index builds for dynamic_shared_memory_type=none. The previous
code failed to realize that this setting effectively disables parallelism, and
would crash if it decided to attempt parallelism anyway. Instead, treat it as
a disabling condition. Kyotaro Horiguchi, who also reported the issue.
Reviewed by Michael Paquier and Peter Geoghegan. Discussion:

Peter Eisentraut pushed:

- Fix typo. Author: Masahiko Sawada <sawada(dot)mshk(at)gmail(dot)com>

- In LDAP test, restart after pg_hba.conf changes. Instead of issuing a reload
after pg_hba.conf changes between test cases, run a full restart. With a
reload, an error in the new pg_hba.conf is ignored and the tests will continue
to run with the old settings, invalidating the subsequent test cases. With a
restart, a faulty pg_hba.conf will lead to the test being aborted, which is
what we'd rather want.

- Fix typo.

- doc: pg_function_is_visible also applies to aggregates and procedures.

- Add tests for pg_get_functiondef.

- Add procedure support to pg_get_functiondef. This also makes procedures work
in psql's \ef and \sf commands. Reported-by: Pavel Stehule

- Rename enable_partition_wise_join to enable_partitionwise_join. Discussion:

- Fix crash when canceling parallel query. elog(FATAL) would end up calling
PortalCleanup(), which would call executor shutdown code, which could fail and
crash, especially under parallel query. This was introduced by
8561e4840c81f7e345be2df170839846814fa004, which did not want to mark an active
portal as failed by a normal transaction abort anymore. But we do need to do
that for an elog(FATAL) exit. Introduce a variable shmem_exit_inprogress
similar to the existing proc_exit_inprogress, so we can tell whether we are in
the FATAL exit scenario. Reported-by: Andres Freund <andres(at)anarazel(dot)de>

- Move function comment to the right place.

- Minor comment fix.

Tom Lane pushed:

- Make plpgsql use its DTYPE_REC code paths for composite-type variables.
Formerly, DTYPE_REC was used only for variables declared as "record";
variables of named composite types used DTYPE_ROW, which is faster for some
purposes but much less flexible. In particular, the ROW code paths are
entirely incapable of dealing with DDL-caused changes to the number or data
types of the columns of a row variable, once a particular plpgsql function has
been parsed for the first time in a session. And, since the stored
representation of a ROW isn't a tuple, there wasn't any easy way to deal with
variables of domain-over-composite types, since the domain constraint checking
code would expect the value to be checked to be a tuple. A lesser, but still
real, annoyance is that ROW format cannot represent a true NULL composite
value, only a row of per-field NULL values, which is not exactly the same
thing. Hence, switch to using DTYPE_REC for all composite-typed variables,
whether "record", named composite type, or domain over named composite type.
DTYPE_ROW remains but is used only for its native purpose, to represent a
fixed-at-compile-time list of variables, for instance the targets of an INTO
clause. To accomplish this without taking significant performance losses,
introduce infrastructure that allows storing composite-type variables as
"expanded objects", similar to the "expanded array" infrastructure introduced
in commit 1dc5ebc90. A composite variable's value is thereby kept (most of
the time) in the form of separate Datums, so that field accesses and updates
are not much more expensive than they were in the ROW format. This holds the
line, more or less, on performance of variables of named composite types in
field-access-intensive microbenchmarks, and makes variables declared "record"
perform much better than before in similar tests. In addition, the logic
involved with enforcing composite-domain constraints against updates of
individual fields is in the expanded record infrastructure not plpgsql proper,
so that it might be reusable for other purposes. In further support of this,
introduce a typcache feature for assigning a unique-within-process identifier
to each distinct tuple descriptor of interest; in particular, DDL alterations
on composite types result in a new identifier for that type. This allows very
cheap detection of the need to refresh tupdesc-dependent data. This improves
on the "tupDescSeqNo" idea I had in commit 687f096ea: that assigned
identifying sequence numbers to successive versions of individual composite
types, but the numbers were not unique across different types, nor was there
support for assigning numbers to registered record types. In passing, allow
plpgsql functions to accept as well as return type "record". There was no
good reason for the old restriction, and it was out of step with most of the
other PLs. Tom Lane, reviewed by Pavel Stehule Discussion:

- Speed up plpgsql function startup by doing fewer pallocs. Previously,
copy_plpgsql_datum did a separate palloc for each variable needing
instance-local storage. In simple benchmarks this made for a noticeable
fraction of the total runtime. Improve it by precalculating the space needed
for all of a function's variables and doing just one palloc for all of them.
In passing, remove PLPGSQL_DTYPE_EXPR from the list of plpgsql "datum" types,
since in fact it has nothing in common with the others, and there is noplace
that needs to discriminate on the basis of dtype between an expression and any
type of datum. And add comments clarifying which datum struct fields are
generic and which aren't. Tom Lane, reviewed by Pavel Stehule Discussion:

- Speed up plpgsql trigger startup by introducing "promises". Over the years
we've accreted quite a few special variables that are predefined in plpgsql
trigger functions. The cost of initializing these variables to their defined
values turns out to be a significant part of the runtime of simple triggers;
but, undoubtedly, most real-world triggers never examine the values of most of
these variables. To improve matters, invent the notion of a variable that has
a "promise" attached to it, specifying which of the predetermined values
should be assigned to the variable if anything ever reads it. This eliminates
all the unneeded startup overhead, in return for a small penalty on accesses
to these variables. Tom Lane, reviewed by Pavel Stehule Discussion:

- Support CONSTANT/NOT NULL/initial value for plpgsql composite variables.
These features were never implemented previously for composite or record
variables ... not that the documentation admitted it, so there's no doc
updates here. This also fixes some issues concerning enforcing DOMAIN NOT
NULL constraints against plpgsql variables, although I'm not sure that that
topic is completely dealt with. I created a new plpgsql test file for these
features, and moved the one relevant existing test case into that file. Tom
Lane, reviewed by Daniel Gustafsson Discussion:

- Fix broken logic for reporting PL/Python function names in errcontext.
plpython_error_callback() reported the name of the function associated with
the topmost PL/Python execution context. This was not merely wrong if there
were nested PL/Python contexts, but it risked a core dump if the topmost one
is an inline code block rather than a named function. That will have proname
= NULL, and so we were passing a NULL pointer to snprintf("%s"). It seems
that none of the PL/Python-testing machines in the buildfarm will dump core
for that, but some platforms do, as reported by Marina Polyakova.
Investigation finds that there actually is an existing regression test that
used to prove that the behavior was wrong, though apparently no one had
noticed that it was printing the wrong function name. It stopped showing the
problem in 9.6 when we adjusted psql to not print CONTEXT by default for
NOTICE messages. The problem is masked (if your platform avoids the core
dump) in error cases, because PL/Python will throw away the originally
generated error info in favor of a new traceback produced at the outer level.
Repair by using ErrorContextCallback.arg to pass the correct context to the
error callback. Add a regression test illustrating correct behavior.
Back-patch to all supported branches, since they're all broken this way.

- Add an assertion that we don't pass NULL to snprintf("%s"). Per commit
e748e902d, we appear to have little or no coverage in the buildfarm of
machines that will dump core when asked to printf a null string pointer.
Let's try to improve that situation by adding an assertion that will make
src/port/snprintf.c behave that way. Since it's just an assertion, it won't
break anything in production builds, but it will help developers find this
type of oversight. Note that while our buildfarm coverage of machines that
use that snprintf implementation is pretty thin on the Unix side (apparently
amounting only to gaur/pademelon), all of the MSVC critters use it.

- Silence assorted "variable may be used uninitialized" warnings. All of these
are false positives, but in each case a fair amount of analysis is needed to
see that, and it's not too surprising that not all compilers are smart enough.
(In particular, in the logtape.c case, a compiler lacking the knowledge
provided by the Assert would almost surely complain, so that this warning will
be seen in any non-assert build.) Some of these are of long standing while
others are pretty recent, but it only seems worth fixing them in HEAD. Jaime
Casanova, tweaked a bit by me Discussion:

- Stabilize new plpgsql_record regression tests. The buildfarm's
CLOBBER_CACHE_ALWAYS animals aren't happy with some of the test cases added in
commit 4b93f5799. There are two different problems: * In two places, a
different CONTEXT stack is shown because the error is detected in a different
place, due to recompiling an expression from scratch rather than re-using a
previously cached plan for it. I fixed these via the expedient of hiding the
CONTEXT stack altogether. * In one place, a test expected to fail (because a
cached plan hadn't been updated) actually succeeds (because the forced
recompile makes it good). I couldn't think of a simple workaround for this,
so I've just commented out that test step altogether. I have hopes of
improving things enough that both of these kluges can be reverted eventually.
The first one is the same kind of problem previously discussed at
https://postgr.es/m/31545.1512924904@sss.pgh.pa.us but there was insufficient
agreement about how to fix it, so we just hacked around the output instability
(commit 9edc97b71). The second issue should be fixed by allowing the plan to
be rebuilt when a type conflict is detected. But for today, let's just make
the buildfarm green again.

- Revert "Stabilize output of new regression test case". This effectively
reverts commit 9edc97b71 (although the test is now in a different place and
has different contents). We don't need that hack anymore, because since
commit 4b93f5799, this test case no longer throws an error and so there's no
displayed CONTEXT that could vary depending on CLOBBER_CACHE_ALWAYS. The
underlying unstable-output problem isn't really gone, of course, but it no
longer manifests here.

- Move the extern declaration for ExceptionalCondition into c.h. This is the
logical conclusion of our decision to support Assert() in both frontend and
backend code: it should be possible to use that after including just c.h. But
as things were arranged before, if you wanted to use Assert() in code that
might be compiled for either environment, you had to include postgres.h for
the backend case. Let's simplify that. Per buildfarm, some of whose members
started throwing warnings after commit 0c62356cc added an Assert in
src/port/snprintf.c. It's possible that some other src/port files that use
the stanza #ifndef FRONTEND #include "postgres.h" #else #include
"postgres_fe.h" #endif could now be simplified to just say '#include "c.h"'.
I have not tested for that, though, and it'd be unlikely to apply for more
than a small number of them.

- Cast to void in StaticAssertExpr, not its callers. Seems a bit silly that
many (in fact all, as of today) uses of StaticAssertExpr would need to cast it
to void to avoid warnings from pickier compilers. Let's just do the cast
right in the macro, instead. In passing, change StaticAssertExpr to
StaticAssertStmt in one place where that seems more apropos. Discussion:

- Doc: fix minor bug in CREATE TABLE example. One example in create_table.sgml
claimed to be showing table constraint syntax, but it was really column
constraint syntax due to the omission of a comma. This is both wrong and
confusing, so fix it in all supported branches. Per report from
neil(at)postgrescompare(dot)com(dot) Discussion:

- Fix plpgsql to enforce domain checks when returning a NULL domain value. If a
plpgsql function is declared to return a domain type, and the domain's
constraints forbid a null value, it was nonetheless possible to return NULL,
because we didn't bother to check the constraints for a null result. I'd
noticed this while fooling with domains-over-composite, but had not gotten
around to fixing it immediately. Add a regression test script exercising this
and various other domain cases, largely borrowed from the plpython_types test.
Although this is clearly a bug fix, I'm not sure whether anyone would thank us
for changing the behavior in stable branches, so I'm inclined not to

- Remove some inappropriate #includes. Other header files should never #include
postgres.h (nor postgres_fe.h, nor c.h), per project policy. Also, there's no
need for any backend .c file to explicitly include elog.h or palloc.h, because
postgres.h pulls those in already. Extracted from a larger patch by Kyotaro
Horiguchi. The rest of the removals he suggests require more study, but these
are no-brainers. Discussion:

Andres Freund pushed:

- Return implementation defined value if pg_$op_s$bit_overflow overflows. Some
older compilers otherwise sometimes complain about undefined values, even
though the return value should not be used in the overflow case. We assume
that any decent compiler will optimize away the unnecessary assignment in
performance critical cases. We do not want to restrain the returned value to
a specific value, e.g. 0 or the wrapped-around value, because some fast ways
to implement overflow detecting math do not easily allow for that (e.g. msvc
intrinsics). As the function documentation already documents the returned
value in case of intrinsics to be implementation defined, no documentation has
to be updated. Per complaint from Tom Lane and his buildfarm member
prairiedog. Author: Andres Freund Discussion:

- Revert "Do execGrouping.c via expression eval machinery.". This reverts
commit 773aec7aa98abd38d6d9435913bb8e14e392c274. There's an unresolved issue
in the reverted commit: It only creates one comparator function, but in for
the nodeSubplan.c case we need more (c.f. FindTupleHashEntry vs
LookupTupleHashEntry calls in nodeSubplan.c). This isn't too difficult to
fix, but it's not entirely trivial either. The fact that the issue only causes
breakage on 32bit systems shows that the current test coverage isn't that
great. To avoid turning half the buildfarm red till those two issues are
addressed, revert.

- Do execGrouping.c via expression eval machinery. This has a performance
benefit on own, although not hugely so. The primary benefit is that it will
allow for to JIT tuple deforming and comparator invocations. Author: Andres
Freund Discussion:

- Do execGrouping.c via expression eval machinery, take two. This has a
performance benefit on own, although not hugely so. The primary benefit is
that it will allow for to JIT tuple deforming and comparator invocations.
Large parts of this were previously committed (773aec7aa), but the commit
contained an omission around cross-type comparisons and was thus reverted.
Author: Andres Freund Discussion:

- Allow tupleslots to have a fixed tupledesc, use in executor nodes. The reason
for doing so is that it will allow expression evaluation to optimize based on
the underlying tupledesc. In particular it will allow to JIT tuple deforming
together with the expression itself. For that expression initialization needs
to be moved after the relevant slots are initialized - mostly unproblematic,
except in the case of nodeWorktablescan.c. After doing so there's no need for
ExecAssignResultType() and ExecAssignResultTypeFromTL() anymore, as all former
callers have been converted to create a slot with a fixed descriptor. When
creating a slot with a fixed descriptor, tts_values/isnull can be allocated
together with the main slot, reducing allocation overhead and increasing cache
density a bit. Author: Andres Freund Discussion:

Magnus Hagander pushed:

- Fix typo in comment.

== Pending Patches ==

Thomas Munro sent in a patch to minor clean-up in dshash.{c,h}.

Thomas Munro sent in a WIP patch to use explicit memory barriers when
manipulating MyProc->subxids and nxids.

Artur Zakirov sent in another revision of a patch to implement a more flexible
configuration for full-text search.

Thomas Munro sent in a patch to remove volatile qualifiers from shm_mq.c.

Álvaro Herrera sent in another revision of a patch to allow unique indexes on
partitioned tables. Peter Eisentraut sent a follow-on patch to clarify the

Rajkumar Raghuwanshi sent in a patch to fix an error which manifested as
expression errors with "FOR UPDATE" and postgres_fdw with partition wise join

Artur Zakirov sent in two more revisions of a patch to fix a bug in

Amit Langote sent in a patch to update the docs to reflect the new capability to
index partitioned tables.

Amit Kapila sent in a patch to fix a failure condition in tuple version locking.

Etsuro Fujita sent in a patch to fix a regression test failure in the PostgreSQL

Michail Nikolaev sent in another revision of a patch to get index scan offsets
faster by using the visibility map.

Amit Kapila sent in another revision of a patch to restrict concurrent
update/delete with UPDATE of partition key.

Rafia Sabih sent in another revision of a patch to implement partition-wise

Marina Polyakova sent in a patch to fix a PL/PythonU breakage on Solaris.

Peter Eisentraut sent in a patch to support Python 3.7 when it arrives.

Anthony Bykov sent in two more revisions of a patch to make TRANSFORMs for
JSONB in PL/Perl.

Amit Langote and Etsuro Fujita traded patches to initialize per-partition
objects during tuple routing.

Michaël Paquier sent in a patch to fix some cache lookup errors with functions
manipulation object addresses.

Amit Langote sent in three more revisions of a patch to speed up partition

Andrew Dunstan sent in two more revisions of a patch to speed up ALTER TABLE ...

Nikita Glukhov sent in another revision of a patch to implement SQL/JSON

Nikita Glukhov sent in another revision of a patch to implement SQL/JSON's

Kyotaro HORIGUCHI sent in a patch to fix the fact that the comment on formdesc
in relcache.c's LookupOpclassInfo() omits pg_subscription.

Nikita Glukhov sent in another revision of a patch to implement SQL/JSON's

Peter Eisentraut sent in a patch to implement a Kerberos test suite.

Nikolay Shaplov sent in another revision of a patch to add an enum reloption

Pierre Ducroquet sent in another revision of a patch atop the existing JIT patch
to enable JIT compiling with LLVM v10.1.

Takayuki Tsunakawa sent in a patch to fix an issue where a cascaded standby
fails to start after a clean shutdown by zero-filling WAL blocks on standbys.

Amit Langote sent in four more revisions of a patch to reorganize the
partitioning code.

Álvaro Herrera sent in another revision of a patch to implement FOR EACH ROW
triggers on partitioned tables.

Amit Kapila sent in another revision of a patch to ensure that parallel paths
include tlist cost.

Kyotaro HORIGUCHI sent in a patch to ensure that extra shared memory isn't
requested when HAVE_SPINLOCKS is enabled.

Ashutosh Bapat sent in a patch to change baserel to foreignrel in some
postgresql_fdw functions.

Claudio Freire sent in another revision of a patch to vacuum to update the FSM
more frequently.

Kyotaro HORIGUCHI sent in a patch to ensure that spin.c includes pg_sema.h only
when necessary.

Kyotaro HORIGUCHI sent in a patch to remove DSM_INPL_NONE.

Konstantin Knizhnik sent in a patch to change the transaction lock behavior from
long chains to pairs, which eliminates a quadratic behavior.

Pavan Deolasee sent in another revision of a patch to implement MERGE.

Ashutosh Bapat sent in another revision of a patch to improve the partition
matching algorithm for partition-wise joins.

Takayuki Tsunakawa sent in a patch to produce a crash dump before main() on

Amul Sul sent in a patch to fix a server crash in the
pg_replication_slot_advance function.

Julian Markwort sent in a patch to add a new auth option to pg_hba.conf:

Anna Akenteva sent in a patch to fix an issue where larger strings can be loaded
into a table than selected from it.

Tom Lane and Marina Polyakova traded patches to stabilize some regression test

Joe Conway sent in two revisions of a patch to ensure that all catalog tables
that need them have appropriate TOAST tables.

Arseny Sher sent in a patch to start decoding from a restart_lsn which
definitely exists.

Oleg Ivanov sent in another revision of a patch to implement generic WAL

David Rowley sent in another revision of a patch to improve runtime partition

Michaël Paquier sent in another revision of a patch to prevent SSL and LDAP
tests from running without support in build and add PROVE_EXTRA_ALLOWED to
control optional test suites.

Michaël Paquier sent in a patch to documenting PROVE_TESTS in section of TAP

Fabien COELHO sent in two revisions of a patch to pgbench to enable specifying
scale as a size.

David Rowley sent in two revisions of a patch to make PartitionClauseInfo a
nonnode type.

Browse pgsql-announce by date

  From Date Subject
Next Message Danilo Endesfelder 2018-02-21 15:13:40 pgconf.de 2018 - Open for Registration and Schedule online
Previous Message Monica Real Amores 2018-02-15 17:41:51 repmgr 4.0.3 Now Available