== PostgreSQL Weekly News - November 05 2017 ==

From: David Fetter <david(at)fetter(dot)org>
To: PostgreSQL Announce <pgsql-announce(at)postgresql(dot)org>
Subject: == PostgreSQL Weekly News - November 05 2017 ==
Date: 2017-11-05 20:32:48
Message-ID: 20171105203248.GA14597@fetter.org
Views: Raw Message | Whole Thread | Download mbox
Lists: pgsql-announce

== PostgreSQL Weekly News - November 05 2017 ==

PGCon 2018 will take place in Ottawa on May 29 - June 2018. The CfP goes out

Prague PostgreSQL Developer Day 2018 (P2D2 2018) is a two-day
conference that will be held on February 14-15 2018 in Prague, Czech Republic.
The CfP is open until January 5, 2018 at https://p2d2.cz/callforpapers

== PostgreSQL Product News ==

PGroonga 2.0.2 a full text search platform for all languages, released.

Pgpool-II 3.6.7, 3.5.11, 3.4.14, and 3.3.18 released.

OmniDB 2.3, a browser-based database management tool, released.

check_postgres 2.23.0, a PostgreSQL plugin for the
Nagios/Icinga monitoring systems, released.

Database Designer for PostgreSQL 1.12 released.

== PostgreSQL Jobs for November ==


== PostgreSQL Local ==

2ndQuadrant PostgreSQL Conference 2017 (2Q PGConf, for short) will be hosted on
November 6th & 7th in New York City, and November 9th in Chicago.

PGConf Local: Seattle will be held November 13 - 14, 2017.

PGDay Australia 2017 will be held on November 17 in Melbourne.

PostgreSQL Session will take place November 17th, 2017, in Paris, France.

PGDAY Brasilia 2017 will take place December 2, 2017. The CfP
is open at

PGConf Local: Austin will be held December 4 - 5, 2017. Call for Papers is
now open at https://www.pgconf.us/conferences/Austin2017

PGConf.ASIA 2017 will take place on December 4-6 2017 in Akihabara, Tokyo,

FOSDEM PGDay 2018, a one day conference held before the main FOSDEM event will
be held in Brussels, Belgium, on Feb 2nd, 2018.

PGConf India 2018 will be on February 22-23, 2018 in Bengaluru, Karnataka.
Proposals are due via https://goo.gl/forms/F9hRjOIsaNasVOAz2 by October 31st, 2017.

PostgreSQL(at)SCaLE is a two day, two track event which takes place on
March 8-9, 2018, at Pasadena Convention Center, as part of SCaLE 16X.
The CfP is open through October 31, 2017 at

PGConf APAC 2018 will be held in Singapore March 22-24, 2018. The CfP is open
at http://2018.pgconfapac.org/cfp through December 4, 2017.

The German-speaking PostgreSQL Conference 2018 will take place on April 13th,
2018 in Berlin. The CfP is open until January 09, 2018 at
http://2018.pgconf.de/de/callforpapers.html and the conference site is at

== PostgreSQL in the News ==

Planet PostgreSQL: http://planet.postgresql.org/

PostgreSQL Weekly News is brought to you this week by David Fetter

Submit news and announcements by Sunday at 3:00pm EST5EDT. Please send English
language ones to david(at)fetter(dot)org, German language to pwn(at)pgug(dot)de, Italian
language to pwn(at)itpug(dot)org(dot)

== Applied Patches ==

Magnus Hagander pushed:

- Fix typo.

- Fix typo in comment. Etsuro Fujita

- Fix comment. Author: Bernd Helmle <mailings(at)oopsware(dot)de>

Álvaro Herrera pushed:

- Fix autovacuum work item error handling. In autovacuum's "work item"
processing, a few strings were allocated in the current transaction's memory
context, which goes away during error handling; if an error happened during
execution of the work item, the pfree() calls to clean up afterwards would try
to release already-released memory, possibly leading to a crash. In branch
master, this was already fixed by commit 335f3d04e4c8, so backpatch that to
REL_10_STABLE to fix the problem there too. As a secondary problem, verify
that the autovacuum worker is connected to the right database for each work
item; otherwise some items would be discarded by workers in other databases.
Reported-by: Justin Pryzby Discussion:

- Revert bogus fixes of HOT-freezing bug. It turns out we misdiagnosed what the
real problem was. Revert the previous changes, because they may have worse
consequences going forward. A better fix is forthcoming. The simplistic test
case is kept, though disabled. Discussion:

- Fix BRIN summarization concurrent with extension. If a process is extending a
table concurrently with some BRIN summarization process, it is possible for
the latter to miss pages added by the former because the number of pages is
computed ahead of time. Fix by determining a fresh relation size after
inserting the placeholder tuple: any process that further extends the table
concurrently will update the placeholder tuple, while previous pages will be
processed by the heap scan. Reported-by: Tomas Vondra Reviewed-by: Tom Lane
Author: Álvaro Herrera Discussion:
Backpatch-to: 9.5

- Fix thinkos in BRIN summarization. The previous commit contained a thinko
that made a single-range summarization request process from there to end of
table. Fix by setting the correct end range point. Per buildfarm.

Tom Lane pushed:

- Doc: call out UPDATE syntax change as a v10 compatibility issue. The change
made by commit 906bfcad7 means that if you're writing a parenthesized column
list in UPDATE ... SET, but that column list is only one column, you now need
to write ROW(expression) on the righthand side, not just a parenthesized
expression. This was an intentional change for spec compatibility and
potential future expansion of the possibilities for the RHS, but I'd neglected
to document it as a compatibility issue, figuring that hardly anyone would
bother with parenthesized syntax for a single target column. I was wrong, as
shown by questions from Justin Pryzby, Adam Brusselback, and others. Move the
release note item into the compatibility section and point out the behavior
change for a single target column. Discussion:

- Fix underqualified cast-target type names in pg_dump and psql queries.
Queries running with some non-pg_catalog schema frontmost in their search path
need to be careful to schema-qualify type names that should be sought in
pg_catalog. Vitaly Burovoy reported an oversight of this sort in pg_dump's
dumpSequence, and grepping detected another one in psql's
describeOneTableDetails, both introduced by sequence-related changes in v10.
In pg_dump, we can fix things by removing the cast altogether, since it
doesn't really matter what data types are reported for these query result
columns. Likewise in psql, the query seemed to be working unduly hard to get
a result that's guaranteed to be exactly 'bigint'. I also changed a couple of
occurrences of "::char" similarly. These are not bugs, since "char" is a
typename keyword and not subject to search_path rules, but it seems better to
use uniform style. Vitaly Burovoy and Tom Lane Discussion:

- Fix ALTER TABLE code to update domain constraints when needed. It's possible
for dropping a column, or altering its type, to require changes in domain
CHECK constraint expressions; but the code was previously only expecting to
find dependent table CHECK constraints. Make the necessary adjustments. This
is a fairly old oversight, but it's a lot easier to encounter the problem in
the context of domains over composite types than it was before. Given the
lack of field complaints, I'm not going to bother with a back-patch, though
I'd be willing to reconsider that decision if someone does complain. Patch by
me, reviewed by Michael Paquier Discussion:

- Allow bitmap scans to operate as index-only scans when possible. If we don't
have to return any columns from heap tuples, and there's no need to recheck
qual conditions, and the heap page is all-visible, then we can skip fetching
the heap page altogether. Skip prefetching pages too, when possible, on the
assumption that the recheck flag will remain the same from one page to the
next. While that assumption is hardly bulletproof, it seems like a good bet
most of the time, and better than prefetching pages we don't need. This
commit installs the executor infrastructure, but doesn't change any planner
cost estimates, thus possibly causing bitmap scans to not be chosen in cases
where this change renders them the best choice. I (tgl) am not entirely
convinced that we need to account for this behavior in the planner, because I
think typically the bitmap scan would get chosen anyway if it's the best bet.
In any case the submitted patch took way too many shortcuts, resulting in too
many clearly-bad choices, to be committable. Alexander Kuzmenkov, reviewed by
Alexey Chernyshov, and whacked around rather heavily by me. Discussion:

- Doc: update URL for check_postgres. Reported by Dan Vianello. Discussion:

- Teach planner to account for HAVING quals in aggregation plan nodes. For some
reason, we have never accounted for either the evaluation cost or the
selectivity of filter conditions attached to Agg and Group nodes (which, in
practice, are always conditions from a HAVING clause). Applying our regular
selectivity logic to post-grouping conditions is a bit bogus, but it's surely
better than taking the selectivity as 1.0. Perhaps someday the
extended-statistics mechanism can be taught to provide statistics that would
help us in getting non-default estimates here. Per a gripe from Benjamin
Coutu. This is surely a bug fix, but I'm hesitant to back-patch because of
the prospect of destabilizing existing plan choices. Given that it took us
this long to notice the bug, it's probably not hurting too many people in the
field. Discussion: https://postgr.es/m/20968.1509486337@sss.pgh.pa.us

- Fix corner-case errors in brin_doupdate(). In some cases the BRIN code
releases lock on an index page, and later re-acquires lock and tries to check
that the tuple it was working on is still there. That check was a couple
bricks shy of a load. It didn't consider that the page might have turned into
a "revmap" page. (The samepage code path doesn't call brin_getinsertbuffer(),
so it isn't protected by the checks for revmap status there.) It also didn't
check whether the tuple offset was now off the end of the linepointer array.
Since commit 24992c6db the latter case is pretty common, but at least in
principle it could have occurred before that. The net result is that
concurrent updates of a BRIN index could fail with errors like "invalid index
offnum" or "inconsistent range map". Per report from Tomas Vondra.
Back-patch to 9.5, since this code is substantially the same in all versions
containing BRIN. Discussion:

- Set the metapage's pd_lower correctly in brin, gin, and spgist indexes.
Previously, these index types left the pd_lower field set to the default
SizeOfPageHeaderData, which is really a lie because it ought to point past
whatever space is being used for metadata. The coding accidentally failed to
fail because we never told xlog.c that the metapage is of standard format ---
but that's not very good, because it impedes WAL consistency checking, and in
some cases prevents compression of full-page images. To fix, ensure that we
set pd_lower correctly, not only when creating a metapage but whenever we
write it out (these apparently redundant steps are needed to cope with
pg_upgrade'd indexes that don't yet contain the right value). This allows
telling xlog.c that the page is of standard format. The WAL consistency check
mask functions are made to mask only if pd_lower appears valid, which I think
is likely unnecessary complication, since any metapage appearing in a v11 WAL
stream should contain valid pd_lower. But it doesn't cost much to be
paranoid. Amit Langote, reviewed by Michael Paquier and Amit Kapila

- pgbench: replace run-time string comparisons with an enum identifier. Minor
refactoring that should yield some performance benefit. Fabien Coelho,
reviewed by Aleksandr Parfenov Discussion:

- Flag index metapages as standard-format in xlog.c calls. btree, hash, and
bloom indexes all set up their metapages in standard format (that is, with
pd_lower and pd_upper correctly delimiting the unused area); but they mostly
didn't inform the xlog routines of this. When calling log_newpage[_buffer],
this is bad because it loses the opportunity to compress unused data out of
the WAL record. When calling XLogRegisterBuffer, it's not such a performance
problem because all of these call sites also use REGBUF_WILL_INIT, preventing
an FPI image from being written. But it's still a good idea to provide the
flag when relevant, because that aids WAL consistency checking. This
completes the project of getting all the in-core index AMs to handle their
metapage WAL operations similarly. Amit Kapila, reviewed by Michael Paquier

- Avoid looping through line pointers twice in PageRepairFragmentation(). There
doesn't seem to be any good reason to do the filling of the itemidbase[] array
separately from the first traversal of the pointers. It's certainly not a win
if there are any line pointers with storage, and even if there aren't, this
change doesn't insert code into the part of the first loop that will be
traversed in that case. So let's just merge the two loops. Yura Sokolov,
reviewed by Claudio Freire Discussion:

- First-draft release notes for 10.1. As usual, the release notes for other
branches will be made by cutting these down, but put them up for community
review first. Note that a fair percentage of the entries apply only to prior
branches because their issue was already fixed in 10.0.

- Release notes for 10.1, 9.6.6, 9.5.10, 9.4.15, 9.3.20, 9.2.24. In the v10
branch, also back-patch the effects of 1ff01b390 and c29c57890 on these files,
to reduce future maintenance issues. (I'd do it further back, except that the
9.X branches differ anyway due to xlog-to-wal link tag renaming.)

Robert Haas pushed:

- Add sanity check for pg_proc.provariadic. Check that the values from
pg_proc.h match what ProcedureCreate would have done. Robert Haas and Amul
Sul Discussion:

- Fix code related to partitioning schemes for dropped columns. The entry in
appinfo->translated_vars can be NULL; if so, we must avoid dereferencing it.
Ashutosh Bapat Discussion:

- Don't exaggerate the number of temporary blocks read. A read that returns
zero bytes (or an error) should not increment the number of temporary blocks
read. Thomas Munro Discussion:

Stephen Frost pushed:

- Remove inbound links to sql-createuser. CREATE USER is an alias for CREATE
ROLE, not its own command any longer, so clean up references to the
'sql-createuser' link to go to 'sql-createrole' instead. In passing, change a
few cases of 'CREATE USER' to be 'CREATE ROLE ... LOGIN'. The remaining
cases appear reasonable and also mention the distinction between 'CREATE ROLE'
and 'CREATE USER'. Also, don't say CREATE USER "assumes" LOGIN, but rather
"includes". Patch-by: David G. Johnston, with assumes->includes by me.

Michael Meskes pushed:

- Make sure ecpglib does accepts digits behind decimal point even for integers
in. Informix mode. Spotted and fixed by 高增琦 <pgf00a(at)gmail(dot)com>

- Fix float parsing in ecpg INFORMIX mode.

- Improve error message for incorrect number inputs in libecpg.

Peter Eisentraut pushed:

- pg_basebackup: Fix comparison handling of tablespace mappings on Windows. A
candidate path needs to be canonicalized before being checked against the
mappings, because the mappings are also canonicalized. This is especially
relevant on Windows Reported-by: nb <nbedxp(at)gmail(dot)com> Author: Michael Paquier
<michael(dot)paquier(at)gmail(dot)com> Reviewed-by: Ashutosh Sharma

- doc: Add to hot standby documentation. Document the order of changing certain
settings when using hot-standby servers. This is just a logical consequence
of what was already documented, but it gives the users some more practical
advice. Author: Yorick Peterse <yorickpeterse(at)gmail(dot)com> Reviewed-by:
Aleksander Alekseev <a(dot)alekseev(at)postgrespro(dot)ru> Reviewed-by: Robert Haas

- doc: Mention pg_stat_wal_receiver in streaming replication docs. Also make
the link to pg_stat_replication more precise. Author: Michael Paquier
<michael(dot)paquier(at)gmail(dot)com> Reviewed-by: Jeff Janes <jeff(dot)janes(at)gmail(dot)com>

- doc: Adjust name in acknowledgments. per request of the named person

- doc: Clarify pgstattuple privileges information. The description has gotten a
bit confusing over time, so rewrite the paragraph a bit. Reported-by: Feike
Steenbergen <feikesteenbergen(at)gmail(dot)com>

- Remove wal_keep_segments from default configuration in PostgresNode.pm. This
is only used in the pg_rewind tests, so only set it there. It's better if
other tests run closer to a default configuration. Author: Michael Paquier

- Simplify new test suite handling of passwordcheck. This changes the use of a
custom configuration file to enforce the value of preload_shared_libraries to
simply load the library during the tests. This removes the restriction of
running installcheck on the tests, and simplifies its makefile contrary to
what has been introduced in af7211e. Author: Michael Paquier

- pg_ctl: Improve message. Change message for restarting a server from a
directory without a PID file. This accounts for the case where a restart
happens after an initdb. The new message indicates that the start has not
completed yet and might fail. Author: Jesper Pedersen

- Don't reset additional columns on subscriber to NULL on UPDATE. When a
publisher table has fewer columns than a subscriber, the update of a row on
the publisher should result in updating of only the columns in common. The
previous coding mistakenly reset the values of additional columns on the
subscriber to NULL because it failed to skip updates of columns not found in
the attribute map. Author: Petr Jelinek <petr(dot)jelinek(at)2ndquadrant(dot)com>

- doc: Convert ids to upper case at build time. This makes the produced HTML
anchors upper case, making it backward compatible with the previous (9.6)
build system. Reported-by: Thomas Kellerer <spam_eater(at)gmx(dot)net>

- Fix incorrect use of bool. NSUnLinkModule() doesn't take a bool as second
argument but one of set of specific constants. The numeric values are the
same in this case, but clean it up while we're cleaning up bool use elsewhere.
Reviewed-by: Michael Paquier <michael(dot)paquier(at)gmail(dot)com>

- ecpg: Use bool instead of int. Use "bool" for Boolean variables, rather than
"int", matching backend change f505edace12655f3491b9c91991731e2b6bf1f0b.
Reviewed-by: Michael Paquier <michael(dot)paquier(at)gmail(dot)com>

- ecpg: Remove useless return values. Remove useless or inconsistently used
return values from functions, matching backend changes
99bf328237d89e0fd22821a940d4af0506353218 and
791359fe0eae83641f0929159d5861359d395e97. Reviewed-by: Michael Paquier

- doc: Update text for new recovery_target_lsn setting. Reported-by: Tomonari
Katsumata <t(dot)katsumata1122(at)gmail(dot)com> Author: Michael Paquier

Noah Misch pushed:

- In client support of v10 features, use standard schema handling. Back-patch
to v10. This continues the work of commit
080351466c5a669bf35a323bdec9e296330a5dbb. Discussion:

- Ignore CatalogSnapshot when checking COPY FREEZE prerequisites. This restores
the ability, essentially lost in commit
ffaa44cb559db332baeee7d25dedd74a61974203, to use COPY FREEZE under REPEATABLE
READ isolation. Back-patch to 9.4, like that commit. Reviewed by Tom Lane.

== Pending Patches ==

Nikolay Shaplov sent in another revision of a patch to move all am-related
reloption code into src/backend/access/[am-name] and get rid of relopt_kind for
custom AM.

Alexander Kuzmenkov sent in another revision of a patch to implement full merge
join on comparison clause.

Alexander Korotkov sent in another revision of a patch to fix the cube contrib
extension's ORDER BY issue and KNN operations.

Christoph Dreis sent in a patch to remove instances of "the the" in the code

Dagfinn Ilmari Mannsåker sent in a patch to fix a typo in a get_collation_name()

Tom Lane sent in a patch to rewrite PL/PythonU's typeio code.

Tomas Vondra sent in another revision of a patch to implement BRIN multi-range
min-max indexes and BRIN bloom indexes.

Andres Freund sent in another revision of a patch to speed up overflow checks
for int and float types.

Kyotaro HORIGUCHI sent in a patch to add several pieces of vacuum information to
the pg_stat_*_tables.

Chris Travers sent in two revisions of a patch to restrict pg_rewind to
whitelisted directories.

Lætitia Avrot and Amit Langote traded patches to add several missing options to
the ALTER TABLE synopsis.

Amul Sul sent in two more revisions of a patch to implement hash partitioning.

Thomas Munro sent in two more revisions of a patch to implement parallel hash.

Amit Langote sent in three more revisions of a patch to prune partitions faster.

Amit Kapila sent in two more revisions of a patch to parallelize queries
containing initplans.

Raúl Marín Rodríguez sent in two revisions of a patch to add support for pow()
in pgbench.

Alexander Korotkov sent in a patch to add VP trees as an SP-GiST extension.

Anastasia Lubennikova sent in another revision of a patch to implement covering
+ unique indexes.

Kyotaro HORIGUCHI sent in another revision of a patch to restrict maximum keep
segments by repslots.

Kyotaro HORIGUCHI sent in another revision of a patch to protect syscache from
bloating with negative cache entries.

Haribabu Kommi sent in another revision of a patch to prepare for pluggable

David Rowley sent in a patch to remove [Merge]Append nodes which contain a
single subpath.

Peter Eisentraut sent in a patch to consistently catch errors from Python _New()

Simon Riggs sent in another revision of a patch to remove the secondary

Vitaly Burovoy sent in two revisions of a patch to fix dumping pre-10 DBs by
pg_dump10 if table "name" exists.

Peter Eisentraut sent in a patch to implement SQL procedures, which can span

Peter Eisentraut sent in a patch to implement transaction control in PL/PythonU

Shubham Barai sent in another revision of a patch to implement predicate locking
in GiST indexes.

Takayuki Tsunakawa sent in another revision of a patch to implement
statement-level rollback.

Peter Eisentraut sent in a patch to add support for INSERT OVERRIDING to the
PostgreSQL FDW.

Peter Eisentraut sent in a patch to allow returning dynamic result sets from

Peter Eisentraut sent in two revisions of a patch to add const decorations to
many char * arguments in functions.

Oleg Ivanov sent in a patch to implement generic WAL compression.

David Rowley sent in a patch to support removing LEFT JOINs with DISTINCT/GROUP

Andreas Karlsson sent in another revision of a patch to implement REINDEX

Pavel Stěhule sent in another revision of a patch to enable specifying sort
order for psql's describe commands when size is present.

Petr Jelínek sent in another revision of a patch to fix walsender timeouts when
decoding large transactions.

Masahiko Sawada sent in another revision of a patch to add an explicit relation
name in VACUUM VERBOSE log.

Ildus Kurbangaliev sent in another revision of a patch to add custom compression

Tom Lane sent in a patch to fix how reltuples are counted in VACUUM.

Andreas Karlsson sent in another revision of a patch to add GnuTLS support.

Fabien COELHO sent in another revision of a patch to pgbench to use enum for
meta commands.

David Rowley sent in a patch to implement ALists (ArrayLists), these being like
a List, only using an array as the data structure rather than a linked list.

Piotr Stefaniak and Nikita Glukhov traded patches to implement SQL-JSON.

Thomas Munro sent in a patch to fix some LDAP URI decoding bugs.

Antonin Houska sent in another revision of a patch to implement aggregate

David Fetter sent in a patch to skip an unneeded temp file in 'make html'.

Thomas Munro sent in two revisions of a patch to add ldaps support for ldap

Andres Freund sent in a patch to fix pruning of locked and updated tuples.

Fabrízio de Royes Mello sent in two more revisions of a patch to implement hooks
for session_start and session_end.

Tom Lane sent in two more revisions of a patch to improve compactify_tuples.

Fabien COELHO sent in another revision of a patch to pgbench to enable storing
select results into variables.

Robert Haas sent in two more revisions of a patch to make Gather nodes faster.

Amit Kapila sent in another revision of a patch to ensure that parallel paths
include tlist cost.

Lucas (lucas75 AT gmail DOT com) sent in a patch to add a --lock-early option to

Emre Hasegeli sent in another revision of a patch to refactor the geometric
functions and operators code, provide a header file for built-in float
datatypes, use the built-in float datatype to implement geometric types, fix
obvious problems around the line datatype, check for float -0 after
multiplications and divisions, and improve test coverage of geometric types.

David Christensen sent in a patch to add a two-argument form of
current_setting(NAME, FALLBACK), which avoids erroring out on, for example,
non-existence of the setting to be set.

Noah Misch sent in a patch to fix some encoding issues encountered with libxml2
functions in UTF-8-encoded databases.

Browse pgsql-announce by date

  From Date Subject
Next Message Daniel Gustafsson 2017-11-06 14:06:54 Nordic PGDay 2018 - Registration, CfP and sponsorships open!
Previous Message Christoph Berg 2017-11-04 18:02:46 check_postgres 2.23.0 released