|From:||David Fetter <david(at)fetter(dot)org>|
|To:||PostgreSQL Announce <pgsql-announce(at)postgresql(dot)org>|
|Subject:||== PostgreSQL Weekly News - March 11 2018 ==|
|Views:||Raw Message | Whole Thread | Download mbox|
== PostgreSQL Weekly News - March 11 2018 ==
== PostgreSQL Product News ==
pgmetrics, a command-line tool to extract and display metrics from a PostgreSQL
server, released as open source.
== PostgreSQL Jobs for March ==
== PostgreSQL Local ==
Nordic PGDay 2018 will be held in Oslo, Norway, at the Radisson Blu Hotel
Nydalen, on March 13, 2018. Registration is open and the schedule is posted.
pgDay Paris 2018 will be held in Paris, France at the Espace Saint-Martin, on
March 15 2018. Registration is open.
PGConf APAC 2018 will be held in Singapore March 22-23, 2018.
The German-speaking PostgreSQL Conference 2018 will take place on April 13th,
2018 in Berlin.
PGConfNepal 2018 will be held May 4-5, 2018 at Kathmandu University, Dhulikhel,
PGCon 2018 will take place in Ottawa on May 29 - June 1, 2018.
Swiss PGDay 2018 will take place in Rapperswil (near Zurich) on June 29, 2018.
The CfP is open February 6, 2018 through April 14, 2018, and registration is
open February 6, 2018 through June 28, 2018.
PGConf.Brazil 2018 will take place in São Paulo, Brazil on August 3-4 2018.
== PostgreSQL in the News ==
Planet PostgreSQL: http://planet.postgresql.org/
PostgreSQL Weekly News is brought to you this week by David Fetter
Submit news and announcements by Sunday at 3:00pm EST5EDT. Please send English
language ones to david(at)fetter(dot)org, German language to pwn(at)pgug(dot)de, Italian
language to pwn(at)itpug(dot)org(dot)
== Applied Patches ==
Magnus Hagander pushed:
- Actually pick .lib file when multiple perl libs are present.
7240962f8626ff09bb8f9e71ecdb074775bdd035 got it right in the comment, but the
code did not actually do what the comment said. Fix that. Issue pointed out
by Noah Misch.
- Clarify initdb --help message for --wal-segsize. Specify that the value is in
megabytes. This aligns the message with what's in the documentation.
Peter Eisentraut pushed:
- Remove some obsolete procedure-specific code from PLs. Since procedures are
now declared to return void, code that handled return values for procedures
separately is no longer necessary.
- doc: Tiny whitespace fix.
- Tests for Kerberos/GSSAPI authentication. Like the LDAP and SSL tests, these
are not run by default but can be selected via PG_TEST_EXTRA. Reviewed-by:
Thomas Munro <thomas(dot)munro(at)enterprisedb(dot)com> Reviewed-by: Michael Paquier
- Raise Test::More version requirement. Since
ed8a7c6fcf92b6b57ed8003bbd4a4eb92a6039bc, version 0.87 is required.
- Fix expected error message in test.
- doc: Add replication parameter to libpq documentation. Author: Michael
Paquier <michael(at)paquier(dot)xyz> Reported-by: Şahap Aşçı <sahapasci(at)gmail(dot)com>
Reviewed-by: Vik Fearing <vik(dot)fearing(at)2ndquadrant(dot)com>
- doc: Improve calculation of vm.nr_hugepages. The previous method worked off
the full virtual address space, not just the shared memory usage. Author:
Tsunakawa, Takayuki <tsunakawa(dot)takay(at)jp(dot)fujitsu(dot)com> Reviewed-by: Justin
Pryzby <pryzby(at)telsasoft(dot)com> Reviewed-by: Vasundhar Boddapati
- Fix costing of parallel hash joins. Commit
1804284042e659e7d16904e7bbb0ad546394b6a3 established that single-batch
parallel-aware hash joins could create one large shared hash table using the
combined work_mem budget of all participants. The costing accidentally
assumed that parallel-oblivious hash joins could also do that. The
documentation for initial_cost_hashjoin() also failed to mention the new
argument. Repair. Author: Thomas Munro Reported-By: Antonin Houska
Reviewed-By: Antonin Houska Discussion:
- Fix test counting in SSL tests. The branch that does not support
tls-server-end-point runs more tests, so we need to structure the test
counting dynamically. Reviewed-by: Michael Paquier <michael(at)paquier(dot)xyz>
- Fix typo. Author: Daniel Gustafsson <daniel(at)yesql(dot)se>
- doc: Add more substructure to SSL documentation. The SSL documentation text
has gotten a bit long, so add some subsections and reorder for better flow.
Fujii Masao pushed:
- Fix pg_rewind to handle relation data files in tablespaces properly.
pg_rewind checks whether each file is a relation data file, from its path.
Previously this check logic had the bug which made pg_rewind fail to recognize
any relation data files in tablespaces. Which also caused an assertion failure
in pg_rewind. Back-patch to 9.5 where pg_rewind was added. Author: Takayuki
Tsunakawa Reviewed-by: Michael Paquier Discussion:
Robert Haas pushed:
- shm_mq: Fix detach race condition. Commit
34db06ef9a1d7f36391c64293bf1e0ce44a33915 adopted a lock-free design for
shm_mq.c, but it introduced a race condition that could lose messages. When
shm_mq_receive_bytes() detects that the other end has detached, it must make
sure that it has seen the final version of mq_bytes_written, or it might miss
a message sent before detaching. Thomas Munro Discussion:
- test_decoding: Remove unused #include directives. Euler Taveira Discussion:
- Correctly assess parallel-safety of tlists when SRFs are used. Since commit
69f4b9c85f168ae006929eec44fc44d569e846b9, the existing code was no longer
assessing the parallel-safety of the real tlist for each upper rel, but rather
the first of possibly several tlists created by split_pathtarget_at_srfs().
Repair. Even though this is clearly wrong, it's not clear that it has any
user-visible consequences at the moment, so no back-patch for now. If we
discover later that it does have user-visible consequences, we might need to
back-patch this to v10. Patch by me, per a report from Rajkumar Raghuwanshi.
Tom Lane pushed:
- Add infrastructure to support server-version-dependent tab completion. Up to
now we've not worried about whether psql's tab completion queries would work
against prior server versions. But since we support older server versions in
describe.c, we really ought to do so here as well. Failing to take care of
this not only leads to loss of tab-completion service when using an older
server, but risks aborting a user's open transaction when we send an
incompatible query to the server. The recent changes in pg_proc.prokind are
one reason to take this more seriously now than before, and the proposed patch
for completion after SELECT needs some such capability as well. Hence, create
some infrastructure to allow selection of one of several versions of a query
depending on server version. For cases where we just need to select one of
several query strings, introduce VersionedQuery and
COMPLETE_WITH_VERSIONED_QUERY(). Likewise extend the SchemaQuery
infrastructure to allow versioning of those. I went ahead and fixed the
prokind issues, to serve as an illustration of how to use versioned
SchemaQuery. To have some illustration of VersionedQuery, change the support
for completion of publication and subscription names so that psql will not
send sure-to-fail queries to pre-v10 servers. There is much more that should
be done to make tab completion more friendly to older servers, but this commit
is mainly meant to get the infrastructure in place, so I stopped here.
- Temporarily instrument postgres_fdw test to look for statistics changes. It
seems fairly hard to explain recent buildfarm failures without the theory that
something is doing an ANALYZE behind our backs. Probe for this directly to
see if it's true. In principle the outputs of these queries should be stable,
since the table in question is small enough that ANALYZE's sample will include
all rows. But even if that turns out to be wrong, we can put up with some
failures for a bit. I don't intend to leave this here indefinitely.
- In initdb, don't bother trying max_connections = 10. The server won't
actually start with that setting anymore, not since we raised the default
max_wal_senders to 10. Per discussion, we don't wish to back down on that
default, so instead raise the effective floor for max_connections (to 20).
It's still possible to configure a smaller setting manually, but initdb won't
set it that way. Since that change happened in v10, back-patch to v10.
Kyotaro Horiguchi Discussion:
- Fix cross-checking of ReservedBackends/max_wal_senders/MaxConnections. We
were independently checking ReservedBackends < MaxConnections and
max_wal_senders < MaxConnections, but because walsenders aren't allowed to use
superuser-reserved connections, that's really the wrong thing. Correct
behavior is to insist on ReservedBackends + max_wal_senders being less than
MaxConnections. Fix the code and associated documentation. This has been
wrong for a long time, but since the situation probably hardly ever arises in
the field (especially pre-v10, when the default for max_wal_senders was zero),
no back-patch. Discussion: https://firstname.lastname@example.org
- Revert "Temporarily instrument postgres_fdw test to look for statistics
changes.". This reverts commit c2c537c56dc30ec3cdc12051f4ea5363aa66d73c.
It's now clear that whatever is going on there, it can't be blamed on
unexpected ANALYZE runs, because the statistics are the same just before the
failing query as they were at the start of the test.
- Add test scaffolding for exercising optimizer's predicate-proof logic. The
predicate-proof code in predtest.c is a bit hard to test as-is: you have to
construct a query whose plan changes depending on the success of a test, and
in tests that have been around for awhile, it's always possible that the plan
shape is now being determined by some other factor. Our existing regression
tests aren't doing real well at providing full code coverage of predtest.c,
either. So, let's add a small test module that allows directly inspecting the
results of predicate_implied_by() and predicate_refuted_by() for arbitrary
boolean expressions. I chose the set of tests committed here in order to get
reasonably complete code coverage of predtest.c just from running this test
module, and to cover some cases called out as being interesting in the
existing comments. We might want to add more later. But this set already
shows a few cases where perhaps things could be improved. Indeed, this
exercise proves that predicate_refuted_by() is buggy for the case of
clause_is_check = true, though fortunately we aren't using that case anywhere
yet. I'll look into doing something about that in a separate commit. For
now, just memorialize the current behavior. Discussion:
- Fix test_predtest's idea of what weak refutation means. I'd initially
supposed that predicate_refuted_by(..., true) ought to say that "A refutes B"
means "non-falsity of A implies non-truth of B". But it seems better to
define it as "truth of A implies non-truth of B". This is more useful in the
current system, slightly easier to prove, and in closer correspondence to the
existing code behavior. With this change, test_predtest no longer claims that
any existing test cases show false proof reports, though there still are cases
where we could prove something and fail to. Discussion:
- Improve predtest.c's internal docs, and enhance its functionality a bit.
Commit b08df9cab left things rather poorly documented as far as the exact
semantics of "clause_is_check" mode went. Also, that mode did not really work
correctly for predicate_refuted_by; although given the lack of specification
as to what it should do, as well as the lack of any actual use-case, that's
perhaps not surprising. Rename "clause_is_check" to "weak" proof mode, and
provide specifications for what it should do. I defined weak refutation as
meaning "truth of A implies non-truth of B", which makes it possible to use
the mode in the part of relation_excluded_by_constraints that checks for
mutually contradictory WHERE clauses. Fix up several places that did things
wrong for that definition. (As far as I can see, these errors would only lead
to failure-to-prove, not incorrect claims of proof, making them not serious
bugs even aside from the fact that v10 contains no use of this mode. So there
seems no need for back-patching.) In addition, teach
predicate_refuted_by_recurse that it can use predicate_implied_by_recurse
after all when processing a strong NOT-clause, so long as it asks for the
correct proof strength. This is an optimization that could have been included
in commit b08df9cab, but wasn't. Also, simplify and generalize the logic that
checks for whether nullness of the argument of IS [NOT] NULL would force
overall nullness of the predicate or clause. (This results in a change in the
partition_prune test's output, as it is now able to prune an all-nulls
partition that it did not recognize before.) In passing, in
PartConstraintImpliedByRelConstraint, remove bogus conversion of the
constraint list to explicit-AND form and then right back again; that
accomplished nothing except forcing a useless extra level of recursion inside
- In psql, restore old behavior of Query_for_list_of_functions. Historically,
tab completion for functions has offered the names of aggregates as well.
This is essential in at least one context, namely GRANT/REVOKE, because there
is no GRANT ON AGGREGATE syntax. There are other cases where a command that
nominally is for functions will allow aggregates as well, though not all do.
Commit fd1a421fe changed this query to disallow aggregates, but that doesn't
seem to have been thought through very carefully. Change it to allow
aggregates (but still ignore procedures). We might at some point tighten this
up, but it'd require sorting through all the uses of this query to see which
ones should offer aggregate names and which shouldn't. Given the lack of
field complaints about the historical laxity here, that's work I'm not eager
to do right now. Discussion:
- Fix improper uses of canonicalize_qual(). One of the things
canonicalize_qual() does is to remove constant-NULL subexpressions of
top-level AND/OR clauses. It does that on the assumption that what it's given
is a top-level WHERE clause, so that NULL can be treated like FALSE. Although
this is documented down inside a subroutine of canonicalize_qual(), it wasn't
mentioned in the documentation of that function itself, and some callers
hadn't gotten that memo. Notably, commit d007a9505 caused
get_relation_constraints() to apply canonicalize_qual() to CHECK constraints.
That allowed constraint exclusion to misoptimize situations in which a CHECK
constraint had a provably-NULL subclause, as seen in the regression test case
added here, in which a child table that should be scanned is not. (Although
this thinko is ancient, the test case doesn't fail before 9.2, for reasons
I've not bothered to track down in detail. There may be related cases that do
fail before that.) More recently, commit f0e44751d added an independent bug by
applying canonicalize_qual() to index expressions, which is even sillier since
those might not even be boolean. If they are, though, I think this could lead
to making incorrect index entries for affected index expressions in v10. I
haven't attempted to prove that though. To fix, add an "is_check" parameter
to canonicalize_qual() to specify whether it should assume WHERE or CHECK
semantics, and make it perform NULL-elimination accordingly. Adjust the
callers to apply the right semantics, or remove the call entirely in cases
where it's not known that the expression has one or the other semantics. I
also removed the call in some cases involving partition expressions, where it
should be a no-op because such expressions should be canonical already ...
and was a no-op, independently of whether it could in principle have done
something, because it was being handed the qual in implicit-AND format which
isn't what it expects. In HEAD, add an Assert to catch that type of mistake
in future. This represents an API break for external callers of
canonicalize_qual(). While that's intentional in HEAD to make such callers
think about which case applies to them, it seems like something we probably
wouldn't be thanked for in released branches. Hence, in released branches,
the extra parameter is added to a new function canonicalize_qual_ext(), and
canonicalize_qual() is a wrapper that retains its old behavior. Patch by me
with suggestions from Dean Rasheed. Back-patch to all supported branches.
Andres Freund pushed:
- Fix HEAP_INSERT_IS_SPECULATIVE to HEAP_INSERT_SPECULATIVE in comments. This
was wrong since 168d5805e4c08bed7b95d351bf097cff7c07dd65, which introduced
speculative inserts. Author: Andres Freund
- Add parenthesized options syntax for ANALYZE. This is analogous to the syntax
allowed for VACUUM. This allows us to avoid making new options reserved
keywords and makes it easier to allow arbitrary argument order. Oh, and it's
consistent with the other commands, too. Author: Nathan Bossart Reviewed-By:
Michael Paquier, Masahiko Sawada Discussion:
- Fix parent node of WCO expressions in partitioned tables. Since
edd44738bc8814 WCO expressions of partitioned tables are initialized with the
first subplan as parent. That's not correct, as the correct context is the
ModifyTableState node. That's also what is used for RETURNING processing,
initialized nearby. This appears not to cause any visible problems for in
core code, but is problematic for in development patch. Discussion:
Álvaro Herrera pushed:
- Clone extended stats in CREATE TABLE (LIKE INCLUDING ALL). The LIKE INCLUDING
ALL clause to CREATE TABLE intuitively indicates cloning of extended
statistics on the source table, but it failed to do so. Patch it up so that
it does. Also include an INCLUDING STATISTICS option to the LIKE clause, so
that the behavior can be requested individually, or excluded individually.
While at it, reorder the INCLUDING options, both in code and in docs, in
alphabetical order which makes more sense than feature-implementation order
that was previously used. Backpatch this to Postgres 10, where extended
statistics were introduced, because this is seen as an oversight in a fresh
feature which is better to get consistent from the get-go instead of changing
only in pg11. In pg11, comments on statistics objects are cloned too. In
pg10 they are not, because I (Álvaro) was too coward to change the parse node
as required to support it. Also, in pg10 I chose not to renumber the parser
symbols for the various INCLUDING options in LIKE, for the same reason. Any
corresponding user-visible changes (docs) are backpatched, though.
Reported-by: Stephen Froehlich Author: David Rowley Reviewed-by: Álvaro
Herrera, Tomas Vondra Discussion:
- Fix bogus Name assignment in CreateStatistics. Apparently, it doesn't work to
use a plain cstring as a Name datum: you may end up having random bytes
because of failing to zero the bytes after the terminating \0, as indicated by
valgrind. I introduced this bug in 5564c1181548, so backpatch this fix to
REL_10_STABLE, like that commit. While at it, fix a slightly misleading
comment, pointed out by David Rowley.
- Refrain from duplicating data in reorderbuffers. If a walsender exits leaving
data in reorderbuffers, the next walsender that tries to decode the same
transaction would append its decoded data in the same spill files without
truncating it first, which effectively duplicate the data. Avoid that by
removing any leftover reorderbuffer spill files when a walsender starts.
Backpatch to 9.4; this bug has been there from the very beginning of logical
decoding. Author: Craig Ringer, revised by me Reviewed by: Álvaro Herrera,
Petr Jelínek, Masahiko Sawada
- Fix typo. Author: Kyotaro HORIGUCHI Discussion:
- Add missing debug lines during bootstrap. Noticed while playing with changes
that mess with the bootstrap sequence; the operations patched here failed to
emit anything, leading the developer to think that the bug was in the previous
operation that did emit a message.
Stephen Frost pushed:
- Fix typo for RangeVarGetRelidExtended. The function is actually
RangeVarGetRelidExtended, so the comment should reflect that. Author: Michael
Paquier Discussion: https://postgr.es/m/20180307035216.GA3184@paquier.xyz
== Pending Patches ==
Amit Langote sent in a patch to fix an infelicity between partitions and WCO
quals on parent tables.
Thomas Munro sent in a patch to fix a typo in src/backend/access/hash/README.
Rajkumar Raghuwanshi sent in another revision of a patch to implement
partition-wise JOIN, accounting for default partitions.
Fabien COELHO sent in another revision of a patch to add a --random-seed option
Fabien COELHO sent in another revision of a patch to add a pgbench "progress"
Michaël Paquier sent in a patch to fix a particle before SQL.
Claudio Freire sent in another revision of a patch to update the FSM more
frequently in VACUUM.
Andrey Borodin sent in a patch to fill in missing pieces of the SLRU checksum
Nikhil Sontakke sent in another revision of a patch to implement logical
decoding of two-phase transactions.
Heikki Linnakangas sent in another revision of a patch to speed up crc32 on
Satyanarayana Narlapuram sent in another revision of a patch to add client
connection redirection support to PostgreSQL.
Michael Banck and Álvaro Herrera traded patches to allow parallel pg_dump to
Daniel Gustafsson sent in another revision of a patch to refactor backend
signalling code and use the new infrastructure to support an optional message in
Peter Eisentraut sent in two more revisions of a patch to allow INOUT parameters
Jeevan Chalke, Robert Haas, and Ashutosh Bapat traded patches to implement
David Steele sent in three more revisions of a patch to make it possible to
allow group read access on the filesystem.
Michael Banck sent in two more revisions of a patch to verify checksums during
Alexander Korotkov sent in another revision of a patch to implement incremental
Pavan Deolasee sent in three more revisions of a patch to implement MERGE.
David Steele sent in a patch to add regression tests for reinit.c.
Dean Rasheed sent in two more revisions of a patch to improve MCV lists for
highly skewed distributions.
David Rowley sent in two more revisions of a patch to make array_agg and
Etsuro Fujita and Amit Langote traded patches to fix an issue where inserts into
a partitioned table could cause a crash.
Michaël Paquier extend lookup routines for FDW and foreign server with NULL
handling, refactor routines for subscription and publication lookups, and
eliminate user-visible cache lookup errors for objaddr SQL functions.
Aleksandr Parfenov sent in another revision of a patch to add a flexible
configuration for full-text search.
Dmitry Dolgov sent in another revision of a patch to implement generic type
Craig Ringer sent in another revision of a patch to clean up reorder buffer
files when starting logical decoding.
Peter Eisentraut sent in a patch to fix test counting in SSL tests.
Stephen Frost sent in another revision of a patch to rewrite the pg_dump TAP
Artur Zakirov sent in another revision of a patch to implement shared ISpell
Nikita Glukhov sent in two more revisions of a patch to implement JSONPATH.
Nikita Glukhov sent in two more revisions of a patch to implement SQL/JSON
Nikita Glukhov sent in two more revisions of a patch to implement JSON_TABLE.
Stephen Frost sent in two more revisions of a patch to add a default role
Edmund Horner sent in two more revisions of a patch to add tab completion for
SELECT in psql.
Sergei Kornilov sent in three more revisions of a patch to skip NOT NULL checks
in ALTER TABLE when appropriate CHECKs already enforce them.
Michail Nikolaev sent in two more revisions of a patch to use indexes for
OFFSETs and GROUP BY where possible.
David Rowley sent in a patch to disable physical tlists in ALTER TABLE ... ADD
COLUMN ... DEFAULT.
Ashutosh Bapat add a missing break statement after transformCallStmt in
Andrey Borodin sent in another revision of a patch to implement GiST VACUUM.
Kyotaro HORIGUCHI sent in a patch to ensure that cursor position is not added
when hide_stmt is set.
Álvaro Herrera sent in another revision of a patch to implement foreign key
Ildar Musin sent in another revision of a patch to add a general purpose hashing
function to pgbench.
Nikolay Shaplov sent in another revision of a patch to add an enum relation
Fabrízio de Royes Mello sent in a patch to fix missing spaces in the docs.
Robert Haas and Rajkumar Raghuwanshi traded patches to fix an infelicity between
parallel append and a simple UNION ALL.
Kyotaro HORIGUCHI sent in two more revisions of a patch to remove catcache
entries that haven't been used for a certain time.
Daniel Vérité sent in another revision of a patch to implement a csv table
format for psql.
Álvaro Herrera sent in four more revisions of a patch to implement FOR EACH ROW
triggers on partitioned tables.
Peter Geoghegan sent in a patch to report query text in parallel CREATE INDEX
Masahiko Sawada sent in another revision of a patch to improve messaging during
logical replication worker.
Anastasia Lubennikova sent in another revision of a patch to implement Covering
+ unique indexes.
Pavel Stěhule sent in another revision of a patch to implement schema variables.
Peter Eisentraut sent in another revision of a patch to support GNUTLS as an SSL
Julian Markwort sent in another revision of a patch to add plans to
Tomas Vondra sent in a patch to use a sort-based approach to speed up
Amit Khandekar sent in a patch to fix a concurrency bug in UPDATEs of partition
Kyotaro HORIGUCHI sent in another revision of a patch to remove
Dilip Kumar sent in a patch to fix fpwupdate.
John Naylor sent in another revision of a patch to rationalize the handling of
Peter Eisentraut sent in another revision of a patch to track statistics for
Masahiko Sawada sent in another revision of a patch to report autovac workitem
Amul Sul sent in another revision of a patch to restrict concurrent
update/delete with UPDATE of partition key.
Alexander Korotkov and Masahiko Sawada traded patches to add a GUC for cleanup
Takayuki Tsunakawa and Michaël Paquier traded patches to fix a bug where
pg_rewind creates corrupt WAL files, and the standby cannot catch up the
Michaël Paquier sent in a patch to fix some missing schema qualifications.
Yura Sokolov sent in another revision of a patch to make a hash table for xip in
Tom Lane sent in a patch to fix bogus use of canonicalize_qual.
Amit Kapila sent in two more revisions of a patch to implement Predicate Locking
in hash index.
Tomas Vondra sent in another revision of a patch to implement multivariate
histograms and MCV lists.
Amit Kapila sent in another revision of a patch to ensure that parallel paths
include tlist cost.
Peter Eisentraut sent in a patch to set libpq sslcompression to off by default.
Peter Geoghegan sent in a patch to fix corruption of backend REINDEX processing
Álvaro Herrera sent in another revision of a patch to allow foreign keys and
|Next Message||Monica Real Amores||2018-03-16 14:27:23||BDR 1.0.5 Now Available|
|Previous Message||Nicolas Thauvin||2018-03-10 13:25:45||pg_back 1.5 released|