== PostgreSQL Weekly News - April 09 2017 ==
== PostgreSQL Jobs for April ==
== PostgreSQL Local ==
PGCon 2017 will take place in Ottawa on 23-26 May.
Postgres Vision will take place in Boston, June 26 - 28, 2017.
Swiss PGDay in Rapperswil will take place June 30, 2017.
The CfP is open through April 14, 2017.
PostgresOpen will occur September 6-8, 2017 in San Francisco.
The CfP is open until May 30, 2017.
PGDay UK 2017 will take place July 4, 2017 in London, England, UK.
The CfP closes April 20th 2017.
== PostgreSQL in the News ==
Planet PostgreSQL: http://planet.postgresql.org/
PostgreSQL Weekly News is brought to you this week by David Fetter
Submit news and announcements by Sunday at 3:00pm Pacific time.
Please send English language ones to david(at)fetter(dot)org, German language
to pwn(at)pgug(dot)de, Italian language to pwn(at)itpug(dot)org(dot)
== Applied Patches ==
Tom Lane pushed:
- Document psql's behavior of recalling the previously executed query. Various
psql slash commands that normally act on the current query buffer will
automatically recall and re-use the most recently executed SQL command
instead, if the current query buffer is empty. Although this behavior is
ancient (dating apparently to commit 77a472993), it was documented nowhere in
the psql reference page. For that matter, we'd never bothered to define the
concept of "current query buffer" explicitly. Fix that. Do some wordsmithing
on relevant command descriptions to improve clarity and consistency.
- Doc: clarify behavior of OT_WHOLE_LINE and OT_FILEPIPE psql slash commands.
This is another bit of ancient behavior that was documented poorly (in a
couple of cases) or not at all (in several others). Discussion:
- Remove reinvention of stringify macro. We already have CppAsString2, there's
no need for the MSVC support to re-invent a macro to do that (and especially
not to inject it in as ugly a way as this). Discussion:
- Fix integer-overflow problems in interval comparison. When using integer
timestamps, the interval-comparison functions tried to compute the overall
magnitude of an interval as an int64 number of microseconds. As reported by
Frazer McLean, this overflows for intervals exceeding about 296000 years,
which is bad since we nominally allow intervals many times larger than that.
That results in wrong comparison results, and possibly in corrupted btree
indexes for columns containing such large interval values. To fix, compute
the magnitude as int128 instead. Although some compilers have native support
for int128 calculations, many don't, so create our own support functions that
can do 128-bit addition and multiplication if the compiler support isn't
there. These support functions are designed with an eye to allowing the
int128 code paths in numeric.c to be rewritten for use on all platforms,
although this patch doesn't do that, or even provide all the int128 primitives
that will be needed for it. Back-patch as far as 9.4. Earlier releases did
not guard against overflow of interval values at all (commit 146604ec4 fixed
that), so it seems not very exciting to worry about overly-large intervals for
them. Before 9.6, we did not assume that unreferenced "static inline"
functions would not draw compiler warnings, so omit functions not directly
referenced by timestamp.c, the only present consumer of int128.h. (We could
have omitted these functions in HEAD too, but since they were written and
debugged on the way to the present patch, and they look likely to be needed by
numeric.c, let's keep them in HEAD.) I did not bother to try to prevent such
warnings in a --disable-integer-datetimes build, though. Before 9.5,
configure will never define HAVE_INT128, so the part of int128.h that exploits
a native int128 implementation is dead code in the 9.4 branch. I didn't
bother to remove it, thinking that keeping the file looking similar in
different branches is more useful. In HEAD only, add a simple test harness
for int128.h in src/tools/. In back branches, this does not change the
float-timestamps code path. That's not subject to the same kind of overflow
risk, since it computes the interval magnitude as float8. (No doubt, when
this code was originally written, overflow was disregarded for exactly that
reason.) There is a precision hazard instead :-(, but we'll avert our eyes
from that question, since no complaints have been reported and that code's
deprecated anyway. Kyotaro Horiguchi and Tom Lane Discussion:
- Clean up psql/describe.c's messy query for extended stats. Remove unnecessary
casts, safely schema-qualify the ones that remain, lose an unnecessary level
of sub-SELECT, reformat for tidiness.
- Clean up after insufficiently-researched optimization of tuple conversions.
tupconvert.c's functions formerly considered that an explicit tuple conversion
was necessary if the input and output tupdescs contained different type OIDs.
The point of that was to make sure that a composite datum resulting from the
conversion would contain the destination rowtype OID in its composite-datum
header. However, commit 3838074f8 entirely misunderstood what that check was
for, thinking that it had something to do with presence or absence of an OID
column within the tuple. Removal of the check broke the no-op conversion path
in ExecEvalConvertRowtype, as reported by Ashutosh Bapat. It turns out that
of the dozen or so call sites for tupconvert.c functions,
ExecEvalConvertRowtype is the only one that cares about the composite-datum
header fields in the output tuple. In all the rest, we'd much rather avoid an
unnecessary conversion whenever the tuples are physically compatible.
Moreover, the comments in tupconvert.c only promise physical compatibility not
a metadata match. So, let's accept the removal of the guarantee about the
output tuple's rowtype marking, recognizing that this is a API change that
could conceivably break third-party callers of tupconvert.c. (So, let's
remember to mention it in the v10 release notes.) However, commit 3838074f8
did have a bit of a point here, in that two tuples mustn't be considered
physically compatible if one has HEAP_HASOID set and the other doesn't. (Some
of the callers of tupconvert.c might not really care about that, but we can't
assume it in general.) The previous check accidentally covered that issue,
because no RECORD types ever have OIDs, while if two tupdescs have the same
named composite type OID then, a fortiori, they have the same tdhasoid
setting. If we're removing the type OID match check then we'd better include
tdhasoid match as part of the physical compatibility check. Without that hack
in tupconvert.c, we need ExecEvalConvertRowtype to take responsibility for
inserting the correct rowtype OID label whenever tupconvert.c decides it need
not do anything. This is easily done with heap_copy_tuple_as_datum, which
will be considerably faster than a tuple disassembly and reassembly anyway; so
from a performance standpoint this change is a win all around compared to what
happened in earlier branches. It just means a couple more lines of code in
ExecEvalConvertRowtype. Ashutosh Bapat and Tom Lane Discussion:
- Fix planner error (or assert trap) with nested set operations. As reported by
Sean Johnston in bug #14614, since 9.6 the planner can fail due to trying to
look up the referent of a Var with varno 0. This happens because we generate
such Vars in generate_append_tlist, for lack of any better way to describe the
output of a SetOp node. In typical situations nothing really cares about
that, but given nested set-operation queries we will call estimate_num_groups
on the output of the subquery, and that wants to know what a Var actually
refers to. That logic used to look at subquery->targetList, but in commit
3fc6e2d7f I'd switched it to look at subroot->processed_tlist, ie the actual
output of the subquery plan not the parser's idea of the result. It seemed
like a good idea at the time :-(. As a band-aid fix, change it back. Really
we ought to have an honest way of naming the outputs of SetOp steps, which
suggests that it'd be a good idea for the parser to emit an RTE corresponding
to each one. But that's a task for another day, and it certainly wouldn't
yield a back-patchable fix. Report:
- Ensure that ExecPrepareExprList's result is all in one memory context. Noted
by Amit Langote. Discussion:
- Optimize joins when the inner relation can be proven unique. If there can
certainly be no more than one matching inner row for a given outer row, then
the executor can move on to the next outer row as soon as it's found one
match; there's no need to continue scanning the inner relation for this outer
row. This saves useless scanning in nestloop and hash joins. In merge joins,
it offers the opportunity to skip mark/restore processing, because we know we
have not advanced past the first possible match for the next outer row. Of
course, the devil is in the details: the proof of uniqueness must depend only
on joinquals (not otherquals), and if we want to skip mergejoin mark/restore
then it must depend only on merge clauses. To avoid adding more planning
overhead than absolutely necessary, the present patch errs in the conservative
direction: there are cases where inner_unique or skip_mark_restore processing
could be used, but it will not do so because it's not sure that the uniqueness
proof depended only on "safe" clauses. This could be improved later. David
Rowley, reviewed and rather heavily editorialized on by me Discussion:
- Add newly-symlinked files to "make clean" target. Oversight in 60f11b87a.
- Clean up bugs in clause_selectivity() cleanup. Commit ac2b09508 was not
terribly carefully reviewed. Band-aid it to not fail on non-RestrictInfo
input, per report from Andreas Seltenreich. Also make it do something more
reasonable with variable-free clauses, and improve nearby comments.
Peter Eisentraut pushed:
- doc: Change xref style to number only. Change the style of links generated by
xrefs to section number only, as it was with DSSSL, instead of number and
title, as is the default of the XSLT stylesheets. Our documentation is mostly
written expecting the old style, so keep that for the time being, per
- Handle change of slot name in logical replication apply. Since change of slot
name is a supported operation, handle it more gracefully, instead of in the
this-should-not-happen way. Author: Petr Jelinek
- Fix whitespace.
- Make header self-contained. Add necessary include files for things used in
- Fix remote position tracking in logical replication. We need to set the
origin remote position to end_lsn, not commit_lsn, as commit_lsn is the start
of commit record, and we use the origin remote position as start position when
restarting replication stream. If we'd use commit_lsn, we could request data
that we already received from the remote server after a crash of a downstream
server. Author: Petr Jelinek <petr(dot)jelinek(at)2ndquadrant(dot)com>
- psql: Add some missing tab completion. Add tab completion for
COMMENT/SECURITY LABEL ON PUBLICATION/SUBSCRIPTION. Reported-by: Stephen
- Fix typo. Author: Masahiko Sawada <sawada(dot)mshk(at)gmail(dot)com>
- Adjust min/max values when changing sequence type. When changing the type of
a sequence, adjust the min/max values of the sequence if it looks like the
previous values were the default values. Previously, it would leave the old
values in place, requiring manual adjustments even in the usual/default cases.
Reviewed-by: Michael Paquier <michael(dot)paquier(at)gmail(dot)com> Reviewed-by: Vitaly
- pageinspect: Add bt_page_items function with bytea argument. Author: Tomas
Vondra <tomas(dot)vondra(at)2ndquadrant(dot)com> Reviewed-by: Ashutosh Sharma
- Capitalize names of PLs consistently. Author: Daniel Gustafsson
- dblink: Small code rearrangement for clarity. suggested by Tom Lane
- doc: Remove remark elements. They were not rendered with DSSSL, but now they
show up by default using XSLT. Just remove them, since they are not useful.
- Use American English in error message. All error messages use the American
English spelling of recognize, apply to the single one not doing so to be
consistent. Author: Daniel Gustafsson <daniel(at)yesql(dot)se>
- Fix compiler warning and add some more comments.
- Fix AclResult vs bool type mix-up. Using AclResult as a bool or vice versa
works by accident, but it's unusual and possibly confusing style, so write it
out more explicitly.
- doc: Formatting fix for XSL-FO PDF build.
- Fix mixup of bool and ternary value. Not currently a problem, but could be
with stricter bool behavior under stdbool or C++. Reviewed-by: Andres Freund
- pg_dump: Rename some typedefs to avoid name conflicts. In struct
_archiveHandle, some of the fields have the same name as a typedef. This is
kind of confusing, so rename the types so they have names distinct from the
struct fields. In C++, the previous coding changes the meaning of the typedef
in the scope of the struct, causing warnings and possibly other problems.
Reviewed-by: Andres Freund <andres(at)anarazel(dot)de>
- Mark immutable functions in information schema as parallel safe. Also add
opr_sanity check that all preloaded immutable functions are parallel safe.
(Per discussion, this does not necessarily have to be true for all possible
such functions, but deviations would be unlikely enough that maintaining such
a test is reasonable.) Reported-by: David Rowley
<david(dot)rowley(at)2ndquadrant(dot)com> Reviewed-by: Robert Haas
<robertmhaas(at)gmail(dot)com> Reviewed-by: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
- Fix logical replication between different encodings. When sending a tuple
attribute, the previous coding erroneously sent the length byte before
encoding conversion, which would lead to protocol failures on the receiving
side if the length did not match the following string. To fix that, use
pq_sendcountedtext() for sending tuple attributes, which takes care of all of
that internally. To match the API of pq_sendcountedtext(), send even text
values without a trailing zero byte and have the receiving end put it in place
instead. This matches how the standard FE/BE protocol behaves. Reported-by:
Kyotaro HORIGUCHI <horiguchi(dot)kyotaro(at)lab(dot)ntt(dot)co(dot)jp>
- Remove use of Jade and DSSSL. All documentation is now built using XSLT.
Remove all references to Jade, DSSSL, also JadeTex and some other outdated
tooling. For chunked HTML builds, this changes nothing, but removes the
transitional "oldhtml" target. The single-page HTML build is ported over to
XSLT. For PDF builds, this removes the JadeTex builds and moves the FOP
builds in their place.
- Avoid using a C++ keyword in header file. per cpluspluscheck
- Fix compiler warning on MSVC 2010. Author: Michael Paquier
- Identity columns. This is the SQL standard-conforming variant of PostgreSQL's
serial columns. It fixes a few usability issues that serial columns have:
* CREATE TABLE / LIKE copies default but refers to same sequence * cannot
add/drop serialness with ALTER TABLE * dropping default does not drop sequence
* need to grant separate privileges to sequence * other slight weirdnesses
because serial is some kind of special macro. Reviewed-by: Vitaly Burovoy
- doc: Add SPFLAGS to osx calls. This enables the same OpenSP warnings on osx
calls that we get from onsgmls (make check) and formerly from openjade. Older
tool chains apparently have some of these warnings on by default (see comment
at SPFLAGS assignment). So users of such tool chains would complain about
warnings or errors that users of newer tool chains would not see, unless they
used "make check".
- doc: Add missing closing tag. Author: Erik Rijkers <er(at)xs4all(dot)nl>
- doc: Add some markup.
Andrew Gierth pushed:
- Try and silence spurious Coverity warning. gset_data (aka gd) in planner.c is
always non-null if and only if parse->groupingSets is non-null, but Coverity
doesn't know that and complains. Feed it an assertion to see if that keeps it
Robert Haas pushed:
- Properly acquire buffer lock for page-at-a-time hash vacuum. In a couple of
places, _hash_kill_items was mistakenly called with the buffer lock not held.
Repair. Ashutosh Sharma, per a report from Andreas Seltenreich Discussion:
- Abstract logic to allow for multiple kinds of child rels. Currently, the only
type of child relation is an "other member rel", which is the child of a
baserel, but in the future joins and even upper relations may have child rels.
To facilitate that, introduce macros that test to test for particular
RelOptKind values, and use them in various places where they help to clarify
the sense of a test. (For example, a test may allow RELOPT_OTHER_MEMBER_REL
either because it intends to allow child rels, or because it intends to allow
simple rels.) Also, remove find_childrel_top_parent, which will not work for a
child rel that is not a baserel. Instead, add a new RelOptInfo member
top_parent_relids to track the same kind of information in a more generic
manner. Ashutosh Bapat, slightly tweaked by me. Review and testing of the
patch set from which this was taken by Rajkumar Raghuwanshi and Rafia Sabih.
- Update comment. Craig Ringer, reviewed by me.
- Print new RelOptInfo field top_parent_relids in outfuncs.c. I intended to
include this adjustment in the previous commit
(7a39b5e4d11229ece930a51fd7cb29e535db44940) but messed up.
- Further corrections and improvements to partitioning documentation. Amit
- Expand hash indexes more gradually. Since hash indexes typically have very
few overflow pages, adding a new splitpoint essentially doubles the on-disk
size of the index, which can lead to large and abrupt increases in disk usage
(and perhaps long delays on occasion). To mitigate this problem to some
degree, divide larger splitpoints into four equal phases. This means that,
for example, instead of growing from 4GB to 8GB all at once, a hash index will
now grow from 4GB to 5GB to 6GB to 7GB to 8GB, which is perhaps still not as
smooth as we'd like but certainly an improvement. This changes the on-disk
format of the metapage, so bump HASH_VERSION from 2 to 3. This will force a
REINDEX of all existing hash indexes, but that's probably a good idea anyway.
First, hash indexes from pre-10 versions of PostgreSQL could easily be
corrupted, and we don't want to confuse corruption carried over from an older
release with any corruption caused despite the new write-ahead logging in v10.
Second, it will let us remove some backward-compatibility code added by commit
293e24e507838733aba4748b514536af2d39d7f2. Mithun Cy, reviewed by Amit Kapila,
Jesper Pedersen and me. Regression test outputs updated by me. Discussion:
- Fix formula in _hash_spareindex. This was correct in earlier versions of the
patch that lead to commit ea69a0dead5128c421140dc53fac165ba4af8520, but
somehow got broken in the last version which I actually committed. Mithun Cy,
per an off-list report from Ashutosh Sharma Discussion:
- Fix thinko in BitmapAdjustPrefetchIterator. Dilip Kumar Discussion:
- hash: Fix write-ahead logging bug. The size of the data is not the same thing
as the size of the size of the data. Reported off-list by Tushar Ahuja. Fix
by Ashutosh Sharma, reviewed by Amit Kapila. Discussion:
- Fix pageinspect failures on hash indexes. Make every page in a hash index
which isn't all-zeroes have a valid special space, so that tools like
pageinspect don't error out. Also, make pageinspect cope with all-zeroes
pages, because _hash_alloc_buckets can leave behind large numbers of those
until they're consumed by splits. Ashutosh Sharma and Robert Haas, reviewed
by Amit Kapila. Original trouble report from Jeff Janes. Discussion:
- Add ProcArrayGroupUpdate wait event. Discussion:
- Use English, instead of internal names, for translatable messages.
Stephen Frost pushed:
- Remove --verbose from PROVE_FLAGS. Per discussion, the TAP tests are really
more verbose than necessary, so remove the --verbose flag from PROVE_FLAGS.
Also add comments to let folks know how they can enable it if they really wish
to, as suggested by Craig Ringer. Author: Michael Paquier, additional
comments by me. Discussion:
Simon Riggs pushed:
- Speedup 2PC recovery by skipping two phase state files in normal path. 2PC
state info held in shmem at PREPARE, then cleaned at COMMIT PREPARED/ABORT
PREPARED, avoiding writing/fsyncing any state information to disk in the
normal path, greatly enhancing replay speed. Prepared transactions that live
past one checkpoint redo horizon will be written to disk as now. Similar
conceptually to 978b2f65aa1262eb4ecbf8b3785cb1b9cf4db78e and building upon the
infrastructure created by that commit. Authors, in equal measure: Stas
Kelvich, Nikhil Sontakke and Michael Paquier Discussion:
- Fix uninitialized variables in twophase.c.
- Make min_wal_size/max_wal_size use MB internally. Previously they were
defined using multiples of XLogSegSize. Remove GUC_UNIT_XSEGS. Introduce
GUC_UNIT_MB Extracted from patch series on XLogSegSize infrastructure. Beena
- Spelling mistake in comment in utility.c.
- Allow --with-wal-segsize=n up to n=1024MB. Other part of Beena Emerson's
patch to allow testing
- Collect and use multi-column dependency stats. Follow on patch in the
multi-variate statistics patch series. CREATE STATISTICS s1 WITH
(dependencies) ON (a, b) FROM t; ANALYZE; will collect dependency stats on (a,
b) and then use the measured dependency in subsequent query planning. Commit
7b504eb282ca2f5104b5c00b4f05a3ef6bb1385b added CREATE STATISTICS with
n-distinct coefficients. These are now specified using the mutually exclusive
option WITH (ndistinct). Author: Tomas Vondra, David Rowley Reviewed-by:
Kyotaro HORIGUCHI, Álvaro Herrera, Dean Rasheed, Robert Haas and many other
comments and contributions Discussion:
- Reduce lock level for CREATE STATISTICS. In line with other lock reductions
related to planning. Simon Riggs
- Avoid SnapshotResetXmin() during AtEOXact_Snapshot(). For normal commits and
aborts we already reset PgXact->xmin, so we can simply avoid running
SnapshotResetXmin() twice. During performance tests by Alexander Korotkov,
diagnosis by Andres Freund showed PgXact array as a bottleneck. After manual
analysis by me of the code paths that touch those memory locations, I was able
to identify extraneous code in the main transaction commit path. Avoiding
touching highly contented shmem improves concurrent performance slightly on
all workloads, confirmed by tests run by Ashutosh Sharma and Alexander
Korotkov. Simon Riggs Discussion:
- Always SnapshotResetXmin() during ClearTransaction(). Avoid corner cases
during 2PC with 6bad580d9e678a0b604883e14d8401d469b06566
- Reset API of clause_selectivity(). Discussion:
Andrew Dunstan pushed:
- Clarify documentation of to_tsvector(json(b)). Per gripe from Sven R. Kunze
- Make json_populate_record and friends operate recursively. With this change
array fields are populated from json(b) arrays, and composite fields are
populated from json(b) objects. Along the way, some significant code
refactoring is done to remove redundancy in the way to populate_record[_set]
and to_record[_set] functions operate, and some significant efficiency gains
are made by caching tuple descriptors. Nikita Glukhov, edited some by me.
Reviewed by Aleksander Alekseev and Tom Lane.
- Remove extraneous comma to satisfy picky compiler. per buildfarm
Andres Freund pushed:
- Force synchronous commit in new-ish test_decoding test. This was missed in
- Fix two valgrind issues in slab allocator. During allocation
VALGRIND_MAKE_MEM_DEFINED was called with a pointer as size. That kind of
works, but makes valgrind exceedingly slow for workloads involving the slab
allocator. Secondly there was an access to memory marked as unreachable
within SlabCheck(). Fix that too. Author: Tomas Vondra Discussion:
- Code review for recent slot.c changes.
- Increase parallel bitmap scan test coverage. Author: Dilip Kumar Discussion:
- Add minimal test for EXPLAIN ANALYZE of parallel query. This displays the
number of workers launched, thus the test is dependant on configuration to
some degree. We'll see whether that turns out ot be a problem. Author: Rafia
- Fix parallel bitmapscan tests on builds without USE_PREFETCH. This was broken
- Allow avoiding tuple copy within tuplesort_gettupleslot(). Add a "copy"
argument to make it optional to receive a copy of caller tuple that is safe to
use following a subsequent manipulating of tuplesort's state. This is a
performance optimization. Most existing tuplesort_gettupleslot() callers are
made to opt out of copying. Existing callers that happen to rely on the
validity of tuple memory beyond subsequent manipulations of the tuplesort
request their own copy. This brings tuplesort_gettupleslot() in line with
tuplestore_gettupleslot(). In the future, a "copy" tuplesort_getdatum()
argument may be added, that similarly allows callers to opt out of receiving
their own copy of tuple. In passing, clarify assumptions that callers of
other tuplesort fetch routines may make about tuple memory validity, per gripe
from Tom Lane. Author: Peter Geoghegan Discussion:
- Fix issues in e8fdbd58fe. When the 64bit atomics simulation is in use, we
can't necessarily guarantee the correct alignment of the atomics due to lack
of compiler support for doing so- that's fine from a safety perspective,
because everything is protected by a lock, but we asserted the alignment in
all cases. Weaken them. Per complaint from Alvaro Herrera. My #ifdefery for
PG_HAVE_8BYTE_SINGLE_COPY_ATOMICITY wasn't sufficient. Fix that. Per
complaint from Alexander Korotkov.
- Improve 64bit atomics support. When adding atomics back in b64d92f1a, I added
64bit support as optional; there wasn't yet a direct user in sight. That
turned out to be a bit short-sighted, it'd already have been useful a number
of times. Add a fallback implementation of 64bit atomics, just like the one
we have for 32bit atomics. Additionally optimize reads/writes to 64bit on a
number of platforms where aligned writes of that size are atomic. This can now
be tested with PG_HAVE_8BYTE_SINGLE_COPY_ATOMICITY. Author: Andres Freund
Reviewed-By: Amit Kapila Discussion:
Kevin Grittner pushed:
- Follow-on cleanup for the transition table patch. Commit 59702716 added
transition table support to PL/pgsql so that SQL queries in trigger functions
could access those transient tables. In order to provide the same level of
support for PL/perl, PL/python and PL/tcl, refactor the relevant code into a
new function SPI_register_trigger_data. Call the new function in the trigger
handler of all four PLs, and document it as a public SPI function so that
authors of out-of-tree PLs can do the same. Also get rid of a second
QueryEnvironment object that was maintained by PL/pgsql. That was previously
used to deal with cursors, but the same approach wasn't appropriate for PLs
that are less tangled up with core code. Instead, have SPI_cursor_open
install the connection's current QueryEnvironment, as already happens for
SPI_execute_plan. While in the docs, remove the note that transition tables
were only supported in C and PL/pgSQL triggers, and correct some ommissions.
Thomas Munro with some work by Kevin Grittner (mostly docs)
- Add isolation test for SERIALIZABLE READ ONLY DEFERRABLE. This improves code
coverage and lays a foundation for testing similar issues in a distributed
environment. Author: Thomas Munro <thomas(dot)munro(at)enterprisedb(dot)com>
Reviewed-by: Michael Paquier <michael(dot)paquier(at)gmail(dot)com>
- Fix the RTE_NAMEDTUPLESTORE case in get_rte_attribute_is_dropped(). Problems
pointed out by Andres Freund and Thomas Munro.
- Add GUCs for predicate lock promotion thresholds. Defaults match the fixed
behavior of prior releases, but now DBAs have better options to tune
serializable workloads. It might be nice to be able to set this per relation,
but that part will need to wait for another release. Author: Dagfinn Ilmari
Heikki Linnakangas pushed:
- Remove dead code and fix comments in fast-path function handling.
HandleFunctionRequest() is no longer responsible for reading the protocol
message from the client, since commit 2b3a8b20c2. Fix the outdated comments.
HandleFunctionRequest() now always returns 0, because the code that used to
return EOF was moved in 2b3a8b20c2. Therefore, the caller no longer needs to
check the return value. Reported by Andres Freund. Backpatch to all supported
versions, even though this doesn't have any user-visible effect, to make
backporting future patches in this area easier. Discussion:
- Remove bogus SCRAM_ITERATION_LEN constant. It was not used for what the
comment claimed, at all. It was actually used as the 'base' argument to
strtol(), when reading the iteration count. We don't need a constant for
base-10, so remove it.
- Remove duplicate assignment. Harmless, but clearly wrong. Kyotaro Horiguchi
- Use SASLprep to normalize passwords for SCRAM authentication. An important
step of SASLprep normalization, is to convert the string to Unicode
normalization form NFKC. Unicode normalization requires a fairly large table
of character decompositions, which is generated from data published by the
Unicode consortium. The script to generate the table is put in
src/common/unicode, as well test code for the normalization. A pre-generated
version of the tables is included in src/include/common, so you don't need the
code in src/common/unicode to build PostgreSQL, only if you wish to modify the
normalization tables. The SASLprep implementation depends on the UTF-8
functions from src/backend/utils/mb/wchar.c. So to use it, you must also
compile and link that. That doesn't change anything for the current users of
these functions, the backend and libpq, as they both already link with
wchar.o. It would be good to move those functions into a separate file in
src/commmon, but I'll leave that for another day. No documentation changes
included, because there is no details on the SCRAM mechanism in the docs
anyway. An overview on that in the protocol specification would probably be
good, even though SCRAM is documented in detail in RFC5802. I'll write that as
a separate patch. An important thing to mention there is that we apply
SASLprep even on invalid UTF-8 strings, to support other encodings. Patch by
Michael Paquier and me. Discussion:
- Fix the new SASLprep tests to work with non-UTF-8 locales. Fix by forcing
database encoding to UTF-8, regardless of the current locale. Pointed out by
Tom Lane. Discussion:
Álvaro Herrera pushed:
- Comment fixes for extended statistics. Clean up some code comments in new
extended statistics code, from 7b504eb282.
- Fix BRIN cost estimation. The original code was overly optimistic about the
cost of scanning a BRIN index, leading to BRIN indexes being selected when
they'd be a worse choice than some other index. This complete rewrite should
be more accurate. Author: David Rowley, based on an earlier patch by Emre
Hasegeli Reviewed-by: Emre Hasegeli Discussion:
- Fix new BRIN desummarize WAL record. The WAL-writing piece was forgetting to
set the pages-per-range value. Also, fix the declared type of struct member
heapBlk, which I mistakenly set as OffsetNumber rather than BlockNumber.
Problem was introduced by commit c655899ba9ae (April 1st). Any system that
tries to replay the new WAL record written before this fix is likely to die on
replay and require pg_resetwal. Reported by Tom Lane. Discussion:
- Reduce the number of pallocs() in BRIN. Instead of allocating memory in
brin_deform_tuple and brin_copy_tuple over and over during a scan, allow reuse
of previously allocated memory. This is said to make for a measurable
performance improvement. Author: Jinyu Zhang, Álvaro Herrera Reviewed by:
Tomas Vondra Discussion:
- Fix printf format to use %zd when printing sizes. Using %ld as we were doing
raises compiler warnings on 32 bit platforms. Reported by Andres Freund.
Joe Conway pushed:
- Silence compiler warning in sepgsql. <selinux/label.h> includes <stdbool.h>,
which creates an incompatible We don't care if <stdbool.h> redefines
"true"/"false"; those are close enough. Complaint and initial patch by Mike
Palmiotto. Final approach per Tom Lane's suggestion, as discussed on hackers.
Backpatching to all supported branches. Discussion:
- Silence uninitialized variable compiler warning in sepgsql. At -Og
optimization gcc warns that variable tclass may be used uninitialized when
relkind == RELKIND_INDEX. Actually that can't happen due to an early return,
but quiet the compiler by initializing tclass to 0. In passing, use uint16_t
consistently for the declaration of tclass. Complaint and initial patch by
Mike Palmiotto. Editorializing by me. Probably not worth backpatching given
that it is cosmetic, so apply to development head only. Discussion:
Magnus Hagander pushed:
- Fix typo in comment. Masahiko Sawada
== Pending Patches ==
Vaishnavi Prabakaran sent in two more revisions of a patch to add
batch/pipelining support to libpq.
Craig Ringer sent in five more revisions of a patch to add logical decoding on
Ashutosh Bapat and Amit Langote traded patches to document some intricacies of
the relationship between declarative partitions and foreign tables.
Takayuki Tsunakawa sent in three more revisions of a patch to support huge pages
Amit Khandekar sent in two more revisions of a patch to enable UPDATEs on
partitioned tables which would cause rows to move among partitions to work.
Rushabh Lathia sent in a patch to add coverage tests for gather merge.
Amit Langote sent in a patch to enable ON CONFLICT DO NOTHING on partitioned
Haribabu Kommi sent in another revision of a patch to refactor handling of
database attributes between pg_dump and pg_dumpall.
Alexander Korotkov sent in another revision of a patch to implement incremental
Daniel Gustafsson sent in a patch to use strcmp() instead of pg_strcasecmp() for
Kyotaro HORIGUCHI sent in a patch to distinguish "aggressive" VACUUMs from non-
Vinayak Pokale sent in another revision of a patch to add an ANALYZE progress
Tatsuo Ishii and Andres Freund traded patches to rearm statement_timeout after
each executed query.
Michaël Paquier sent in two revisions of a patch to rewrite the test of
pg_upgrade as a TAP test.
Kuntal Ghosh sent in two revisions of a patch to fix parallel worker counts
after a crash.
Amit Langote sent in a patch to remove pg_stat_progress_vacuum from Table 28.2.
David Rowley sent in three more revisions of a patch to make clausesel smarter.
Antonin Houska sent in another revision of a patch to implement aggregation
Kyotaro HORIGUCHI sent in another revision of a patch to implement asynchronous
Maksim Milyutin sent in a patch to create a "local" index for partitioned
Fabien COELHO sent in a patch to add a special variable in psql to reflect the
last query status.
Ashutosh Bapat sent in two more revisions of a patch to implement partition-wise
joins for declaratively partitioned tables.
Rahila Syed sent in three more revisions of a patch to add support for a default
partition in declarative partitions.
Craig Ringer and Jim Nasby traded patches to add SPI_execute_callback and
documentation for same, then use it in PL/PythonU to speed up SPI results.
Dmitry Dolgov sent in two more revisions of a patch to implement generic type
Amit Khandekar sent in another revision of a patch to implement parallel append.
Etsuro Fujita sent in another revision of a patch to fix a bug in the PostgreSQL
Pavan Deolasee sent in two more revisions of a patch to implement WARM.
Mark Dilger sent in a patch to create a generic PG_GETARG_*_P infrastructure.
Robert Haas sent in another revision of a patch to add pgstathashindex() to get
hash index table statistics.
Etsuro Fujita sent in another revision of a patch to support parameterized
Fabien COELHO sent in three more revisions of a patch to enable pgbench to store
select results into variables.
Jeff Davis sent in a patch to implement range merge join.
Ashutosh Bapat sent in a patch to implement constraint exclusion for partitioned
Kyotaro HORIGUCHI sent in a patch to ensure that enabling subscription starts a
Michaël Paquier sent in a patch to use base64-based encoding for stored and
server keys in SCRAM verifiers, move the routine to build the SCRAM verifier
into src/common/, refactor frontend-side random number generation, extend
PQencryptPassword with a hashing method, and extend psql's \password and
createuser to handle SCRAM verifier creation.
Tatsuro Yamada sent in a patch to make postgresGetForeignPlan use foreignrel
instead of baserel.
Thomas Munro sent in a patch to implement hash tables in dynamic shared memory.
Thomas Munro sent in a PoC patch to enable sharing record typmods between
Vitaly Burovoy sent in a patch to implement SET IDENTITY ... IF NOT EXISTS.
Kyotaro HORIGUCHI sent in a patch to fix distclean of
Yorick Peterse sent in two revisions of a patch to update the hot-standby
documentation (in the high availability section) so it explicitly mentions that
certain settings need to be applied to servers in a particular order.
Beena Emerson sent in a patch to update the initdb regression tests to include
increasing the default WAL segment size.
Aleksander Alekseev sent in another revision of a patch to remove an unused
argument in btree_xlog_split.
Tatsuo Ishii sent in a patch to document the fact that pg_export_snapshot()
cannot be used during recovery (i.e. on standby servers).
Aleksander Alekseev sent in a patch to warn users about duplicate configuration
parameters in postgresql.conf.
Álvaro Herrera sent in another revision of a patch to fix wal_level_minimal.
Thomas Munro sent in a patch to initialise a freed segment counter in dsa.c.
Claudio Freire sent in another revision of a patch to allow VACUUM to use more
than 1GB work mem in, and make it free dead tuples array as early as possible.
Thomas Munro sent in a patch to fix and document some snapshot issues.
Thomas Munro sent in a patch to add a pg_waiting_for_safe_snapshot() function.
Tom Lane and Mark Dilger traded patches to use IsA checks for bitmapsets.
Michael Banck sent in another revision of a patch to add an option to create a
replication slot in pg_basebackup if not yet present.
pgsql-announce by date
|Next:||From: Martín Marqués||Date: 2017-04-10 12:39:59|
|Subject: PgDay Argentina 2017 in Santa Fe - Call For Papers is Open|
|Previous:||From: David Fetter||Date: 2017-04-02 22:50:15|
|Subject: == PostgreSQL Weekly News - April 02 2017 ==|