== PostgreSQL Weekly News - November 15 2015 ==

== PostgreSQL Weekly News - November 15 2015 ==

PostgreSQL 9.5 Beta 2 released. Test!
http://www.postgresql.org/about/news/1625/

== PostgreSQL Product News ==

pg_wintools, a PostgreSQL extension for interfacing with Windows OS,
released.
https://github.com/jmguazzo/pg_wintools

== PostgreSQL Jobs for November ==

http://archives.postgresql.org/pgsql-jobs/2015-11/

== PostgreSQL Local ==

PGConf Silicon Valley 2015 is November 17-18 at the South San
Francisco Convention Center.
http://www.pgconfsv.com

PGBR2015 (The Brazilian PostgreSQL Conference) will take place in Porto
Alegre, Rio Grande do Sul, on November 18, 19 and 20. Registration is
open.
http://pgbr.postgresql.org.br/2015/en/

Postgres Conference China 2015 will be on November 20-22, 2015 in
Beijing.
http://postgres2015.eventdove.com/ (Chinese)
http://postgrescluster2015.eventdove.com/ (English)

PGConf.DE will be held on November 26-27, 2015 in Hamburg, Germany, at
the Lindner Hotel am Michel.
http://2015.pgconf.de/

PostgreSQL(at)SCaLE is a two day, two track event which takes place on Jan.
21-22, 2016, at Pasadena Convention Center, as part of SCaLE 14X.
https://www.socallinuxexpo.org/scale/14x/cfp

== PostgreSQL in the News ==

Planet PostgreSQL: http://planet.postgresql.org/

PostgreSQL Weekly News is brought to you this week by David Fetter

Submit news and announcements by Sunday at 3:00pm Pacific time.
Please send English language ones to david(at)fetter(dot)org, German language
to pwn(at)pgug(dot)de, Italian language to pwn(at)itpug(dot)org(dot) Spanish language
to pwn(at)arpug(dot)com(dot)ar(dot)

== Applied Patches ==

Andres Freund pushed:

- Add paragraph about ON CONFLICT interaction with partitioning.
Author: Peter Geoghegan and Andres Freund Discussion:
CAM3SWZScpWzQ-7EJC77vwqzZ1GO8GNmURQ1QqDQ3wRn7AbW1Cg(at)mail(dot)gmail(dot)com,
CAHGQGwFUCWwSU7dtc2aRdRk73ztyr_jY5cPOyts+K8xKJ92X4Q(at)mail(dot)gmail(dot)com
Backpatch: 9.5, where UPSERT was introduced
http://git.postgresql.org/pg/commitdiff/c31f1dc55992bc8b2c35c0ff99795547d4df323a

Robert Haas pushed:

- Add a dummy return statement to TupleQueueRemap. This is
unreachable for multiple reasons, but per Amit Kapila the Windows
compiler he is using still thinks we can get there.
http://git.postgresql.org/pg/commitdiff/89ff5c7f7512a919e76407b4f8269e5b0b39dd2b

- Fix rebasing mistake in nodeGather.c The patches committed as
6e71dd7ce9766582da453f493bc371d64977282f and
3a1f8611f2582df0a16bcd35caed2e1526387643 were developed in parallel
but dependent on each other in a way that I failed to notice. This
patch to fix the problem was prepared by Amit Kapila.
http://git.postgresql.org/pg/commitdiff/bf3d0156311f0d8fa7705d8de0d07c4e0298051a

- Comment update. Adjust to account for
5fc4c26db5120bd90348b6ee3101fcddfdf54800. Etsuro Fujita
http://git.postgresql.org/pg/commitdiff/5c90a2ffddd2504a7441e80a22ba0fe8d484291a

- Add outfuncs.c support for GatherPath. I dunno how commit
3bd909b220930f21d6e15833a17947be749e7fde missed this, but it
evidently did.
http://git.postgresql.org/pg/commitdiff/f764ecd81b2a8a1e9000d43a73ca5eec8e8008bc

- Make sequential scans parallel-aware. In addition, this path fills
in a number of missing bits and pieces in the parallel
infrastructure. Paths and plans now have a parallel_aware flag
indicating whether whatever parallel-aware logic they have should be
engaged. It is believed that we will need this flag for a number of
path/plan types, not just sequential scans, which is why the flag is
generic rather than part of the SeqScan structures specifically.
Also, execParallel.c now gives parallel nodes a chance to initialize
their PlanState nodes from the DSM during parallel worker startup.
Amit Kapila, with a fair amount of adjustment by me. Review of
previous patch versions by Haribabu Kommi and others.
http://git.postgresql.org/pg/commitdiff/f0661c4e8c44c0ec7acd4ea7c82e85b265447398

- Generate parallel sequential scan plans in simple cases. Add a new
flag, consider_parallel, to each RelOptInfo, indicating whether a
plan for that relation could conceivably be run inside of a parallel
worker. Right now, we're pretty conservative: for example, it might
be possible to defer applying a parallel-restricted qual in a
worker, and later do it in the leader, but right now we just don't
try to parallelize access to that relation. That's probably the
right decision in most cases, anyway. Using the new flag, generate
parallel sequential scan plans for plain baserels, meaning that we
now have parallel sequential scan in PostgreSQL. The logic here is
pretty unsophisticated right now: the costing model probably isn't
right in detail, and we can't push joins beneath Gather nodes, so
the number of plans that can actually benefit from this is pretty
limited right now. Lots more work is needed. Nevertheless, it
seems time to enable this functionality so that all this code can
actually be tested easily by users and developers. Note that, if
you wish to test this functionality, it will be necessary to set
max_parallel_degree to a value greater than the default of 0. Once
a few more loose ends have been tidied up here, we might want to
consider changing the default value of this GUC, but I'm leaving it
alone for now. Along the way, fix a bug in cost_gather: the
previous coding thought that a Gather node's transfer overhead
should be costed on the basis of the relation size rather than the
number of tuples that actually need to be passed off to the leader.
Patch by me, reviewed in earlier versions by Amit Kapila.
http://git.postgresql.org/pg/commitdiff/80558c1f5aa109d08db0fbd76a6d370f900628a8

- Provide readfuncs support for custom scans. Commit
a0d9f6e434bb56f7e5441b7988f3982feead33b3 added this support for all
other plan node types; this fills in the gap. Since
TextOutCustomScan complicates this and is pretty well useless,
remove it. KaiGai Kohei, with some modifications by me.
http://git.postgresql.org/pg/commitdiff/a05dc4d7fd57d4ae084c1f0801973e5c1a1aa26e

- Make idle backends exit if the postmaster dies. Letting backends
continue to run if the postmaster has exited prevents PostgreSQL
from being restarted, which in many environments is catastrophic.
Worse, if some other backend crashes, we no longer have any
protection against shared memory corruption. So, arrange for them
to exit instead. We don't want to expend many cycles on this, but
including postmaster death in the set of things that we wait for
when a backend is idle seems cheap enough. Rajeev Rastogi and
Robert Haas
http://git.postgresql.org/pg/commitdiff/ac1d7945f866b1928c2554c0f80fd52d7f977772

- Move each SLRU's lwlocks to a separate tranche. This makes it
significantly easier to identify these lwlocks in LWLOCK_STATS or
Trace_lwlocks output. It's also arguably better from a modularity
standpoint, since lwlock.c no longer needs to know anything about
the LWLock needs of the higher-level SLRU facility. Ildus
Kurbangaliev, reviewd by Álvaro Herrera and by me.
http://git.postgresql.org/pg/commitdiff/fe702a7b3f9f2bc5bf6d173166d7d55226af82c8

- libpq: Notice errors a backend may have sent just before dying. At
least since the introduction of Hot Standby, the backend has
sometimes sent fatal errors even when no client query was in
progress, assuming that the client would receive it. However,
pqHandleSendFailure was not in sync with this assumption, and only
tries to catch notices and notifies. Add a parseInput call to the
loop there to fix. Andres Freund suggested the fix. Comments are
by me. Reviewed by Michael Paquier.
http://git.postgresql.org/pg/commitdiff/c3e7c24a1d60dc6ad56e2a0723399f1570c54224

- Remove accidentally-committed debugging code. Amit Kapila
http://git.postgresql.org/pg/commitdiff/179c97bf587df710f98aa3ce2d95f503b0757e15

Tom Lane pushed:

- Improve our workaround for 'TeX capacity exceeded' in building PDF
files. In commit a5ec86a7c787832d28d5e50400ec96a5190f2555 I wrote a
quick hack that reduced the number of TeX string pool entries
created while converting our documentation to PDF form. That held
the fort for awhile, but as of HEAD we're back up against the same
limitation. It turns out that the original coding of
\FlowObjectSetup actually results in *three* string pool entries
being generated for every "flow object" (that is, potential
cross-reference target) in the documentation, and my previous hack
only got rid of one of them. With a little more care, we can reduce
the string count to one per flow object plus one per
actually-cross-referenced flow object (about 115000 + 5000 as of
current HEAD); that should work until the documentation volume
roughly doubles from where it is today. As a not-incidental side
benefit, this change also causes pdfjadetex to stop emitting
unreferenced hyperlink anchors (bookmarks) into the PDF file. It
had been making one willy-nilly for every flow object; now it's just
one per actually-cross-referenced object. This results in close to
a 2X savings in PDF file size. We will still want to run the output
through "jpdftweak" to get it to be compressed; but we no longer
need removal of unreferenced bookmarks, so we might be able to find
a quicker tool for that step. Although the failure only affects
HEAD and US-format output at the moment, 9.5 cannot be more than a
few pages short of failing likewise, so it will inevitably fail
after a few rounds of minor-version release notes. I don't have a
lot of faith that we'll never hit the limit in the older branches;
and anyway it would be nice to get rid of jpdftweak across the
board. Therefore, back-patch to all supported branches.
http://git.postgresql.org/pg/commitdiff/944b41fc00600b74f518005ac314cc222bf6abd5

- Add missing "static" qualifier. Per buildfarm member pademelon.
http://git.postgresql.org/pg/commitdiff/b05ae27e9a57a1c8f1f44720c23c4828106b7205

- Docs: fix misleading example. Commit 8457d0beca731bf0 introduced an
example which, while not incorrect, failed to exhibit the behavior
it meant to describe, as a result of omitting an E'' prefix that
needed to be there. Noticed and fixed by Peter Geoghegan. I (tgl)
failed to resist the temptation to wordsmith nearby text a bit while
at it.
http://git.postgresql.org/pg/commitdiff/7b6fb76349fccc38f0ef96166f207c6acd21968c

- Be more noisy about "wrong number of nailed relations" initfile
problems. In commit 5d1ff6bd559ea8df1b7302e245e690b01b9a4fa4 I
added some logic to relcache.c to try to ensure that the regression
tests would fail if we made a mistake about which relations belong
in the relcache init files. I'm quite sure I tested that, but I
must have done so only for the non-shared-catalog case, because a
report from Adam Brightwell showed that the regression tests still
pass just fine if we bollix the shared-catalog init file in the way
this code was supposed to catch. The reason is that that file gets
loaded before we do client authentication, so the WARNING is not
sent to the client, only to the postmaster log, where it's far too
easily missed. The least Rube Goldbergian answer to this is to put
an Assert(false) after the elog(WARNING). That will certainly get
developers' attention, while not breaking production builds' ability
to recover from corner cases with similar symptoms. Since this is
only of interest to developers, there seems no need for a
back-patch, even though the previous commit went into all branches.
http://git.postgresql.org/pg/commitdiff/da3751c8ea6f833d8e69fbdd168d29d328674398

- Improve documentation around autovacuum-related storage parameters.
These were discussed in three different sections of the manual,
which unsurprisingly had diverged over time; and the descriptions of
individual variables lacked stylistic consistency even within each
section (and frequently weren't in very good English anyway). Clean
up the mess, and remove some of the redundant information in hopes
that future additions will be less likely to re-introduce
inconsistency. For instance I see no need for maintenance.sgml to
include its very own list of all the autovacuum storage parameters,
especially since that list was already incomplete.
http://git.postgresql.org/pg/commitdiff/6404751ce91fb02999e46700a1ac62b3c6319abf

- Do a round of copy-editing on the 9.5 release notes. Also fill in
the previously empty "major enhancements" list. YMMV as to which
items should make the cut, but it's past time we had something more
than a placeholder here. (I meant to get this done before beta2 was
wrapped, but got distracted by PDF build problems. Better late than
never.)
http://git.postgresql.org/pg/commitdiff/39b9978d9cac34ad799a5fa3ff3846f3e0372b0a

- Fix unwanted flushing of libpq's input buffer when socket EOF is
seen. In commit 210eb9b743c0645d I centralized libpq's logic for
closing down the backend communication socket, and made the new
pqDropConnection routine always reset the I/O buffers to empty.
Many of the call sites previously had not had such code, and while
that amounted to an oversight in some cases, there was one place
where it was intentional and necessary *not* to flush the input
buffer: pqReadData should never cause that to happen, since we
probably still want to process whatever data we read. This is the
true cause of the problem Robert was attempting to fix in
c3e7c24a1d60dc6a, namely that libpq no longer reported the backend's
final ERROR message before reporting "server closed the connection
unexpectedly". But that only accidentally fixed it, by invoking
parseInput before the input buffer got flushed; and very likely
there are timing scenarios where we'd still lose the message before
processing it. To fix, pass a flag to pqDropConnection to tell it
whether to flush the input buffer or not. On review I think
flushing is actually correct for every other call site. Back-patch
to 9.3 where the problem was introduced. In HEAD, also improve the
comments added by c3e7c24a1d60dc6a.
http://git.postgresql.org/pg/commitdiff/c405918858c09d4264ffc5a3c73d57dc3efb0213

- Improve type numeric's calculations for ln(), log(), exp(), pow().
Set the "rscales" for intermediate-result calculations to ensure
that suitable numbers of significant digits are maintained
throughout. The previous coding hadn't thought this through in any
detail, and as a result could deliver results with many inaccurate
digits, or in the worst cases even fail with divide-by-zero errors
as a result of losing all nonzero digits of intermediate results.
In exp_var(), get rid entirely of the logic that separated the
calculation into integer and fractional parts: that was neither
accurate nor particularly fast. The existing range-reduction method
of dividing by 2^n can be applied across the full input range
instead of only 0..1, as long as we are careful to set an
appropriate rscale for each step. Also fix the logic in mul_var()
for shortening the calculation when the caller asks for fewer output
digits than an exact calculation would require. This bug doesn't
affect simple multiplications since that code path asks for an exact
result, but it does contribute to accuracy issues in the
transcendental math functions. In passing, improve performance of
mul_var() a bit by forcing the shorter input to be on the left, thus
reducing the number of iterations of the outer loop and probably
also reducing the number of carry-propagation steps needed. This is
arguably a bug fix, but in view of the lack of field complaints, it
does not seem worth the risk of back-patching. Dean Rasheed
http://git.postgresql.org/pg/commitdiff/7d9a4737c268f61fb8800957631f12d3f13be218

- Fix ruleutils.c's dumping of whole-row Vars in ROW() and VALUES()
contexts. Normally ruleutils prints a whole-row Var as "foo.*". We
already knew that that doesn't work at top level of a SELECT list,
because the parser would treat the "*" as a directive to expand the
reference into separate columns, not a whole-row Var. However,
Joshua Yanovski points out in bug #13776 that the same thing happens
at top level of a ROW() construct; and some nosing around in the
parser shows that the same is true in VALUES(). Hence, apply the
same workaround already devised for the SELECT-list case, namely to
add a forced cast to the appropriate rowtype in these cases. (The
alternative of just printing "foo" was rejected because it is
difficult to avoid ambiguity against plain columns named "foo".)
Back-patch to all supported branches.
http://git.postgresql.org/pg/commitdiff/7745bc352a82bd588be986479c7aabc3b076a375

Álvaro Herrera pushed:

- vacuumdb: don't prompt for passwords over and over. Having the
script prompt for passwords over and over was a preexisting problem
when it processed multiple databases or when it processed multiple
analyze stages, but the parallel mode introduced in commit
a179232047 made it worse. Fix the annoyance by keeping a copy of
the password used by the first connection that requires one. Since
users can (currently) only have a single password, there's no need
for more complex arrangements (such as remembering one password per
database). Per bug #13741 reported by Eric Brown. Patch authored
and cross-reviewed by Haribabu Kommi and Michael Paquier, slightly
tweaked by Álvaro Herrera. Discussion:
http://www.postgresql.org/message-id/20151027193919.931.54948@wrigleys.postgresql.org
Backpatch to 9.5, where parallel vacuumdb was introduced.
http://git.postgresql.org/pg/commitdiff/83dec5a712af251af15effbf781ddaedc3bf6b3b

Stephen Frost pushed:

- Correct sepgsql docs with regard to RLS The sepgsql docs included a
comment that PG doesn't support RLS. That is only true for versions
prior to 9.5. Update the docs for 9.5 and master to say that PG
supports RLS but that sepgsql does not yet. Pointed out by Heikki.
Back-patch to 9.5
http://git.postgresql.org/pg/commitdiff/42aa1c032e3520d608a1e89abbf36409b022d4b7

Bruce Momjian pushed:

- pg_upgrade: properly detect file copy failure on Windows.
Previously, file copy failures were ignored on Windows due to an
incorrect return value check. Report by Manu Joye Backpatch through
9.1
http://git.postgresql.org/pg/commitdiff/025106e314627f5aacdf72fca52ba0ed985d696e

- Fix spelling error in postgresql.conf. Report by Greg Clough
http://git.postgresql.org/pg/commitdiff/e57646e96284d4bad79f87a5c1dfbcd8f5434020

== Rejected Patches (for now) ==

No one was disappointed this week :-)

== Pending Patches ==

Adam Brightwell sent in two revisions of a patch to bootstrap
pg_shseclabel in relcache initialization.

Kaigai Kouhei sent in another revision of a patch to reconcile Foreign
join pushdown with EvalPlanQual.

Ashutosh Bapat sent in another revision of a patch to add a
transaction manager for foreign servers.

Marco Nenciarini sent in a patch to fix an incompatibility between
9.3's WAL stream and pg_receivelog.

Kaigai Kouhei sent in a patch for each of 9.5 and git master to allow
CustomScan on copyfuncs.

Pavel Stěhule sent in another revision of a patch to allow psql to
send multiple SQL statements via the command line.

Pavel Stěhule and Catalin Iacob traded patches to add an ereport
function to PL/PythonU.

Artur Zakirov sent in another revision of a patch to enhance the
performance of Hunspell dictionaries in full-text search.

Peter Geoghegan sent in a patch to fix some erroneous documentation of
char(n).

Michael Paquier sent in another revision of a patch to add
trigonometric functions which take degree values as input.

Andres Freund sent in another revision of a patch to control
continuous flushing of checkpoints.

Thomas Munro sent in two revisions of a patch to create a "causal
reads" capability.

Peter Eisentraut sent in another revision of a patch to handle missing
pieces of compilation infrastructure more gracefully.

Michael Paquier sent in another revision of a patch to show tuple data
in pageinspect.

Craig Ringer sent in a patch to implement pg_logical, a plugin based
on logical WAL streams.

Kyotaro HORIGUCHI sent in a patch to allow using regexes in frontend
code, in particular tab completion for psql.

Thomas Munro sent in another revision of a patch to simplify tab
completion in psql.

Alexander Korotkov sent in another revision of a patch to rework the
access method interface.

Peter Geoghegan sent in a patch to expand SortSupport to char(n),
bytea, and alternative opclasses.

Kaigai Kouhei sent in a patch to add serialization support to
CustomScan.

SAWADA Masahiko sent in two more revisions of a patch to add a
"frozen" bit to the visibility map.

Amit Kapila sent in a patch to fix an issue in the parallel seq scan
code.

Robert Haas sent in a patch to implement Append in parallel.

Jeff Janes and Tom Lane traded patches to speed up set_rtable_names.

Amit Kapila sent in a patch to fix some issues in parallel seq scan.

Robert Haas sent in a patch to tranche-ify buffer locks.

Haribabu Kommi sent in another revision of a patch to skip ALTER x SET
SCHEMA if the schema didn't change.