== PostgreSQL Weekly News - May 13 2012 ==
PostgreSQL Session will be held on October 4th, 2012, in Paris,
France. More information at:
== PostgreSQL Product News ==
AnySQL Maestro 12.5, an ODBC-based management tool which works with
pgXplorer, a GUI for postgres 8.4+, released.
The code is open sourced under the ISC license and can be found on github at:
Postgres-XC 1.0beta2, a write-scalable multi-master symmetric cluster
based on PostgreSQL, released.
PL/Proxy 2.4, a database partitioning system implemented as a PL, released.
SIDU 3.5 for PostgreSQL, a web-based tool for database administration, released.
Version 1.25.1 of tail_n_mail, a Postgres log watcher program, released:
== PostgreSQL Jobs for May ==
== PostgreSQL Local ==
PGCon 2012 will be held 17-18 May 2012, in Ottawa at the University of
Ottawa. It will be preceded by two days of tutorials on 15-16 May 2012.
PGDay France will be in Lyon on June 7, 2012.
PostgreSQL conference China 2012 will be on June 14-17, 2012 in Beijing.
PostgreSQL Conference Europe 2012 will be in Prague, Czech Republic
on October 23-26. The call for sponsors is open.
== PostgreSQL in the News ==
Planet PostgreSQL: http://planet.postgresql.org/
PostgreSQL Weekly News is brought to you this week by David Fetter
Submit news and announcements by Sunday at 3:00pm Pacific time.
Please send English language ones to david(at)fetter(dot)org, German language
to pwn(at)pgug(dot)de, Italian language to pwn(at)itpug(dot)org(dot) Spanish language
== Reviews ==
== Applied Patches ==
Magnus Hagander pushed:
- Make "unexpected EOF" messages DEBUG1 unless in an open transaction.
"Unexpected EOF on client connection" without an open transaction is
mostly noise, so turn it into DEBUG1. With an open transaction it's
still indicating a problem, so keep those as ERROR, and change the
message to indicate that it happened in a transaction.
- Only attempt to show collations on servers >= 9.1. Show a proper
error message instead of a SQL error. Josh Kupershmidt
Peter Eisentraut pushed:
- Fix pg_config.h make rule. According to the Autoconf documentation,
there should be a make rule pg_config.h: stamp-h so that with the
right setup around this, a change in pg_config.h.in will trigger a
rebuild of everything that depends on pg_config.h. But this doesn't
always work, sometimes you need to run make twice to get everything
up to date after a change of pg_config.h.in. The fix is to write
the rule as pg_config.h: stamp-h ; instead (with an empty command
instead of no command). This is what Automake-generated makefiles
effectively do, so it seems safe to be on this side. It's not
actually clear why this is (apparently) more correct. It's been
without response so far.
- Remove strdup, strtol, strtoul from libpgport. These should not be
needed anymore, at least after the recent port removals. So let's
see whether we can do without them.
- Fix misleading comments. Josh Kupershmidt
- Fix dependency tracking for src/port/%_srv.o files. Because they
use their own compilation rule, they don't use the dependency
tracking logic from Makefile.global. To make sure that dependency
tracking works anyway for the *_srv.o files, depend on their *.o
siblings as well, which do have proper dependencies. It's a hack
that might fail someday if there is a *_srv.o without a
corresponding *.o, but it works for now (and those would probably go
into src/backend/port/ anyway).
- Update config.guess and config.sub
- psql: Add variable to control keyword case in tab completion. This
adds the variable COMP_KEYWORD_CASE, which controls in what case
keywords are completed. This is partially to let users configure
the change from commit 69f4f1c3576abc535871c6cfa95539e32a36120f, but
it also offers more behaviors than were available before.
- Tweak contrib --help output to match common style. Placeholders
such as OPTION are typically kept in singular.
- Update documentation to say that gmake check in contrib is supported
- Split contrib documentation into extensions and programs. Create
separate appendixes for contrib extensions and other server plugins
on the one hand, and utility programs on the other. Recast the
documentation of the latter as refentries, so that man pages are
- Add make dependency so that postgres.bki is rebuilt in major version
change. Every time since the current rule for postgres.bki was put
in place when we change the major version, people complain that
their tests fail in strange ways. This is because the version
number in postgres.bki is not updated, because it has no dependency
for that. And you can't even force the rebuild manually if you
don't happen to know which file has the problem. Fix that now
before it will happen again. The only remaining problem with
switching major versions, as far as the regression tests are
concerned, is that contrib needs to be rebuilt. But that's easily
invoked, and in any case the failure modes are more friendly if you
- Python 2.2 is no longer supported. It was already on its last legs,
and it turns out that it was accidentally broken in commit
89e850e6fda9e4e441712012abe971fe938d595a and no one cared. So
remove the rest the support for it and update the documentation to
indicate that Python 2.3 is now required.
- Whitespace cleanup
- PL/Python: Fix slicing support for result objects for Python 3. The
old way of implementing slicing support by implementing
PySequenceMethods.sq_slice no longer works in Python 3. You now
have to implement PyMappingMethods.mp_subscript. Do this by simply
proxying the call to the wrapped list of result dictionaries.
Consolidate some of the subscripting regression tests. Jan Urbański
- PL/Python: Update incorrect comment. Jan Urbański
- Properly capitalize the TimeZone parameter. For better or worse, it
is implemented in mixed case, so document it as such, at least in
the main listing, like DateStyle. Josh Kupershmidt
- PL/Python: Adjust the regression tests for Python 3.3. The string
representation of ImportError changed. Remove printing that; it's
not necessary for the test. The order in which members of a dict
are printed changed. But this was always implementation-dependent,
so we have just been lucky for a long time. Do the printing the
hard way to ensure sorted order.
- Fix obsolescent C declaration syntax. gcc
-Wextra/-Wold-style-declaration thinks that "inline" should go
before the function return type.
- Remove unused AC_SUBST variables. These were apparently never used.
The AC_SUBST was probably just added in a copy-and-paste manner.
(The shell variables continue to be used inside configure. The
change is just that we don't need them outside of configure.)
- pg_ctl: Improve --help output. All other --help output has = signs
between long options and their arguments, so do it here as well.
- Small punctuation editing of postgresql.conf.sample
- Remove leftovers of BeOS port. These should have been removed when
the BeOS port was removed in
Tom Lane pushed:
- Reduce idle power consumption of walwriter and checkpointer
processes. This patch modifies the walwriter process so that, when
it has not found anything useful to do for many consecutive wakeup
cycles, it extends its sleep time to reduce the server's idle power
consumption. It reverts to normal as soon as it's done any
successful flushes. It's still true that during any async commit,
backends check for completed, unflushed pages of WAL and signal the
walwriter if there are any; so that in practice the walwriter can
get awakened and returned to normal operation sooner than the sleep
time might suggest. Also, improve the checkpointer so that it uses
a latch and a computed delay time to not wake up at all except when
it has something to do, replacing a previous hardcoded 0.5 sec
wakeup cycle. This also is primarily useful for reducing the
server's power consumption when idle. In passing, get rid of the
dedicated latch for signaling the walwriter in favor of using its
procLatch, since that comports better with possible generic signal
handlers using that latch. Also, fix a pre-existing bug with
failure to save/restore errno in walwriter's signal handlers. Peter
Geoghegan, somewhat simplified by Tom
- Reduce idle power consumption of stats collector process. Latch-ify
the stats collector, so that it does not need an arbitrary wakeup
cycle to check for postmaster death. The incremental savings in
idle power is pretty marginal, since we only had it waking every two
seconds; but I believe that this patch may also improve the
collector's performance under load, by reducing the number of kernel
calls made per message when messages are arriving constantly (we now
avoid a select/poll call except when we need to sleep). The change
also reduces the time needed for a normal database shutdown on
platforms where signals don't interrupt select().
- Fix an issue in recent walwriter hibernation patch. Users of
asynchronous-commit mode expect there to be a guaranteed maximum
delay before an async commit's WAL records get flushed to disk. The
original version of the walwriter hibernation patch broke that. Add
an extra shared-memory flag to allow async commits to kick the
walwriter out of hibernation mode, without adding any noticeable
overhead in cases where no action is needed.
- Improve control logic for bgwriter hibernation mode. Commit
6d90eaaa89a007e0d365f49d6436f35d2392cfeb added a hibernation mode to
the bgwriter to reduce the server's idle-power consumption.
However, its interaction with the detailed behavior of
BgBufferSync's feedback control loop wasn't very well thought out.
That control loop depends primarily on the rate of buffer
allocation, not the rate of buffer dirtying, so the hibernation mode
has to be designed to operate only when no new buffer allocations
are happening. Also, the check for whether the system is
effectively idle was not quite right and would fail to detect a
constant low level of activity, thus allowing the bgwriter to go
into hibernation mode in a way that would let the cycle time vary
quite a bit, possibly further confusing the feedback loop. To fix,
move the wakeup support from MarkBufferDirty and
SetBufferCommitInfoNeedsSave into StrategyGetBuffer, and prevent the
bgwriter from entering hibernation mode unless no buffer allocations
have happened recently. In addition, fix the delaying logic to
remove the problem of possibly not responding to signals promptly,
which was basically caused by trying to use the process latch's
is_set flag for multiple purposes. I can't prove it but I'm
suspicious that that hack was responsible for the intermittent
"postmaster does not shut down" failures we've been seeing in the
buildfarm lately. In any case it did nothing to improve the
readability or robustness of the code. In passing, express the
hibernation sleep time as a multiplier on BgWriterDelay, not a
constant. I'm not sure whether there's any value in exposing the
longer sleep time as an independently configurable setting, but we
can at least make it act like this for little extra code.
- Further tweaking of nomenclature in checkpointer.c. Get rid of some
more naming choices that only make sense if you know that this code
used to be in the bgwriter, as well as some stray comments
referencing the bgwriter.
- Improve tests for postmaster death in auxiliary processes. In
checkpointer and walwriter, avoid calling PostmasterIsAlive unless
WaitLatch has reported WL_POSTMASTER_DEATH. This saves a kernel
call per iteration of the process's outer loop, which is not all
that much, but a cycle shaved is a cycle earned. I had already
removed the unconditional PostmasterIsAlive calls in bgwriter and
pgstat in previous patches, but forgot that WL_POSTMASTER_DEATH is
supposed to be treated as untrustworthy (per comment in
unix_latch.c); so adjust those two cases to match. There are a few
other places where the same idea might be applied, but only after
substantial code rearrangement, so I didn't bother.
- Remove unportable use of SGML character-code entity. It'd be nice
to be able to spell Jan Urbanski's name with the correct accent
marks, but we haven't yet found a way that works in everybody's docs
toolchain. This way definitely doesn't.
- Improve Windows implementation of WaitLatch/WaitLatchOrSocket.
Ensure that signal handlers are serviced before this function
returns. This should make the behavior more like Unix. Also, add
some more error checking, and make some other cosmetic improvements.
No back-patch since it's not clear whether this is fixing any live
bug that would affect 9.1. I'm more concerned about 9.2 anyway
given our considerable recent expansions in the usage of WaitLatch.
- Fix Windows implementation of PGSemaphoreLock. The original coding
failed to reset ImmediateInterruptOK before returning, which would
potentially allow a subsequent query-cancel interrupt to be accepted
at an unsafe point. This is a really nasty bug since it's so hard
to predict the consequences, but they could be unpleasant. Also,
ensure that signal handlers are serviced before this function
returns, even if the semaphore is already set. This should make the
behavior more like Unix. Back-patch to all supported versions.
- Make WaitLatch's WL_POSTMASTER_DEATH result trustworthy; simplify
callers. Per a suggestion from Peter Geoghegan, make WaitLatch
responsible for verifying that the WL_POSTMASTER_DEATH bit it
returns is truthful (by testing PostmasterIsAlive). Then simplify
its callers, who no longer need to do that for themselves. Remove
weasel wording about falsely-set result bits from WaitLatch's API
- Temporarily revert stats collector latch changes so we can ship
beta1. This patch reverts commit
49340037ee3ab46cb24144a86705e35f272c24d5 and some follow-on tweaking
in pgstat.c. While the basic scheme of latch-ifying the stats
collector seems sound enough, it's failing on most Windows buildfarm
members for unknown reasons, and there's no time left to debug that
before 9.2beta1. Better to ship a beta version without this
improvement. I hope to re-revert this once beta1 is out, though.
- Tweak documentation wording to avoid "pdfendlink" failure. HEAD
documentation was failing to build as US PDF for me, because a link
to "CREATE CAST" was getting split across pages. Adjust wording to
remove this rather gratuitous cross-reference.
- Stamp 9.2beta1.
- Improve discussion of setting server parameters. Rewrite
description of "include_if_exists" for clarity. Add subsection
headings to make the structure of the page a little clearer. A
couple other minor improvements too. Josh Kupershmidt and Tom Lane
- Fix contrib/citext's upgrade script to handle array and domain
cases. We previously recognized that citext wouldn't get marked as
collatable during pg_upgrade from a pre-9.1 installation, and hacked
its create-from-unpackaged script to manually perform the necessary
catalog adjustments. However, we overlooked the fact that domains
over citext, as well as the citext array type, need the same
adjustments. Extend the script to handle those cases. Also, the
documentation suggested that this was only an issue in pg_upgrade
scenarios, which is quite wrong; loading any dump containing citext
from a pre-9.1 server will also result in the type being wrongly
marked. I approached the documentation problem by changing the
9.1.2 release note paragraphs about this issue, which is
historically inaccurate. But it seems better than having the
information scattered in multiple places, and leaving incorrect info
in the 9.1.2 notes would be bad anyway. We'll still need to mention
the issue again in the 9.1.4 notes, but perhaps they can just
reference 9.1.2 for fix instructions. Per report from Evan Carroll.
Back-patch into 9.1.
- Cosmetic adjustments for postmaster's handling of checkpointer.
Correct some comments, order some operations a bit more
consistently. No functional changes.
- Update example of process titles shown by "ps". This example was
quite old: it lacked the WAL writer and autovac launcher as well as
the more recently added checkpointer. Linux "ps" seems to show
slightly different stuff now too.
- Explain compatibility item about language names a bit more. Since
we've got an "open items" list item about this, apparently some
people are pretty worried about it. In passing remove a lot of
- Fix WaitLatchOrSocket to handle EOF on socket correctly. When using
poll(), EOF on a socket is reported with the POLLHUP not POLLIN flag
(at least on Linux). WaitLatchOrSocket failed to check this bit,
causing it to go into a busy-wait loop if EOF occurs. We earlier
fixed the same mistake in the test for the state of the
postmaster_alive socket, but missed it for the caller-supplied
socket. Fortunately, this error is new in 9.2, since 9.1 only had a
select() based code path not a poll() based one.
- Avoid unnecessary process wakeups in the log collector. syslogger
was coded to wake up once per second whether there was anything
useful to do or not. As part of our campaign to reduce the server's
idle power consumption, change it to use a latch for waiting. Now,
in the absence of any data to log or any signals to service, it will
only wake up at the programmed logfile rotation times (if any).
- Fix bogus declaration of local variable. rc should be an int here,
not a pgsocket. Fairly harmless as long as pgsocket is an integer
type, but nonetheless wrong. Error introduced in commit
- Attempt to fix some issues in our Windows socket code. Make sure
WaitLatchOrSocket regards FD_CLOSE as a read-ready condition. We
might want to tweak this further, but it was surely wrong as-is.
Make pgwin32_waitforsinglesocket detach its private event object
from the passed socket before returning. I suspect that failure to
do so leads to race conditions when other code (such as
WaitLatchOrSocket) attaches a different event object to the same
socket. Moreover, the existing coding meant that repeated calls to
pgwin32_waitforsinglesocket would perform ResetEvent on an event
actively connected to a socket, which is rumored to be an unsafe
practice; the WSAEventSelect documentation appears to recommend
against this, though it does not say not to do it in so many words.
Also, uniformly use the coding pattern "WSAEventSelect(s, NULL, 0)"
to detach events from sockets, rather than passing the event in the
second parameter. The WSAEventSelect documentation says that the
second parameter is ignored if the third is 0, so theoretically this
should make no difference. However, elsewhere on the same reference
page the use of NULL in this context is recommended, and I have
found suggestions on the net that some versions of Windows have bugs
with a non-NULL second parameter in this usage. Some other
mostly-cosmetic cleanup, such as using the right one of
WSAGetLastError and GetLastError for reporting errors from these
- Re-revert stats collector latch changes. This reverts commit
cb2f2873d6b81ad7f0a9733ba738bfac0746fb7b, restoring the latch-ified
stats collector logic. We'll soon see if this works any better on
the Windows buildfarm machines.
- Fix DROP TABLESPACE to unlink symlink when directory is not there.
If the tablespace directory is missing entirely, we allow DROP
TABLESPACE to go through, on the grounds that it should be possible
to clean up the catalog entry in such a situation. However, we
forgot that the pg_tblspc symlink might still be there. We should
try to remove the symlink too (but not fail if it's no longer
there), since not doing so can lead to weird behavior subsequently,
as per report from Michael Nolan. There was some discussion of
adding dependency links to prevent DROP TABLESPACE when the catalogs
still contain references to the tablespace. That might be worth
doing too, but it's an orthogonal question, and in any case wouldn't
be back-patchable. Back-patch to 9.0, which is as far back as the
logic looks like this. We could possibly do something similar in
8.x, but given the lack of reports I'm not sure it's worth the
trouble, and anyway the case could not arise in the form the logic
is meant to cover (namely, a post-DROP transaction rollback having
resurrected the pg_tablespace entry after some or all of the
filesystem infrastructure is gone).
- Add some temporary instrumentation to pgstat.c. Log main-loop
blocking events and the results of inquiry messages. This is to get
some clarity as to what's happening on those Windows buildfarm
members that still don't like the latch-ified stats collector. This
bulks up the postmaster log a tad, so I won't leave it in place for
Bruce Momjian pushed:
- Document how to find non-ASCII characters in the release notes.
- Add another URL for HTML entity codes.
- Publish draft version of Postgres 9.2 release notes.
- Update 9.2 SSL release note links.
- Improve two 9.2 release note items.
- Release notes adjustments from Erik Rijkers
- Remove URL that suggested numeric HTML entities.
- Another doc fix for HTML entities.
- Update entitiy escapes docs; must be Latin1.
- 9.2 release note adjustments from Robert Haas.
- Add 9.2 release note suggestions from Heikki Linnakangas,
- New 9.2 SSL parameters are GUC, not libpq, per Magnus's observation.
- 9.2 release note updates from Peter Geoghegan
- Add comma after "Previously" as suggested by Josh Kupershmidt
- Merge visibility map crash-safety into index-only scan release note
item, per Robert Haas.
- Update comment for 'name' data type to say 63 "bytes". Catalog
version bump so everyone has the same comment for beta1.
- Revert catalog bump; was post-beta1, and unnecessary.
- Update group commit release note item.
Simon Riggs pushed:
- Avoid xid error from age() function when run on Hot Standby
- Rename BgWriterCommLock to CheckpointerCommLock
- Rename BgWriterShmem/Request to CheckpointerShmem/Request
- Ensure age() returns a stable value rather than the latest value
- Remove extraneous #include "storage/proc.h"
- Ensure backwards compatibility for GetStableLatestTransactionId()
Joe Conway pushed:
- PL/pgSQL RETURN NEXT was leaking converted tuples, causing out of
memory when looping through large numbers of rows. Flag the
converted tuples to be freed. Complaint and patch by Joe.
Heikki Linnakangas pushed:
- Fix outdated comment. Multi-insert records observe
XLOG_HEAP_INIT_PAGE flag too, as Andres Freund pointed out.
- Copy-editing 9.2 release notes. Fix a couple of typos, add missing
filename tags, and add a link to the doc section on timezone guc.
- Fix a couple of typos in 9.2 release notes. Thom Brown
- On GiST page split, release the locks on child pages before
recursing up. When inserting the downlinks for a split gist page,
we used hold the locks on the child pages until the insertion into
the parent - and recursively its parent if it had to be split too -
were all completed. Change that so that the locks on child pages are
released after the insertion in the immediate parent is done, before
recursing further up the tree. This reduces the number of lwlocks
that are held simultaneously. Holding many locks is bad for
concurrency, and in extreme cases you can even hit the limit of 100
simultaneously held lwlocks in a backend. If you're really unlucky,
you can hit the limit while in a critical section, which brings down
the whole system. This fixes bug #6629 reported by Tom Forbes.
Backpatch to 9.1. The page splitting code was rewritten in 9.1, and
the old code did not have this problem.
Robert Haas pushed:
- Prevent loss of init fork when truncating an unlogged table. Fixes
bug #6635, reported by Akira Kurosawa.
== Rejected Patches (for now) ==
No one was disappointed this week :-)
== Pending Patches ==
Robert Haas sent in several approaches to optimizing CLOG background
Zoltan Boszormenyi sent in two more revisions of the patch to create
and use a locktimeout and SIGALARM framework.
Pavel Stehule sent in another revision of the patch to add an enhanced
ErrorData structure to PL/pgsql.
Magnus Hagander sent in a patch to fix a bug where pg_receivelog
didn't handle timeouts correctly.
Antonin Houska sent in a patch to implement some more of the LATERAL
functionality via functions.
Jeff Janes sent in a patch against pgbench which adds a --foreign-keys
option to initialization mode which creates all the relevant
constraints between the default tables.
Noah Misch sent in a patch which updates code comments per
pgsql-announce by date
|Next:||From: josh||Date: 2012-05-14 22:48:49|
|Subject: PostgreSQL 9.2 Beta 1 Now Available for Testing|
|Previous:||From: Michael Paquier||Date: 2012-05-13 03:59:11|
|Subject: Postgres-XC 1.0beta2 is released|