== PostgreSQL Weekly News - May 09 2009 ==

== PostgreSQL Weekly News - May 09 2009 ==

New Survey: Which 8.4 Feature are you most excited about?
www.postgresql.org/community

PostgreSQL's web site needs an overhaul, and $2000 is on the table to
get it. Gain fame and (a small) fortune with your skills.
http://wiki.postgresql.org/wiki/Website_Overhaul_Requirements

PGDay Junin / Buenos Aires Unnoba will be on June 6, 2009.
http://www.arpug.com.ar/trac/wiki/PgDayUnnoba

PGDay.EU 2009 will be at Telecom ParisTech in Paris, France on
November 6-7, 2009.
http://www.pgday.eu/

== PostgreSQL Product News ==

check_postgres 2.8.0, a Nagios plug-in for monitoring PostgreSQL, released.
http://bucardo.org/check_postgres/

Jopr 2.2, a server management and monitoring application, released.
http://www.jboss.org/jopr/

PostgreSQL RPM Building Project has released another version of The
PostgreSQL Live CD.
http://www.pglivecd.org

Npgsql2 2.0.5, a .NET data provider for PostgreSQL, released.
http://www.npgsql.org

pgAdmin 1.10 beta 3, a GUI management tool for PostgreSQL, released.
http://www.postgresql.org/ftp/pgadmin3/release/v1.10.0-beta3/

pgpool-II 2.2.2, a connection pooler and more, released.
http://pgfoundry.org/projects/pgpool/

The United States PostgreSQL Association is now a 501(c)(3)
non-profit. Kudos!
http://www.postgresql.us/

PostGIS 1.3.6, a geospatial system for PostgreSQL, released.
http://postgis.refractions.net/

Slony 1.2.16 and 2.0.2, a cascading master-slave replication system
for PostgreSQL, released.
http://slony.info/

== PostgreSQL 8.4 Feature of the Week ==

Windowing Functions. Those reporting queries just got much, much
easier. PARTITION BY is especially helpful here.

== PostgreSQL Tip of the Week ==

The referencing ("child") side of foreign key fields should generally
be indexed, unless the referenced value is very low cardinality.
PostgreSQL does not do this automatically.

== PostgreSQL Jobs for May ==

http://archives.postgresql.org/pgsql-jobs/2009-05/threads.php

== PostgreSQL Local ==

On May 20th 2009 in Pisa, Gabriele Bartolini of 2ndQuadrant Italia
will participate to the event "Open-Source: companies meet the
students" organised by the Tuscany Open Source Software Laboratory.
More information (in Italian):
http://www.tosslab.it/open-source-aziende-e-studenti-si-incontrano

PGCon 2009 will be held 21-22 May 2009, in Ottawa at the University of
Ottawa. It will be preceded by two days of tutorials on 19-20 May
2009.
http://www.pgcon.org/2009/

PgDay Florianopolis will be May 22, 2009. Contact Dickson S. Guedes
(guedes AT guedesoft DOT net) to participate or submit a paper.
http://www.postgresql.org.br/eventos/pgday/sc

Save The Date: pgDay San Jose. Sunday, July 19th 2009 immediately
before OSCON. CfP, more info TBA!

PGCon Brazil will be take place October 23-24 2009 at Unicamp in
Campinas, Sao Paulo state.

== PostgreSQL in the News ==

Planet PostgreSQL: http://planet.postgresql.org/

PostgreSQL Weekly News is brought to you this week by David Fetter and
Josh Berkus.

Submit news and announcements by Sunday at 3:00pm Pacific time.
Please send English language ones to david(at)fetter(dot)org, German language
to pwn(at)pgug(dot)de, Italian language to pwn(at)itpug(dot)org(dot)

== Applied Patches ==

Peter Eisentraut committed:

- Replace a couple of references to files that no longer exist in the
source tree with references to the appropriate URLs. Robert Haas

- Disable the use of Unicode escapes in string constants (U&'') when
standard_conforming_strings is not on, for security reasons.

Magnus Hagander committed:

- In pgsql/src/backend/port/win32_shmem.c, call SetLastError(0) before
calling the file mapping functions to make sure that the error code
is reset, as a precaution in case the API doesn't properly reset it
on success. This could be necessary, since we check the error value
even if the function doesn't fail for specific success cases.

- In pgsql/src/port/dirmod.c, update no longer current comments in
header.

- In pgsql/src/backend/port/win32_shmem.c, make the win32 shared
memory code try 10 times instead of one if it fails because the
shared memory segment already exists. This means it can take up to
10 seconds before it reports the error if it *does* exist, but
hopefully it will make the system capable of restarting even when
the server is under high load.

Heikki Linnakangas committed:

- Fix the query used for \d against 8.2 and 8.3 servers.

- Request XLOG switch before writing checkpoint in pg_start_backup().
Otherwise you can end up with an unrecoverable backup if you start a
new base backup right after finishing archive recovery. In that
scenario, the redo pointer of the checkpoint that pg_start_backup()
writes points to the XLOG segment where the timeline-changing
end-of-archive-recovery checkpoint is. The beginning of that segment
contains pages with the old timeline ID, and we don't accept that in
recovery unless we find a history file covering the old timeline ID.
If you omit pg_xlog from the base backup and clear the archive
directory before starting the backup, there will be no such history
file available. The bug is present in all versions since PITR was
introduced in 8.0, but I'm back-patching only back to 8.2. Earlier
versions didn't have XLOG switch records, making this fix
unfeasible. Given the lack of reports until now, it doesn't seem
worthwhile to spend more effort to fix 8.0 and 8.1. Per report and
suggestion by Mikael Krantz

- Add alternative expected output files for cs_CZ locale for
btree_gist and tsearch2 tests. This should make 'comet_moth'
buildfarm member pass contrib check. Zdenek Kotala.

Tom Lane committed:

- In pgsql/src/bin/psql/describe.c, code review for \df rewrite: fix
assorted bugs, make type and volatility columns localizable.

- In pgsql/src/backend/commands/analyze.c, avoid integer overflow in
the loop that extracts histogram entries from ANALYZE's total
sample. The original coding is at risk of overflow for statistics
targets exceeding about 2675; this was not a problem before 8.4 but
it is now. Per bug #4793 from Dennis Noordsij.

- In pgsql/src/backend/access/nbtree/nbtpage.c, update comment for
_bt_relandgetbuf.

- Insert CHECK_FOR_INTERRUPTS() calls into btree and hash index scans
at the points where we step right or left to the next page. This
should ensure reasonable response time to a query cancel request
during an unsuccessful index scan, as seen in recent gripe from Marc
Cousin. It's a bit trickier than it might seem at first glance,
because CHECK_FOR_INTERRUPTS() is a no-op if executed while holding
a buffer lock. So we have to do it just at the point where we've
dropped one page lock and not yet acquired the next. Remove
CHECK_FOR_INTERRUPTS calls at the top level of btgetbitmap and
hashgetbitmap, since they're pointless given the added checks. I
think that GIST is okay already --- at least, there's a
CHECK_FOR_INTERRUPTS at a plausible-looking place in gistnext(). I
don't claim to know GIN well enough to try to poke it for this, if
indeed it has a problem at all. This is a pre-existing issue, but
in view of the lack of prior complaints I'm not going to risk
back-patching.

- Install a "dead man switch" to allow the postmaster to detect cases
where a backend has done exit(0) or exit(1) without having
disengaged itself from shared memory. We are at risk for this
whenever third-party code is loaded into a backend, since such code
might not know it's supposed to go through proc_exit() instead.
Also, it is reported that under Windows there are ways to externally
kill a process that cause the status code returned to the postmaster
to be indistinguishable from a voluntary exit (thank you,
Microsoft). If this does happen then the system is probably hosed
--- for instance, the dead session might still be holding locks. So
the best recovery method is to treat this like a backend crash. The
dead man switch is armed for a particular child process when it
acquires a regular PGPROC, and disarmed when the PGPROC is released;
these should be the first and last touches of shared memory
resources in a backend, or close enough anyway. This choice means
there is no coverage for auxiliary processes, but I doubt we need
that, since they shouldn't be executing any user-provided code
anyway. This patch also improves the management of the EXEC_BACKEND
ShmemBackendArray array a bit, by reducing search costs. Although
this problem is of long standing, the lack of field complaints seems
to mean it's not critical enough to risk back-patching; at least not
till we get some more testing of this mechanism.

- In pgsql/src/backend/storage/ipc/ipc.c, install an atexit(2)
callback that ensures that proc_exit's cleanup processing will still
be performed if something in a backend process calls exit()
directly, instead of going through proc_exit() as we prefer. This
is a second response to the issue that we might load third-party
code that doesn't know it should not call exit(). Such a call will
now cause a reasonably graceful backend shutdown, if possible. (Of
course, if the reason for the exit() call is out-of-memory or some
such, we might not be able to recover, but at least we will try.)

- Make new complaint about unsafe Unicode literals include an error
location. Every other ereport in scan.l has one, this should too.

- In pgsql/src/backend/port/win32_shmem.c, minor improvement: avoid
assuming that GetLastError value cannot be affected by CloseHandle()
or Sleep().

- In pgsql/src/backend/commands/dbcommands.c, tweak a comment to agree
a bit better with the new dispensation that locales are
database-wide, not server-wide.

- Fix ecpg tests for change that disallowed Unicode literals unless
standard_conforming_strings is on.

- Modify CREATE DATABASE to enforce that the source database's
encoding setting must be used for the new database, except when
copying from template0. This is the same rule that we now enforce
for locale settings, and it has the same motivation: databases other
than template0 might contain data that would be invalid according to
a different setting. This represents another step in a continuing
process of locking down ways in which encoding violations could
occur inside the backend. Per discussion of a few days ago. In
passing, fix pre-existing breakage of mbregress.sh, and fix up a
couple of ereport() calls in dbcommands.c that failed to specify
sqlstate codes.

- In pgsql/src/backend/optimizer/plan/initsplan.c, tweak
distribute_qual_to_rels so that when we decide a pseudoconstant qual
can be pushed to the top of the join tree, we update both the relids
and qualscope variables to keep them in sync. This prevents a
possible later failure of an Assert clause, and affects nothing else
since qualscope isn't used later except for that Assert. At the
moment the Assert shouldn't be reachable when we've pushed the qual
up; but this is cheap insurance, and it's more sensible anyway in
terms of the overall logic of the routine. Per analysis of a bug
report from Stefan Huehner. I'm not back-patching this since it's
just future-proofing; but if anyone gets tempted to change
check_outerjoin_delay again in the back branches, this might be
needed.

- In pgsql/src/backend/optimizer/plan/initsplan.c, ooops ...
make_outerjoininfo wasn't actually enforcing the join order
restrictions specified for semijoins in optimizer/README, to wit
that you can't reassociate outer joins into or out of the RHS of a
semijoin. Per report from Heikki Linnakangas.

- Change pgbench to use the table names pgbench_accounts,
pgbench_branches, pgbench_history, and pgbench_tellers, rather than
just accounts, branches, history, and tellers. This is to prevent
accidental conflicts with real application tables, as has been
reported to happen at least once. Also remove the automatic "SET
search_path = public" that it did at startup, as this seems to
restrict testing flexibility without actually buying much. Per
proposal by Joshua Drake and ensuing discussion. Joshua Drake and
Tom Lane.

- Add an option to AlterTableCreateToastTable() to allow its caller to
force a toast table to be built, even if the sum-of-column-widths
calculation indicates one isn't needed. This is needed by
pg_migrator because if the old table has a toast table, we have to
migrate over the toast table since it might contain some live data,
even though subsequent column drops could mean that no
recently-added rows could require toasting.

Bruce Momjian committed:

- In pgsql/src/include/access/transam.h, 'PGDLLIMPORT'
ShmemVariableCache, needed for pg_migrator.so function linkage on
Win32. Tested by Hiroshi Saito.

- In pgsql/src/tools/fsync/test_fsync.c, add missing third argument to
open().

== Rejected Patches (for now) ==

No one was disappointed this week :-)

== Pending Patches ==

Robert Haas sent in another revision of the ALTER TABLE ... ALTER
COLUMN ... SET DISTINCT patch.

Andres Freund sent in two revisions of a patch which implements ALTER
TABLE ... DROP [COLUMN | CONSTRAINT] IF EXISTS ...

Dickson S. Guedes sent in a patch to add a way to display both client
and server versions in the psql prompt.

Seth Robertson sent in a patch to automate selection of client
certificates in libpq.

Andrew (RhodiumToad) Gierth sent in a patch to change
src/backend/libpq/be-secure.c from calling
SSL_CTX_use_certificate_file to SSL_CTX_use_certificate_chain_file.

Joshua Drake sent in a patch to make version_stamp.pl strict-clean.

Khee Chin sent in a patch to make psql's \di show the type of index
used.

David Fetter sent in a patch to make all the perl programs
strict-clean.