== PostgreSQL Weekly News - April 20 2008 ==

== PostgreSQL Weekly News - April 20 2008 ==

Many patches have been arriving on the -patches list.

The PostgreSQL booth at FISL was another success, this time with a new
training outfit, new faces staffing, and some tragedy at the end.
FISL had almost 7500 participants this year, making it the world's
largest FLOSS conference.

== PostgreSQL Product News ==

Archiveopteryx 2.08 released.
http://www.archiveopteryx.org/2.08

Federico (rotellaro) Campoli's course on PITR will be June 9, 2008.
http://www.pghost.eu/node/59

== PostgreSQL Jobs for April ==

http://archives.postgresql.org/pgsql-jobs/2008-04/threads.php

== PostgreSQL Local ==

PostgreSQL Day Unicamp 2008 will be on April 22 in Campinas, SP, Brazil.
http://www.dextra.com.br/dia-postgresql.htm

Corporate Databases 2008 will be April 24-25 in Moscow.
http://citforum.ru/seminars/cbd2008/

PGCon 2008 will be May 20-23 in Ottawa.
http://www.pgcon.org/2008/

PGDay will be in Portland the day before OSCON.
http://pugs.postgresql.org/taxonomy/term/53

Utah Open Source Conference 2008's CfP is open through June 1.
This 2nd annual conference is August 28-30, 2008 in Salt Lake City, UT
http://2008.utosc.com/

== PostgreSQL in the News ==

Planet PostgreSQL: http://www.planetpostgresql.org/

General Bits, Archives and occasional new articles:
http://www.varlena.com/GeneralBits/

PostgreSQL Weekly News is brought to you this week by David Fetter

Submit news and announcements by Sunday at 3:00pm Pacific time.
Please send English language ones to david(at)fetter(dot)org, German language
to pwn(at)pgug(dot)de, Italian language to pwn(at)itpug(dot)org(dot)

== Applied Patches ==

Tom Lane committed:

- Push index operator lossiness determination down to GIST/GIN opclass
"consistent" functions, and remove pg_amop.opreqcheck, as per recent
discussion. The main immediate benefit of this is that we no longer
need 8.3's ugly hack of requiring @@@ rather than @@ to test
weight-using tsquery searches on GIN indexes. In future it should
be possible to optimize some other queries better than is done now,
by detecting at runtime whether the index match is exact or not.
Tom Lane, after an idea of Heikki Linnakangas's, and with some help
from Teodor Sigaev.

- In pgsql/src/backend/utils/cache/relcache.c, fix LOAD_CRIT_INDEX()
macro to take out AccessShareLock on the system index it is trying
to build a relcache entry for. This is an oversight in my 8.2 patch
that tried to ensure we always took a lock on a relation before
trying to build its relcache entry. The implication is that if
someone committed a reindex of a critical system index at about the
same time that some other backend were starting up without a valid
pg_internal.init file, the second one might PANIC due to not seeing
any valid version of the index's pg_class row. Improbable case, but
definitely not impossible.

- Repair two places where SIGTERM exit could leave shared memory state
corrupted. (Neither is very important if SIGTERM is used to shut
down the whole database cluster together, but there's a problem if
someone tries to SIGTERM individual backends.) To do this,
introduce new infrastructure macros
PG_ENSURE_ERROR_CLEANUP/PG_END_ENSURE_ERROR_CLEANUP that take care
of transiently pushing an on_shmem_exit cleanup hook. Also use this
method for createdb cleanup --- that wasn't a
shared-memory-corruption problem, but SIGTERM abort of createdb
could leave orphaned files lying around. Backpatch as far as 8.2.
The shmem corruption cases don't exist in 8.1, and the createdb
usage doesn't seem important enough to risk backpatching further.

- In pgsql/src/backend/commands/explain.c, add some code to EXPLAIN to
show the targetlist (ie, output columns) of each plan node. For the
moment this is debug support only and is not enabled unless
EXPLAIN_PRINT_TLISTS is defined at build time. Later I'll see about
the idea of letting EXPLAIN VERBOSE do it.

- Fix a couple of oversights associated with the "physical tlist"
optimization: we had several code paths where a physical tlist could
be used for the input to a Sort node, which is a dumb idea because
any unneeded table columns will increase the volume of data the sort
has to push around. (Unfortunately the easy-looking fix of calling
disuse_physical_tlist during make_sort_xxx doesn't work because in
most cases we're already committed to the current input tlist ---
it's been marked with sort column numbers, or we've built grouping
column numbers using it, etc. The tlist has to be selected properly
at the calling level before we start constructing sort-col
information. This is easy enough to do, we were just failing to
take the point into consideration.) Back-patch to 8.3. I believe
the problem probably exists clear back to 7.4 when the physical
tlist optimization was added, but I'm afraid to back-patch further
than 8.3 without a great deal more study than I want to put into it.
The code in this area has drifted a lot over time. The real-world
importance of these code paths is uncertain anyway --- I think in
many cases we'd probably prefer hash-based methods.

- Cause EXPLAIN's VERBOSE option to print the target list (output
column list) of each plan node, instead of its former behavior of
dumping the internal representation of the plan tree. The latter
display is still available for those who really want it (see
debug_print_plan), but uses for it are certainly few and and far
between. Per discussion. This patch also removes the
explain_pretty_print GUC, which is obsoleted by the change.

- Fix rmtree() so that it keeps going after failure to remove any
individual file; the idea is that we should clean up as much as we
can, even if there's some problem removing one file. Make the error
messages a bit less misleading, too. In passing, const-ify function
arguments.

- Fix typo in pgsql/contrib/seg/uninstall_seg.sql.

- In pgsql/contrib/seg/seg.c, seg_size() has to be V1 calling
convention, too.

- In pgsql/contrib/earthdistance/earthdistance.c, convert
earthdistance's only C function to v1 call convention,
to future-proof it against pass-by-value float8.

- Allow float8, int8, and related datatypes to be passed by value on
machines where Datum is 8 bytes wide. Since this will break
old-style C functions (those still using version 0 calling
convention) that have arguments or results of these types, provide a
configure option to disable it and retain the old pass-by-reference
behavior. Likewise, provide a configure option to disable the
recently-committed float4 pass-by-value change. Zoltan Boszormenyi,
plus configurability stuff by me.

- Make earthdistance use version-0 calling convention if not
USE_FLOAT8_BYVAL, and version-1 if USE_FLOAT8_BYVAL. This might
seem a bit pointless, but the idea is to have at least one
regression test that will fail if we ever accidentally break
version-0 functions that return float8. However, they're already
broken, or at least hopelessly unportable, in the USE_FLOAT8_BYVAL
case. Per a recent suggestion from Greg Stark.

- In pgsql/src/include/pg_config.h.win32, add
FLOAT4PASSBYVAL/FLOAT8PASSBYVAL to pg_config.h.win32, as a stopgap
measure to get the Windows buildfarm members working again. I don't
know if it's worth exposing these as configurables, or exactly how
to do it in the MSVC build system ...

- In pgsql/src/backend/executor/execMain.c, fix a couple of places in
execMain that erroneously assumed that SELECT FOR UPDATE/SHARE
couldn't occur as a subquery in a query with a non-SELECT top-level
operation. Symptoms included outright failure (as in report from
Mark Mielke) and silently neglecting to take the requested row
locks. Back-patch to 8.3, because the visible failure in the INSERT
... SELECT case is a regression from 8.2. I'm a bit hesitant to
back-patch further given the lack of field complaints.

Alvaro Herrera committed:

- In pgsql/src/bin/psql/describe.c, fix indentation in new REFERENCED
BY psql output, per Brendan Jurd.

- Clean up a few places where Datums were being treated as pointers
(and vice versa) without going through DatumGetPointer. Gavin
Sherry, with Feng Tian.

- Modify the float4 datatype to be pass-by-val. Along the way, remove
the last uses of the long-deprecated float32 in contrib/seg; the
definitions themselves are still there, but no longer used.
fmgr/README updated to match. I added a CREATE FUNCTION to account
for existing seg_center() code in seg.c too, and some tests for it
and the neighbor functions. At the same time, remove checks for
NULL which are not needed (because the functions are declared
STRICT). I had to do some adjustments to contrib's btree_gist too.
The choices for representation there are not ideal for changing the
underlying types :-( Original patch by Zoltan Boszormenyi, with
some adjustments by me.

- In pgsql/contrib/seg/seg.c, change the float4-returning functions in
contrib/seg to fmgr v1 calling conventions. I also changed seg_in
and seg_out, which was probably unnecessary, but it can't harm.

Bruce Momjian committed:

- Add URL for TODO: "Consider automatic caching of statements at
various levels."

- Add pg_terminate_backend() to allow terminating only a single
session.

- Mark TODO as DONE: "Allow administrators to safely terminate
individual sessions."

- Add to TODO: "Allow XML to accept more liberal DOCTYPE
specifications."

- Move LISTEN/NOTIFY items to separate TODO section.

- Split TODO into two items: "Allow NOTIFY in rules involving
conditionals" and "Improve LISTEN concurrency."

- Revert addition of pg_terminate_backend() because of race
conditions. Unmark TODO.

- Re-add terminate TODO item.

- Add to TODO: "Implement the non-threaded Avahi service discovery
protocol."

- Update TODO wording on psql.

- In pgsql/src/bin/psql/mbprint.c, fix comment typo. Bryce Nesbitt.

- In FAQ, update most recent release to 8.3.1.

- In pgsql/src/tools/pgindent/pgindent, ignore blank lines in typedef
file.

- Re-enable pg_terminate_backend() using SIGTERM. SIGTERM testing
still needed.

- Mark TODO as DONE: "Allow administrators to safely terminate
individual sessions."

- Remove TODO.detail references; instead add TODO URLs pointing to
the archives. We have been using URLs for a while for new items.

- Remove TODO.detail directory. All URLs now in TODO file as
references.

- Update TODO using new script.

- Add TODO2html tool to convert TODO to HTML.

- Add script FAQ2txt to convert HTML files to txt. Add comment to
TODO2html.

- Update doc script comments.

Andrew Dunstan committed:

- Make integer_datetimes the default for MSVC even if not mentioned in
config.pl.

- In pgsql/src/tools/msvc/Project.pm, add multi-line flag to regex
that needs it. Backpatch to 8.2. Fix from Andreas Zeugswetter.

- Avoid using unnecessary pgwin32_safestat in libpq.

- Fix MinGW warnings re: formats and unused variables. Per ITAGAKI
Takahiro.

Heikki Linnakangas committed:

- Fix two race conditions between the pending unlink mechanism that
was put in place to prevent reusing relation OIDs before next
checkpoint, and DROP DATABASE. First, if a database was dropped,
bgwriter would still try to unlink the files that the rmtree() call
by the DROP DATABASE command has already deleted, or is just about
to delete. Second, if a database is dropped, and another database is
created with the same OID, bgwriter would in the worst case delete a
relation in the new database that happened to get the same OID as a
dropped relation in the old database. To fix these race conditions:
1. make rmtree() ignore ENOENT errors. This fixes the 1st race
condition. 2. make ForgetDatabaseFsyncRequests forget unlink
requests as well. 3. force checkpoint on in dropdb on all platforms
Since ForgetDatabaseFsyncRequests() is asynchronous, the 2nd change
isn't enough on its own to fix the problem of dropping and creating
a database with same OID, but forcing a checkpoint on DROP DATABASE
makes it sufficient. Per Tom Lane's bug report and proposal.
Backpatch to 8.3.

Teodor Sigaev committed:

- In pgsql/src/backend/utils/adt/tsquery_gist.c, fix broken compare
function for tsquery_ops. Per Tom Lane's report. I never understood
why initial authors GiST in pgsql choose so stgrange signature for
'same' method: bool *sameFn(Datum a, Datum b, bool* result) instead
of simple, logical bool sameFn(Datum a, Datum b) This change will
break any existing GiST extension, so we still live with it and will
live.

== Rejected Patches (for now) ==

No one was disappointed this week :-)

== Pending Patches ==

Brendan Jurd sent in another revision of his printTable API.

Alvaro Herrera sent in another revision of his patch to track
ActiveSnapshot.

Bruce Momjian and Bryce Nesbitt updated Bryce Nesbitt's patch in
several iterations. The patch optionally sets a maximum width for
psql output and wraps the overflow.

Brendan Jurd sent in a new revision of his patch to show INHERIT in
\du.

Brendan Jurd sent in a new revision of his patch to add conversions
Text <-> C string.

Tom Lane sent in a WIP patch to remove lossy-operator RECHECK flags in
some places.

Teodor Sigaev sent in another revision of a patch which implements
partial match in GIN indexes.

Merlin Moncure sent in a patch to add hooks for the libpqtypes
proposal.

ITAGAKI Takahiro sent in a patch which sorts writes during a
checkpoint.

Brendan Jurd sent in a patch to remove the typename field from
A_Const.

Merlin Moncure sent in a patch to add PQmakeResult, PQsetValue and
PQresultAlloc to libpq.

Andrew Chernow sent in two revisions of his libpq object hooks patch.

Pavel Stehule sent in a patch which adds new options (SQLSTATE,
DETAIL, DETAIL_LOG and HINT) to PL/PgSQL's RAISE statement.

Alex Hunsaker sent in a revised version of Joshua Drake's patch which
adds a command line option --use-statement-timeout to pg_dump and
pg_restore.

Brendan Jurd sent in two revisions of a patch which makes \z
privileges multi-line in psql.