== PostgreSQL Weekly News - September 24 2007 ==

== PostgreSQL Weekly News - September 24 2007 ==

Postgres releases 8.2.5, 8.1.10, 8.0.14, 7.4.18, and 7.3.20 are out.
Upgrade ASAP!

Following the release of PostgreSQL 8.3 later this year, PostgreSQL
8.1.x and 8.0.x will cease to be officially supported on Windows.

== PostgreSQL Product News ==

MS Access to PostgreSQL Converter 2.0 released.
http://pgfoundry.org/projects/access2postgres/

PgPool 3.4.1 released.
http://pgfoundry.org/projects/pgpool/

New PostgreSQL RPM sets are available for Fedora / RHEL
http://www.postgresql.org/download/

== PostgreSQL Jobs for September ==

http://archives.postgresql.org/pgsql-jobs/2007-09/threads.php

== PostgreSQL Local ==

PostgreSQL Fall Conference will be October 20, 2007 in Portland.
http://www.postgresqlconference.org/

There will be a PostgreSQL booth at Ohio Linuxfest on September 29th,
and Robert Treat will be speaking.
http://www.ohiolinux.org/speakers.html#talk1_06

== PostgreSQL in the News ==

Planet PostgreSQL: http://www.planetpostgresql.org/

General Bits, Archives and occasional new articles:
http://www.varlena.com/GeneralBits/

PostgreSQL Weekly News is brought to you this week by David Fetter

Submit news and announcements by Sunday at 3:00pm Pacific time.
Please send English language ones to david(at)fetter(dot)org, German to
pwn(at)pgug(dot)de(dot)

== Applied Patches ==

Michael Meskes committed:

- Removed superfluous ECPGfree() call from
pgsql/src/interfaces/ecpg/ecpglib/execute.c.

Neil Conway committed:

- In pgsql/src/backend/utils/adt/float.c, prevent corr() from
returning the wrong results for negative correlation values. The
previous coding essentially assumed that x = sqrt(x*x), which does
not hold for x < 0. Thanks to Jie Zhang at Greenplum and Gavin
Sherry for reporting this issue.

Andrew Dunstan committed:

- Add perl replacements for build.bat and vcregress.bat. In due course
the .bat files will be altered to become tiny wrappers for these
scripts, and one or two other .bat files will disappear.

- In pgsql/src/tools/msvc/Install.pm, restrict overly broad searches,
and fix typo, in recent fix. Per Hannes Eder.

- In pgsql/src/tools/msvc/Install.pm, replace calls to external dir
program with perlish globs and File::Find calls. Fixes complaint
fron Hannes Eder, whose environment found a different dir program.

- Go back to using a separate method for doing ILIKE for single byte
character encodings that doesn't involve calling lower(). This
should cure the performance regression in this case complained of by
Guillaume Smet. It still leaves the horrid performance for
multi-byte encodings introduced in 8.2, but there's no obvious
solution for that in sight.

- Cleanup items from csvlog changes, per ITAGAKI Takahiro.

- Close previously open holes for invalidly encoded data to enter the
database via builtin functions, as recently discussed on -hackers.
chr() now returns a character in the database encoding. For UTF8
encoded databases the argument is treated as a Unicode code point.
For other multi-byte encodings the argument must designate a strict
ascii character, or an error is raised, as is also the case if the
argument is 0. ascii() is adjusted so that it remains the inverse
of chr().

Teodor Sigaev committed:

- In pgsql/src/backend/utils/adt/tsrank.c, fix msvc warnings, patch by
Hannes Eder.

- In pgsql/src/backend/tsearch/dict_thesaurus.c, avoid
possibly-unportable initializer, per Gregory Stark and buildfarm
warning.

Bruce Momjian committed:

- In FAQ_DEV, fix typo per Brendan Jurd.

- Mark TODOs as done: "Consider shrinking expired tuples to just their
headers," "Allow heap reuse of UPDATEd rows if no indexed columns
are changed, and old and new versions are on the same heap page,"
and remove no-longer-needed TODO: "Reuse index tuples that point to
heap tuples that are not visible to anyone?"

- In pgsql/doc/src/sgml/func.sgml, remove tabs from SGML file.

- In pgsql/src/tools/RELEASE_CHANGES, add reminder that integer pg
version also needs updating.

- Remove extra tab in postgresql.conf

Tom Lane committed:

- Replace linear searches with binary searches in pg_dump's code to
lookup objects by OID. Per gripe from nikitathespider.

- In pgsql/src/backend/utils/adt/xml.c, fix bugs in XML binary I/O
functions. Heikki and Tom

- Make autovacuum report the start time of its current activity in
pg_stat_activity. Per gripe from Jim Nasby.

- In pgsql/src/backend/storage/ipc/procarray.c,
transactionIdIsInProgress can skip scanning the ProcArray if the
target XID is later than latestCompletedXid, per Florian Pflug.
Also some minor improvements in the XIDCACHE_DEBUG code --- make
sure each call of TransactionIdIsInProgress is counted one way or
another.

- Temporarily modify tsearch regression tests to suppress notice that
comes out at erratic times, because it is creating a totally
unacceptable level of noise in our buildfarm results. This patch
can be reverted when and if the code is fixed to not issue notices
during cache reload events.

- Fix cost estimates for EXISTS subqueries that are evaluated as
initPlans (because they are uncorrelated with the immediate parent
query). We were charging the full run cost to the parent node,
disregarding the fact that only one row need be fetched for EXISTS.
While this would only be a cosmetic issue in most cases, it might
possibly affect planning outcomes if the parent query were itself a
subquery to some upper query. Per recent discussion with Steve
Crawford.

- In pgsql/doc/src/sgml/config.sgml, document the translations from
Postgres message severity levels to syslog and eventlog severity
levels, per suggestion from Josh Drake. Also, some wordsmithing for
the csvlog documentation.

- In pgsql/src/backend/postmaster/syslogger.c, fix erroneous Assert()
in syslogger process start in EXEC_BACKEND case, per ITAGAKI
Takahiro. Also, rewrite syslogger_forkexec() in hopes of
eliminating the confusion in the first place.

- In pgsql/src/backend/utils/adt/oracle_compat.c, fix bogus
calculation of potential output string length in translate().

- In pgsql/src/include/fmgr.h, parenthesize macro arguments safely. I
see no bug among the current uses of PG_DETOAST_DATUM_SLICE, but
it's clearly trouble waiting to happen.

- Although I'd misdiagnosed the reason for the recent failures on
buildfarm member grebe, I see no reason to revert the
1-byte-header-friendly changes I made in varlena.c. Instead, tweak
the code a little bit to get more advantage out of that.

- In pgsql/src/backend/utils/adt/regexp.c, defend against the case
where malloc returns NULL for malloc(0).

- Fix varlena.c routines to allow 1-byte-header text values. This is
now demonstrably necessary for text_substring() since regexp_split
functions may pass it such a value; and we might as well convert the
whole file at once. Per buildfarm results (though I wonder why most
machines aren't showing a failure).

- Fix regex, LIKE, and some other second-rank text-manipulation
functions to not cause needless copying of text datums that have
1-byte headers. Greg Stark, in response to performance gripe from
Guillaume Smet and ITAGAKI Takahiro.

- Improve handling of prune/no-prune decisions by storing a page's
oldest unpruned XMAX in its header. At the cost of 4 bytes per
page, this keeps us from performing heap_page_prune when there's no
chance of pruning anything. Seems to be necessary per Heikki's
preliminary performance testing.

- In pgsql/src/backend/utils/time/tqual.c, change tqual.c tests to use
!TransactionIdIsCurrentTransactionId, rather than
TransactionIdDidAbort, when handling the case that xmin is one of
the current transaction's XIDs and the tuple has been deleted. xmax
must also be one of the current transaction's XIDs, since no one
else can see it yet, and it's cheaper to look at local state than
shared state to find out if xmax aborted. Per an idea of Heikki's.

- In pgsql/src/backend/storage/ipc/procarray.c, make some simple
performance improvements in TransactionIdIsInProgress(). For XIDs
of our own transaction and subtransactions, it's cheaper to ask
TransactionIdIsCurrentTransactionId() than to look in shared memory.
Also, the xids[] work array is always the same size within any given
process, so malloc it just once instead of doing a palloc/pfree on
every call; aside from being faster this lets us get rid of some
goto's, since we no longer have any end-of-function pfree to do.
Both ideas by Heikki Linnakangas.

- In pgsql/src/backend/access/transam/twophase.c, fix comments that
misspelled TransactionIdIsInProgress, per Heikki Linnakangas.

- If we're gonna provide an --enable-profiling configure option,
surely it ought to know that you need -DLINUX_PROFILE on Linux.

- Insert a hack in pl/tcl to disable Tcl's built-in Notifier
subsystem, which has a bad habit of launching multiple threads
within the backend and thereby causing all kinds of havoc.
Fortunately, we don't need it, and recent Tcl versions provide an
easy way to disable it. Diagnosis and fix by Steve Marshall, Paul
Bayer, and Doug Knight of WSI Corporation.

- Solaris portability fix that was previously made in contrib/tsearch2
but got lost from the version committed to main tree. Per Greg
Stark.

- Revert ill-fated patch to release exclusive lock early after vacuum
truncates a table. Introduces race condition, as shown by buildfarm
failures.

- HOT updates. When we update a tuple without changing any of its
indexed columns, and the new version can be stored on the same heap
page, we no longer generate extra index entries for the new version.
Instead, index searches follow the HOT-chain links to ensure they
find the correct tuple version. In addition, this patch introduces
the ability to "prune" dead tuples on a per-page basis, without
having to do a complete VACUUM pass to recover space. VACUUM is
still needed to clean up dead index entries, however. Pavan
Deolasee, with help from a bunch of other people.

- Remove Assert(BgWriterShmem != NULL), which is rather pointless
since we'd dump core anyway immediately afterward if it were null;
and it seems to confuse some versions of icc into generating bad
code. Per report from Sergey Koposov. Patched in HEAD only, for
the moment, since this is only likely to affect developers.

- Fix overflow in extract(epoch from interval) for intervals exceeding
68 years. Seems to have been introduced in 8.1 by careless
SECS_PER_DAY search-and-replace.

- Update release notes for last-minute fix.

- Fix aboriginal mistake in lazy VACUUM's code for truncating away
no-longer-needed pages at the end of a table. We thought we could
throw away pages containing HEAPTUPLE_DEAD tuples; but this is not
so, because such tuples very likely have index entries pointing at
them, and we wouldn't have removed the index entries. The problem
only emerges in a somewhat unlikely race condition: the dead tuples
have to have been inserted by a transaction that later aborted, and
this has to have happened between VACUUM's initial scan of the page
and then rechecking it for empty in count_nondeletable_pages. But
that timespan will include an index-cleaning pass, so it's not all
that hard to hit. This seems to explain a couple of previously
unsolved bug reports.

== Rejected Patches (for now) ==

No one was disappointed this week :-)

== Pending Patches ==

Bernd Helmle sent in a patch which adds a "Login" field to psql's \du
command, displaying wether a role has the LOGIN privilege granted or
not.

Heikki Linnakangas sent in a patch which optimizes pg_next_dst_boundary.