"ERROR: could not read block 6 ...: read only 0 of 8192 bytes" after autovacuum cancelled

A customer of ours recently hit a problem where after an autovacuum was
cancelled on a table, the app started getting the message in $subject:

ERROR: could not read block 6 of relation 1663/35078/1761966: read only 0 of 8192 bytes

(block numbers vary from 1 to 6). Things remained in this state until
another autovacuum came along and cleaned up the table, 4 minutes later
(this is a high traffic table; there are several inserts per second).

The log looks like this:

2009-10-20 04:02:07 PDT [27396]: [1-1] LOG: automatic vacuum of table "database.public.tabname": index scans: 1
pages: 6 removed, 1 remain
tuples: 755 removed, 2 remain
system usage: CPU 0.00s/0.00u sec elapsed 1.42 sec
2009-10-20 04:02:07 PDT [27396]: [2-1] ERROR: canceling autovacuum task
2009-10-20 04:02:07 PDT [27396]: [3-1] CONTEXT: automatic vacuum of table "database.public.tabname"

What I thought could have happened is that the table was truncated, and
then the sinval message telling that to other backends was not sent due
to the rollback. When they tried to insert to the page they had
recorded as rd_targblock, they try to read the page but it's no longer
there.

I can reproduce this by adding a sleep and CHECK_FOR_INTERRUPTS after
lazy_vacuum_rel() returns, and before CommitTransactionCommand.

So far as I can see, what we need is to make sure the sinval message is
sent regardless of transaction commit/abort. How can that be done? It
is quite ugly to have an untimely autovacuum cancel disrupt the ability
to insert into a table.

Thoughts?

--
Alvaro Herrera http://www.CommandPrompt.com/
The PostgreSQL Company - Command Prompt, Inc.