Re: postgres server crashes unexpectedly

Hi there,

Sorry about the lack of information on the system. We're running fedora (not
for sure what version though) core (whitebox).

I did as you said and this is the result:

DETAIL: 0 index pages have been deleted, 0 are currently reusable.
CPU 0.00s/0.00u sec elapsed 0.01 sec.
INFO: "grp_member": moved 0 row versions, truncated 4 to 4 pages
DETAIL: CPU 0.00s/0.00u sec elapsed 0.00 sec.
INFO: vacuuming "public.story_member"
INFO: "story_member": found 603570 removable, 9903 nonremovable row
versions in 43011 pages
DETAIL: 0 dead row versions cannot be removed yet.
Nonremovable row versions range from 44 to 44 bytes long.
There were 6139208 unused item pointers.
Total free space (including removable row versions) is 323999824 bytes.
42732 pages are or will become empty, including 0 at the end of the table.
42958 pages containing 323999400 free bytes are potential move destinations.
CPU 0.52s/0.18u sec elapsed 5.91 sec.
INFO: index "fkx_story__story_member" now contains 9903 row versions in
17736 pages
DETAIL: 64 index row versions were removed.
15219 index pages have been deleted, 15219 are currently reusable.
CPU 0.29s/0.06u sec elapsed 26.88 sec.
PANIC: corrupted item pointer: offset = 0, size = 0
server closed the connection unexpectedly
This probably means the server terminated abnormally
before or while processing the request.
The connection to the server was lost. Attempting reset: WARNING:
terminating connection because of crash of another server process
DETAIL: The postmaster has commanded this server process to roll back the
current transaction and exit, because another server process exited
abnormally and possibly corrupted shared memory.
HINT: In a moment you should be able to reconnect to the database and
repeat your command.
Failed.
!>
!>

I keep getting this error:

WARNING: terminating connection because of crash of another server process
DETAIL: The postmaster has commanded this server process to roll back the
current transaction and exit, because another server process exited
abnormally and possibly corrupted shared memory.
HINT: In a moment you should be able to reconnect to the database and
repeat your command.
server closed the connection unexpectedly
This probably means the server terminated abnormally
before or while processing the request.
The connection to the server was lost. Attempting reset: Succeeded.

What could be doing this? It just started out of the blue... I reindexed the
index it mentioned and it seems to error out more...

-Chadwick

----- Original Message -----
From: "Tom Lane" <tgl(at)sss(dot)pgh(dot)pa(dot)us>
To: "Chadwick Horn" <chadhorn(at)gmail(dot)com>
Cc: <pgsql-sql(at)postgresql(dot)org>
Sent: Monday, March 17, 2008 7:32 PM
Subject: Re: [SQL] postgres server crashes unexpectedly

> "Chadwick Horn" <chadhorn(at)gmail(dot)com> writes:
>> PANIC: corrupted item pointer: offset = 0, size = 0
>> LOG: autovacuum process (PID 3037) was terminated by signal 6
>
> Hmm ... the only instances of that error text are in PageIndexTupleDelete
> and PageIndexMultiDelete, so we can fairly safely say that you have a
> partially zeroed-out page in some index somewhere. If that's the only
> damage then you're in luck: you can recover by reindexing.
>
> What I'd do is turn off autovacuum and instead do a manual VACUUM
> VERBOSE to see where it crashes; then you could just reindex the one
> problem table instead of the whole database.
>
> You ought to look into why this happened, too. Since you've provided
> precisely 0 context about PG version or platform, it's hard to speculate
> about that ...
>
> regards, tom lane