Re: unexpected shutdown

On 6/18/07, developer(at)wexwarez(dot)com <developer(at)wexwarez(dot)com> wrote:
>
> My database has shutdown several times in the last couple days. I have no
> idea why. I am running centos and I have not rebooted the server or made
> any configuration changes. I am running postgres 8.2 and it has been
> stable since I installed it about 5 months ago. The databases crashes and
> so my software application goes down. When I restart my application
> everything seems to work fine. But then it crashes again, something
> appears to be corrupt. Here are my logs:
>
>
> LOG: server process (PID 501) was terminated by signal 9
> LOG: terminating any other active server processes
> WARNING: terminating connection because of crash of another server
> process
> DETAIL: The postmaster has commanded this server process to roll back the
> current transaction and exit, because another server process exited
> abnormally and possibly corrupted shared memory.
> HINT: In a moment you should be able to reconnect to the database and
> repeat your command.
> WARNING: terminating connection because of crash of another server
> process
> DETAIL: The postmaster has commanded this server process to roll back the
> current transaction and exit, because another server process exited
> abnormally and possibly corrupted shared memory.
> HINT: In a moment you should be able to reconnect to the database and
> repeat your command.
> WARNING: terminating connection because of crash of another server
> process
> DETAIL: The postmaster has commanded this server process to roll back the
> current transaction and exit, because another server process exited
> abnormally and possibly corrupted shared memory.
> HINT: In a moment you should be able to reconnect to the database and
> repeat your command.
> WARNING: terminating connection because of crash of another server
> process
> DETAIL: The postmaster has commanded this server process to roll back the
> current transaction and exit, because another server process exited
> abnormally and possibly corrupted shared memory.
> HINT: In a moment you should be able to reconnect to the database and
> repeat your command.
> WARNING: terminating connection because of crash of another server
> process
> DETAIL: The postmaster has commanded this server process to roll back the
> current transaction and exit, because another server process exited
> abnormally and possibly corrupted shared memory.
> HINT: In a moment you should be able to reconnect to the database and
> repeat your command.
> FATAL: the database system is in recovery mode
> FATAL: the database system is in recovery mode
> LOG: all server processes terminated; reinitializing
> LOG: database system was interrupted at 2007-06-17 10:55:32 PDT
> LOG: checkpoint record is at 0/72F41748
> LOG: redo record is at 0/72F41748; undo record is at 0/0; shutdown FALSE
> LOG: next transaction ID: 0/2638157; next OID: 52761
> LOG: next MultiXactId: 4; next MultiXactOffset: 7
> LOG: database system was not properly shut down; automatic recovery in
> progress
> LOG: record with zero length at 0/72F41790
> LOG: redo is not required
> LOG: database system is ready
>
>
> LOG: server process (PID 13904) was terminated by signal 9
> LOG: terminating any other active server processes
> WARNING: terminating connection because of crash of another server
> process
> DETAIL: The postmaster has commanded this server process to roll back the
> current transaction and exit, because another server process exited
> abnormally and possibly corrupted shared memory.
> HINT: In a moment you should be able to reconnect to the database and
> repeat your command.
> WARNING: terminating connection because of crash of another server
> process
> DETAIL: The postmaster has commanded this server process to roll back the
> current transaction and exit, because another server process exited
> abnormally and possibly corrupted shared memory.
> HINT: In a moment you should be able to reconnect to the database and
> repeat your command.
> WARNING: terminating connection because of crash of another server
> process
> DETAIL: The postmaster has commanded this server process to roll back the
> current transaction and exit, because another server process exited
> abnormally and possibly corrupted shared memory.
> HINT: In a moment you should be able to reconnect to the database and
> repeat your command.
> WARNING: terminating connection because of crash of another server
> process
> DETAIL: The postmaster has commanded this server process to roll back the
> current transaction and exit, because another server process exited
> abnormally and possibly corrupted shared memory.
> HINT: In a moment you should be able to reconnect to the database and
> repeat your command.
> LOG: all server processes terminated; reinitializing
> LOG: database system was interrupted at 2007-06-18 10:09:51 PDT
> LOG: checkpoint record is at 0/73609D18
> LOG: redo record is at 0/73609D18; undo record is at 0/0; shutdown FALSE
> LOG: next transaction ID: 0/2645768; next OID: 52761
> LOG: next MultiXactId: 4; next MultiXactOffset: 7
> LOG: database system was not properly shut down; automatic recovery in
> progress
> LOG: record with zero length at 0/73609D60
> LOG: redo is not required
> LOG: database system is ready
>
>
>
> ---------------------------(end of broadcast)---------------------------
> TIP 6: explain analyze is your friend

Looks like someone is doing a kill -9 on one of the database processes
causing postgres to do a restart.

LOG: server process (PID 501) was terminated by signal 9