fd.c doesn't remove files on a crash-restart

From: "Joshua D(dot) Drake" <jd(at)commandprompt(dot)com>
To: pgsql-hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: fd.c doesn't remove files on a crash-restart
Date: 2016-03-16 17:53:42
Message-ID: 56E99DA6.6080501@commandprompt.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hello,

fd.c[1] will remove files from pgsql_tmp on a restart but not a
crash-restart per this comment:

/*
* NOTE: we could, but don't, call this during a post-backend-crash restart
* cycle. The argument for not doing it is that someone might want to
examine
* the temp files for debugging purposes. This does however mean that
* OpenTemporaryFile had better allow for collision with an existing temp
* file name.
*/

I understand that this is designed this way. I think it is a bad idea
because:

1. The majority crash-restarts in the wild are going to be diagnosed
rather easily within the OS itself. They fall into things like OOM
killer and out of disk space.

2. It can cause significant issues, we ran into this yesterday:

-bash-4.1$ ls pgsql_tmp31227*|du -sh
250G

There is no active process/backend with PID 31227. The database itself
is only 55G, but we are taking up an 5x that with dead files.

3. The problem can get worse over time. If you have a very long running
instance, any time the backend crash-restarts you have to potential to
increase disk space used for no purpose.

Sincerely,

JD

P.S. Thanks to AndrewG for his assistance in finding this.

1. http://doxygen.postgresql.org/fd_8c_source.html

--
Command Prompt, Inc. http://the.postgres.company/
+1-503-667-4564
PostgreSQL Centered full stack support, consulting and development.
Everyone appreciates your honesty, until you are honest with them.

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Andres Freund 2016-03-16 17:56:29 Re: fd.c doesn't remove files on a crash-restart
Previous Message Robert Haas 2016-03-16 17:45:24 Re: POC: Cache data in GetSnapshotData()