Idle sessions keep deleted files open

From: "Peter J(dot) Holzer" <hjp-pgsql(at)hjp(dot)at>
To: pgsql-general(at)postgresql(dot)org
Subject: Idle sessions keep deleted files open
Date: 2020-03-29 22:49:13
Message-ID: 20200329224913.GA11265@hjp.at
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-general

I noticed this with PostgreSQL 11.7 on Linux:

I did a «vacuum full» on a table which had become quite bloated (size
before the vacuum: 392 GB, after: 291 GB). I expected the disk usage to
increase as the new table was written and then drop sharply after the
vacuum was finished.

However, that didn't happen. While usage increased by about 300 GB
during the vacuum, it stayed the same afterwards (as reported by «df»).

However, «du» reported about 400 GB less than «df». So apparently, files
had been unlinked but were still in use by some process.

«lsof» confirmed this: There were some idle sessions which were keeping
the files open. As soon as I got those sessions to execute some request,
they closed the files, causing the disk space to be freed.

I think I noticed that before when dropping tables, but didn't draw the
right conclusion at the time.

So, if you are reorganizing large tables, keep an eye on idle sessions -
they may keep deleted files around for quite some time.

hp

--
_ | Peter J. Holzer | Story must make more sense than reality.
|_|_) | |
| | | hjp(at)hjp(dot)at | -- Charles Stross, "Creative writing
__/ | http://www.hjp.at/ | challenge!"

Responses

Browse pgsql-general by date

  From Date Subject
Next Message Peter J. Holzer 2020-03-29 23:21:09 Re: Ident authentication failed
Previous Message Andrus 2020-03-29 21:47:22 Re: could not determine encoding for locale "et_EE.UTF-8": codeset is "CPUTF-8" in pg_restore