Re: pg_restore causing ENOSPACE on the WAL partition. Fundamental issue?

On Tuesday, June 10, 2025, Dimitrios Apostolou <jimis(at)gmx(dot)net> wrote:

> Hello list,
>
> I have previously raised an issue on pgsql-general, where PostgreSQL is
> logging without any boundaries to the WAL directory, even if other writer
> processes can't catch up with it. It ends up with WAL partition becoming
> full and a bad crash. Read more at the thread at:
>
> https://www.postgresql.org/message-id/flat/076464ad-3d70-dd2
> 5-9e8f-e84f27decfba%40gmx.net
>
> If it's true, I consider this a critical and under-documented issue, thus
> I'm bringing it up here. Otherwise please let me know if any of my
> assumptions are wrong.
>
> In short, this is how it can easily happen, but in principle it's much
> more generic:
>
> + The WAL is on a dedicated high-perf drive
>
> + pg_restore is writing to a slow tablespace on a network drive
>
> + data is logged to the WAL faster than what can be written to that
> tablespace
>
> + it eventually ends up with the WAL filling up, regardless of
> max_wal_size...
>
>
> In other words, I'm surprised that there is no mechanism for the backends
> to block while the WAL is overflowing. Am I wrong here?
>
>
Probably not. max_wal_size is documented as being a soft limit that can be
exceeded due to this very thing. If the performance of your data disk is
so much worse than your WAL disk that you cannot checkpoint fast enough to
prevent WAL from overflowing its disk during that period then you do indeed
have an issue that PostgreSQL doesn’t provide protections for - it’s being
optimistic and helpful to the point of letting you hang yourself with the
rope it’s given you.

David J.