Re: Hard limit on WAL space used (because PANIC sucks)

On Tue, Jan 21, 2014 at 9:35 AM, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:

> Simon Riggs <simon(at)2ndQuadrant(dot)com> writes:
> > On 6 June 2013 16:00, Heikki Linnakangas <hlinnakangas(at)vmware(dot)com>
> wrote:
> >> The current situation is that if you run out of disk space while writing
> >> WAL, you get a PANIC, and the server shuts down. That's awful.
>
> > I don't see we need to prevent WAL insertions when the disk fills. We
> > still have the whole of wal_buffers to use up. When that is full, we
> > will prevent further WAL insertions because we will be holding the
> > WALwritelock to clear more space. So the rest of the system will lock
> > up nicely, like we want, apart from read-only transactions.
>
> I'm not sure that "all writing transactions lock up hard" is really so
> much better than the current behavior.
>
> My preference would be that we simply start failing writes with ERRORs
> rather than PANICs. I'm not real sure ATM why this has to be a PANIC
> condition. Probably the cause is that it's being done inside a critical
> section, but could we move that?
>

My understanding is that if it runs out of buffer space while in an
XLogInsert, it will be holding one or more buffer content locks
exclusively, and unless it can complete the xlog (or scrounge up the info
to return that buffer to its previous state), it can never release that
lock. There might be other paths were it could get by with an ERROR, but
if no one can write xlog anymore, all of those paths must quickly converge
to the one that cannot simply ERROR.

Cheers,

Jeff