Re: Reducing power consumption on idle servers

On Fri, Jan 27, 2023 at 7:37 PM Bharath Rupireddy
<bharath(dot)rupireddyforpostgres(at)gmail(dot)com> wrote:
> On Wed, Jan 25, 2023 at 2:10 AM Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
> > It's kind of moot until we've reached the point where we can
> > credibly claim to have explicit wakeups for every event of
> > interest. I don't think we're very close to that today, and
> > I do think we should try to get closer. There may come a point
> > of diminishing returns though.
>
> IIUC, we're discussing here whether or not to get rid of hibernate
> loops, IOW, sleep-wakeup-doworkifthereisany-sleep loops and rely on
> other processes' wakeup signals to reduce the overall power
> consumption, am I right?
>
> I'm trying to understand this a bit - can the signals (especially,
> SIGURG that we use to set latches to wake up processes) ever get lost
> on the way before reaching the target process? If yes, how? How
> frequently can it happen? Is there any history of reported issues in
> postgres because a signal got lost?
>
> I'm reading about Pending Signals and queuing of signals with
> sigqueue() (in linux), can any of these guarantee that signals sent
> never get lost?

Signals don't get lost. At least with the current definition of so
called "reliable" signals, in the POSIX standard. (There was a
previous system of signals in ancient UNIX that was harder to program
with; we still see echos of that today, for example in C standard
there are basic signals, and Windows implements those, but they're
borderline useless; before a signal handler runs, the handler is also
de-registered, so many interesting handlers would have to re-register
it, but signals can't be atomically blocked at the same time and there
is a window of time where the default behaviour applies, which (from
our modern perspective) is clearly insane as there is literally no way
to close that race and avoid some fatal default behaviour). BTW
sigqueue (signals that are queued up without being consolidated, and
carry a user-controlled value with them) are "realtime signals" AKA
"signal queuing", which I guess we'll never be able to use because at
least Apple didn't implement them; the "reliable signals" we use are
the plain kind where signals don't carry any kind of payload and are
collapsed into a single "pending" bit, so that if 2 people do
kill(SIGFOO) around the same time your handler might only run once (if
you registered one and its not blocked), but it will definitely run >
0 times (or be delievered via various other means such as sigwait(),
etc...).

Then there is the question of races as you go into the wait primitive.
I mean in between our reading latch->is_set and entering the kernel,
which we defend against using various techniques which you can read
about at the top of latch.c; the short version is that if that
happens, those system calls will immediately return.

The uncertainty discussed here comes from the comment beginning "There
is a race condition here, ..." in bgwriter.c, referring to the
protocol implemented by StrategyNotifyBgWriter(). I haven't really
looked into it. I *guess* it's probably approximately OK if the
bgwriter misses a wakeup from time to time, because there's another
chance to wake it up as soon as someone needs another buffer, and if
you don't need any more buffers soon your system is probably idle; but
really "needing another buffer" isn't a great proxy for "more buffers
are being made dirty" at all! Someone would need to think about it
and construct a complete argument for why it's OK to go to sleep for
60 seconds, or infinity, with that sloppiness in place.

There's also the walwriter to look into; from memory it was a little
less fuzzy but I haven't looked recently.