Re: Sending notifications from the master to the standby

Joachim Wieland <joe(at)mcknight(dot)de> writes:
> [ send NOTIFYs to slaves by means of: ]
> In the patch I added a new WAL message type, XLOG_NOTIFY that writes
> out WAL records when the notifications are written into the pages of
> the SLRU ring buffer. Whenever an SLRU page is found to be full, a new
> WAL record will be created, that's just a more or less arbitrary form
> of batching a bunch of them together but that's easy to do and most
> often, I think there won't be more than at most a few record per
> transaction anyway.

I'm having a hard time wrapping my mind around why you'd do it that way.
ISTM there are two fairly serious problems:

1. Emitting WAL records for NOTIFY traffic results in significantly
more overhead, with no benefit whatever, for existing non-replicated
NOTIFY-using applications. Those folk are going to see a performance
degradation, and they're going to complain.

2. Batching NOTIFY traffic will result in a delay in receipt, which will
annoy anybody who's trying to make actual use of the notifications on
standby servers. The worst case here happens if notify traffic on the
master is bursty: the last few messages in a burst might not get to the
slave for a long time, certainly long after the commits that the
messages were supposed to be telling people about.

So this design is non-optimal both for existing uses and for the
proposed new uses, which means nobody will like it. You could
ameliorate #1 by adding a GUC that determines whether NOTIFY actually
writes WAL, but that's pretty ugly. In any case ISTM that problem #2
means this design is basically broken.

I wonder whether it'd be practical to not involve WAL per se in this
at all, but to transmit NOTIFY messages by having walsender processes
follow the notify stream (as though they were listeners) and send the
notify traffic as a separate message stream interleaved with the WAL
traffic. We already have, as of a few days ago, the concept of
additional traffic in the walsender stream besides the WAL data itself,
so adding notify traffic as another message type should be
straightforward. It might be a bit tricky to get walreceivers to inject
the data into the slave-side ring buffer at the right time, ie, not
until after the commit a given message describes has been replayed;
but I don't immediately see a reason to think that's infeasible.

Going in this direction would mean that slave-side LISTEN only works
when using walsender/walreceiver, and not with old-style log shipping.
But personally I don't see a problem with that. If you're trying to
LISTEN you probably want pretty up-to-date data anyway.

regards, tom lane