Re: Listen / Notify rewrite

On Thu, Nov 12, 2009 at 4:25 AM, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
>> One possible solution would be to write to the queue before committing
>> and adding the TransactionID.  Then other backends can check if our
>> TransactionID has successfully committed or not. Not sure if this is
>> worth the overhead however...
>
> That sounds reasonable, and it's certainly no more overhead than the
> tuple visibility checks that happen in the current implementation.

I am not too concerned about the runtime of the visibility checks,
instead I suppose that most of the overhead will come from waiting for
another transaction to either commit or abort...

If transaction t1 scans the queue and at some point finds
notifications from t2, then it will ask for the status of t2. If it
finds out that t2 is still running, then it has no other option than
to stop working on the queue and wait (it will be signalled again
later once t2 has finished).

This also means that we cannot at the same time write notifications to
the queue and read from it and if a transaction places a few million
notifications into the queue, readers need to wait until it has
finished and only after that they can continue and read the
notifications...

And it means that if the queue is full, we might run into a
deadlock... A transaction adding notifications will wait for the
readers to proceed and the readers wait for the transaction to commit
or abort...

One option could be to write new notifications to a subdirectory, and
create a bunch of new segment files there. Once this is done, the
segment files could be moved over and renamed, so that they continue
the slru queue... If we run out of disk space while filling that
temporary subdirectory, then we can just delete the subdirectory and
nobody has been blocked. We could still run into errors moving and
renaming the segment files (e.g. permission problems) so that we still
might need to abort the transaction...

>> 2. The payload parameter is optional. A notifying client can either call
>> "NOTIFY foo;" or "NOTIFY foo 'payload';". The length of the payload is
>> currently limited to 128 characters... Not sure if we should allow longer
>> payload strings...
>
> Might be a good idea to make the max the same as the max length for
> prepared transaction GUIDs?  Not sure anyone would be shipping those
> around, but it's a pre-existing limit of about the same size.

Yes, sounds reasonable to have the same limit for user-defined identifiers...

Joachim