Re: shared-memory based stats collector

Hi,

On 2018-09-20 09:55:27 +0200, Antonin Houska wrote:
> I've spent some time reviewing this version.
>
> Design
> ------
>
> 1. Even with your patch the stats collector still uses an UDP socket to
> receive data. Now that the shared memory API is there, shouldn't the
> messages be sent via shared memory queue? [1] That would increase the
> reliability of message delivery.
>
> I can actually imagine backends inserting data into the shared hash tables
> themselves, but that might make them wait if the same entries are accessed
> by another backend. It should be much cheaper just to insert message into
> the queue and let the collector process it. In future version the collector
> can launch parallel workers so that writes by backends do not get blocked
> due to full queue.

I don't think either of these is right. I think it's crucial to get rid
of the UDP socket, but I think using a shmem queue is the wrong
approach. Not just because postgres' shm_mq is single-reader/writer, but
also because it's plainly unnecessary. Backends should attempt to
update the shared hashtable, but acquire the necessary lock
conditionally, and leave the pending updates of the shared hashtable to
a later time if they cannot acquire the lock.

Greetings,

Andres Freund