Re: Stats collector performance improvement

Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us> writes:
> I found if I put a pg_usleep(100) in the buffer process the backend
> speed was good, but packets were lost. What I found worked well was to
> do multiple recv() calls in a loop. The previous code did a select(),
> then perhaps a recv() and pipe write() based on the results of the
> select(). This caused many small packets to be written to the pipe and
> the pipe write overhead seems fairly large. The best fix I found was to
> loop over the recv() call at most 25 times, collecting a group of
> packets that can then be sent to the collector in one pipe write. The
> recv() socket is non-blocking, so a zero return indicates there are no
> more packets available. Patch attached.

This seems incredibly OS-specific. How many platforms did you test it
on?

A more serious objection is that it will cause the stats machinery to
work very poorly if there isn't a steady stream of incoming messages.
You can't just sit on 24 messages until the 25th one arrives next week.

regards, tom lane