Re: [HACKERS] [POC] Faster processing at Gather node

From: Robert Haas <robertmhaas(at)gmail(dot)com>
To: Andres Freund <andres(at)anarazel(dot)de>
Cc: Amit Kapila <amit(dot)kapila16(at)gmail(dot)com>, Rafia Sabih <rafia(dot)sabih(at)enterprisedb(dot)com>, PostgreSQL Developers <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: [HACKERS] [POC] Faster processing at Gather node
Date: 2018-02-28 15:06:27
Views: Raw Message | Whole Thread | Download mbox
Lists: pgsql-hackers

On Tue, Feb 27, 2018 at 4:06 PM, Andres Freund <andres(at)anarazel(dot)de> wrote:
>> OK, I'll try to check how feasible that would be.
> cool.

It's not too hard, but it doesn't really seem to help, so I'm inclined
to leave it alone. To make it work, you need to keep two separate
counters in the shm_mq_handle, one for the number of bytes since we
did an increment and the other for the number of bytes since we sent a
signal. I don't really want to introduce that complexity unless there
is a benefit.

With just 0001 and 0002: 3968.899 ms, 4043.428 ms, 4042.472 ms, 4142.226 ms
With two-separate-counters.patch added: 4123.841 ms, 4101.917 ms,
4063.368 ms, 3985.148 ms

If you take the total of the 4 times, that's an 0.4% slowdown with the
patch applied, but I think that's just noise. It seems possible that
with a larger queue -- and maybe a different query shape it would
help, but I really just want to get the optimizations that I've got
committed, provided that you find them acceptable, rather than spend a
lot of time looking for new optimizations, because:

1. I've got other things to get done.

2. I think that the patches I've got here capture most of the available benefit.

3. This case isn't super-common in the first place -- we generally
want to avoid feeding tons of tuples through the Gather.

4. We might abandon the shm_mq approach entirely and switch to
something like sticking tuples in DSA using the flexible tuple slot
stuff you've proposed elsewhere.

Robert Haas
The Enterprise PostgreSQL Company

Attachment Content-Type Size
0002-shm-mq-reduce-receiver-latch-set-v3.patch application/octet-stream 7.5 KB
0001-shm-mq-less-spinlocks-v4.patch application/octet-stream 14.3 KB
two-separate-counters.patch application/octet-stream 6.1 KB

In response to


Browse pgsql-hackers by date

  From Date Subject
Next Message Tom Lane 2018-02-28 15:11:32 Re: Direct converting numeric types to bool
Previous Message n.zhuchkov 2018-02-28 15:06:11 Re: Direct converting numeric types to bool