pgsql: Glue layer to connect the executor to the shm_mq mechanism.

From: Robert Haas <rhaas(at)postgresql(dot)org>
To: pgsql-committers(at)postgresql(dot)org
Subject: pgsql: Glue layer to connect the executor to the shm_mq mechanism.
Date: 2015-09-19 01:57:31
Message-ID: E1Zd7P9-0005Ya-Sk@gemulon.postgresql.org
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-committers

Glue layer to connect the executor to the shm_mq mechanism.

The shm_mq mechanism was built to send error (and notice) messages and
tuples between backends. However, shm_mq itself only deals in raw
bytes. Since commit 2bd9e412f92bc6a68f3e8bcb18e04955cc35001d, we have
had infrastructure for one message to redirect protocol messages to a
queue and for another backend to parse them and do useful things with
them. This commit introduces a somewhat analogous facility for tuples
by adding a new type of DestReceiver, DestTupleQueue, which writes
each tuple generated by a query into a shm_mq, and a new
TupleQueueFunnel facility which reads raw tuples out of the queue and
reconstructs the HeapTuple format expected by the executor.

The TupleQueueFunnel abstraction supports reading from multiple tuple
streams at the same time, but only in round-robin fashion. Someone
could imaginably want other policies, but this should be good enough
to meet our short-term needs related to parallel query, and we can
always extend it later.

This also makes one minor addition to the shm_mq API that didn'
seem worth breaking out as a separate patch.

Extracted from Amit Kapila's parallel sequential scan patch. This
code was originally written by me, and then it was revised by Amit,
and then it was revised some more by me.

Branch
------
master

Details
-------
http://git.postgresql.org/pg/commitdiff/4a4e6893aa080b9094dadbe0e65f8a75fee41ac6

Modified Files
--------------
src/backend/executor/Makefile | 2 +-
src/backend/executor/tqueue.c | 262 ++++++++++++++++++++++++++++++++++++++
src/backend/storage/ipc/shm_mq.c | 9 ++
src/backend/tcop/dest.c | 7 +
src/include/executor/tqueue.h | 31 +++++
src/include/storage/shm_mq.h | 3 +
src/include/tcop/dest.h | 3 +-
src/tools/pgindent/typedefs.list | 1 +
8 files changed, 316 insertions(+), 2 deletions(-)

Browse pgsql-committers by date

  From Date Subject
Next Message Peter Eisentraut 2015-09-19 02:07:24 pgsql: Remove trailing slashes from directories in find command
Previous Message Jeff Janes 2015-09-18 20:20:36 Re: [COMMITTERS] pgsql: Add pages deleted from pending list to FSM