Parallel query execution

From: Bruce Momjian <bruce(at)momjian(dot)us>
To: PostgreSQL-development <pgsql-hackers(at)postgreSQL(dot)org>
Subject: Parallel query execution
Date: 2013-01-15 22:14:19
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

I mentioned last year that I wanted to start working on parallelism:

Years ago I added thread-safety to libpq. Recently I added two parallel
execution paths to pg_upgrade. The first parallel path allows execution
of external binaries pg_dump and psql (to restore). The second parallel
path does copy/link by calling fork/thread-safe C functions. I was able
to do each in 2-3 days.

I believe it is time to start adding parallel execution to the backend.
We already have some parallelism in the backend:
effective_io_concurrency and helper processes. I think it is time we
start to consider additional options.

Parallelism isn't going to help all queries, in fact it might be just a
small subset, but it will be the larger queries. The pg_upgrade
parallelism only helps clusters with multiple databases or tablespaces,
but the improvements are significant.

I have summarized my ideas by updating our Parallel Query Execution wiki

Please consider updating the page yourself or posting your ideas to this
thread. Thanks.

Bruce Momjian <bruce(at)momjian(dot)us>

+ It's impossible for everything to be true. +


Browse pgsql-hackers by date

  From Date Subject
Next Message Stephen Frost 2013-01-15 22:19:58 Re: [PATCH] COPY .. COMPRESSED
Previous Message Robert Haas 2013-01-15 21:54:51 Re: unlogged tables vs. GIST