From: | Korry Douglas <korry(at)me(dot)com> |
---|---|
To: | PostgreSQL-development Hackers <pgsql-hackers(at)postgresql(dot)org> |
Subject: | Parallel Foreign Scans - need advice |
Date: | 2019-05-15 16:55:33 |
Message-ID: | 6ED3BCAA-C4A8-44E4-86EC-6309E41C6352@me.com |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-hackers |
Hi all, I’m working on an FDW that would benefit greatly from parallel foreign scan. I have implemented the callbacks described here:https://www.postgresql.org/docs/devel/fdw-callbacks.html#FDW-CALLBACKS-PARALLEL. and I see a big improvement in certain plans.
My problem is that I can’t seem to get a parallel foreign scan in a query that does not contain an aggregate.
For example:
SELECT count(*) FROM foreign table;
Gives me a parallel scan, but
SELECT * FROM foreign table;
Does not.
I’ve been fiddling with the costing GUCs, foreign scan row estimates, and foreign scan cost estimates - I can force the cost of a partial path to be much lower than a sequential foreign scan, but no luck.
Any troubleshooting advice?
A second related question - how can I find the actual number of workers chose for my ForeignScan? At the moment, I looking at ParallelContext->nworkers (inside of the InitializeDSMForeignScan() callback) because that seems to be the first callback function that might provide the worker count. I need the *actual* worker count in order to evenly distribute my workload. I can’t use the usual trick of having each worker grab the next available chunk (because I have to avoid seek operations on compressed data). In other words, it is of great advantage for each worker to read contiguous chunks of data - seeking to another part of the file is prohibitively expensive.
Thanks for all help.
— Korry
From | Date | Subject | |
---|---|---|---|
Next Message | Alvaro Herrera | 2019-05-15 17:05:30 | Re: New EXPLAIN option: ALL |
Previous Message | Daniel Verite | 2019-05-15 16:41:32 | RE: psql - add SHOW_ALL_RESULTS option |