| From: | Rafia Sabih <rafia(dot)pghackers(at)gmail(dot)com> |
|---|---|
| To: | Robert Haas <robertmhaas(at)gmail(dot)com> |
| Cc: | Jelte Fennema-Nio <postgres(at)jeltef(dot)nl>, Yilin Zhang <jiezhilove(at)126(dot)com>, KENAN YILMAZ <kenan(dot)yilmaz(at)localus(dot)com(dot)tr>, Andy Fan <zhihuifan1213(at)163(dot)com>, PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org> |
| Subject: | Re: Bypassing cursors in postgres_fdw to enable parallel plans |
| Date: | 2026-11-03 13:24:53 |
| Message-ID: | CA+FpmFcwiJrj6r_Thac3OEBaAEVs9dppC3N8QHAQCAtcoQLWDg@mail.gmail.com |
| Views: | Whole Thread | Raw Message | Download mbox | Resend email |
| Thread: | |
| Lists: | pgsql-hackers |
On Fri, 9 Oct 2026 at 22:57, Robert Haas <robertmhaas(at)gmail(dot)com> wrote:
> On Tue, Sep 15, 2026 at 9:30 AM Rafia Sabih <rafia(dot)pghackers(at)gmail(dot)com>
> wrote:
> > The patches required a minor rebase.
> > The latest patches attached.
>
> If you enter drain_active_scan() with tuplestore_lost_rows = true, the
> function will set it to false, which is wrong. I think that
> drain_active_scan() should store the previous value of
> tuplestore_lost_rows into a local variable just before setting
> tuplestore_lost_rows = true, and restore that value in the places
> where it previously stores false.
>
>
> http://postgr.es/m/CA+Tgmoa-FXH9jZaxm039eWf8=nhnoBfDS=oeKfm50sbGEAZ58g@mail.gmail.com
> pointed out that pgfdw_subxact_callback() needs to drain active scans
> before calling RELEASE SAVEPOINT. It still doesn't. You should
> probably add some transaction-control tests to the regression test
> suite for postgres_fdw, so that these kinds of issues can't come back
> once fixed.
>
> drain_active_scan() switches to active_fsstate->batch_cxt while it's
> executing, which causes allocations to be performed in the correct
> memory context. But that only handles memory allocation, and
> tuplestores also use files, which are controlled by the resource owner
> mechanism, not the memory context mechanism. Any files created here
> should end up owned by whatever the correct resource owner is for the
> scan we're draining, but instead they end up using the resource owner
> that's in effect at the time we need to perform the drain. You'll need
> to save the resource owner and temporarily switch to it here as you do
> for the memory context.
>
> I attach Claude-written test cases for all three issues.
>
Thank you for providing the tests and your comments on this.
The latest version of the patches are attached.
I have added tests to cover these issues, and also compacted the tests.
--
Regards,
Rafia Sabih
CYBERTEC PostgreSQL International GmbH
| Attachment | Content-Type | Size |
|---|---|---|
| v19-0001-postgres_fdw-Rename-cursor_exists-flag-to-scan_i.patch | application/octet-stream | 4.0 KB |
| v19-0002-postgres_fdw-Add-streaming_fetch-option-for-curs.patch | application/octet-stream | 95.2 KB |
| From | Date | Subject | |
|---|---|---|---|
| Next Message | Jeevan Chalke | 2026-11-03 13:26:41 | Re: PL/pgSQL: apply the too_many_rows extra check to EXECUTE ... INTO |
| Previous Message | Manu | 2026-11-03 12:48:08 | Re: [(known) BUG] DELETE/UPDATE more than one row in partitioned foreign table |