| From: | Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> |
|---|---|
| To: | assam258(at)gmail(dot)com |
| Cc: | tharakan(at)gmail(dot)com, pgsql-bugs(at)lists(dot)postgresql(dot)org |
| Subject: | Re: BUG #19911: Window aggregate over ROWS n FOLLOWING: "unexpected end of tuplestore" after spill |
| Date: | 2026-11-18 02:03:13 |
| Message-ID: | 601850.1794967393@sss.pgh.pa.us |
| Views: | Whole Thread | Raw Message | Download mbox | Resend email |
| Thread: | |
| Lists: | pgsql-bugs |
Henson Choi <assam258(at)gmail(dot)com> writes:
> Thanks for the report. I can reproduce it on master, and the attached
> patch fixes it. Below are the cause and fix, why this kind of bug
> keeps turning up in WindowAgg, and two follow-up ideas for master.
I've spent the last little while poking at this problem too.
I concur with your root-cause diagnosis: WindowAgg's dead reckoning
about where the tuplestore read pointers are positioned is getting
out of sync with their actual positions. I've got very little faith
in your proposed patch though. WindowAgg maintains a bunch of
read pointers and is less than consistent about which one is the
"active" one when it calls tuplestore_puttupleslot(), so I suspect
that there are more places than this one that would need resync logic
if we follow this fix approach.
I experimented with some alternative fix ideas, but couldn't come
up with anything very workable. I think what we need to do is
what you called a mitigation but seems to me to be a full fix:
get rid of WindowAgg's local tracking of tuplestore positions
in favor of extending the tuplestore API with some more-appropriate
operations. In particular, if tuplestore had an API call to the
effect of "set the read pointer to absolute tuple number N", we
could get rid of a lot of silliness in WindowAgg. Having a call
to read out the position as an absolute tuple number might help
too, but I'm not sure we'd even need it if we had the first one.
The reason that tuplestore hasn't got that already is long obsolete:
I think it's mainly that we wanted to support tuplestores with more
than 4G tuples on machines without int64 arithmetic, so dealing in
absolute rather than relative tuple numbers didn't work. There are
some fine points to be dealt with, like compensating for
"tuplestore_trim" operations, but I think it's not going to be too
complicated. And fortunately we never exposed the tuplestore state
structs in any header, so there's no ABI breakage risk to worry about
from adding or expanding fields.
I would not bother with the intermediate management layer that you
talked about; that seems like it's just adding bureaucracy.
regards, tom lane
| From | Date | Subject | |
|---|---|---|---|
| Next Message | Hayato Kuroda (Fujitsu) | 2026-11-18 02:06:52 | RE: Streaming decoding fails with "unexpected table_index_fetch_tuple call during logical decoding" when a relation has a TOASTed conbin (follow-up to BUG #18641) |
| Previous Message | Tender Wang | 2026-11-18 01:42:41 | Re: BUG #19912: MERGE NOT MATCHED BY SOURCE sees stale source values after concurrent update (EPQ) |