Re: logical decoding : exceeded maxAllocatedDescs for .spill files

On Tue, 26 Nov 2019 at 10:49, Amit Kapila <amit(dot)kapila16(at)gmail(dot)com> wrote:
>
> On Fri, Nov 22, 2019 at 7:38 PM Amit Khandekar <amitdkhan(dot)pg(at)gmail(dot)com> wrote:
> >
> > On Fri, 22 Nov 2019 at 4:26 PM, Amit Kapila <amit(dot)kapila16(at)gmail(dot)com> wrote:
> >>
> >> I think this is exactly the reason for the problem. In my test [1],
> >> the error "permission denied" occurred when I second time executed
> >> pg_logical_slot_get_changes() which means on first execution the
> >> unlink would have been successful but the files are still not removed
> >> as they were not closed. Then on second execution, it gets an error
> >> "Permission denied" when it again tries to unlink files via
> >> ReorderBufferCleanupSerializedTXNs().
> >>
> >>
> >> .
> >> > But what you are seeing is "Permission denied" errors. Not sure why
> >> > unlink() is failing.
> >> >
> >>
> >> In your test program, if you try to unlink the file second time, you
> >> should see the error "Permission denied".
> >
> > I tested using the sample program and indeed I got the error 5 (access denied) when I called unlink the second time.
> >
>
> So, what is the next step here? How about if we somehow check whether
> the file exists before doing unlink, say by using stat?
But the thing is, the behaviour is so much in a grey area, that we
cannot reliably say for instance that when stat() says there is no
such file, there is indeed no such file, and if we re-create the same
file when it is still open, it is always going to open a new file,
etc.

> If that doesn't work, I think we might need to go in the direction of tracking
> file handles in some way, so that they can be closed during an abort.
Yeah, that is one way. I am still working on different approaches.
WIll get back with proposals.

--
Thanks,
-Amit Khandekar
EnterpriseDB Corporation
The Postgres Database Company