Re: logical decoding : exceeded maxAllocatedDescs for .spill files

On Thu, Nov 21, 2019 at 5:02 AM Amit Kapila <amit(dot)kapila16(at)gmail(dot)com> wrote:

> On Wed, Nov 20, 2019 at 5:41 PM Juan José Santamaría Flecha
> <juanjo(dot)santamaria(at)gmail(dot)com> wrote:
> >
> > On Wed, Nov 20, 2019 at 9:48 AM Amit Khandekar <amitdkhan(dot)pg(at)gmail(dot)com>
> wrote:
> >>
> >> On Wed, 20 Nov 2019 at 13:10, Amit Kapila <amit(dot)kapila16(at)gmail(dot)com>
> wrote:
> >> > See comment in pgunlink() "We need to loop because even though
> >> > PostgreSQL uses flags that allow unlink while the file is open, other
> >> > applications might have the file
> >> > open without those flags.". Can you once see if there is any flag
> >> > that you have missed to pass to allow this?
> >>
> >> > If there is nothing we
> >> > can do about it, then we might need to use some different API or maybe
> >> > define a new API that can handle this.
> >>
> >> There were objections against modifying the vfd api only for this
> >> replication-related use-case. Having a new API will require all the
> >> changes required to enable the virtual FDs feature that we need from
> >> vfd. If nothing works out from the FILE_SHARE_DELETE thing, I am
> >> thinking, we can use VFD, plus we can keep track of per-subtransaction
> >> vfd handles, and do something similar to AtEOSubXact_Files().
> >>
> >
> > The comment about "other applications might have the file open without
> those flags." is surely due to systems working with an antivirus touching
> Postgres files.
> >
> > I was not able to reproduce the Permission denied error with current
> HEAD,
> >
>
> I am not sure what exactly you tried. Can you share the steps and
> your environment details?
>
>
Sure, I was trying to reproduce the Permission denied error after
the ERROR_HANDLE_EOF fix.

1. Using a clean environment [1] the spill.sql script produces the expected
output.
2. I manually injected a negative value for readBytes after @@ -2611,10
+2627,11 @@ ReorderBufferRestoreChanges(ReorderBuffer *rb, ReorderBufferTXN
*txn. In doing so pg_logical_slot_get_changes() failed, but following
executions did not run into Permission denied.
3. During the cleanup of some of the tests, pg_drop_replication_slot()
failed because the "pg_replslot/regression_slot" folder was is use.

[1] Win10 (1903) MSVC 19.22.27905

Regards,

Juan José Santamaría Flecha