Re: logical decoding : exceeded maxAllocatedDescs for .spill files

From: Amit Kapila <amit(dot)kapila16(at)gmail(dot)com>
To: Amit Khandekar <amitdkhan(dot)pg(at)gmail(dot)com>
Cc: Alvaro Herrera from 2ndQuadrant <alvherre(at)alvh(dot)no-ip(dot)org>, Andres Freund <andres(at)anarazel(dot)de>, Juan José Santamaría Flecha <juanjo(dot)santamaria(at)gmail(dot)com>, PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>, Robert Haas <robertmhaas(at)gmail(dot)com>, Thomas Munro <thomas(dot)munro(at)gmail(dot)com>, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>, Tomas Vondra <tomas(dot)vondra(at)2ndquadrant(dot)com>
Subject: Re: logical decoding : exceeded maxAllocatedDescs for .spill files
Date: 2019-12-14 06:29:43
Message-ID: CAA4eK1JLKiJCS+VqWZmvXKySMvikHDhEtXcLf3i9Z1Kb+veZXw@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Thu, Dec 12, 2019 at 9:50 PM Amit Khandekar <amitdkhan(dot)pg(at)gmail(dot)com> wrote:
>
> Attached is a v4 patch that also addresses your code comments so far.
> I have included the test case in 006_logical_decoding.pl. I observed
> that the test case just adds only about 0.5 to 1 sec time. Please
> verify on your env also, and also whether the test reproduces the
> issue without the code changes.
>

It takes roughly the same time on my machine as well. I have checked
on Windows as well, it increases the time from 14 to 16 (17) seconds
for this test. I don't think this is any big increase considering the
timing of other tests and it would be good to have a test for such
boundary conditions. I have slightly change the comments in the patch
and ran pgindent. Attached, find the patch with a proposed commit
message.

I have also made minor changes related to below code in patch:
- else if (readBytes != sizeof(ReorderBufferDiskChange))
+
+ file->curOffset += readBytes;
+
+ if (readBytes !=
sizeof(ReorderBufferDiskChange))

Why the size is added before the error check? I think it should be
after that check, so changed accordingly. Similarly, I don't see why
we need to change 'else if' to 'if' in this code, so changed back.

I think we need to change/tweak the test for back branches as there we
don't have logical_decoding_work_mem. Can you please look into that
and see if you can run perltidy for the test file.

--
With Regards,
Amit Kapila.
EnterpriseDB: http://www.enterprisedb.com

Attachment Content-Type Size
use_vfd_for_logrep_v5.patch application/octet-stream 11.3 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Dent John 2019-12-14 12:09:09 Re: [WIP] UNNEST(REFCURSOR): allowing SELECT to consume data from a REFCURSOR
Previous Message Thomas Munro 2019-12-14 06:15:31 Re: checkpointer: PANIC: could not fsync file: No such file or directory