From: | Ajin Cherian <itsajin(at)gmail(dot)com> |
---|---|
To: | Amit Kapila <amit(dot)kapila16(at)gmail(dot)com> |
Cc: | Dilip Kumar <dilipbalaut(at)gmail(dot)com>, Erik Rijkers <er(at)xs4all(dot)nl>, Kuntal Ghosh <kuntalghosh(dot)2007(at)gmail(dot)com>, Tomas Vondra <tomas(dot)vondra(at)2ndquadrant(dot)com>, Michael Paquier <michael(at)paquier(dot)xyz>, Peter Eisentraut <peter(dot)eisentraut(at)2ndquadrant(dot)com>, PostgreSQL Hackers <pgsql-hackers(at)postgresql(dot)org> |
Subject: | Re: PATCH: logical_work_mem and logical streaming of large in-progress transactions |
Date: | 2020-07-09 02:48:17 |
Message-ID: | CAFPTHDa-LedPs7vNwMyWw_dediE50vPXiN2NDqv32ei=1KKk3A@mail.gmail.com |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-hackers |
On Thu, Jul 9, 2020 at 12:28 PM Amit Kapila <amit(dot)kapila16(at)gmail(dot)com> wrote:
> On Wed, Jul 8, 2020 at 7:31 PM Ajin Cherian <itsajin(at)gmail(dot)com> wrote:
>
> Thanks for showing the interest in patch. How have you ensured that
> streaming is happening? I don't think the proposed patch can ensure
> it for every case because we also rely on logical_decoding_work_mem to
> decide whether to stream/spill, see ReorderBufferCheckMemoryLimit. I
> think with your patch it will allow streaming for cases where we have
> large amount of WAL to decode.
>
>
Maybe I missed something but I looked at ReorderBufferCheckMemoryLimit,
even there it checks the same function ReorderBufferCanStream () and
decides whether to stream or spill. Did I miss something?
while (rb->size >= logical_decoding_work_mem * 1024L)
{
/*
* Pick the largest transaction (or subtransaction) and evict it from
* memory by streaming, if supported. Otherwise, spill to disk.
*/
if (ReorderBufferCanStream(rb) &&
(txn = ReorderBufferLargestTopTXN(rb)) != NULL)
{
/* we know there has to be one, because the size is not zero */
Assert(txn && !txn->toptxn);
Assert(txn->total_size > 0);
Assert(rb->size >= txn->total_size);
ReorderBufferStreamTXN(rb, txn);
}
else
{
I will also add debug and test as you suggested.
regards,
Ajin Cherian
Fujitsu Australia
From | Date | Subject | |
---|---|---|---|
Next Message | Amit Kapila | 2020-07-09 03:11:03 | Re: Resetting spilled txn statistics in pg_stat_replication |
Previous Message | Tom Lane | 2020-07-09 02:40:42 | Re: Is this a bug in pg_current_logfile() on Windows? |