Re: [PATCH] Use indexes on the subscriber when REPLICA IDENTITY is full on the publisher

From: Peter Smith <smithpb2250(at)gmail(dot)com>
To: Amit Kapila <amit(dot)kapila16(at)gmail(dot)com>
Cc: Önder Kalacı <onderkalaci(at)gmail(dot)com>, vignesh C <vignesh21(at)gmail(dot)com>, "shiy(dot)fnst(at)fujitsu(dot)com" <shiy(dot)fnst(at)fujitsu(dot)com>, "wangw(dot)fnst(at)fujitsu(dot)com" <wangw(dot)fnst(at)fujitsu(dot)com>, Marco Slot <marco(dot)slot(at)gmail(dot)com>, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>, Andres Freund <andres(at)anarazel(dot)de>, PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>
Subject: Re: [PATCH] Use indexes on the subscriber when REPLICA IDENTITY is full on the publisher
Date: 2023-03-12 21:14:03
Message-ID: CAHut+Ps8gS3cXTTA7iBu3Rm6fioiAD7hqKWp=68U4Cr9kBt0Zg@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Sat, Mar 11, 2023 at 6:05 PM Amit Kapila <amit(dot)kapila16(at)gmail(dot)com> wrote:
>
> On Fri, Mar 10, 2023 at 7:58 PM Önder Kalacı <onderkalaci(at)gmail(dot)com> wrote:
> >
> >
> > I think one option could be to drop some cases altogether, but not sure we'd want that.
> >
> > As a semi-related question: Are you aware of any setting that'd make pg_stat_all_indexes
> > reflect the changes sooner? It is hard to debug what is the bottleneck in the tests, but
> > I have a suspicion that there might be several poll_query_until() calls on
> > pg_stat_all_indexes, which might be the reason?
> >
>
> Yeah, I also think poll_query_until() calls on pg_stat_all_indexes is
> the main reason for these tests taking more time. When I commented
> those polls, it drastically reduces the test time. On looking at
> pgstat_report_stat(), it seems we don't report stats sooner than 1s
> and as most of this patch's test relies on stats, it leads to taking
> more time. I don't have a better idea to verify this patch without
> checking whether the index scan is really used by referring to
> pg_stat_all_indexes. I think trying to reduce the poll calls may help
> in reducing the test timings further. Some ideas on those lines are as
> follows:

If the reason for the stats polling was only to know if some index is
chosen or not, I was wondering if you can just convey the same
information to the TAP test via some conveniently placed (DEBUG?)
logging.

This way the TAP test can do a 'wait_for_log' instead of the
'poll_query_until'. It will probably generate lots of extra logging
but it still might be lots faster that current code because it won't
incur the 1s overheads of the stats.

------
Kind Regards,
Peter Smith.
Fujitsu Australia.

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Attila Soki 2023-03-12 21:56:06 Re: WIP Patch: pg_dump structured
Previous Message Tom Lane 2023-03-12 20:50:06 Re: WIP Patch: pg_dump structured