Re: Logical replication CPU-bound with TRUNCATE/DROP/CREATE many tables

On Thu, Oct 1, 2020 at 2:43 PM Keisuke Kuroda
<keisuke(dot)kuroda(dot)3862(at)gmail(dot)com> wrote:
>
> > > I test the patch on the master HEAD(9796f455) and it works fine.
> > > * make installcheck-world: passed
> > > * walsender process continues to use 100% of the CPU 1core when
> > > TRUNCATE 1000 partition: 1s or less
> > > ** before patch : 230s
> >
> > Does this result indicate that it is still CPU bound but it does the
> > actual decoding and completes in 1s instead of spending 230s mainly to
> > execute unnecessary invalidations?
>
> Okay, to check the decodeing and invalidation times,
> I got the time to return the results to pg_recvlogical and perf.
>
> Before the patch, most of the processing was done
> by hash_seq_search in the ReferenodeMapInvalidateCallback.
> After the patch, this problem has been resolved.
>
> # test
>
> (1) start pg_recvlogical
> (2) INSERT to 2000 partition for creating RelfilenodemapHash
> (3) TRUNCATE 1000 partition
>
> (1)
> pg_recvlogical --create-slot --start -f - --if-not-exists \
> --plugin=test_decoding --slot=test1 --dbname=postgres --username=postgres \
> --option=include-timestamp | gawk '{ print strftime("%Y-%m-%d
> %H:%M:%S"), $0; fflush(); }'
>
> (2)
> DO $$
> DECLARE
> i INT;
> j INT;
> schema TEXT;
> tablename TEXT;
> BEGIN
> FOR i IN 1 .. 2 LOOP
> schema := 'nsp_' || to_char(i, 'FM000');
> tablename := 'tbl_' || to_char(i, 'FM000');
> EXECUTE 'INSERT INTO ' || schema || '.' || tablename || '
> SELECT i,i,i,0,0,0,0,0,0,0,0,0,''a''::bytea,''a''::bytea FROM
> generate_series(0,9999,1) AS i';
> END LOOP;
> END;
> $$ LANGUAGE plpgsql;
>
> (3)
> TRUNCATE TABLE nsp_001.tbl_001;
>
> # before
>
> (3) TRUNCATE
> decode + invalidation = 222s
>
> 2020-10-01 16:58:29 BEGIN 529
> 2020-10-01 17:02:11 COMMIT 529 (at 2020-10-01 16:58:29.281747+09)
>
> (Before the patch was applied, I was concerned that
> the "table nsp_001.tbl_001 ... TRUNCATE" was not outputting.)
>

Why is it that before the patch these were not displayed and after the
patch, we started to see this?

--
With Regards,
Amit Kapila.