Re: row filtering for logical replication

From: "Euler Taveira" <euler(at)eulerto(dot)com>
To: "Rahila Syed" <rahilasyed90(at)gmail(dot)com>
Cc: "Peter Eisentraut" <peter(dot)eisentraut(at)enterprisedb(dot)com>, Önder Kalacı <onderkalaci(at)gmail(dot)com>, japin <japinli(at)hotmail(dot)com>, "Michael Paquier" <michael(at)paquier(dot)xyz>, "David Steele" <david(at)pgmasters(dot)net>, "Craig Ringer" <craig(at)2ndquadrant(dot)com>, "Tomas Vondra" <tomas(dot)vondra(at)2ndquadrant(dot)com>, "Amit Langote" <amitlangote09(at)gmail(dot)com>, "PostgreSQL Hackers" <pgsql-hackers(at)lists(dot)postgresql(dot)org>
Subject: Re: row filtering for logical replication
Date: 2021-03-29 13:16:44
Message-ID: b02b43c7-3b84-4ca4-8218-94ea486c2fce@www.fastmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Mon, Mar 29, 2021, at 6:45 AM, Rahila Syed wrote:
> While running some tests on v13 patches, I noticed that, in case the published table data
> already exists on the subscriber database before creating the subscription, at the time of
> CREATE subscription/table synchronization, an error as seen as follows
>
> With the patch:
>
> 2021-03-29 14:32:56.265 IST [78467] STATEMENT: CREATE_REPLICATION_SLOT "pg_16406_sync_16390_6944995860755251708" LOGICAL pgoutput USE_SNAPSHOT
> 2021-03-29 14:32:56.279 IST [78467] LOG: could not send data to client: Broken pipe
> 2021-03-29 14:32:56.279 IST [78467] STATEMENT: COPY (SELECT aid, bid, abalance, filler FROM public.pgbench_accounts WHERE (aid > 0)) TO STDOUT
> 2021-03-29 14:32:56.279 IST [78467] FATAL: connection to client lost
> 2021-03-29 14:32:56.279 IST [78467] STATEMENT: COPY (SELECT aid, bid, abalance, filler FROM public.pgbench_accounts WHERE (aid > 0)) TO STDOUT
> 2021-03-29 14:33:01.302 IST [78470] LOG: logical decoding found consistent point at 0/4E2B8460
> 2021-03-29 14:33:01.302 IST [78470] DETAIL: There are no running transactions.
Rahila, I tried to reproduce this issue with the attached script but no luck. I always get

> Without the patch:
>
> 2021-03-29 15:05:01.581 IST [79029] ERROR: duplicate key value violates unique constraint "pgbench_branches_pkey"
> 2021-03-29 15:05:01.581 IST [79029] DETAIL: Key (bid)=(1) already exists.
> 2021-03-29 15:05:01.581 IST [79029] CONTEXT: COPY pgbench_branches, line 1
> 2021-03-29 15:05:01.583 IST [78538] LOG: background worker "logical replication worker" (PID 79029) exited with exit code 1
> 2021-03-29 15:05:06.593 IST [79031] LOG: logical replication table synchronization worker for subscription "test_sub2", table "pgbench_branches" has started
... this message. The code that reports this error is from the COPY command.
Row filter modifications has no control over it. It seems somehow your
subscriber close the replication connection causing this issue. Can you
reproduce it consistently? If so, please share your steps.

--
Euler Taveira
EDB https://www.enterprisedb.com/

Attachment Content-Type Size
test-row-filter-pgbench.sh application/x-shellscript 3.0 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Matthias van de Meent 2021-03-29 13:18:37 Re: Calendar support in localization
Previous Message Amit Langote 2021-03-29 12:41:49 Re: making update/delete of inheritance trees scale better