| From: | Akashkiran Shivakumar <akashkiran(dot)foss(at)gmail(dot)com> |
|---|---|
| To: | pgsql-general(at)lists(dot)postgresql(dot)org |
| Subject: | Re: Can postgres replication slot using pgoutput release multiple CDC records for a single update to a particular row |
| Date: | 2025-08-10 03:00:28 |
| Message-ID: | CADTnmPjxCQMLGMw5+t2qY0inhJoY85u8MNPRGzsZK7fPBevH7Q@mail.gmail.com |
| Views: | Whole Thread | Raw Message | Download mbox | Resend email |
| Thread: | |
| Lists: | pgsql-general |
Can someone take a look at this ? Any help would be appreciated
Regards,
Akashkiran
On Tue, Jul 29, 2025 at 3:37 PM Akashkiran Shivakumar <
akashkiran(dot)foss(at)gmail(dot)com> wrote:
> Hello,
> I have a postgres database (major version 13) and am doing CDC by using a
> replication slot with pgoutput. In our data lake, we see that there are
> multiple updates (3 in this case) happening to the same row as part of the
> same transaction. This doesn't make sense if we look at them as separate
> updates. The expectation was that the row was updated once and a single CDC
> record was pushed out. I haven't completely ruled out whether multiple
> updates happened in that transaction but I wanted to ask the community if
> it is possible that a single update statement could spill over as multiple
> CDC update records by pgoutput / postgres ?
>
> If yes, could you possibly point to the testcases or code where this might
> happen? Any blogs or suggestions are welcome. LMK if you need further
> information
>
> P.S: Each update seen in the data lake changes at least one field in the
> row. The entire row with all the columns are pushed out for the 3 updates
> seen in the data lake. From the application perspective, it makes sense if
> the 3 updates were merged into a single update record
>
> Regards,
> Akashkiran
>
| From | Date | Subject | |
|---|---|---|---|
| Next Message | jian he | 2025-08-10 15:20:12 | Re: Emitting JSON to file using COPY TO |
| Previous Message | Adrian Klaver | 2025-08-08 22:43:13 | Re: When UPDATE a row in a table with BEFORE ROW UPDATE trigger, the XMAX of new tuple is set to current XID |