Re: Parallel Apply

From: wenhui qiu <qiuwenhuifx(at)gmail(dot)com>
To: Tomas Vondra <tomas(at)vondra(dot)me>
Cc: houzj(dot)fnst(at)fujitsu(dot)com, pgsql-hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: Parallel Apply
Date: 2025-11-21 02:31:01
Message-ID: CAGjGUAKKA=raicKoKU4-sviUp=mEw6B8sLd22H4N-twtWVrY1g@mail.gmail.com
Views: Whole Thread | Raw Message | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hi Tomas
> discussing some of this stuff. I'm not saying we should copy all of
> this, but it seems like a good source of inspiration what (not) to do.
I'm not saying we should copy MySQL's implementation. MySQL’s parallel
replication is based on group commit, and PostgreSQL can’t directly adopt
that approach. However, MySQL hashes transactions within the same commit
group by primary and unique keys, assuming that transactions with different
hashes do not conflict (since MySQL's row locks are based on index ). This
allows transactions to be safely replayed in parallel on replicas, and
their execution order within the group doesn’t matter.

Thanks

On Thu, Nov 20, 2025 at 10:50 PM Tomas Vondra <tomas(at)vondra(dot)me> wrote:

> On 11/20/25 14:10, wenhui qiu wrote:
> > Hi
> >> 1) The way the patch determines dependencies seems to be the "writeset"
> >> approach from other replication systems (e.g. MySQL does that). Maybe we
> >> should stick to the same naming?
> >
> >> OK, I did not research the design in MySQL in detail but will try to
> > analyze it.
> > I have some documents for mysql parallel apply binlog event.But after
> > MySQL 8.4, only the writeset mode is available. In scenarios with a
> > primary key or unique key, the replica replay is not ordered, but the
> > data is eventually consistent."
> > https://dev.mysql.com/worklog/task/?id=9556 <https://dev.mysql.com/
> > worklog/task/?id=9556>
> > https://dev.mysql.com/blog-archive/improving-the-parallel-applier-with-
> > writeset-based-dependency-tracking/ <https://dev.mysql.com/blog-archive/
> > improving-the-parallel-applier-with-writeset-based-dependency-tracking/>
> > https://medium.com/airtable-eng/optimizing-mysql-replication-lag-with-
> > parallel-replication-and-writeset-based-dependency-tracking-1fc405cf023c
> > <https://medium.com/airtable-eng/optimizing-mysql-replication-lag-with-
> > parallel-replication-and-writeset-based-dependency-tracking-1fc405cf023c>
> >
>
> FWIW there was a talk about MySQL replication at pgconf.dev 2024
>
> https://www.youtube.com/watch?v=eOfUqh5PltM
>
> discussing some of this stuff. I'm not saying we should copy all of
> this, but it seems like a good source of inspiration what (not) to do.
>
>
> regards
>
> --
> Tomas Vondra
>
>

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Ian Lawrence Barwick 2025-11-21 03:14:00 [PATCH] psql: tab completion for ALTER ROLE ... IN DATABASE ...
Previous Message Peter Smith 2025-11-21 02:05:43 Re: Proposal: Conflict log history table for Logical Replication