Re: Proposal: Support Logical replication of large objects

From: Dilip Kumar <dilipbalaut(at)gmail(dot)com>
To: Bernd Helmle <mailings(at)oopsware(dot)de>
Cc: PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>
Subject: Re: Proposal: Support Logical replication of large objects
Date: 2025-12-04 15:11:48
Message-ID: CAFiTN-sJ9JvRHBryPJ08pzz3GA2BnbYr4GuOOTpxYTC66fUr0w@mail.gmail.com
Views: Whole Thread | Raw Message | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Tue, Dec 2, 2025 at 6:41 PM Bernd Helmle <mailings(at)oopsware(dot)de> wrote:
>
> Am Montag, dem 01.12.2025 um 09:27 +0530 schrieb Dilip Kumar:
> > The decoder will then translate the standard single-row
> > insert/update WAL records generated for the internal pg_largeobject
> > table rows into logical operations formatted as LOWRITE: OID, OFFSET,
> > DATA, LENGTH, where the OFFSET is calculated as pageno (corresponding
> > to pg_largeobject row) * LOBLKSIZE. Subsequently, the apply worker on
> > the subscriber side converts this logical operation into lo_open(),
> > lo_seek() and lowrite() operation. While there is potential for
> > further optimization by generating the LOWRITE operation only for the
> > modified data instead of for the entire LOBLKSIZE bytes this behavior
> > is consistent with how updates are currently logged for standard user
> > tables.
>
>
> Thanks for this, i think this is a long awaited feature, at least for
> those workloads that can't easily get rid of LOs...

Right

> I didn't look into your POC (yet), but what happens if the subscriber
> database concurrently does lo_create()? Would that cause conflicting
> OIDs, preventing applying the records decoded from the publisher?

I've implemented decoding for lowrite operations, specifically
handling INSERT statements into pg_largeobject by translating them
into corresponding lowrite operations. However, lo_create is currently
not decoded. The patch expects the user to execute lo_create on the
subscriber explicitly, similar to how DDL for tables is handled.

But I understand the lo_create can be a very frequent activity so
instead of treating them as DDL we should decode and replicate the
lo_create as well and I have asked for the suggestion in my email.
And if we replicate the lo_create as well then any existing
conflicting OID on the subscriber will create conflict and I believe
we can handle that as part of conflict detection.

--
Regards,
Dilip Kumar
Google

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Tom Lane 2025-12-04 15:17:28 Re: Use func(void) for functions with no parameters
Previous Message Álvaro Herrera 2025-12-04 14:36:01 Re: IPC/MultixactCreation on the Standby server