RE: Multi-insert for logical replication apply

Dear Bharath,

> The narrower the row, the more rows fit on a page and the bigger the saving.
> The WAL reduction is the main benefit - less I/O, fewer WAL files, faster crash
> recovery, and less WAL to ship for cascading replication. The apply-time change
> is small (under 5%). My hypothesis is that the > other per-row work in the apply
> path outweighs the WAL write, so fewer WAL records alone doesn't move it much.
> I'll also test with streaming=off to see how that compares. (I also have
> small-scale results from 1 to 100K rows showing the gain grows with row count;
> happy to share if useful.)

Hmm. Actually I'm bit unclear the outcome wins the complexity of
adding codes. The benefit might be specific: apply-time cannot be
improved and the amount of WAL cannot be reduced for large tuples. But
it's good to measure the recovery time before and after the patching.
How do others think?

BTW, your patch looks like table_multi_insert() is called even if only
a few tuples are accumulated. I'm not familiar with the TableAM, but
are there any overheads that to use instead of tuple_insert()?

> For some background, I proposed multi-insert for CREATE TABLE AS and materialized
> views earlier [1].

FYI, the thread seems to be re-started.

> Currently the subscriber batches single inserts (this patch) and does a
> multi-insert. Another idea is to add a new message type for multi-insert on the
> publisher when decoding XLOG_HEAP2_MULTI_INSERT, with the subscriber using
> multi-insert only on that path. That keeps things simpler, but it only kicks
in when the publisher used multi-insert itself (e.g. COPY FROM), and it requires
> protocol changes.

I think it has enough motivation to introduce MULTI_INSERT replication
message. It might be narrower window to apply, but it's ensured that
there are lots of tuples to be inserted.

Best regards,
Hayato Kuroda
FUJITSU LIMITED