RE: Failed transaction statistics to measure the logical replication progress

On Friday, December 3, 2021 3:12 PM vignesh C <vignesh21(at)gmail(dot)com> wrote:
> Thanks for the updated patch.
> Currently we are storing the commit count, error_count and abort_count for
> each table of the table sync operation. If we have thousands of tables, we will
> be storing the information for each of the tables.
> Shouldn't we be storing the consolidated information in this case.
> diff --git a/src/backend/replication/logical/tablesync.c
> b/src/backend/replication/logical/tablesync.c
> index f07983a..02e9486 100644
> --- a/src/backend/replication/logical/tablesync.c
> +++ b/src/backend/replication/logical/tablesync.c
> @@ -1149,6 +1149,11 @@ copy_table_done:
> MyLogicalRepWorker->relstate_lsn = *origin_startpos;
> SpinLockRelease(&MyLogicalRepWorker->relmutex);
>
> + /* Report the success of table sync. */
> + pgstat_report_subworker_xact_end(MyLogicalRepWorker->subid,
> +
> MyLogicalRepWorker->relid,
> +
> 0 /* no logical message type */ );
Okay.

I united all stats into that of apply worker.
In line with this change, I fixed the TAP tests as well
to cover the updates of stats done by table sync workers.

Also, during my self-review, I noticed that
I should call pgstat_report_subworker_xact_end() before
process_syncing_tables() because it can lead to process
exit, which results in missing one increment of the stats columns.
I noted this point in a comment as well.

Best Regards,
Takamichi Osumi