Re: PATCH: logical_work_mem and logical streaming of large in-progress transactions

On Thu, Jul 16, 2020 at 12:23 PM Dilip Kumar <dilipbalaut(at)gmail(dot)com> wrote:
>
> On Wed, Jul 15, 2020 at 6:59 PM Amit Kapila <amit(dot)kapila16(at)gmail(dot)com> wrote:
> >
> >
> > Let me know what you think of the changes?
>
> I have reviewed the changes and looks fine to me.
>

Thanks, I am planning to start committing a few of the infrastructure
patches (especially first two) by early next week as we have resolved
all the open issues and done an extensive review of the entire
patch-set. In the attached version, there is a slight change in one
of the commit messages as compared to the previous version. I would
like to describe in brief the first two patches for the sake of
convenience. Let me know if you or anyone else sees any problems with
these.

The first patch in the series allows us to WAL-log subtransaction and
top-level XID association. The logical decoding infrastructure needs
to know which top-level
transaction the subxact belongs to, in order to decode all the
changes. Until now that might be delayed until commit, due to the
caching (GPROC_MAX_CACHED_SUBXIDS), preventing features requiring
incremental decoding. So we also write the assignment info into WAL
immediately, as part of the next WAL record (to minimize overhead)
only when *wal_level=logical*. We can not remove the existing
XLOG_XACT_ASSIGNMENT WAL as that is required for avoiding overflow in
the hot standby snapshot.

The second patch writes WAL for invalidations at command end with
wal_level=logical. When wal_level=logical, write invalidations at
command end into WAL so that decoding can use this information. This
patch is required to allow the streaming of in-progress transactions
in logical decoding. We still add the invalidations to the cache and
write them to WAL at commit time in RecordTransactionCommit(). This
uses the existing XLOG_INVALIDATIONS xlog record type, from the
RM_STANDBY_ID resource manager (see LogStandbyInvalidations for
details). So existing code relying on those invalidations (e.g. redo)
does not need to be changed. The invalidations written at command end
uses a new xlog record type XLOG_XACT_INVALIDATIONS, from RM_XACT_ID
resource manager. See LogLogicalInvalidations for details. These new
xlog records are ignored by existing redo procedures, which still rely
on the invalidations written to commit records. The invalidations are
decoded and accumulated in top-transaction, and then executed during
replay. This obviates the need to decode the invalidations as part of
a commit record.

The performance testing has shown that there is no performance penalty
with either of the patches but there is some additional WAL which in
most cases is 2-5% but in worst cases and for some specific DDL's it
is up to 15% with the second patch, however, that happens at
wal_level=logical only. We have considered an alternative to blow up
all caches on any DDL in WALSenders and that will have both CPU and
network overhead. For detailed results and analysis see [1][2].

[1] - https://www.postgresql.org/message-id/CAKYtNAqWkPpPFrdEbpPrCan3G_QAcankZarRKKd7cj6vQigM7w%40mail.gmail.com
[2] - https://www.postgresql.org/message-id/CAA4eK1L3PoiBw6uogB7jD5rmdT-GmEF4kOEccS1AWKuBhSkQkQ%40mail.gmail.com

--
With Regards,
Amit Kapila.
EnterpriseDB: http://www.enterprisedb.com