Re: PATCH: logical_work_mem and logical streaming of large in-progress transactions

Hello.

At Fri, 13 Dec 2019 14:46:20 +0530, Amit Kapila <amit(dot)kapila16(at)gmail(dot)com> wrote in
> On Wed, Dec 11, 2019 at 11:46 PM Robert Haas <robertmhaas(at)gmail(dot)com> wrote:
> >
> > On Mon, Dec 2, 2019 at 3:32 AM Dilip Kumar <dilipbalaut(at)gmail(dot)com> wrote:
> > > I have rebased the patch set on the latest head.
> >
> > 0001 looks like a clever approach, but are you sure it doesn't hurt
> > performance when many small XLOG records are being inserted? I think
> > XLogRecordAssemble() can get pretty hot in some workloads.
> >
> > With regard to 0002, logging a separate WAL record for each
> > invalidation seems painful; I think most operations that generate
> > invalidations generate a bunch of them all at once. Perhaps you could
> > just queue up invalidations as they happen, and then force anything
> > that's been queued up to be emitted into WAL just before you emit any
> > WAL record that might need to be decoded.
> >
>
> I feel we can log the invalidations of the entire command at one go if
> we log at CommandEndInvalidationMessages. We already have all the
> invalidations of current command in
> transInvalInfo->CurrentCmdInvalidMsgs. This can save us the effort of
> maintaining a new separate list/queue for invalidations and to a good
> extent, it will ameliorate your concern of logging each invalidation
> separately.

I have a question on this. Does that mean that the current logical
decoder (or reorderbuffer) may emit incorrect result if it made a
catalog change during the current transaction being decoded? If so,
this is not a feature but a bug fix.

regards.

--
Kyotaro Horiguchi
NTT Open Source Software Center