Re: logical streaming of xacts via test_decoding is broken

On Thu, Nov 12, 2020 at 8:45 AM Amit Kapila <amit(dot)kapila16(at)gmail(dot)com> wrote:
>
> On Wed, Nov 11, 2020 at 7:05 PM Dilip Kumar <dilipbalaut(at)gmail(dot)com> wrote:
> >
> > On Wed, Nov 11, 2020 at 6:59 PM Amit Kapila <amit(dot)kapila16(at)gmail(dot)com> wrote:
> > >
> > > On Wed, Nov 11, 2020 at 10:00 AM Dilip Kumar <dilipbalaut(at)gmail(dot)com> wrote:
> > > >
> > > > On Tue, Nov 10, 2020 at 7:20 PM Amit Kapila <amit(dot)kapila16(at)gmail(dot)com> wrote:
> > > > >
> > > > > On Tue, Nov 10, 2020 at 2:25 PM Dilip Kumar <dilipbalaut(at)gmail(dot)com> wrote:
> > > > > >
> > > > >
> > > > > You can probably add a comment as to why we are disallowing this case.
> > > > > I thought of considering 'stream-changes' parameter here because it
> > > > > won't make sense to give this parameter without it, however, it seems
> > > > > that is not necessary but maybe adding a comment
> > > > > here in that regard would be a good idea.
> > > >
> > > > Should we also consider the case that if the user just passed
> > > > skip_empty_streams to true then we should automatically set
> > > > skip_empty_xacts to true?
> > > >
> > >
> > > Is there any problem if we don't do this? Actually, adding
> > > dependencies on parameters is confusing so I want to avoid that unless
> > > it is really required.
> > >
> > > > And we will allow the 'skip_empty_streams' parameter only if
> > > > stream-changes' is true.
> >
> > The reason behind this thought is that if the user doesn't pass any
> > value for 'skip_empty_xacts' then the default value will be false and
> > if the user only pass 'skip_empty_streams' to true then we will error
> > out assuming that skip_empty_xacts is false but skip_empty_streams is
> > true. So it seems instead of error out we can assume that
> > skip_empty_streams true mean skip_empty_xacts is also true if nothing
> > is passed for that.
> >
>
> So, let's see the overall picture here. We can have four options:
> skip_empty_xacts = true, skip_empty_stream = false;
> skip_empty_xacts = true, skip_empty_stream = true;
> skip_empty_xacts = false, skip_empty_stream = false;
> skip_empty_xacts = false, skip_empty_stream = true;
>
> I think we want to say the first three could be supported and for the
> last one either we can either give an error or make its behavior
> similar to option-2? Is this what your understanding as well?

For the last one if the user has specifically passed false for the
skip_empty_xacts then error and if the user did not pass anything for
skip_empty_xacts then make its behavior similar to option-2.

> Another thing I am thinking let's just not expose skip_empty_stream to
> the user and consider the behavior based on the value of
> skip_empty_xacts. Internally, in the code, we can still have different
> variables to distinguish between empty_xacts and empty_streams.

Yeah, even I think in most of the cases it makes more sense to have
skip_empty_xacts and skip_empty_stream similar values. So better we
don't expose skip_empty_stream. I agree that we need to keep two
variables to track the empty stream and empty xacts.

--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com