Re: Assertion failure while streaming toasted data

On Tue, May 25, 2021 at 2:34 PM Dilip Kumar <dilipbalaut(at)gmail(dot)com> wrote:
>
> > When the transaction is streamed, I see:
> > ```
> > + opening a streamed block for transaction
> > + table public.toasted: INSERT: id[integer]:9001 other[text]:'bbb' data[text]:'ccc'
> > + table public.toasted: INSERT: id[integer]:9002 other[text]:'ddd' data[text]:'eee'
> > + table public.toasted: INSERT: id[integer]:9003 other[text]:'bar' data[text]:unchanged-toast-datum
> > <snipped>
> > ```
> >
> > For a non-streamed case, the `data[text]` column shows the actual data. That probably manifests into NULL data when downstream handles it.
>
> Yes, I am able to reproduce this, basically, until we get the last
> tuple of the multi insert we can not clear the toast data otherwise we
> can never form a complete tuple. So the only possible fix I can think
> of is to consider the multi-insert WAL without the final multi-insert
> tuple as partial data then we will avoid streaming until we get the
> complete WAL of one multi-insert. I am working on the patch to fix
> this, I will share that in some time.

The attached patch should fix the issue, now the output is like below

===
opening a streamed block for transaction
table public.toasted: INSERT: id[integer]:9001 other[text]:'bbb'
data[text]:'ccc'
table public.toasted: INSERT: id[integer]:9002 other[text]:'ddd'
data[text]:'eee'
table public.toasted: INSERT: id[integer]:9003 other[text]:'bar'
data[text]:'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
<repeat >
===

--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com