| From: | SATYANARAYANA NARLAPURAM <satyanarlapuram(at)gmail(dot)com> |
|---|---|
| To: | Antonin Houska <ah(at)cybertec(dot)at> |
| Cc: | PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>, alvherre(at)kurilemu(dot)de |
| Subject: | Re: [PATCH] Compressed TOAST data corruption with REPACK CONCURRENTLY |
| Date: | 2026-04-17 17:40:39 |
| Message-ID: | CAHg+QDf+yGGizLHAOm_q8Y9SR-tuDa4vWYp+riJ1QWnWxeeLQw@mail.gmail.com |
| Views: | Whole Thread | Raw Message | Download mbox | Resend email |
| Thread: | |
| Lists: | pgsql-hackers |
Hi
On Fri, Apr 17, 2026 at 8:45 AM Antonin Houska <ah(at)cybertec(dot)at> wrote:
> SATYANARAYANA NARLAPURAM <satyanarlapuram(at)gmail(dot)com> wrote:
>
> > restore_tuple() in repack.c uses SET_VARSIZE() to reconstruct the
> varlena header when
> > reading back external attributes from the spill file. In this process,
> looks like the flag
> > SET_VARSIZE_COMPRESSED is silently lost. Because of this, when REPACK
> CONCURRENTLY
> > run any concurrently updated column whose value was TOAST-compressed
> ends up with raw
> > compressed bytes behind an "uncompressed" header returning garbled data
> on subsequent reads.
> > It appears that existing tests are using random chars which are
> uncompressable.
> >
> > Please find the attached
> 0001-Fix-restore_tuple-losing-varlena-compression-flag.patch to fix this.
> > Additionally I updated the existing repack_toast test to include the
> scenario I was talking about.
>
> Good catch, thanks!
>
> I'd slightly prefer to fix it w/o checking the varlena type, as
> attached. However, your test fails to reproduce the issue here, so I'm not
> able to verify the fix. I'll take a closer look early next week.
>
I started with that but tried to follow the existing code pattern. This
LGTM.
Please add a comment as well.
>
> --
> Antonin Houska
> Web: https://www.cybertec-postgresql.com
>
>
| From | Date | Subject | |
|---|---|---|---|
| Next Message | DaeMyung Kang | 2026-04-17 17:44:50 | [PATCH] Use direct hash lookup in logicalrep_partmap_invalidate_cb() |
| Previous Message | Maxym Kharchenko | 2026-04-17 17:03:49 | Re: Truncate logs by max_log_size |