Re: Historic snapshot doesn't track txns committed in BUILDING_SNAPSHOT state

On Fri, Jun 27, 2025 at 3:30 AM Ajin Cherian <itsajin(at)gmail(dot)com> wrote:
>
> On Fri, Jun 27, 2025 at 3:48 PM cca5507 <cca5507(at)qq(dot)com> wrote:
> >
> > Hi,
> >
> > I refactor the code and fix the git apply warning according to [1].
> >
> > Here are the new version patches.
> >
> > --
> > Regards,
> > ChangAo Chen
> >
> > [1] https://www.postgresql.org/message-id/Zrmh7X8jYCbFYXjH%40ip-10-97-1-34.eu-west-3.compute.internal
>
> I see this problem is similar to the bug reported in [1], and your fix
> also addresses the issue reported there. Although I like your approach
> of tracking changes starting from the BUILDING_SNAPSHOT state, I’d
> like to suggest an alternative.
>
> While debugging that issue, my plan was not to track catalog changes
> prior to SNAPBUILD_CONSISTENT, but instead to ensure we don’t use
> snapshots built before SNAPBUILD_CONSISTENT, since we don’t track
> catalog changes in those states. We should discard previously built
> snapshots and rebuild them once we reach the SNAPBUILD_CONSISTENT
> state. At that point, all necessary transactions would have been
> committed, and builder->xmin would have advanced enough to decode all
> transactions from then on.
>
> The problem is that previously built snapshots hang around without the
> latest xmin and xmax, and we tend to reuse them. We should ensure that
> all txn->base_snapshot and builder->snapshot snapshots built in the
> SNAPBUILD_FULL_SNAPSHOT state are rebuilt once we reach
> SNAPBUILD_CONSISTENT. For this, we need to track when the snapshot was
> built. There is already a field in ReorderBufferTXN -
> 'base_snapshot_lsn' which we can use. If base_snapshot_lsn <
> builder->start_decoding_at, then we should rebuild the snapshot. Just
> a thought.

How does this idea address the distributed snapshots? Replaying a
transaction could use multiple snapshots if catalog-change
transactions are committed when the transaction was running. I think
that's true also for transactions that started in FULL state and were
committed in CONSISTENT state.

Regards,

--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com