Re: Potential data loss of 2PC files

On Fri, Dec 30, 2016 at 11:22 AM, Michael Paquier
<michael(dot)paquier(at)gmail(dot)com> wrote:
> On Thu, Dec 29, 2016 at 6:41 PM, Ashutosh Bapat
> <ashutosh(dot)bapat(at)enterprisedb(dot)com> wrote:
>> I agree with this.
>> If no prepared transactions were required to be fsynced
>> CheckPointTwoPhase(), do we want to still fsync the directory?
>> Probably not.
>>
>> May be you want to call fsync_fname(TWOPHASE_DIR, true); if
>> serialized_xacts > 0.
>
> Definitely true for the non-recovery code path. But not for restart
> points as there is no GXACT entry created by the redo routine of 2PC
> prepare. We could have a static counter tracking how many 2PC files
> have been flushed since last restart point or not but I am not
> convinced if that's worth the facility.

As per the prologue of the function, it doesn't expect any 2PC files
to be written out in the function i.e. between two checkpoints. Most
of those are created and deleted between two checkpoints. Same would
be true for recovery as well. Thus in most of the cases we shouldn't
need to flush the two phase directory in this function whether during
normal operation or during the recovery. So, we should avoid flushing
repeatedly when it's not needed. I agree that serialized_xacts > 0 is
not the right condition during recovery on standby to flush the two
phase directory.

During crash recovery, 2PC files are present on the disk, which means
the two phase directory has correct record of it. This record can not
change. So, we shouldn't need to flush it again. If that's true
serialized_xacts will be 0 during recovery thus serialized_xacts > 0
condition will still hold.

On a standby however we will have to flush the two phase directory as
part of checkpoint if there were any files left behind in that
directory. We need a different condition there.

>
> That's not true for recovery. So I could go for something like that:
> "If any 2PC files have been flushed, do the same for the parent
> directory to make this information durable on disk. On recovery, issue
> the fsync() anyway, individual 2PC files have already been flushed whe
> replaying their respective XLOG_XACT_PREPARE record.
>

We need to specify recovery (log replay) on standby specifically, I guess.

--
Best Wishes,
Ashutosh Bapat
EnterpriseDB Corporation
The Postgres Database Company