Re: Detecting File Damage & Inconsistencies

On Thu, Mar 18, 2021 at 6:20 AM Craig Ringer
<craig(dot)ringer(at)enterprisedb(dot)com> wrote:
>
> On Mon, 15 Mar 2021 at 21:01, David Steele <david(at)pgmasters(dot)net> wrote:
>>
>> On 11/18/20 5:23 AM, Simon Riggs wrote:
>> > On Wed, 18 Nov 2020 at 06:42, Craig Ringer
>> > <craig(dot)ringer(at)enterprisedb(dot)com> wrote:
>> >>
>> >> On Fri, Nov 13, 2020 at 7:24 PM Simon Riggs <simon(at)2ndquadrant(dot)com> wrote:
>> >>>
>> >>>
>> >>> What I'm proposing is an option to add 16 bytes onto each COMMIT
>> >>> record
>> >>
>> >>
>> >> Would it make sense to write this at the time we write a topxid assignment to WAL instead?
>> >>
>> >> Otherwise it won't be accessible to streaming-mode logical decoding.
>> >
>> > Do you mean extend the xl_xact_assignment record? My understanding is
>> > that is not sent in all cases, so not sure what you mean by "instead".
>>
>> Craig, can you clarify?
>
>
> Right. Or write a separate WAL record when the feature is enabled. But it's probably sufficient to write it as an optional chunk on xl_xact_assignment records. We often defer writing them so we can optimise away xacts that never actually wrote anything, but IIRC we still write one before we write any WAL that references the xid. That'd be fine, since we don't need the info any sooner than that during decoding. I'd have to double check that we write it in all cases and won't get to that too soon, but I'm pretty sure we do...

The commit record is optimized away if no xid is assigned, though is
still present if we didn't write any WAL records.

But if a commit record exists in the WAL stream, we want to know where
it came from.

A later patch will add PITR capability based on this information so
attaching it directly to the commit record is fairly important, IMHO.

--
Simon Riggs http://www.EnterpriseDB.com/