Re: WAL replay bugs

From: Michael Paquier <michael(dot)paquier(at)gmail(dot)com>
To: Heikki Linnakangas <hlinnakangas(at)vmware(dot)com>
Cc: PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: WAL replay bugs
Date: 2014-06-13 08:00:30
Message-ID: CAB7nPqSxxEthw_+Zy=BXv6fGZrS-7=PZvXb+4v6+uDyF=ZcH-A@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Fri, Jun 13, 2014 at 4:50 PM, Michael Paquier
<michael(dot)paquier(at)gmail(dot)com> wrote:
> On Fri, Jun 13, 2014 at 4:48 PM, Heikki Linnakangas
> <hlinnakangas(at)vmware(dot)com> wrote:
>> On 06/13/2014 10:14 AM, Michael Paquier wrote:
>>>
>>> On Mon, Jun 2, 2014 at 9:55 PM, Michael Paquier
>>> <michael(dot)paquier(at)gmail(dot)com> wrote:
>>>>
>>>> On Wed, Apr 23, 2014 at 9:43 PM, Heikki Linnakangas
>>>> <hlinnakangas(at)vmware(dot)com> wrote:
>>>> Perhaps there are parts of what is proposed here that could be made
>>>> more generalized, like the masking functions. So do not hesitate if
>>>> you have any opinion on the matter.
>>>
>>> OK, attached is the result of this hacking:
>>>
>>> Buffer capture facility: check WAL replay consistency
>>>
>>> It is a tool aimed to be used by developers and buildfarm machines
>>> that can be used to check for consistency at page level when replaying
>>> WAL files among several nodes of a cluster (generally master and
>>> standby node).
>>>
>>> This facility is made of two parts:
>>> - A server part, where all the changes happening at page level are
>>> captured and inserted in a file called buffer_captures located at the
>>> root of PGDATA. Each buffer entry is masked to make the comparison
>>> across node consistent (flags like hint bits for example) and then
>>> each buffer is captured is with the following format as a single line
>>> of the output file:
>>> LSN: %08X/%08X page: PAGE_IN_HEXA
>>> Hexadecimal format makes it easier to detect differences between
>>> pages, and format is chosen to facilitate comparison between buffer
>>> entries.
>>> - A client part, located in contrib/buffer_capture_cmp, that can be
>>> used to compare buffer captures between nodes.
>>
>>
>> Oh, you moved the masking code from the client tool to the backend. Why?
>> When debugging, it's useful to have the genuine, non-masked page image
>> available.
> My thought is to share the CPU effort of masking between backends...
And that having a set of API to do page masking on the server side
would be useful for extensions as well. Now that I recall this was one
of the first things that came to my mind when looking at this
facility, thinking that it would be useful to have them in a separate
file, with a dedicated header.
--
Michael

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Kyotaro HORIGUCHI 2014-06-13 08:08:18 pg_resetxlog to clear backup start/end locations.
Previous Message Michael Paquier 2014-06-13 07:50:22 Re: WAL replay bugs