Re: pgsql: Generic Messages for Logical Decoding

From: Fujii Masao <masao(dot)fujii(at)gmail(dot)com>
To: Andres Freund <andres(at)anarazel(dot)de>
Cc: Simon Riggs <simon(at)2ndquadrant(dot)com>, pgsql-committers <pgsql-committers(at)postgresql(dot)org>
Subject: Re: pgsql: Generic Messages for Logical Decoding
Date: 2016-04-07 03:26:28
Message-ID: CAHGQGwEXrJbouHH9XaJH18J2A+0Uq6Ek9Z=BXsiOCK=9RczR+w@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-committers

On Thu, Apr 7, 2016 at 12:06 AM, Andres Freund <andres(at)anarazel(dot)de> wrote:
>
>
> On April 6, 2016 5:00:54 PM GMT+02:00, Fujii Masao <masao(dot)fujii(at)gmail(dot)com> wrote:
>>On Wed, Apr 6, 2016 at 6:08 PM, Simon Riggs <simon(at)2ndquadrant(dot)com>
>>wrote:
>>> Generic Messages for Logical Decoding
>>>
>>> API and mechanism to allow generic messages to be inserted into WAL
>>that are
>>> intended to be read by logical decoding plugins. This commit adds an
>>optional
>>> new callback to the logical decoding API.
>>
>>When I specify very long text message, pg_xlogdump failed to dump
>>correctly the WAL file which should contain that inserted text message.
>>Isn't this a bug?
>>
>>You can reproduce the problem by the following steps.
>>
>>=# SELECT pg_xlogfile_name(pg_switch_xlog());
>> pg_xlogfile_name
>>--------------------------
>> 000000010000000000000003
>>(1 row)
>>
>>=# SELECT pg_xlogfile_name(pg_current_xlog_location());
>> pg_xlogfile_name
>>--------------------------
>> 000000010000000000000003
>>(1 row)
>>
>>=# SELECT pg_logical_emit_message(true, 'test',
>>repeat('0123456789ABCDEFG', 1024*1024));
>> pg_logical_emit_message
>>-------------------------
>> 0/510CD40
>>(1 row)
>>
>>=# SELECT pg_xlogfile_name(pg_current_xlog_location());
>> pg_xlogfile_name
>>--------------------------
>> 000000010000000000000005
>>(1 row)
>>
>>=# insert into t values(0,0);
>>INSERT 0 1
>>postgres=# SELECT pg_xlogfile_name(pg_current_xlog_location());
>> pg_xlogfile_name
>>--------------------------
>> 000000010000000000000005
>>(1 row)
>>
>>The WAL record of pg_logical_emit_message() should be stored in
>>000000010000000000000004 and 000000010000000000000005.
>>The WAL record of last insertion should be stored in
>>000000010000000000000005.
>>But the results of pg_xlogdump were wrong as follows.
>>
>>$ pg_xlogdump data/pg_xlog/000000010000000000000004
>>rmgr: Standby len (rec/tot): 24/ 50, tx: 0, lsn:
>>0/04000028, prev 0/030146F8, desc: RUNNING_XACTS nextXid 863
>>latestCompletedXid 862 oldestRunningXid 863
>>
>>$ pg_xlogdump data/pg_xlog/000000010000000000000005
>>pg_xlogdump: FATAL: could not find a valid record after 0/5000000
>
> If you specify a file it only looks at records in that file. Try with -s.

In my example, the WAL record of INSERT that I executed last should be in
000000010000000000000005. But pg_xlogdump could not display that.
The output of pg_xlogdump was:

$ pg_xlogdump data/pg_xlog/000000010000000000000005
pg_xlogdump: FATAL: could not find a valid record after 0/5000000

ISTM that if a WAL file starts with the latter half of LOGICAL MESSAGE
WAL data, pg_xlogdump treats it as invalid and gives up dumping the
remaining WAL data in the file.

Regards,

--
Fujii Masao

In response to

Responses

Browse pgsql-committers by date

  From Date Subject
Next Message Stephen Frost 2016-04-07 03:43:31 Re: [COMMITTERS] pgsql: Use GRANT system to manage access to sensitive functions
Previous Message Fujii Masao 2016-04-07 03:13:56 Re: pgsql: Use GRANT system to manage access to sensitive functions