Re: Non-replayable WAL records through overflows and >MaxAllocSize lengths

>> A little confused here, does this patch V3 intend to solve this problem "record length 2145386550 at 0/3000060 too long"?
> No, not once the record exists. But it does remove Postgres' ability
> to create such records, thereby solving the problem for all systems
> that generate WAL through Postgres' WAL writing APIs.
>
>> I set up a simple Primary and Standby stream replication environment, and use the above query to run the test for before and after patch v3. The error message still exist, but with different message.
>>
>> Before patch v3, the error is showing below,
>>
>> 2022-06-10 15:32:25.307 PDT [4253] LOG: record length 2145386550 at 0/3000060 too long
>> 2022-06-10 15:32:47.763 PDT [4257] FATAL: terminating walreceiver process due to administrator command
>> 2022-06-10 15:32:47.763 PDT [4253] LOG: record length 2145386550 at 0/3000060 too long
>>
>> After patch v3, the error displays differently
>>
>> 2022-06-10 15:53:53.397 PDT [12848] LOG: record length 2145386550 at 0/3000060 too long
>> 2022-06-10 15:54:07.249 PDT [12852] FATAL: could not receive data from WAL stream: ERROR: requested WAL segment 000000010000000000000045 has already been removed
>> 2022-06-10 15:54:07.275 PDT [12848] LOG: record length 2145386550 at 0/3000060 too long
>>
>> And once the error happens, then the Standby can't continue the replication.
> Did you initiate a new cluster or otherwise skip the invalid record
> you generated when running the instance based on master? It seems to
> me you're trying to replay the invalid record (len > MaxAllocSize),
> and this patch does not try to fix that issue. This patch just tries
> to forbid emitting records larger than MaxAllocSize, as per the check
> in XLogRecordAssemble, so that we wont emit unreadable records into
> the WAL anymore.
>
> Reading unreadable records still won't be possible, but that's also
> not something I'm trying to fix.

Thanks a lot for the clarification. My testing environment is pretty
simple, initdb for Primary, run basebackup and set the connection string
for Standby, then run the "pg_logical_emit_message" query and tail the
log on standby side.

Best regards,

--
David

Software Engineer
Highgo Software Inc. (Canada)
www.highgo.ca