RE: Perform streaming logical transactions by background workers and parallel apply

From: "Hayato Kuroda (Fujitsu)" <kuroda(dot)hayato(at)fujitsu(dot)com>
To: "houzj(dot)fnst(at)fujitsu(dot)com" <houzj(dot)fnst(at)fujitsu(dot)com>
Cc: "wangw(dot)fnst(at)fujitsu(dot)com" <wangw(dot)fnst(at)fujitsu(dot)com>, Peter Smith <smithpb2250(at)gmail(dot)com>, Dilip Kumar <dilipbalaut(at)gmail(dot)com>, "shiy(dot)fnst(at)fujitsu(dot)com" <shiy(dot)fnst(at)fujitsu(dot)com>, PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>, Masahiko Sawada <sawada(dot)mshk(at)gmail(dot)com>, Amit Kapila <amit(dot)kapila16(at)gmail(dot)com>, "Hayato Kuroda (Fujitsu)" <kuroda(dot)hayato(at)fujitsu(dot)com>
Subject: RE: Perform streaming logical transactions by background workers and parallel apply
Date: 2022-11-04 09:46:55
Message-ID: TYAPR01MB5866E0165FBBFEA66436E3E6F53B9@TYAPR01MB5866.jpnprd01.prod.outlook.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

> While testing yours, I found that the leader apply worker has been crashed in the
> following case.
> I will dig the failure more, but I reported here for records.

I found a reason why the leader apply worker crasehes.
In parallel_apply_free_worker() the leader sends the pending message to parallel apply worker:

```
+ /*
+ * Resend the pending message to parallel apply worker to cleanup the
+ * queue. Note that parallel apply worker will just ignore this message
+ * as it has already handled this message while applying spooled
+ * messages.
+ */
+ result = shm_mq_send(winfo->mq_handle, strlen(winfo->pending_msg),
+ winfo->pending_msg, false, true);
```

...but the message length should not be calucarete by strlen() because the logicalrep message has '\0'.
PSA the patch to fix it. It can be applied on v42 patch set.

Best Regards,
Hayato Kuroda
FUJITSU LIMITED

Attachment Content-Type Size
0001-fix-wrong-message-length-estimation.patch application/octet-stream 2.2 KB

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Laurenz Albe 2022-11-04 09:49:42 Re: Version 14/15 documentation Section "Alter Default Privileges"
Previous Message vignesh C 2022-11-04 09:36:48 Re: Support logical replication of DDLs