| From: | Tatsuo Ishii <ishii(at)postgresql(dot)org> |
|---|---|
| To: | emond(dot)papegaaij(at)gmail(dot)com |
| Cc: | pgpool-hackers(at)lists(dot)postgresql(dot)org |
| Subject: | Re: Segfault when a watchdog sync changes the primary under running children |
| Date: | 2026-11-10 06:27:03 |
| Message-ID: | 20261110.152703.1432512828592324147.ishii@postgresql.org |
| Views: | Whole Thread | Raw Message | Download mbox | Resend email |
| Thread: | |
| Lists: | pgpool-hackers |
Hi Emond,
> Hi,
>
> We hit a child segfault in pgpool-II 4.7.2 on a 3-node watchdog cluster
> during a planned switchover of the primary. The backtrace:
>
> #0 read_packets_and_process (...) at protocol/pool_process_query.c:5148
> #1 pool_process_query (...) at protocol/pool_process_query.c:299
> #2 do_child (...) at protocol/child.c:464
>
> Line 5148 is FD_ISSET(MAIN(backend)->fd, &exceptmask). The slot for the
> node returned by pool_virtual_main_db_node_id() was NULL.
>
> The crashing node was a watchdog standby. When the leader performed the
> failover, the standby applied it in sync_backend_from_watchdog(). That
> function updates Req_info->primary_node_id and main_node_id, and only then
> kills and re-forks the children. Unlike failover(), it does not set
> Req_info->switching.
>
> A child that was processing a query at that moment got the new primary from
> pool_virtual_main_db_node_id(). Its session had no connection to that node,
> because the node was down when the session started. Later failbacks only
> set need_to_restart on the child, and the client's pooled connection never
> ended.
>
> The attached patch sets Req_info->switching for the duration of the sync,
> as failover() does, so children wait until the sync is done or they are
> killed. While switching is set, register_node_operation_request() doesn't
> signal the main process. So when requests were queued during the sync, the
> patch raises the failover interrupt at the end.
>
> The code is the same on master. With the patch, the watchdog and failover
> regression tests (003, 004, 011–015, 018, 028, 037, 125, 131) pass on
> V4_7_STABLE.
>
> Best regards,
> Emond
Sorry for late response. I am looking into this now.
Regards,
--
Tatsuo Ishii
SRA OSS K.K.
English: http://www.sraoss.co.jp/index_en/
Japanese:http://www.sraoss.co.jp
| From | Date | Subject | |
|---|---|---|---|
| Next Message | Koshino Taiki | 2026-11-11 08:57:22 | Re: Proposal: Recent mutated table tracking in memory |
| Previous Message | Koshino Taiki | 2026-11-04 01:27:13 | Re: [PATCH v1] psql: Do not complete JSON format for COPY FROM |