Re: Segfault when a watchdog sync changes the primary under running children

From: Tatsuo Ishii <ishii(at)postgresql(dot)org>
To: emond(dot)papegaaij(at)gmail(dot)com
Cc: pgpool-hackers(at)lists(dot)postgresql(dot)org
Subject: Re: Segfault when a watchdog sync changes the primary under running children
Date: 2026-11-10 06:27:03
Message-ID: 20261110.152703.1432512828592324147.ishii@postgresql.org
Views: Whole Thread | Raw Message | Download mbox | Resend email
Thread:
Lists: pgpool-hackers

Hi Emond,

> Hi,
>
> We hit a child segfault in pgpool-II 4.7.2 on a 3-node watchdog cluster
> during a planned switchover of the primary. The backtrace:
>
> #0 read_packets_and_process (...) at protocol/pool_process_query.c:5148
> #1 pool_process_query (...) at protocol/pool_process_query.c:299
> #2 do_child (...) at protocol/child.c:464
>
> Line 5148 is FD_ISSET(MAIN(backend)->fd, &exceptmask). The slot for the
> node returned by pool_virtual_main_db_node_id() was NULL.
>
> The crashing node was a watchdog standby. When the leader performed the
> failover, the standby applied it in sync_backend_from_watchdog(). That
> function updates Req_info->primary_node_id and main_node_id, and only then
> kills and re-forks the children. Unlike failover(), it does not set
> Req_info->switching.
>
> A child that was processing a query at that moment got the new primary from
> pool_virtual_main_db_node_id(). Its session had no connection to that node,
> because the node was down when the session started. Later failbacks only
> set need_to_restart on the child, and the client's pooled connection never
> ended.
>
> The attached patch sets Req_info->switching for the duration of the sync,
> as failover() does, so children wait until the sync is done or they are
> killed. While switching is set, register_node_operation_request() doesn't
> signal the main process. So when requests were queued during the sync, the
> patch raises the failover interrupt at the end.
>
> The code is the same on master. With the patch, the watchdog and failover
> regression tests (003, 004, 011–015, 018, 028, 037, 125, 131) pass on
> V4_7_STABLE.
>
> Best regards,
> Emond

Sorry for late response. I am looking into this now.

Regards,
--
Tatsuo Ishii
SRA OSS K.K.
English: http://www.sraoss.co.jp/index_en/
Japanese:http://www.sraoss.co.jp

In response to

Responses

Browse pgpool-hackers by date

  From Date Subject
Next Message Koshino Taiki 2026-11-11 08:57:22 Re: Proposal: Recent mutated table tracking in memory
Previous Message Koshino Taiki 2026-11-04 01:27:13 Re: [PATCH v1] psql: Do not complete JSON format for COPY FROM