Re: Segfault when a watchdog sync changes the primary under running children

From: Bo Peng <pengbo(at)sraoss(dot)co(dot)jp>
To: Emond Papegaaij <emond(dot)papegaaij(at)gmail(dot)com>
Cc: "pgpool-hackers(at)lists(dot)postgresql(dot)org" <pgpool-hackers(at)lists(dot)postgresql(dot)org>
Subject: Re: Segfault when a watchdog sync changes the primary under running children
Date: 2026-10-22 09:22:53
Message-ID: OSOP286MB7806335AB9E3FEE945354582F2652@OSOP286MB7806.JPNP286.PROD.OUTLOOK.COM
Views: Whole Thread | Raw Message | Download mbox | Resend email
Thread:
Lists: pgpool-hackers

Hi Emond,

Thank you very much for the detailed analysis and the patch!

I really appreciate your efforts in identifying the root cause,
providing a fix, and running the regression tests.

I will review your patch and get back to you.

Best regards,
---
Bo Peng <pengbo(at)sraoss(dot)co(dot)jp>
SRA OSS K.K.
URL: https://www.sraoss.co.jp/

________________________________________
差出人: Emond Papegaaij <emond(dot)papegaaij(at)gmail(dot)com>
送信: 2026 年 10 月 19 日 (月曜日) 19:46
宛先: pgpool-hackers(at)lists(dot)postgresql(dot)org <pgpool-hackers(at)lists(dot)postgresql(dot)org>
件名: Segfault when a watchdog sync changes the primary under running children

Hi,

We hit a child segfault in pgpool-II 4.7.2 on a 3-node watchdog cluster during a planned switchover of the primary. The backtrace:

#0  read_packets_and_process (...) at protocol/pool_process_query.c:5148
#1  pool_process_query (...) at protocol/pool_process_query.c:299
#2  do_child (...) at protocol/child.c:464

Line 5148 is FD_ISSET(MAIN(backend)->fd, &exceptmask). The slot for the node returned by pool_virtual_main_db_node_id() was NULL.

The crashing node was a watchdog standby. When the leader performed the failover, the standby applied it in sync_backend_from_watchdog(). That function updates Req_info->primary_node_id and main_node_id, and only then kills and re-forks the children. Unlike failover(), it does not set Req_info->switching.

A child that was processing a query at that moment got the new primary from pool_virtual_main_db_node_id(). Its session had no connection to that node, because the node was down when the session started. Later failbacks only set need_to_restart on the child, and the client's pooled connection never ended.

The attached patch sets Req_info->switching for the duration of the sync, as failover() does, so children wait until the sync is done or they are killed. While switching is set, register_node_operation_request() doesn't signal the main process. So when requests were queued during the sync, the patch raises the failover interrupt at the end.

The code is the same on master. With the patch, the watchdog and failover regression tests (003, 004, 011–015, 018, 028, 037, 125, 131) pass on V4_7_STABLE.

Best regards,
Emond

In response to

Browse pgpool-hackers by date

  From Date Subject
Next Message Bo Peng 2026-10-22 09:41:03 Re: target_session_attrs
Previous Message Khem Raj 2026-10-21 15:31:04 [PATCH] configure: Check the size of long and long long