BUG #16208: background worker "logical replication worker" was terminated by signal 11: Segmentation

From: PG Bug reporting form <noreply(at)postgresql(dot)org>
To: pgsql-bugs(at)lists(dot)postgresql(dot)org
Cc: mr(dot)kaleek(at)gmail(dot)com
Subject: BUG #16208: background worker "logical replication worker" was terminated by signal 11: Segmentation
Date: 2020-01-16 12:26:08
Message-ID: 16208-784ecaeb269906e4@postgresql.org
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-bugs

The following bug has been logged on the website:

Bug reference: 16208
Logged by: Firdaus Firdaus
Email address: mr(dot)kaleek(at)gmail(dot)com
PostgreSQL version: 12.1
Operating system: Ubuntu 16.04.6 LTS
Description:

I got error when replicate with native logical method from PostgreSQL 11.6
to PostgreSQL 12.1, I will connecting table partition to table partition
from Version 11.6 to Version 12.1, but after all tables already replicated
around more than 2 hours, server can't access and recovery mode.

Actually we are facing problem when replicating partition table from 11.6 to
12.1, but if there is no partition table the replication works so well.
As FYI, our server spec is 16 core and 64GB. And we are using this ubuntu

LSB Version:
core-9.20160110ubuntu0.2-amd64:core-9.20160110ubuntu0.2-noarch:security-9.20160110ubuntu0.2-amd64:security-9.20160110ubuntu0.2-noarch
Distributor ID: Ubuntu
Description: Ubuntu 16.04.6 LTS
Release: 16.04
Codename: xenial

Can you help us to find out the solution and configuration?

please find below :

sudo gdb -p 15803
GNU gdb (Ubuntu 7.11.1-0ubuntu1~16.5) 7.11.1
Copyright (C) 2016 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later
<http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word".
Attaching to process 15803
Reading symbols from /usr/lib/postgresql/12/bin/postgres...(no debugging
symbols found)...done.
Reading symbols from /lib/x86_64-linux-gnu/libpthread.so.0...Reading symbols
from
/usr/lib/debug/.build-id/b1/7c21299099640a6d863e423d99265824e7bb16.debug...done.
done.
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Reading symbols from /usr/lib/x86_64-linux-gnu/libxml2.so.2...(no debugging
symbols found)...done.
Reading symbols from /lib/x86_64-linux-gnu/libpam.so.0...(no debugging
symbols found)...done.
Reading symbols from /lib/x86_64-linux-gnu/libssl.so.1.0.0...(no debugging
symbols found)...done.
Reading symbols from /lib/x86_64-linux-gnu/libcrypto.so.1.0.0...(no
debugging symbols found)...done.
Reading symbols from /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2...(no
debugging symbols found)...done.
Reading symbols from /lib/x86_64-linux-gnu/librt.so.1...Reading symbols from
/usr/lib/debug//lib/x86_64-linux-gnu/librt-2.23.so...done.
done.
Reading symbols from /lib/x86_64-linux-gnu/libdl.so.2...Reading symbols from
/usr/lib/debug//lib/x86_64-linux-gnu/libdl-2.23.so...done.
done.
Reading symbols from /lib/x86_64-linux-gnu/libm.so.6...Reading symbols from
/usr/lib/debug//lib/x86_64-linux-gnu/libm-2.23.so...done.
done.
Reading symbols from /usr/lib/x86_64-linux-gnu/libldap_r-2.4.so.2...(no
debugging symbols found)...done.
Reading symbols from /usr/lib/x86_64-linux-gnu/libicui18n.so.55...(no
debugging symbols found)...done.
Reading symbols from /usr/lib/x86_64-linux-gnu/libicuuc.so.55...(no
debugging symbols found)...done.
Reading symbols from /lib/x86_64-linux-gnu/libsystemd.so.0...(no debugging
symbols found)...done.
Reading symbols from /lib/x86_64-linux-gnu/libc.so.6...Reading symbols from
/usr/lib/debug//lib/x86_64-linux-gnu/libc-2.23.so...done.
done.
Reading symbols from /lib64/ld-linux-x86-64.so.2...Reading symbols from
/usr/lib/debug//lib/x86_64-linux-gnu/ld-2.23.so...done.
done.
Reading symbols from /lib/x86_64-linux-gnu/libz.so.1...(no debugging symbols
found)...done.
Reading symbols from /lib/x86_64-linux-gnu/liblzma.so.5...(no debugging
symbols found)...done.
Reading symbols from /lib/x86_64-linux-gnu/libaudit.so.1...(no debugging
symbols found)...done.
Reading symbols from /usr/lib/x86_64-linux-gnu/libkrb5.so.3...(no debugging
symbols found)...done.
Reading symbols from /usr/lib/x86_64-linux-gnu/libk5crypto.so.3...(no
debugging symbols found)...done.
Reading symbols from /lib/x86_64-linux-gnu/libcom_err.so.2...(no debugging
symbols found)...done.
Reading symbols from /usr/lib/x86_64-linux-gnu/libkrb5support.so.0...(no
debugging symbols found)...done.
Reading symbols from /usr/lib/x86_64-linux-gnu/liblber-2.4.so.2...(no
debugging symbols found)...done.
Reading symbols from /lib/x86_64-linux-gnu/libresolv.so.2...Reading symbols
from /usr/lib/debug//lib/x86_64-linux-gnu/libresolv-2.23.so...done.
done.
Reading symbols from /usr/lib/x86_64-linux-gnu/libsasl2.so.2...(no debugging
symbols found)...done.
Reading symbols from /usr/lib/x86_64-linux-gnu/libgssapi.so.3...(no
debugging symbols found)...done.
Reading symbols from /usr/lib/x86_64-linux-gnu/libgnutls.so.30...(no
debugging symbols found)...done.
Reading symbols from /usr/lib/x86_64-linux-gnu/libstdc++.so.6...(no
debugging symbols found)...done.
Reading symbols from /lib/x86_64-linux-gnu/libgcc_s.so.1...(no debugging
symbols found)...done.
Reading symbols from /usr/lib/x86_64-linux-gnu/libicudata.so.55...(no
debugging symbols found)...done.
Reading symbols from /lib/x86_64-linux-gnu/libselinux.so.1...(no debugging
symbols found)...done.
Reading symbols from /lib/x86_64-linux-gnu/libgcrypt.so.20...(no debugging
symbols found)...done.
Reading symbols from /lib/x86_64-linux-gnu/libkeyutils.so.1...(no debugging
symbols found)...done.
Reading symbols from /usr/lib/x86_64-linux-gnu/libheimntlm.so.0...(no
debugging symbols found)...done.
Reading symbols from /usr/lib/x86_64-linux-gnu/libkrb5.so.26...(no debugging
symbols found)...done.
Reading symbols from /usr/lib/x86_64-linux-gnu/libasn1.so.8...(no debugging
symbols found)...done.
Reading symbols from /usr/lib/x86_64-linux-gnu/libhcrypto.so.4...(no
debugging symbols found)...done.
Reading symbols from /usr/lib/x86_64-linux-gnu/libroken.so.18...(no
debugging symbols found)...done.
Reading symbols from /usr/lib/x86_64-linux-gnu/libp11-kit.so.0...(no
debugging symbols found)...done.
Reading symbols from /usr/lib/x86_64-linux-gnu/libidn.so.11...(no debugging
symbols found)...done.
Reading symbols from /usr/lib/x86_64-linux-gnu/libtasn1.so.6...(no debugging
symbols found)...done.
Reading symbols from /usr/lib/x86_64-linux-gnu/libnettle.so.6...(no
debugging symbols found)...done.
Reading symbols from /usr/lib/x86_64-linux-gnu/libhogweed.so.4...(no
debugging symbols found)...done.
Reading symbols from /usr/lib/x86_64-linux-gnu/libgmp.so.10...(no debugging
symbols found)...done.
Reading symbols from /lib/x86_64-linux-gnu/libpcre.so.3...(no debugging
symbols found)...done.
Reading symbols from /lib/x86_64-linux-gnu/libgpg-error.so.0...(no debugging
symbols found)...done.
Reading symbols from /usr/lib/x86_64-linux-gnu/libwind.so.0...(no debugging
symbols found)...done.
Reading symbols from /usr/lib/x86_64-linux-gnu/libheimbase.so.1...(no
debugging symbols found)...done.
Reading symbols from /usr/lib/x86_64-linux-gnu/libhx509.so.5...(no debugging
symbols found)...done.
Reading symbols from /usr/lib/x86_64-linux-gnu/libsqlite3.so.0...(no
debugging symbols found)...done.
Reading symbols from /lib/x86_64-linux-gnu/libcrypt.so.1...Reading symbols
from /usr/lib/debug//lib/x86_64-linux-gnu/libcrypt-2.23.so...done.
done.
Reading symbols from /usr/lib/x86_64-linux-gnu/libffi.so.6...(no debugging
symbols found)...done.
Reading symbols from /usr/lib/postgresql/12/lib/pg_stat_statements.so...(no
debugging symbols found)...done.
Reading symbols from /usr/lib/postgresql/12/lib/pg_partman_bgw.so...(no
debugging symbols found)...done.
Reading symbols from /lib/x86_64-linux-gnu/libnss_files.so.2...Reading
symbols from
/usr/lib/debug//lib/x86_64-linux-gnu/libnss_files-2.23.so...done.
done.
0x00007f5ce4bf85b3 in __select_nocancel () at
../sysdeps/unix/syscall-template.S:84
84 ../sysdeps/unix/syscall-template.S: No such file or directory.
(gdb) cont
Continuing.

Program received signal SIGUSR1, User defined signal 1.
0x00007f5ce4b30730 in __sigprocmask (how=2, set=0x56135acfba40 <UnBlockSig>,
oset=0x0)
at ../sysdeps/unix/sysv/linux/x86_64/sigprocmask.c:36
36 ../sysdeps/unix/sysv/linux/x86_64/sigprocmask.c: No such file or
directory.
(gdb) detach
Detaching from program: /usr/lib/postgresql/12/bin/postgres, process 15803
(gdb) b errfinish
Breakpoint 1 at 0x56135a814a94

$ for f in `pgrep postgres`; do ls -l /usr/lib/postgresql/12/bin/postgres;
done
-rwxr-xr-x 1 root root 7978032 Nov 12 19:34
/usr/lib/postgresql/12/bin/postgres
-rwxr-xr-x 1 root root 7978032 Nov 12 19:34
/usr/lib/postgresql/12/bin/postgres
-rwxr-xr-x 1 root root 7978032 Nov 12 19:34
/usr/lib/postgresql/12/bin/postgres

$./pg_ctl -D /data/postgres/data -l logfile -c start
-su: ./pg_ctl: No such file or directory
$ pg_ctl -D /data/postgres/data -l logfile -c start
pg_ctl: command not found

on datadir I did not see core.PID, let me know where the dir.
$ ls -lrth
total 144K
drwx------ 2 postgres postgres 4.0K Jan 2 16:34 pg_twophase
drwx------ 2 postgres postgres 4.0K Jan 2 16:34 pg_tblspc
drwx------ 2 postgres postgres 4.0K Jan 2 16:34 pg_snapshots
drwx------ 2 postgres postgres 4.0K Jan 2 16:34 pg_serial
drwx------ 4 postgres postgres 4.0K Jan 2 16:34 pg_multixact
drwx------ 2 postgres postgres 4.0K Jan 2 16:34 pg_dynshmem
drwx------ 2 postgres postgres 4.0K Jan 2 16:34 pg_commit_ts
-rw------- 1 postgres postgres 3 Jan 2 16:34 PG_VERSION
-rw------- 1 postgres postgres 88 Jan 2 16:34 postgresql.auto.conf
drwx------ 2 postgres postgres 4.0K Jan 2 16:34 pg_xact
drwx------ 2 postgres postgres 4.0K Jan 2 16:34 pg_subtrans
drwx------ 6 postgres postgres 4.0K Jan 2 16:44 base
drwx------ 5 postgres postgres 4.0K Jan 3 18:29 pg_replslot
-rw------- 1 postgres postgres 127 Jan 15 12:38 postmaster.opts
drwx------ 2 postgres postgres 4.0K Jan 16 01:00 pg_log
-rw------- 1 postgres postgres 47 Jan 16 01:00 current_logfiles
drwx------ 3 postgres postgres 52K Jan 16 11:00 pg_wal
drwx------ 4 postgres postgres 4.0K Jan 16 11:40 pg_logical
-rw------- 1 postgres postgres 93 Jan 16 11:40 postmaster.pid
drwx------ 2 postgres postgres 4.0K Jan 16 11:40 pg_stat_tmp
drwx------ 2 postgres postgres 4.0K Jan 16 11:40 pg_notify
drwx------ 2 postgres postgres 4.0K Jan 16 11:40 global
drwx------ 2 postgres postgres 4.0K Jan 16 11:40 pg_stat
$ cd postmaster.pid
-su: cd: postmaster.pid: Not a directory
$ more postmaster.pid
15803
/data/postgres/data
1579063112
5432
/var/run/postgresql
*
5432001 169312256
ready

Responses

Browse pgsql-bugs by date

  From Date Subject
Next Message PG Bug reporting form 2020-01-16 13:07:46 BUG #16209: cant install via https://www.postgresql.org/download/linux/redhat/
Previous Message Oleksandr Shulgin 2020-01-16 08:17:23 Re: libpq parameter parsing problem