Kernel Tainted

Hi,

We are helping a client test an application On Power8 using Postgres 9.5.4 which has been compiled specifically for the Power.

This is running on sles12sp1 the current kernel is 3.12.49-11

We are getting these kernel warning associated with the postmaster process. The application is handling around 15000TPS It appears that one of these messages is generated for each each transaction which fills up the warn.log quite quickly.

I’m trying to understand what is causing the Tainted kernel messages. the warning is at 'WARNING: at ../net/core/dst.c:287’.
I’ve found one link that indicates that this is ip6 related. https://brunomgalmeida.wordpress.com/2015/07/23/disable-ipv6-postgres-and-pgbouncer/ <https://brunomgalmeida.wordpress.com/2015/07/23/disable-ipv6-postgres-and-pgbouncer/>
Is this accurate? And if these action resolve the error, is it more of a bandaid then an actual fix?

Any comments are appreciated.

A sample of the error is below.

Reiner

2016-10-05T15:08:50.219292+02:00 PPDLMREB04 kernel: ------------[ cut here ]------------
2016-10-05T15:08:50.219335+02:00 PPDLMREB04 kernel: WARNING: at ../net/core/dst.c:287
2016-10-05T15:08:50.219341+02:00 PPDLMREB04 kernel: Modules linked in: af_packet xfs libcrc32c ibmveth(X) rtc_generic btrfs xor raid6_pq dm_service_time sr_mod sd_mod cdrom crc_t10dif ibmvfc(X) scsi_transport_fc ibmvscsi(X) scsi_transport_srp scsi_tgt dm_multipath scsi_dh_rdac scsi_dh_emc scsi_dh_alua scsi_dh dm_mod sg scsi_mod autofs4
2016-10-05T15:08:50.219346+02:00 PPDLMREB04 kernel: Supported: Yes, External
2016-10-05T15:08:50.219350+02:00 PPDLMREB04 kernel: CPU: 28 PID: 113041 Comm: postmaster Tainted: G X 3.12.49-11-default #1
2016-10-05T15:08:50.219355+02:00 PPDLMREB04 kernel: task: c0000003c31100d0 ti: c0000003c1f08000 task.ti: c0000003c1f08000
2016-10-05T15:08:50.219362+02:00 PPDLMREB04 kernel: NIP: c0000000005c0bb0 LR: c000000000594bc4 CTR: c0000000006a0ae0
2016-10-05T15:08:50.219415+02:00 PPDLMREB04 kernel: REGS: c0000003c1f0b630 TRAP: 0700 Tainted: G X (3.12.49-11-default)
2016-10-05T15:08:50.219424+02:00 PPDLMREB04 kernel: MSR: 8000000000029033 <SF,EE,ME,IR,DR,RI,LE> CR: 24022288 XER: 00000016
2016-10-05T15:08:50.219430+02:00 PPDLMREB04 kernel: CFAR: c000000000594bc0 SOFTE: 1
2016-10-05T15:08:50.219438+02:00 PPDLMREB04 kernel: GPR00: c000000000594bc4 c0000003c1f0b8b0 c000000000e8ff00 c0000003c35a1980
2016-10-05T15:08:50.219444+02:00 PPDLMREB04 kernel: GPR04: 0000000000000002 0000000000000000 0000000000000001 00000000ffff0000
2016-10-05T15:08:50.219448+02:00 PPDLMREB04 kernel: GPR08: 0000000000000000 0000000000000001 ffffffffffffffff c000000000710810
2016-10-05T15:08:50.219452+02:00 PPDLMREB04 kernel: GPR12: c0000000006a0ae0 c000000007b2fc00 7fffffffffffffff 000000000000003c
2016-10-05T15:08:50.219457+02:00 PPDLMREB04 kernel: GPR16: 0000000000000000 0000000010735620 0000000000000000 0000000000000000
2016-10-05T15:08:50.219462+02:00 PPDLMREB04 kernel: GPR20: c0000003c1746d80 0000000000000001 0000000000000000 00000000000003a8
2016-10-05T15:08:50.219466+02:00 PPDLMREB04 kernel: GPR24: c0000003c1f0b9f0 0000000000000000 0000000000000000 00000000000003a8
2016-10-05T15:08:50.219470+02:00 PPDLMREB04 kernel: GPR28: 0000000000000002 c0000003c1746a00 0000000000000000 c0000003c35a1980
2016-10-05T15:08:50.219475+02:00 PPDLMREB04 kernel: NIP [c0000000005c0bb0] dst_release+0x50/0xa0
2016-10-05T15:08:50.219479+02:00 PPDLMREB04 kernel: LR [c000000000594bc4] sk_dst_check+0x174/0x180
2016-10-05T15:08:50.219484+02:00 PPDLMREB04 kernel: Call Trace:
2016-10-05T15:08:50.219489+02:00 PPDLMREB04 kernel: [c0000003c1f0b8b0] [c0000000016f7d80] 0xc0000000016f7d80 (unreliable)
2016-10-05T15:08:50.219495+02:00 PPDLMREB04 kernel: [c0000003c1f0b8e0] [c000000000594bc4] sk_dst_check+0x174/0x180
2016-10-05T15:08:50.219501+02:00 PPDLMREB04 kernel: [c0000003c1f0b920] [c00000000068d4cc] ip6_sk_dst_lookup_flow+0x4c/0x2a0
2016-10-05T15:08:50.219506+02:00 PPDLMREB04 kernel: [c0000003c1f0b970] [c0000000006b16b8] udpv6_sendmsg+0x688/0xb20
2016-10-05T15:08:50.219511+02:00 PPDLMREB04 kernel: [c0000003c1f0baf0] [c00000000064b30c] inet_sendmsg+0x9c/0x120
2016-10-05T15:08:50.219515+02:00 PPDLMREB04 kernel: [c0000003c1f0bb40] [c00000000058eabc] sock_sendmsg+0xec/0x140
2016-10-05T15:08:50.219519+02:00 PPDLMREB04 kernel: [c0000003c1f0bc60] [c0000000005921a8] SyS_sendto+0x108/0x150
2016-10-05T15:08:50.219524+02:00 PPDLMREB04 kernel: [c0000003c1f0bd80] [c000000000592240] SyS_send+0x50/0x70
2016-10-05T15:08:50.219530+02:00 PPDLMREB04 kernel: [c0000003c1f0bdc0] [c0000000005933f0] SyS_socketcall+0x2a0/0x440
2016-10-05T15:08:50.219534+02:00 PPDLMREB04 kernel: [c0000003c1f0be30] [c00000000000a17c] syscall_exit+0x0/0x7c
2016-10-05T15:08:50.219541+02:00 PPDLMREB04 kernel: Instruction dump:
2016-10-05T15:08:50.219547+02:00 PPDLMREB04 kernel: 60000000 2fbf0000 419e0038 395f0080 7c2004ac 7d205028 3129ffff 7d20512d
2016-10-05T15:08:50.219553+02:00 PPDLMREB04 kernel: 40c2fff4 7c0004ac 7d2a07b4 55290ffe <0b090000> a13f0060 7928e7e3 40c20018
2016-10-05T15:08:50.219557+02:00 PPDLMREB04 kernel: ---[ end trace a13ca043356ba634 ]---
2016-10-05T15:08:54.230104+02:00 PPDLMREB04 kernel: ------------[ cut here ]------------
2016-10-05T15:08:54.230134+02:00 PPDLMREB04 kernel: WARNING: at ../net/core/dst.c:287
2016-10-05T15:08:54.230138+02:00 PPDLMREB04 kernel: Modules linked in: af_packet xfs libcrc32c ibmveth(X) rtc_generic btrfs xor raid6_pq dm_service_time sr_mod sd_mod cdrom crc_t10dif ibmvfc(X) scsi_transport_fc ibmvscsi(X) scsi_transport_srp scsi_tgt dm_multipath scsi_dh_rdac scsi_dh_emc scsi_dh_alua scsi_dh dm_mod sg scsi_mod autofs4
2016-10-05T15:08:54.230142+02:00 PPDLMREB04 kernel: Supported: Yes, External
2016-10-05T15:08:54.230171+02:00 PPDLMREB04 kernel: CPU: 30 PID: 113103 Comm: postmaster Tainted: G W X 3.12.49-11-default #1
2016-10-05T15:08:54.230174+02:00 PPDLMREB04 kernel: task: c0000003ba9d03d0 ti: c0000003bcb20000 task.ti: c0000003bcb20000
2016-10-05T15:08:54.230176+02:00 PPDLMREB04 kernel: NIP: c0000000005c0bb0 LR: c000000000594bc4 CTR: c0000000006a0ae0
2016-10-05T15:08:54.230178+02:00 PPDLMREB04 kernel: REGS: c0000003bcb23630 TRAP: 0700 Tainted: G W X (3.12.49-11-default)
2016-10-05T15:08:54.230180+02:00 PPDLMREB04 kernel: MSR: 8000000000029033 <SF,EE,ME,IR,DR,RI,LE> CR: 24022488 XER: 00000011
2016-10-05T15:08:54.230182+02:00 PPDLMREB04 kernel: CFAR: c000000000594bc0 SOFTE: 1
2016-10-05T15:08:54.230184+02:00 PPDLMREB04 kernel: GPR00: c000000000594bc4 c0000003bcb238b0 c000000000e8ff00 c0000003c35a1980
2016-10-05T15:08:54.230185+02:00 PPDLMREB04 kernel: GPR04: 0000000000000002 0000000000000000 0000000000000001 00000000ffff0000
2016-10-05T15:08:54.230187+02:00 PPDLMREB04 kernel: GPR08: 0000000000000000 0000000000000001 ffffffffffffffff c000000000710810
2016-10-05T15:08:54.230189+02:00 PPDLMREB04 kernel: GPR12: c0000000006a0ae0 c000000007b30e00 7fffffffffffffff 000000000000003c
2016-10-05T15:08:54.230191+02:00 PPDLMREB04 kernel: GPR16: 0000000000000000 0000000010735620 0000000000000000 0000000000000000
2016-10-05T15:08:54.230192+02:00 PPDLMREB04 kernel: GPR20: c0000003c1746d80 0000000000000001 0000000000000000 00000000000003a8
2016-10-05T15:08:54.230194+02:00 PPDLMREB04 kernel: GPR24: c0000003bcb239f0 0000000000000000 0000000000000000 00000000000003a8
2016-10-05T15:08:54.230196+02:00 PPDLMREB04 kernel: GPR28: 0000000000000002 c0000003c1746a00 0000000000000000 c0000003c35a1980
2016-10-05T15:08:54.230198+02:00 PPDLMREB04 kernel: NIP [c0000000005c0bb0] dst_release+0x50/0xa0
2016-10-05T15:08:54.230199+02:00 PPDLMREB04 kernel: LR [c000000000594bc4] sk_dst_check+0x174/0x180
2016-10-05T15:08:54.230201+02:00 PPDLMREB04 kernel: Call Trace:
2016-10-05T15:08:54.230202+02:00 PPDLMREB04 kernel: [c0000003bcb238b0] [c000000001657d80] 0xc000000001657d80 (unreliable)
2016-10-05T15:08:54.230205+02:00 PPDLMREB04 kernel: [c0000003bcb238e0] [c000000000594bc4] sk_dst_check+0x174/0x180
2016-10-05T15:08:54.230207+02:00 PPDLMREB04 kernel: [c0000003bcb23920] [c00000000068d4cc] ip6_sk_dst_lookup_flow+0x4c/0x2a0
2016-10-05T15:08:54.230209+02:00 PPDLMREB04 kernel: [c0000003bcb23970] [c0000000006b16b8] udpv6_sendmsg+0x688/0xb20
2016-10-05T15:08:54.230212+02:00 PPDLMREB04 kernel: [c0000003bcb23af0] [c00000000064b30c] inet_sendmsg+0x9c/0x120
2016-10-05T15:08:54.230214+02:00 PPDLMREB04 kernel: [c0000003bcb23b40] [c00000000058eabc] sock_sendmsg+0xec/0x140
2016-10-05T15:08:54.230216+02:00 PPDLMREB04 kernel: [c0000003bcb23c60] [c0000000005921a8] SyS_sendto+0x108/0x150
2016-10-05T15:08:54.230217+02:00 PPDLMREB04 kernel: [c0000003bcb23d80] [c000000000592240] SyS_send+0x50/0x70
2016-10-05T15:08:54.230219+02:00 PPDLMREB04 kernel: [c0000003bcb23dc0] [c0000000005933f0] SyS_socketcall+0x2a0/0x440
2016-10-05T15:08:54.230221+02:00 PPDLMREB04 kernel: [c0000003bcb23e30] [c00000000000a17c] syscall_exit+0x0/0x7c
2016-10-05T15:08:54.230222+02:00 PPDLMREB04 kernel: Instruction dump:
2016-10-05T15:08:54.230225+02:00 PPDLMREB04 kernel: 60000000 2fbf0000 419e0038 395f0080 7c2004ac 7d205028 3129ffff 7d20512d
2016-10-05T15:08:54.230227+02:00 PPDLMREB04 kernel: 40c2fff4 7c0004ac 7d2a07b4 55290ffe <0b090000> a13f0060 7928e7e3 40c20018
2016-10-05T15:08:54.230229+02:00 PPDLMREB04 kernel: ---[ end trace a13ca043356ba635 ]---