[prev in list] [next in list] [prev in thread] [next in thread]
List: net-snmp-bugs
Subject: [ net-snmp-Bugs-3165949 ] snmpd crashes on AIX 6.1.6.2
From: SourceForge.net <noreply () sourceforge ! net>
Date: 2012-06-15 1:07:28
Message-ID: E1SfL0b-0001Mw-Uu () sfs-ml-4 ! v29 ! ch3 ! sourceforge ! com
[Download RAW message or body]
Bugs item #3165949, was opened at 2011-01-26 07:04
Message generated for change (Comment added) made by blentz
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=112694&aid=3165949&group_id=12694
Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: agent
Group: aix
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: blentz (blentz)
Assigned to: Nobody/Anonymous (nobody)
Summary: snmpd crashes on AIX 6.1.6.2
Initial Comment:
We're experiencing core dumps in snmpd on AIX 6.1 TL6 SP2. Looks like an issue with \
var_ipRouteEntry. Can someone take a look and advise?
I will gladly provide a test platform if the developers don't have one (I'm aware \
that AIX isn't prevalent).
load_rtentries(0x0) at 0xd4b2d3f8
load_rtentries(0x0) at 0xd4b2d3f8
Route_Scan_Reload() at 0xd4b2d998
var_ipRouteEntry(0x2ff1c068, 0x2ff1c27c, 0x2ff1c47c, 0x0, 0x2ff1c278, 0x2ff1c274) at \
0xd4b2cbb0 netsnmp_old_api_helper(0x2001b808, 0x2001bca8, 0x20127468, 0x20127488) at \
0xd4c301a0 netsnmp_call_handler(0x2001b808, 0x2001bca8, 0x20127468, 0x20127488) at \
0xd4c5069c netsnmp_call_handlers(0x2001bca8, 0x20127468, 0x20127488) at 0xd4c50a9c
handle_var_requests(0x20126df8) at 0xd4c5eedc
handle_getnext_loop(0x20126df8) at 0xd4c5fd40
handle_pdu(0x20126df8) at 0xd4c60954
netsnmp_handle_request(0x20126df8, 0x0) at 0xd4c603e0
handle_snmp_packet(0x1, 0x20115a78, 0x63e6e980, 0x20126af8, 0x0) at 0xd4c5cef0
_sess_process_packet(0x20115a18, 0x20115a78, 0x20115a38, 0x201158c8, 0x201159c8, \
0x14, 0x20116ab8, 0x2c) at 0xd4cb089c _sess_read(0x20115a18, 0x2ff1c9d4) at \
0xd4cb1fa0 snmp_sess_read(0x20115a18, 0x2ff1c9d4) at 0xd4cb2044
snmp_read(0x2ff1c9d4) at 0xd4cb0a90
receive() at 0x10002c2c
main(0x5, 0x2ff22bb8) at 0x100025f0
(dbx)
----------------------------------------------------------------------
> Comment By: blentz (blentz)
Date: 2012-06-14 18:07
Message:
Bump?
----------------------------------------------------------------------
Comment By: blentz (blentz)
Date: 2012-05-16 11:26
Message:
Problem also exists on AIX 6.1 TL6 SP7.
We've tried upgrading 5.4.2.1 to 5.5, 5.6, 5.7.1, tried SVN nightlies, RCs
and PREs, but because of the missing HOST-RESOURCES data for the process
table and storage table, we've continued to exclude .1.3.6.1.2.1.4.21 in
most of our configs to prevent them from crashing.
I appreciate any help anyone can provide.
----------------------------------------------------------------------
Comment By: blentz (blentz)
Date: 2012-05-16 11:23
Message:
Are there any updates? This still continues to plague over 50% of our
systems. I managed to find another system, recently installed, and took a
backtrace:
$ /opt/local/net-snmp/sbin/snmpd --version
NET-SNMP version: 5.4.2.1
Web: http://www.net-snmp.org/
Email: net-snmp-coders@lists.sourceforge.net
$ sudo /opt/local/net-snmp/sbin/snmpd -Le -p
/opt/local/net-snmp/var/run/snmpd.pid -a -f 0.0.0.0:16161
nlist err: neither proc nor _proc found.
NET-SNMP version 5.4.2.1
Received SNMP packet(s) from UDP: [0.0.0.0]->[10.0.17.14]:-5505
Memory fault(coredump)
$ sudo file core
core: AIX core file fulldump 32-bit, snmpd
$ sudo dbx /opt/local/net-snmp/sbin/snmpd core
Type 'help' for help.
[using memory image in core]
reading symbolic information ...warning: no source compiled with -g
Segmentation fault in read at 0xd013910c ($t1)
0xd013910c (read+0x2c) 9421ff20 stwu r1,-224(r1)
(dbx) where
read(0x3, 0x2df23128, 0x80000000) at 0xd013910c
klread(0x2df23128, 0x18) at 0xd1ad5efc
klookup(0x0, 0x2df23128, 0x18) at 0xd1ad5fdc
load_rtentries(0x0) at 0xd19d135c
load_rtentries(0x0) at 0xd19d13f8
load_rtentries(0x0) at 0xd19d13f8
load_rtentries(0x0) at 0xd19d13f8
<Thousands of these>
load_rtentries(0x0) at 0xd19d13f8
load_rtentries(0x0) at 0xd19d13f8
load_rtentries(0x0) at 0xd19d13f8
Route_Scan_Reload() at 0xd19d1998
var_ipRouteEntry(0x2ff1c1b8, 0x2ff1c3cc, 0x2ff1c5cc, 0x0, 0x2ff1c3c8,
0x2ff1c3c4) at 0xd19d0bb0
netsnmp_old_api_helper(0x2001bd98, 0x2001c238, 0x2012ab58, 0x2012ab78) at
0xd1a991a0
netsnmp_call_handler(0x2001bd98, 0x2001c238, 0x2012ab58, 0x2012ab78) at
0xd1ab969c
netsnmp_call_handlers(0x2001c238, 0x2012ab58, 0x2012ab78) at 0xd1ab9a9c
handle_var_requests(0x2011a008) at 0xd1ac7edc
handle_getnext_loop(0x2011a008) at 0xd1ac8d40
handle_pdu(0x2011a008) at 0xd1ac9954
netsnmp_handle_request(0x2011a008, 0x0) at 0xd1ac93e0
handle_snmp_packet(0x1, 0x2011a0e8, 0x65654897, 0x2012a258, 0x0) at
0xd1ac5ef0
_sess_process_packet(0x2011a088, 0x2011a0e8, 0x2011a0a8, 0x20119f38,
0x20119f88, 0x14, 0x2011a248, 0x2c) at 0xd1b1989c
_sess_read(0x2011a088, 0x2ff1cb24) at 0xd1b1afa0
snmp_sess_read(0x2011a088, 0x2ff1cb24) at 0xd1b1b044
snmp_read(0x2ff1cb24) at 0xd1b19a90
receive() at 0x10002c2c
main(0x7, 0x2ff22d00) at 0x100025f0
(dbx)
----------------------------------------------------------------------
Comment By: blentz (blentz)
Date: 2011-09-02 08:18
Message:
This is 5.4.2.1.
----------------------------------------------------------------------
Comment By: Niels Baggesen (nba)
Date: 2011-05-29 04:59
Message:
What version of Net-SNMP are you testing?
----------------------------------------------------------------------
Comment By: blentz (blentz)
Date: 2011-05-29 04:49
Message:
Okay, still dumps core with the same errors even while compiled using
--without-kmem-usage
----------------------------------------------------------------------
Comment By: blentz (blentz)
Date: 2011-05-29 04:45
Message:
I recompiled on AIX 5.3 TL 12 SP 2 using --without-kmem-usage and make test
reports the following:
58: testing if snmpnetstat finds the running agent in udpTable... FAIL
59: testing full snmpwalk (SNMPv2c) against agent (may take time)... ..ok
Summary: 58 / 59 succeeded.
make: 1254-004 The error code from the last command is 1.
Stop.
make: 1254-004 The error code from the last command is 2.
Stop.
----------------------------------------------------------------------
Comment By: blentz (blentz)
Date: 2011-05-29 04:28
Message:
It seems both the iproute and the hrstorage issues are similar.
I can run net-snmp on my affected AIX systems if I use:
view systemview included .1
view systemview excluded .1.3.6.1.2.1.4
view systemview excluded .1.3.6.1.2.1.25.2.3.1
My compile is still running. The vast majority of my systems (175+) work
fine with this same copy of binaries, on the same version of AIX, at the
same TL and SP levels.
It's very odd.
----------------------------------------------------------------------
Comment By: blentz (blentz)
Date: 2011-05-29 04:20
Message:
-DALL gives:
wtest: oid in: HOST-RESOURCES-MIB::hrStorageType.39
trace: netsnmp_subtree_find_prev(): agent_registry.c, 1529:
wtest: oid cmp: HOST-RESOURCES-MIB::hrStorageDescr --- off = 11, in off =
0 test = 1
trace: netsnmp_add_varbind_to_cache(): snmp_agent.c, 1870:
snmp_agent: add_vb_to_cache(2012d8b8, 1,
HOST-RESOURCES-MIB::hrStorageType.39, 2006e668)
trace: _callback_lock(): callback.c, 130:
9:callback:lock: locked (APP,null)
trace: snmp_call_callbacks(): callback.c, 322:
callback: START calling callbacks for maj=1 min=12
trace: snmp_call_callbacks(): callback.c, 336:
callback: calling a callback for maj=1 min=12
trace: vacm_check_view(): mibgroup/mibII/vacm_conf.c, 1300:
mibII/vacm_vars: vacm_in_view: ver=1, community=aix2011
trace: netsnmp_udp_getSecName(): snmpUDPDomain.c, 1259:
netsnmp_udp_getSecName: resolve <"aix2011", 0x0a00110e>
trace: netsnmp_udp_getSecName(): snmpUDPDomain.c, 1264:
netsnmp_udp_getSecName: compare <"aix2011", 0x00000000/0x00000000>...
SUCCESS
trace: netsnmp_subtree_find_first(): agent_registry.c, 156:
subtree: looking for subtree for context: ""
trace: netsnmp_subtree_find_first(): agent_registry.c, 160:
subtree: found one for: ""
trace: vacm_check_view(): mibgroup/mibII/vacm_conf.c, 1427:
mibII/vacm_vars: vacm_in_view: sn=notConfigUser, gn=notConfigGroup,
vn=systemview
trace: netsnmp_view_subtree_check(): vacm.c, 532:
9:vacm:checkSubtree: view systemview
trace: netsnmp_view_subtree_check(): vacm.c, 569:
9:vacm:checkSubtree:
systemview matched?
trace: netsnmp_view_subtree_check(): vacm.c, 630:
9:vacm:checkSubtree: matched
trace: netsnmp_view_subtree_check(): vacm.c, 653:
vacm:checkSubtree: , included
trace: snmp_call_callbacks(): callback.c, 348:
callback: END calling callbacks for maj=1 min=12 (1 called)
trace: _callback_unlock(): callback.c, 164:
9:callback:lock: unlocked (APP,null)
trace: netsnmp_add_varbind_to_cache(): snmp_agent.c, 1941:
snmp_agent: tp->start HOST-RESOURCES-MIB::hrStorageType, tp->end
HOST-RESOURCES-MIB::hrStorageDescr,
trace: netsnmp_add_varbind_to_cache(): snmp_agent.c, 1961:
verbose:asp: asp 2012d8b8 reqinfo 2012df28 assigned to request
trace: netsnmp_add_varbind_to_cache(): snmp_agent.c, 1968:
verbose:asp: asp 2012d8b8 reqinfo 2012df28 assigned to request
trace: netsnmp_call_handlers(): agent_handler.c, 510:
handler:calling: main handler bulk_to_next
trace: netsnmp_call_handler(): agent_handler.c, 430:
handler:calling: calling handler bulk_to_next for mode GETNEXT
trace: netsnmp_call_handler(): agent_handler.c, 438:
handler:returned: handler bulk_to_next returned 0
trace: netsnmp_call_handler(): agent_handler.c, 430:
handler:calling: calling handler old_api for mode GETNEXT
trace: header_hrstoreEntry(): host/hr_storage.c, 373:
host/hr_storage: var_hrstoreEntry: request
HOST-RESOURCES-MIB::hrStorageType.39 exact=0
trace: header_hrstoreEntry(): host/hr_storage.c, 381:
host/hr_storage: var_hrstoreEntry: compare
HOST-RESOURCES-MIB::hrStorageType => 1
(index 31 ....HOST-RESOURCES-MIB::hrStorageType.31
(index 32 ....HOST-RESOURCES-MIB::hrStorageType.32
(index 33 ....HOST-RESOURCES-MIB::hrStorageType.33
(index 34 ....HOST-RESOURCES-MIB::hrStorageType.34
(index 35 ....HOST-RESOURCES-MIB::hrStorageType.35
(index 36 ....HOST-RESOURCES-MIB::hrStorageType.36
(index 37 ....HOST-RESOURCES-MIB::hrStorageType.37
(index 38 ....HOST-RESOURCES-MIB::hrStorageType.38
(index 39 ....HOST-RESOURCES-MIB::hrStorageType.39
(index 40 ....HOST-RESOURCES-MIB::hrStorageType.40
trace: header_hrstoreEntry(): host/hr_storage.c, 476:
host/hr_storage: var_hrstoreEntry: process
HOST-RESOURCES-MIB::hrStorageType.40 (ffffffff)
Memory fault(coredump)
----------------------------------------------------------------------
Comment By: blentz (blentz)
Date: 2011-05-29 04:20
Message:
-DALL gives:
wtest: oid in: HOST-RESOURCES-MIB::hrStorageType.39
trace: netsnmp_subtree_find_prev(): agent_registry.c, 1529:
wtest: oid cmp: HOST-RESOURCES-MIB::hrStorageDescr --- off = 11, in off =
0 test = 1
trace: netsnmp_add_varbind_to_cache(): snmp_agent.c, 1870:
snmp_agent: add_vb_to_cache(2012d8b8, 1,
HOST-RESOURCES-MIB::hrStorageType.39, 2006e668)
trace: _callback_lock(): callback.c, 130:
9:callback:lock: locked (APP,null)
trace: snmp_call_callbacks(): callback.c, 322:
callback: START calling callbacks for maj=1 min=12
trace: snmp_call_callbacks(): callback.c, 336:
callback: calling a callback for maj=1 min=12
trace: vacm_check_view(): mibgroup/mibII/vacm_conf.c, 1300:
mibII/vacm_vars: vacm_in_view: ver=1, community=aix2011
trace: netsnmp_udp_getSecName(): snmpUDPDomain.c, 1259:
netsnmp_udp_getSecName: resolve <"aix2011", 0x0a00110e>
trace: netsnmp_udp_getSecName(): snmpUDPDomain.c, 1264:
netsnmp_udp_getSecName: compare <"aix2011", 0x00000000/0x00000000>...
SUCCESS
trace: netsnmp_subtree_find_first(): agent_registry.c, 156:
subtree: looking for subtree for context: ""
trace: netsnmp_subtree_find_first(): agent_registry.c, 160:
subtree: found one for: ""
trace: vacm_check_view(): mibgroup/mibII/vacm_conf.c, 1427:
mibII/vacm_vars: vacm_in_view: sn=notConfigUser, gn=notConfigGroup,
vn=systemview
trace: netsnmp_view_subtree_check(): vacm.c, 532:
9:vacm:checkSubtree: view systemview
trace: netsnmp_view_subtree_check(): vacm.c, 569:
9:vacm:checkSubtree:
systemview matched?
trace: netsnmp_view_subtree_check(): vacm.c, 630:
9:vacm:checkSubtree: matched
trace: netsnmp_view_subtree_check(): vacm.c, 653:
vacm:checkSubtree: , included
trace: snmp_call_callbacks(): callback.c, 348:
callback: END calling callbacks for maj=1 min=12 (1 called)
trace: _callback_unlock(): callback.c, 164:
9:callback:lock: unlocked (APP,null)
trace: netsnmp_add_varbind_to_cache(): snmp_agent.c, 1941:
snmp_agent: tp->start HOST-RESOURCES-MIB::hrStorageType, tp->end
HOST-RESOURCES-MIB::hrStorageDescr,
trace: netsnmp_add_varbind_to_cache(): snmp_agent.c, 1961:
verbose:asp: asp 2012d8b8 reqinfo 2012df28 assigned to request
trace: netsnmp_add_varbind_to_cache(): snmp_agent.c, 1968:
verbose:asp: asp 2012d8b8 reqinfo 2012df28 assigned to request
trace: netsnmp_call_handlers(): agent_handler.c, 510:
handler:calling: main handler bulk_to_next
trace: netsnmp_call_handler(): agent_handler.c, 430:
handler:calling: calling handler bulk_to_next for mode GETNEXT
trace: netsnmp_call_handler(): agent_handler.c, 438:
handler:returned: handler bulk_to_next returned 0
trace: netsnmp_call_handler(): agent_handler.c, 430:
handler:calling: calling handler old_api for mode GETNEXT
trace: header_hrstoreEntry(): host/hr_storage.c, 373:
host/hr_storage: var_hrstoreEntry: request
HOST-RESOURCES-MIB::hrStorageType.39 exact=0
trace: header_hrstoreEntry(): host/hr_storage.c, 381:
host/hr_storage: var_hrstoreEntry: compare
HOST-RESOURCES-MIB::hrStorageType => 1
(index 31 ....HOST-RESOURCES-MIB::hrStorageType.31
(index 32 ....HOST-RESOURCES-MIB::hrStorageType.32
(index 33 ....HOST-RESOURCES-MIB::hrStorageType.33
(index 34 ....HOST-RESOURCES-MIB::hrStorageType.34
(index 35 ....HOST-RESOURCES-MIB::hrStorageType.35
(index 36 ....HOST-RESOURCES-MIB::hrStorageType.36
(index 37 ....HOST-RESOURCES-MIB::hrStorageType.37
(index 38 ....HOST-RESOURCES-MIB::hrStorageType.38
(index 39 ....HOST-RESOURCES-MIB::hrStorageType.39
(index 40 ....HOST-RESOURCES-MIB::hrStorageType.40
trace: header_hrstoreEntry(): host/hr_storage.c, 476:
host/hr_storage: var_hrstoreEntry: process
HOST-RESOURCES-MIB::hrStorageType.40 (ffffffff)
Memory fault(coredump)
----------------------------------------------------------------------
Comment By: blentz (blentz)
Date: 2011-05-29 04:18
Message:
I am recompiling now and will run make test again. My development
environment was moved by another admin, so I don't have the same build as
before.
I am also getting cores while walking
HOST-RESOURCES-TYPES::hrStorageFixedDisk
$ sudo dbx /opt/local/net-snmp/sbin/snmpd core
Type 'help' for help.
[using memory image in core]
reading symbolic information ...warning: no source compiled with -g
Segmentation fault in . at 0xd410 ($t1)
0x0000d410 warning: Unable to access address 0xd410 from core
(dbx) where
.() at 0xd410
hasmntopt(??, ??) at 0xd03e4350
var_hrstore(0x2ff1c1c8, 0x2ff1c3dc, 0x2ff1c5dc, 0x0, 0x2ff1c3d8,
0x2ff1c3d4) at 0xd48472f4
netsnmp_old_api_helper(0x2006f688, 0x2006f508, 0x2011f048, 0x2011f068) at
0xd48a41a0
netsnmp_call_handler(0x2006f688, 0x2006f508, 0x2011f048, 0x2011f068) at
0xd48c469c
netsnmp_call_handlers(0x2006f508, 0x2011f048, 0x2011f068) at 0xd48c4a9c
handle_var_requests(0x2011e9d8) at 0xd48d2edc
handle_pdu(0x2011e9d8) at 0xd48d4914
netsnmp_handle_request(0x2011e9d8, 0x0) at 0xd48d43e0
handle_snmp_packet(0x1, 0x2011d1f8, 0xc4a525, 0x2011e6d8, 0x0) at
0xd48d0ef0
_sess_process_packet(0x2011d198, 0x2011d1f8, 0x2011d1b8, 0x2011d048,
0x2011d178, 0x14, 0x20214ee8, 0x2f) at 0xd4a2189c
_sess_read(0x2011d198, 0x2ff1cad4) at 0xd4a22fa0
snmp_sess_read(0x2011d198, 0x2ff1cad4) at 0xd4a23044
snmp_read(0x2ff1cad4) at 0xd4a21a90
receive() at 0x10002c2c
main(0x9, 0x2ff22cb8) at 0x100025f0
(dbx)
----------------------------------------------------------------------
Comment By: Thomas Anders (tanders)
Date: 2011-01-30 04:04
Message:
Please try again with --without-kmem-usage and report back. Also:
- Does "make test" succeed?
- Please attach the full output of "snmpwalk ... localhost .1". If you
feel if contains sensitive information,please send it privately to me.
----------------------------------------------------------------------
Comment By: blentz (blentz)
Date: 2011-01-29 23:22
Message:
Regarding README.aix:
- I have my own workaround for -brtl using:
export CC="gcc -Wl,-blibpath:$WHERE/lib:/usr/lib:/lib,-brtl"
- I built using GNU grep and GNU sed
- I didn't use --without-kmem-usage, maybe that's my issue?
- I don't use or need IPv6
- I don't need the perl module (at this time)
Thanks for your help!!
----------------------------------------------------------------------
Comment By: blentz (blentz)
Date: 2011-01-29 23:19
Message:
/opt/local/net-snmp/bin/net-snmp-config --configure-options
'--prefix=/opt/local/net-snmp' '--enable-shared=yes' '--enable-static=no'
'--with-defaults' 'CC=gcc
-Wl,-blibpath:/opt/local/net-snmp/lib:/usr/lib:/lib,-brtl'
'CFLAGS=-I/opt/local/net-snmp/include'
'CPPFLAGS=-I/opt/local/net-snmp/include'
'LDFLAGS=-L/opt/local/net-snmp/lib'
----------------------------------------------------------------------
Comment By: Thomas Anders (tanders)
Date: 2011-01-29 22:33
Message:
What configure options did you use when building net-snmp on AIX 6.1?
("net-snmp-config --configure-options")
Did you read and follow the notes in README.aix?
----------------------------------------------------------------------
Comment By: blentz (blentz)
Date: 2011-01-26 07:10
Message:
Excluding .1.3.6.1.2.1.4.21.1 appears to provide a workaround.
----------------------------------------------------------------------
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=112694&aid=3165949&group_id=12694
------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
Net-snmp-bugs mailing list
Net-snmp-bugs@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/net-snmp-bugs
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic