[prev in list] [next in list] [prev in thread] [next in thread] 

List:       lustre-devel
Subject:    [Lustre-devel] [Bug 11491] open_req->rq_type != LI_POISON asserts
From:       th () llnl ! gov (th () llnl ! gov)
Date:       2006-12-29 21:12:54
Message-ID: 200612300412.kBU4CrAt008271 () shell ! clusterfs ! com
[Download RAW message or body]

Please don't reply to lustre-devel. Instead, comment in Bugzilla by using the following link:
https://bugzilla.lustre.org/show_bug.cgi?id=11491



Two alc/production client nodes (1.4.6.95_17.2llnl)
hit this ASSERTION today.
The ep0 messages might indicate an underlying
elan problem.

alc952:

2006-12-29 18:37:18 LustreError: 4656:0:(mdc_request.c:600:mdc_commit_close())
ASSERTION(open_req->rq_type != LI_POISON) failed
2006-12-29 18:37:18 LustreError: 4656:0:(linux-debug.c:130:lbug_with_loc()) LBUG
2006-12-29 18:37:18 Lustre: 4656:0:(linux-debug.c:155:libcfs_debug_dumpstack())
showing stack for process 4656
2006-12-29 18:37:18 run           R running  4896  4656   4650          4657 
4655 (NOTLB)
2006-12-29 18:37:18 fab8c06a de4f7d34 de4f7d48 c0106390 c02b31e3 c02b31e3
de4f7d20 00000190 
2006-12-29 18:37:18        fab8a852 fab8c06a de4f7d5c fab81aff f6710880 e8d59c00
fd2780e5 de4f7d64 
2006-12-29 18:37:18        fab87c31 de4f7d84 fd26c90b 00000258 e55e0a00 f65fc640
e224b200 e55e0a00 
2006-12-29 18:37:18 Call Trace:
2006-12-29 18:37:18  [<c01063a8>] show_stack+0x76/0x7e
2006-12-29 18:37:18  [<fab81aff>] lbug_with_loc+0x8b/0xb2 [libcfs]
2006-12-29 18:37:18  [<fab87c31>] collect_pages_on_cpu+0x0/0x98 [libcfs]
2006-12-29 18:37:18  [<fd26c90b>] mdc_commit_close+0x28b/0x4f8 [mdc]
2006-12-29 18:37:18  [<fcf78e04>] ptlrpc_free_committed+0xbb8/0xc74 [ptlrpc]
2006-12-29 18:37:18  [<fcf74358>] after_reply+0x7b9/0x85c [ptlrpc]
2006-12-29 18:37:18  [<fcf7bc7f>] ptlrpc_queue_wait+0x21c6/0x2a78 [ptlrpc]
2006-12-29 18:37:18  [<fd26d249>] mdc_close+0x6d1/0xbfe [mdc]
2006-12-29 18:37:18  [<fd1ab55b>] ll_close_inode_openhandle+0x55b/0x7ea [llite]
2006-12-29 18:37:18  [<fd1ab9f2>] ll_mdc_real_close+0x208/0x35a [llite]
2006-12-29 18:37:18  [<fd1abdec>] ll_mdc_close+0x2a8/0x3e0 [llite]
2006-12-29 18:37:18  [<fd1ac19f>] ll_file_release+0x27b/0x30e [llite]
2006-12-29 18:37:18  [<c0156cd8>] __fput+0x56/0x104
2006-12-29 18:37:18  [<c01558f0>] filp_close+0x5b/0x65
2006-12-29 18:37:18  [<c02a8c4f>] syscall_call+0x7/0xb
2006-12-29 18:37:18 LDec 29 18:37:18 alc952 LustreError:
4656:0:(mdc_request.c:600:mdc_commit_close()) ASSERTION(openr_req->rq_type !=
LI_POISON) failed
2006-12-29 18:37:18 Dec 29 18:37:18 alc952 LustreError:
4656:0:(linux-debug.c:130:lbug_with_loc()u) LBUG
2006-12-29 18:37:18 mping log to /var/tmp/lustre-log.1167446238.4656
2006-12-29 18:37:19 Lustre: 4656:0:(linux-debug.c:96:libcfs_run_upcall())
Invoked LNET upcall /usr/lib/lustre/lnet_upcall
LBUG,/tmp/root.10816/rpm/BUILD/lustre-1.4.6.95_17.2llnl/lnet/lib
cfs/tracefile.c,libcfs_assertion_failed,400
2006-12-29 18:39:04 ep0[952]: manager thread stuck - scheduled
2006-12-29 18:39:04 ep0[952]: REJOINING at level 0 because of manager thread
2006-12-29 18:39:06 ep0[952]: Withdraw at Level 0
2006-12-29 18:39:06 ep0[952]: Withdraw at Level 1
2006-12-29 18:39:06 ep0[952]: Withdraw at Level 2
2006-12-29 18:39:06 ep0[952]: Withdraw at Level 3
2006-12-29 18:39:06 ep0[952]: Withdraw at Level 4
2006-12-29 18:39:06 ep0[952]: Withdraw from [953-955]
2006-12-29 18:39:06 ep0[952]: Withdraw from [944-951][956-959]
2006-12-29 18:39:06 ep0[952]: Withdraw from [864-943]
2006-12-29 18:39:06 ep0[952]: Withdraw from [768-863][960-1151]
2006-12-29 18:39:06 ep0[952]: Withdraw from [0-767][1152-1535]




alc956:

2006-12-29 18:37:18 LustreError: 30851:0:(mdc_request.c:600:mdc_commit_close())
ASSERTION(open_req->rq_type != LI_POISON) failed
2006-12-29 18:37:18 LustreError: 30851:0:(linux-debug.c:130:lbug_with_loc()) LBUG
2006-12-29 18:37:18 Lustre: 30851:0:(linux-debug.c:155:libcfs_debug_dumpstack())
showing stack for process 30851
2006-12-29 18:37:18 run           R running  4896 30851  30845         30852
30850 (NOTLB)
2006-12-29 18:37:18 fab8c06a d1625d34 d1625d48 c0106390 c02b31e3 c02b31e3
d1625d20 00000190 
2006-12-29 18:37:18        fab8a852 fab8c06a d1625d5c fab81aff f0515b80 ca75f000
fd25a0e5 d1625d64 
2006-12-29 18:37:18        fab87c31 d1625d84 fd24e90b 00000258 cc27e600 d5f7afc0
dd09ce00 cc27e600 
2006-12-29 18:37:18 Call Trace:
2006-12-29 18:37:18  [<c01063a8>] show_stack+0x76/0x7e
2006-12-29 18:37:18  [<fab81aff>] lbug_with_loc+0x8b/0xb2 [libcfs]
2006-12-29 18:37:18  [<fab87c31>] collect_pages_on_cpu+0x0/0x98 [libcfs]
2006-12-29 18:37:18  [<fd24e90b>] mdc_commit_close+0x28b/0x4f8 [mdc]
2006-12-29 18:37:18  [<fcf78e04>] ptlrpc_free_committed+0xbb8/0xc74 [ptlrpc]
2006-12-29 18:37:18  [<fcf74358>] after_reply+0x7b9/0x85c [ptlrpc]
2006-12-29 18:37:18  [<fcf7bc7f>] ptlrpc_queue_wait+0x21c6/0x2a78 [ptlrpc]
2006-12-29 18:37:18  [<fd24f249>] mdc_close+0x6d1/0xbfe [mdc]
2006-12-29 18:37:18  [<fd18d55b>] ll_close_inode_openhandle+0x55b/0x7ea [llite]
2006-12-29 18:37:18  [<fd18d9f2>] ll_mdc_real_close+0x208/0x35a [llite]
2006-12-29 18:37:18  [<fd18ddec>] ll_mdc_close+0x2a8/0x3e0 [llite]
2006-12-29 18:37:18  [<fd18e19f>] ll_file_release+0x27b/0x30e [llite]
2006-12-29 18:37:18  [<c0156cd8>] __fput+0x56/0x104
2006-12-29 18:37:18  [<c01558f0>] filp_close+0x5b/0x65
2006-12-29 18:37:18  [<c02a8c4f>] syscall_call+0x7/0xb
2006-12-29 18:37:18 LustreErDec 29 18:37:18 ralc956 LustreError:
30851:0:(mdc_request.c:600:mdc_commit_close()) ASSERTION(open_req->rq_type !=
LI_POISON) failed
2006-12-29 18:37:18 Dec 29 18:37:18 alc956 LustreError:
30851:0:(linux-debug.c:130:lbug_with_locl()) LBUG
2006-12-29 18:37:18 og to /var/tmp/lustre-log.1167446238.30851
2006-12-29 18:37:21 Lustre: 30851:0:(linux-debug.c:96:libcfs_run_upcall())
Invoked LNET upcall /usr/lib/lustre/lnet_upcall
LBUG,/tmp/root.10816/rpm/BUILD/lustre-1.4.6.95_17.2llnl/lnet/li
bcfs/tracefile.c,libcfs_assertion_failed,400
2006-12-29 18:37:21 Lustre: 30851:0:(linux-debug.c:96:libcfs_run_upcall())
Skipped 16 previous similar messages
2006-12-29 18:38:34 ep0[956]: manager thread stuck - scheduled
2006-12-29 18:38:34 ep0[956]: REJOINING at level 0 because of manager thread
2006-12-29 18:38:36 ep0[956]: Withdraw at Level 0
2006-12-29 18:38:36 ep0[956]: Withdraw at Level 1
2006-12-29 18:38:36 ep0[956]: Withdraw at Level 2
2006-12-29 18:38:36 ep0[956]: Withdraw at Level 3
2006-12-29 18:38:36 ep0[956]: Withdraw at Level 4
2006-12-29 18:38:36 ep0[956]: Withdraw from [957-959]
2006-12-29 18:38:36 ep0[956]: Withdraw from [944-955]
2006-12-29 18:38:36 ep0[956]: Withdraw from [864-943]
2006-12-29 18:38:36 ep0[956]: Withdraw from [768-863][960-1151]
2006-12-29 18:38:37 ep0[956]: Withdraw from [0-767][1152-1535]

[prev in list] [next in list] [prev in thread] [next in thread]