[prev in list] [next in list] [prev in thread] [next in thread] 

List:       xen-users
Subject:    Re: [Xen-users] trouble with xenserver and xfs (soft lockup - CPU#0
From:       Sebastian Gosenheimer <s.gosenheimer () proio ! com>
Date:       2010-04-30 12:45:01
Message-ID: 4BDAD0CD.4030106 () proio ! com
[Download RAW message or body]

Am 27.04.2010 10:51, schrieb Yann Cezard:
> Sebastian Gosenheimer a écrit :
> > Hi everybody,
> > 
> > i hope that i'm on the right mailing list. I'm having some trouble with
> > xfs and xen. We are running the newest xenserver version with ha on dell
> > servers and a dell equalogic.
> > 
> > I set up a fileserver (debian lenny 2.6.29-xs5.5.0.17) with one ext3
> > partiton for the os and one xfs partition for the data. Two webserver
> > are using this fileserver with nfs.
> > 
> > Now i run the second time into following problem when rsync was running
> > for backup:
> > 
> > [212521.428003] BUG: soft lockup - CPU#0 stuck for 61s! [rsync:29921]
> > [212521.428003] Modules linked in: ipv6 nfsd nfs lockd nfs_acl
> > auth_rpcgss sunrpc xenfs xfs exportfs loop evdev xen_netfront pcspkr
> > ext3 jbd mbcache xen_blkfront thermal_sys
> > [212521.428003]
> > [212521.428003] Pid: 29921, comm: rsync Tainted: G      D
> > (2.6.29-xs5.5.0.17 #1)
> > [212521.428003] EIP: 0061:[<c02f2221>] EFLAGS: 00000287 CPU: 0
> > [212521.428003] EIP is at __write_lock_failed+0x9/0x1c
> > [212521.428003] EAX: e1875cbc EBX: c1351e1a ECX: e2407880 EDX: dd004c50
> > [212521.428003] ESI: 00000000 EDI: c536fd40 EBP: e1875cbc ESP: c1351dac
> > [212521.428003]  DS: 007b ES: 007b FS: 00d8 GS: 0033 SS: 0069
> > [212521.428003] CR0: 8005003b CR2: b806e348 CR3: 20d99000 CR4: 00002620
> > [212521.428003] DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
> > [212521.428003] DR6: ffff0ff0 DR7: 00000400
> > [212521.428003] Call Trace:
> > [212521.428003]  [<c02f23a3>] ? _write_lock+0xe/0xf
> > [212521.428003]  [<e33f7409>] ? xfs_iget+0x328/0x44f [xfs]
> > [212521.428003]  [<e340e925>] ? xfs_lookup+0x69/0x97 [xfs]
> > [212521.428003]  [<e341638b>] ? xfs_vn_lookup+0x36/0x6e [xfs]
> > [212521.428003]  [<c0194ee1>] ? do_lookup+0xa6/0x116
> > [212521.428003]  [<c019584d>] ? __link_path_walk+0x524/0x631
> > [212521.428003]  [<c01057fc>] ? xen_force_evtchn_callback+0xc/0x10
> > [212521.428003]  [<c0195d5c>] ? path_walk+0x4f/0xa3
> > [212521.428003]  [<c0196aa0>] ? do_path_lookup+0x132/0x178
> > [212521.428003]  [<c019732e>] ? getname+0x5e/0xb0
> > [212521.428003]  [<c0197b05>] ? user_path_at+0x37/0x5f
> > [212521.428003]  [<c0198e80>] ? filldir64+0x0/0xc5
> > [212521.428003]  [<c019187f>] ? vfs_lstat_fd+0x12/0x3
> > [212455.928003]  [<c0191912>] ? sys_lstat64+0xf/0x23
> > [212455.928003]  [<c0199124>] ? vfs_readdir+0x7c/0x8c
> > [212455.928003]  [<c0198e80>] ? filldir64+0x0/0xc5
> > [212455.928003]  [<c01991cf>] ? sys_getdents64+0x9b/0xa5
> > [212455.928003]  [<c0107f23>] ? sysenter_past_esp+0x3c/0x62
> > [212455.928003]  [<c0107f5b>] ? sysenter_do_call+0x12/0x2f
> > 
> > I have to force reboot to bring the vm back online. Are there any known
> > problems with xfs and xen? Because i know from another deployment that
> > they are having nearly the same issues with xfs. There are no problems
> > with the ext3 filesystem.
> > 
> > Thank you for your help!
> > 
> > 
> Hi Sebastian,
> 
> I'm having similar issues with two Lenny domU (amd64 and i386),
> both with XFS + quite heavy load (I should say normal load for
> what they do : database server and fileserver).
> They are running the standard Lenny Xen kernel (2.6.26-2-xen-*)
> 
> After searching a little, I found this thread :
> http://developer.amazonwebservices.com/connect/thread.jspa?threadID=28968&start=0&tstart=0
>  
> Not sure that it is connected to my (or your) problem, but the domUs
> I have problems with both have XFS log version 2, i have others domUs
> with XFS filesystems and quite heavy charge (mail servers basically),
> but they are still in Etch and XFS log version 1, no problem with them.
> 
> So I think I will give the log version workaround a try.
> 
> Perhaps could you give it a try to ?
> 
> In any case, if you find something about that problem, I would be happy
> to have feedback from you.
> 
> Cheers,
> 

Hi Yann,

sorry for the late answer. I was messing arround the week with some 
other things.

I have now since four days up and running a new kernel on the fileserver 
(2.6.32.12-xs5.5.0.17). Since this time everything seems to be fine.

Right now i don't know if there is a xfs or nfs problem in the kernel 
2.6.26-2-xen-*. I will update all our machines to a newer kernel know 
which are using nfs and/or xfs.

 From my opionion the thread you sent isn't connected to my problem.

Did the workaround work for you or do you still have problems.

--sg


Diese E-Mail enthält vertrauliche und/oder rechtlich geschützte Informationen. 
Wenn Sie nicht der richtige Adressat sind oder diese E-Mail irrtümlich erhalten \
haben,  informieren Sie bitte sofort den Absender und vernichten Sie diese Mail. 
Das unerlaubte Kopieren sowie die unbefugte Weitergabe dieser Mail sind nicht \
gestattet. 

This e-mail may contain confidential and/or privileged information. 
If you are not the intended recipient (or have received this e-mail in error) please \
notify  the sender immediately and destroy this e-mail.  
Any unauthorized copying, disclosure or distribution of the material in this e-mail \
is strictly forbidden. 



_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users


[prev in list] [next in list] [prev in thread] [next in thread]