[prev in list] [next in list] [prev in thread] [next in thread] 

List:       kvm
Subject:    Re: Virtio network performance problem
From:       Dor Laor <dlaor () redhat ! com>
Date:       2008-12-04 22:38:11
Message-ID: 49385BD3.6060609 () redhat ! com
[Download RAW message or body]

Adrian Schmitz wrote:
>>> On Wed, Dec 03, 2008 at 11:20:08AM -0800, Chris Wedgwood wrote:
>>>
>>>       
>>>> TSC instability?  Is this an SMP guest?
>>>>         
>
> Ok, I tried pinning the kvm process to two cores (0,2) on a single
> socket, but that didn't seem to make any difference for my virtio
> network performance. I also tried pinning the process to a single core,
> which also didn't seem to have any effect.
>
>   
I think it is an unsync tsc problem.
First, make sure you pin all of the process threads. There is thread per 
vcpu + io thread +more non relevant.
You can do it by adding the taskset before the cmdline.
Second, you said that you use smp guest. So windows also sees unsync tsc.
So, either test with UP guest or learn how to pin windows receiving ISR, 
DPC and the user app.

Well, testing on Intel or newer AMD is another option.
I tested it again now on Intel with UP guest and there is no such a problem.
Hope to test it next week on AMD SMP guest.

Regards,
Dor
> Someone on IRC suggested that it sounded like a clocking issue, since
> some of my ping times are negative. He suggested trying a different
> clock source. I tried it with dynticks, rtc, and unix. None of them seem
> better, although all of them seem "different" in terms of patterns in
> the ping times. Sorry if this makes it a long post, but I don't know how
> to describe it other than to paste an example (below). Not sure if this
> indicates that it is clock-related or if it is meaningless.
>
> In any event, I'm not sure where to go from here. Another suggestion
> from IRC was that it was due to the age of my host kernel (2.6.18) and
> the fact that it doesn't support high-res timers. If I can avoid
> replacing the distro kernel, I'd like to, but I'll do what I have to, I
> suppose.
>
> With dynticks (these are all with -net user, as I had some trouble with
> my tap interface last night while testing this. The results are roughly
> the same as when I was using tap before, though):
>
> Reply from 10.0.2.2: bytes=32 time=1ms TTL=255
> Reply from 10.0.2.2: bytes=32 time=1ms TTL=255
> Reply from 10.0.2.2: bytes=32 time=1ms TTL=255
> Reply from 10.0.2.2: bytes=32 time=143ms TTL=255
> Reply from 10.0.2.2: bytes=32 time=143ms TTL=255
> Reply from 10.0.2.2: bytes=32 time=1ms TTL=255
> Reply from 10.0.2.2: bytes=32 time=143ms TTL=255
> Reply from 10.0.2.2: bytes=32 time=1ms TTL=255
> Reply from 10.0.2.2: bytes=32 time=1ms TTL=255
> Reply from 10.0.2.2: bytes=32 time=1ms TTL=255
> Reply from 10.0.2.2: bytes=32 time=-139ms TTL=255
> Reply from 10.0.2.2: bytes=32 time=-141ms TTL=255
> Reply from 10.0.2.2: bytes=32 time=-133ms TTL=255
> Reply from 10.0.2.2: bytes=32 time=1ms TTL=255
> Reply from 10.0.2.2: bytes=32 time=143ms TTL=255
> Reply from 10.0.2.2: bytes=32 time=1ms TTL=255
>
> With rtc:
>
> Reply from 10.0.2.2: bytes=32 time=-224ms TTL=255
> Reply from 10.0.2.2: bytes=32 time=-223ms TTL=255
> Reply from 10.0.2.2: bytes=32 time=4ms TTL=255
> Reply from 10.0.2.2: bytes=32 time<1ms TTL=255
> Reply from 10.0.2.2: bytes=32 time<1ms TTL=255
> Reply from 10.0.2.2: bytes=32 time<1ms TTL=255
> Reply from 10.0.2.2: bytes=32 time=225ms TTL=255
> Reply from 10.0.2.2: bytes=32 time=-223ms TTL=255
> Reply from 10.0.2.2: bytes=32 time=-224ms TTL=255
> Reply from 10.0.2.2: bytes=32 time<1ms TTL=255
> Reply from 10.0.2.2: bytes=32 time<1ms TTL=255
> Reply from 10.0.2.2: bytes=32 time=225ms TTL=255
> Reply from 10.0.2.2: bytes=32 time<1ms TTL=255
> Reply from 10.0.2.2: bytes=32 time<1ms TTL=255
> Reply from 10.0.2.2: bytes=32 time<1ms TTL=255
> Reply from 10.0.2.2: bytes=32 time=225ms TTL=255
> Reply from 10.0.2.2: bytes=32 time=225ms TTL=255
>
> With unix:
>
> Reply from 10.0.2.2: bytes=32 time=-191ms TTL=255
> Reply from 10.0.2.2: bytes=32 time<1ms TTL=255
> Reply from 10.0.2.2: bytes=32 time<1ms TTL=255
> Reply from 10.0.2.2: bytes=32 time=-191ms TTL=255
> Reply from 10.0.2.2: bytes=32 time<1ms TTL=255
> Reply from 10.0.2.2: bytes=32 time<1ms TTL=255
> Reply from 10.0.2.2: bytes=32 time<1ms TTL=255
> Reply from 10.0.2.2: bytes=32 time<1ms TTL=255
> Reply from 10.0.2.2: bytes=32 time=-190ms TTL=255
> Reply from 10.0.2.2: bytes=32 time=-191ms TTL=255
> Reply from 10.0.2.2: bytes=32 time=1ms TTL=255
> Reply from 10.0.2.2: bytes=32 time=192ms TTL=255
> --
> To unsubscribe from this list: send the line "unsubscribe kvm" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>   

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
[prev in list] [next in list] [prev in thread] [next in thread]