[prev in list] [next in list] [prev in thread] [next in thread] 

List:       fuse-devel
Subject:    Re: [fuse-devel] stat performance on multicore systems: measurements
From:       Stef Bon <stefbon () gmail ! com>
Date:       2012-09-05 10:37:41
Message-ID: CANXojcx1vqe19hmo5HWqfBwTs8U1ZKSve=X=ZqeYMjgO4vKoFA () mail ! gmail ! com
[Download RAW message or body]

It's a good idea, but I think, as the examples have shown, that you
CANNOT tie the userspace proces activating the fuse event and the
thread processing that request to one cpu.

The descision to assign a thread/process/task to a cpu lies in the
kernel/scheduler, which bases that descision on various things, and
the desire to have the fuse thread and the activating process on the
same cpu is far beyond the scope of the scheduler.

And it would be very hard to make it work I think. You should have some
switches to let the scheduler take into account. And even then so, the
scheduler may take a descision which is not optimal from the fuse
point of view.

I think that's the way to go, are there "switches" to the scheduler to
set a preferred cpu to task?

You cannot do this from userspace. The scheduler will alway go it's
own way and it's going to be a mess.

Stef



2012/8/7 Stefan Westerfeld <stefan@space.twc.de>:
>    Hi!
>
> On Tue, Aug 07, 2012 at 03:17:23PM +0200, Miklos Szeredi wrote:
>> Anand Avati <anand.avati@gmail.com> writes:
>> > I think we really need multiple request queues (as many number of CPUs) which
>> > are processed by multiple user-space fuse threads which bind themselves to one
>> > CPU each and syscalls add their request to the queue belonging to the CPU they
>> > are happening in -- all of this working without any cross-cpu shared data. We
>> > could optionally have some starvation avoidance by making another CPU's fuse
>> > thread pick this CPU's requests if it is too idle and this CPU has too many
>> > requests, but it need not be an initial feature. If all this really improves
>> > performance to the same extent (and hence corroborates the theory), the extra
>> > complexity would be well worth it.
>>
>> Yeah, that would be an interesting experiment.
>
> I can try to predict the outcome, too: what would happen if you implemented
> this code is the following (cpu numbers are of course invented):
>
> - the userspace process would run on cpu 0
> - it would access the fuse fs (i.e. stat)
> - the fuse kernel code would run on cpu 0
> - it would wakeup a worker thread on cpu 0
> - the worker thread would perform the request (i.e. getattr)
> - the worker thread would wake up the kernel code again
> - the userspace process would receive the data
>
> so far so good. But eventually the linux scheduler would decide that the
> userspace process is running on a cpu that is not very idle (since the fuse
> worker is running on cpu 0), so it would move the userspace process to cpu 1
> (which is completely idle). Now, the same code would run again, but this time
> on cpu 1.
>
> Which would cause the linux scheduler to move the userspace process to another
> cpu again, since the userspace process would be running on the only cpu that is
> quite busy, cpu 1.
>
> And so on.
>
> Why do I say this? Because I tried a very simple version of the suggested changes,
> which I am attaching. My patch would simply move the fuse worker thread to the
> same cpu than the userspace process, and I observed: no improvement in performance,
> but the userspace process constantly shifting cpus (which is bad for performance).
>
> So I think there is one small missing piece of code, that somehow ensures that
> once the fuse worker and the userspace code are scheduled on the same cpu, the
> linux scheduler will not move the userspace process away.
>
>    Cu... Stefan
> --
> Stefan Westerfeld, Hamburg/Germany, http://space.twc.de/~stefan
>
> ------------------------------------------------------------------------------
> Live Security Virtual Conference
> Exclusive live event will cover all the ways today's security and
> threat landscape has changed and how IT managers can respond. Discussions
> will include endpoint security, mobile security and the latest in malware
> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
> _______________________________________________
> fuse-devel mailing list
> fuse-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/fuse-devel
>

------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and 
threat landscape has changed and how IT managers can respond. Discussions 
will include endpoint security, mobile security and the latest in malware 
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
fuse-devel mailing list
fuse-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/fuse-devel
[prev in list] [next in list] [prev in thread] [next in thread]