[prev in list] [next in list] [prev in thread] [next in thread] 

List:       kde-devel
Subject:    Re: CPU load control
From:       Manuel Amador <rudd-o () amautacorp ! com>
Date:       2005-08-20 11:39:01
Message-ID: 1124537941.7250.32.camel () master ! amauta
[Download RAW message or body]

El mar, 16-08-2005 a las 22:58 -0500, Henry Miller escribió:

> What operating system, how many CPUs, and what priority are the other 
> processes running at? 

One CPU, standard 0 priority.

>   On my dual processor machine a load of 1 means that 
> one CPU is sitting idle.   If I had a 64 processor machine, then a load of 60 
> would be perfect acceptable.

No.  The load is not proportional to the amount of CPU cycles your tasks
are using.

A load of 1 means that one task, on average, is running or waiting for
I/0 (which usually does not count towards CPU usage for that task, but
naturally that depends).  A load of two means two tasks on the Running
state.  And so on.  Thus load takes into account whether there was any
stalled I/O or not, which is actually the biggest culprit of interactive
jerkiness and the like.

Modern Linuxes do a hell (good) of a job arbitrating CPU usage.  But I/O
contention is not that great.  Run a heavily I/O bound task and watch
you interactive performance go the way of the dodo.  Now run the task
and watch the load: if you're lucky it'll hit 1 or maybe 1.2 (that would
be for the shell, X and the top command running).  Nevertheless, hitting
load 1 with an I/O bound task (or heavy swapping, which has the same
effect) would mean you can kiss your interactive performance goodbye.

Which is why I find a loadavg of 1 a pretty accurate descriptor of when
a background indexer should start throttling its activity.

> 
> Some systems handle load better than others.   I've had a load > 4 on my 
> FreeBSD machine, and it was useable, while my linux machine wasn't.   (Note, 
> invalid comparison, the linux machine was a 386 running a 2.0.x kernel, while 
> the FreeBSD machine is an Athalon 1 ghz, with a 5.x kernel) 

I can bet FreeBSD is better at that, having used FreeBSD and watching
startx gnome-session start in 3 seconds, while Linux on the same machine
took 12.  But we should try to be architecting for the lowest common
denominator.

> 
> > It all depends on what level of interactive responsiveness you're
> > aiming.  An indexing daemon should, IMO, throttle as soon as the system
> > begins to feel lagging, even if just a bit.  That's why I recommend
> > loadavg = 1.
> 
> No, it should run at a low priority, one above idle.

I agree on that.  nice() should be used, definitely.  But nice() will
NOT help you with I/O contention!

>   (We want to be higher 
> than distributed.net clients, but lower than everything else).    A good OS 
> scheduler should have no problem preventing the indexer from interfering with 
> active users.

I've tried cfq and as and none have satisfied me.  Sure, XMMS won't stop
playing music, even with a load of 15 or 20, but other apps are
UNUSABLE.

> 
> > > Speaking of arts, arts _does_ load average monitoring and bails on CPU
> > > overload (it assumes any CPU overload is its own fault), so you could
> > > take a look at what code is in there.
> 
> which isn't a good idea, but if there are bugs in aRts it might be all we can 
> do.    It fails when there are several processors kept busy enough that you 
> have the right amount of CPUs.

I always wondered why artsd has this tendency of consuming 3~5% of CPU,
even when it is doing nothing.  Please don't raise the famous "auto-
suspend" argument here, most of us know about it... thing is artsd
should consume 0 (that's right, ZERO) percent CPU when it's not playing
sound, whether it is suspended or not.  Maybe there's a busy-wait in
artsd code, I have no idea.

> 
> Dual-core is already here on the high end, and multi-processor systems are old 
> news.    It won't be long as the average KDE user will have more than one 
> processor.   Anything that doesn't account for this will be problems.

As I said, load has nothing to do with CPUs.  And interactive
responsiveness has very very little to do with CPU usage, and everything
to do with I/O starvation.

Once again, you can have a 3GHz dual core CPU, and I/O starvation will
still kill you, whether it's caused by heavy swapping or apps loading
thousands of files upon loading (easily cured by RAM, hardly a solution
for most cash-strapped people and businesses).  It's the latter case
that's more common nowadays.

CPUs are so fast these days.  Disks are so relatively slow.  Think of a
common ATA disk, and you'll think of 45 or 50 MBs per second, much less
when the disk is seeking like mad to supply all I/O waiting tasks.  It's
just not enough for apps to throttle according to CPU usage, or schedule
themselves at 20 nice level.  A task in 20 nice level can still kill
interactive performance in modern Linuxes.

You start a background task which starts to read directories and files
(the regular activity of an indexer), even with a nice 20 level, and
pretty soon apps you wanna work with need to page their data from the
hard disk.  This is caused by two phenomenons: disk cache eviction (the
OS evicts useful files from the cache because the indexer starts to read
files, and the OS services those requests, caching stuff for the
indexer) and memory pressure (which happens because while the indexer
runs, stuff will get paged in to disk to service the memory requests
from the indexer).

So, say you wanna work with evo, which you left inactive for 50 seconds,
and evo suddenly needs, 4KB from the swap area or from a shared library.
While it tries to get their 4KB of data, you'll see it freezing and
going blank, for *seconds* at a time!.  Sure, tasks which are loaded
completely in physical RAM will remain responsive.  But for that to
happen, you'll need a really really large amount of RAM, and having swap
disabled altogether.  Or not, because a shared lib was evicted from the
disk cache and needs to be read again.

> 
> 
>  
> >> Visit http://mail.kde.org/mailman/listinfo/kde-devel#unsub to unsubscribe <<
-- 
Manuel Amador                   <rudd-o@amautacorp.com>
http://www.amautacorp.com/            +593 (4) 220-7010
 
>> Visit http://mail.kde.org/mailman/listinfo/kde-devel#unsub to unsubscribe <<

[prev in list] [next in list] [prev in thread] [next in thread]