[prev in list] [next in list] [prev in thread] [next in thread] 

List:       kde-devel
Subject:    Re: FAM and 3.1rc3
From:       Josef Weidendorfer <Josef.Weidendorfer () gmx ! de>
Date:       2002-11-29 19:54:21
[Download RAW message or body]

On Friday 29 November 2002 18:21, Manuel Amador wrote:
> El jue, 28-11-2002 a las 03:46, Michael Kreitzer escribió:
> > I've noticed with FAM enabled in kde 3.1rc3 none of the io slaves seem to
> > realize when fam is inappropriate... I got stuck with an unresponsive
> > system until I figured out FAM was using 100% cpu while KDE tried to use
> > it over 3 sftp connections and 2 smb connections.... this should not
> > happen.  My question is... is this a KDE bug or a FAM bug or a FAM
> > misconfiguration on my part?
>
> Oh, and FAM causes millions of updates on a local open directory with
> Konqueror, just when I retag an MP3/Ogg file with EasyTAG.  After it's
> finished tagging, Konqueror keeps flickering like mad for 10 seconds and
> I can't use any other Konqueror windows.  Disabling service sgi-fam in
> xinetd does the trick, and Konqueror updates directories again, at a
> sane speed.  But isn't fam supposed to help performance by reducing
> reads to find out whether an element has changed?

Hi,

the reasoning behind FAM is to keep CPU resources low for applications that 
want reaction on file changes.
The need for this is *not* given when only one application is looking for 
changes in a local directory: it can do polling at reasonable intervals. This 
is the case with Konqui when you disable FAM and dnotify. As you say, you get 
perfect and reasonable behaviour.

The reason for FAM is given when

* there is a large number of applications looking for changes in the same 
directory. FAM can centralize the actions needed, but notify each application 
on a change.
This *would* be needed if KDE applications keep an eye on their config files: 
Each KDE app would need to listen for changes to the file with global KDE 
configuration, too. Here, the centralized approach of FAM is helpful.
You can specify FAM to only do STAT polling instead of using kernel support 
(e.g. DNOTIFY).

* the directory to watch for changes is remotely mounted. As STATs over NFS 
are slow, you create unneeded network loading. When the FAM daemon is running 
on the server machine, the local FAM on the file server looks for *local* 
changes (perhaps again using only STAT polling, but that's fast when local). 
When a change is detected, the server FAM notifies all watching client FAMs 
and these demultiplex the notification for each application on their machine 
which are interested in changes.
The centralized approach of FAM is needed here, too: Suppose 1000 clients 
doing STAT polling over NFS instead of using FAM...

How does FAM know about changes on *local* files?
There is either
- regular polling with STAT
- using some kernel support: imon (IRIX), dnotify (LINUX), kqueue (BSD?)

STAT polling every 0,5 seconds is mostly enough (configured when compiling 
FAM). Disadvantage is a maximal delay of 0,5 s for events to be delivered.
STAT does *not* open the watched files are make a watched dir its cwd, so 
there will *never* be any problems with "device busy". Umounting devices is 
no problem with STAT polling.

If there are huge numbers of files to be watched, or you want immediatly 
notified about changes, use kernel support:

* imon uses the /dev/imon pseudo device to get notifications from the kernel. 
There's also a implementation for LINUX, but because of bad shape of this 
code, it's not in any legacy kernel and AFAIK no distribution has it build 
in. BTW, this is the IRIX way on SGI machines.

* Don't know about kqueue (is this the right name for this service on BSD?)

* dnotify on LINUX: This is in the legacy kernel. And that's the reason all 
distributions seem to have FAM built to use it (SUSE, Red Hat, Mandrake...).

Drawbacks of DNOTIFY:
- DNOTIFY forces FAM to open the directory which contains files to be watched.
This means problems with unmounting devices. Simple advice: Kill the FAM 
daemon before unmounting ;-)
- DNOTIFY has *no* event delaying. For every write syscall to a watched file, 
DNOTIFY produces immediatly a signal (for FAM). Worse, all DNOTIFY 
applications (FAM, KDirWatch with dnotify support) use realtime signals. 
These aren't merged by the kernel but queued and could lead to overrunning 
this queue (thus, the additional need for SIGIO handler). But this is needed 
as an attribute for the signal is the changed directory file descriptor, and 
thus the application knows the directory where the change happened. When 
using a normal signal for dnotify, events could be missed: You would check 
*all* directories to be watched with STAT for actual change. That's not 
really better than STAT polling alone.

So if your MP3 tagging does 1000 1-char-writes, all watching applications (FAM 
or KDirWatch itself) will get 1000 signals delivered. This can make the whole 
system unusable for a few seconds :-(

Note: You can disable FAM and switch on DNOTIFY support in KDirWatch when 
compiling kdelibs. This has the same disadvantages as using FAM with dnotify.

My advice:
- Use FAM, but recompile *without* dnotify support.
- Blame your distribution for using the problematic dnotify support for FAM.
- Mail the LINUX kernel list for adding a "delay" option for dnotify (e.g. 
signal generation is delayed by 200 ms after the event happend, merging 
events) and for dnotify not being in the way for unmounting (e.g. by using a 
single /dev/dnotify for notifications or forcing closing of dnotify file 
descriptors on unmounting).

And don't blame FAM for the buggy behaviour of dnotify you noted.
This is a fine system tool.

Josef

PS:
Waldo, what do you think of requesting SIGIO signals for dnotify in KDirWatch?
You get signal merging, but have to STAT each watched entry (as already needed 
with SIGIO when the realtime-signal-queue runs over).
At least this wouldn't keep konqui busy seconds after a huge number of signals 
from dnotify.
But perhaps event merging/delaying in KDirWatch will be fine. (go down from a 
10 second lasting flickering in konqui with 100% load to only 1 second 100% 
load without flickering).

The unmounting problem will still be there. When unmounting from KDE, there 
could be a DCOP broadcast to stop watching in all KDE applications. But this 
does not work with umount from the shell or supermount.

Perhaps I should try to hack dnotify myself, sent the patch to the kernel list 
and hope somebody correctifies that bad hack of myself to perhaps allow it 
for application to the legacy LINUX kernel.
Or try to cleanup the imon patch for LINUX.


>> Visit http://mail.kde.org/mailman/listinfo/kde-devel#unsub to unsubscribe <<
[prev in list] [next in list] [prev in thread] [next in thread]