[prev in list] [next in list] [prev in thread] [next in thread] 

List:       kde-devel
Subject:    Re: FAM and 3.1rc3
From:       Manuel Amador <amadorm () usm ! edu ! ec>
Date:       2002-11-29 23:46:59
[Download RAW message or body]


> Hi,
> 
> the reasoning behind FAM is to keep CPU resources low for applications that 
> want reaction on file changes.
> The need for this is *not* given when only one application is looking for 
> changes in a local directory: it can do polling at reasonable intervals. This 
> is the case with Konqui when you disable FAM and dnotify. As you say, you get 
> perfect and reasonable behaviour.
Yes but FAM should also behave sanely, if it's enabled.  I really don't
care about the option to disable it, 100% CPU usage with DD is a bug,
and should be treated like that.

> 
> The reason for FAM is given when
> 
> * there is a large number of applications looking for changes in the same 
> directory. FAM can centralize the actions needed, but notify each application 
> on a change.
> This *would* be needed if KDE applications keep an eye on their config files: 
> Each KDE app would need to listen for changes to the file with global KDE 
> configuration, too. Here, the centralized approach of FAM is helpful.
> You can specify FAM to only do STAT polling instead of using kernel support 
> (e.g. DNOTIFY).
Yes.  But FAM should work FOR the user, not AGAINST it.  Hence it's up
to FAM to determine the *best* way to do it.  Being a system service,
users expect it to work 100% of the time, not when it feels like doing a
good job.

> 
> * the directory to watch for changes is remotely mounted. As STATs over NFS 
> are slow,
why?
> The centralized approach of FAM is needed here, too: Suppose 1000 clients 
> doing STAT polling over NFS instead of using FAM...
> 
> How does FAM know about changes on *local* files?
> There is either
> - regular polling with STAT
> - using some kernel support: imon (IRIX), dnotify (LINUX), kqueue (BSD?)
> 
> STAT polling every 0,5 seconds is mostly enough (configured when compiling 
> FAM). Disadvantage is a maximal delay of 0,5 s for events to be delivered.
> STAT does *not* open the watched files are make a watched dir its cwd, so 
> there will *never* be any problems with "device busy". Umounting devices is 
> no problem with STAT polling.
> 
> If there are huge numbers of files to be watched, or you want immediatly 
> notified about changes, use kernel support:
> 
> * imon uses the /dev/imon pseudo device to get notifications from the kernel. 
> There's also a implementation for LINUX, but because of bad shape of this 
> code, it's not in any legacy kernel and AFAIK no distribution has it build 
> in. BTW, this is the IRIX way on SGI machines.
> 
> * Don't know about kqueue (is this the right name for this service on BSD?)
> 
> * dnotify on LINUX: This is in the legacy kernel. And that's the reason all 
> distributions seem to have FAM built to use it (SUSE, Red Hat, Mandrake...).
> 
> Drawbacks of DNOTIFY:
> - DNOTIFY forces FAM to open the directory which contains files to be watched.
> This means problems with unmounting devices. Simple advice: Kill the FAM 
> daemon before unmounting ;-)
OH!  That's the sucker!  This pinpoints to errors in the conceptual
implementation.
> - DNOTIFY has *no* event delaying. For every write syscall to a watched file, 
> DNOTIFY produces immediatly a signal (for FAM).
FAM should keep track of signals and throttle them.  I know this is a
dnotify bug, but barring swift corrections to the kernel code, FAM
should make do with what it has.
 Worse, all DNOTIFY 
> applications (FAM, KDirWatch with dnotify support) use realtime signals. 
> These aren't merged by the kernel but queued and could lead to overrunning 
> this queue (thus, the additional need for SIGIO handler). But this is needed 
> as an attribute for the signal is the changed directory file descriptor, and 
> thus the application knows the directory where the change happened. When 
> using a normal signal for dnotify, events could be missed: You would check 
> *all* directories to be watched with STAT for actual change. That's not 
> really better than STAT polling alone.
> 
> So if your MP3 tagging does 1000 1-char-writes, all watching applications (FAM 
> or KDirWatch itself) will get 1000 signals delivered. This can make the whole 
> system unusable for a few seconds :-(
yep exactly my behavior.
> 
> Note: You can disable FAM and switch on DNOTIFY support in KDirWatch when 
> compiling kdelibs. This has the same disadvantages as using FAM with dnotify.
> 
> My advice:
> - Use FAM, but recompile *without* dnotify support.
> - Blame your distribution for using the problematic dnotify support for FAM.
> - Mail the LINUX kernel list for adding a "delay" option for dnotify (e.g. 
> signal generation is delayed by 200 ms after the event happend, merging 
> events) and for dnotify not being in the way for unmounting (e.g. by using a 
> single /dev/dnotify for notifications or forcing closing of dnotify file 
> descriptors on unmounting).
isn't it easier for FAM to detect multiple close events and merge them
into one single application event to be sent?  There might be a reason
why you want real time notification in apps using dnotify...
> 
> And don't blame FAM for the buggy behaviour of dnotify you noted.
> This is a fine system tool.
> 
> Josef
> 
> PS:
> Waldo, what do you think of requesting SIGIO signals for dnotify in KDirWatch?
> You get signal merging, but have to STAT each watched entry (as already needed 
> with SIGIO when the realtime-signal-queue runs over).
> At least this wouldn't keep konqui busy seconds after a huge number of signals 
> from dnotify.
> But perhaps event merging/delaying in KDirWatch will be fine. (go down from a 
> 10 second lasting flickering in konqui with 100% load to only 1 second 100% 
> load without flickering).
> 
> The unmounting problem will still be there. When unmounting from KDE, there 
> could be a DCOP broadcast to stop watching in all KDE applications. But this 
> does not work with umount from the shell or supermount.
yes.



> 
> Perhaps I should try to hack dnotify myself, sent the patch to the kernel list 
> and hope somebody correctifies that bad hack of myself to perhaps allow it 
> for application to the legacy LINUX kernel.
> Or try to cleanup the imon patch for LINUX.
> 
>  
> >> Visit http://mail.kde.org/mailman/listinfo/kde-devel#unsub to unsubscribe <<


 
>> Visit http://mail.kde.org/mailman/listinfo/kde-devel#unsub to unsubscribe <<
[prev in list] [next in list] [prev in thread] [next in thread]