[prev in list] [next in list] [prev in thread] [next in thread] 

List:       fuse-devel
Subject:    Re: [fuse-devel] FUSE_FORGET problem.
From:       Terje Malmedal <terje.malmedal () usit ! uio ! no>
Date:       2010-12-17 16:10:57
Message-ID: s1wd3p0fkha.fsf () cornavin ! uio ! no
[Download RAW message or body]


[Miklos Szeredi]
> On Tue, 07 Dec 2010, Goswin von Brederlow wrote:
>> Terje Malmedal <terje.malmedal@usit.uio.no> writes:
>> 
>> > [Miklos Szeredi]
>> >> On Thu, 02 Dec 2010, Terje Malmedal wrote:
>> >>> [Terje Malmedal]
>> >>> > Thankyou very much, but unfortunately it does not seem to work, I still
>> >>> 
>> >>> Oopsie, my fault. gcc found an old fuse.h in /usr/include instead of the
>> >>> one I wanted it to use.
>> >>> 
>> >>> Now it seems that things are working correctly. 
>> >
>> >> Thanks for testing.
>> >
>> >> The 30 minutes processing time for 32M FORGETs made me think about
>> >> what else could be slowing this down.  One thing that I found is that
>> >> libfuse uses a rather small hash table for hashing inodes.  At such a
>> >> large number of cached nodes, this could mean significant performance
>> >> degradation not just for FORGET but for all operations.  This only
>> >> applies to the high level API in <fuse.h>, you are using that aren't
>> >> you?
>> >
>> > Yes, that's right. 
>> >
>> >> If so you could try increasing the value of f->name_table_size and
>> > f-> id_table_size in lib/fuse.c to some larger prime value
>> >> (e.g. 1572869).
>> >
>> > OK, I am trying that number first. Is there any reason not to go any
>> > higher, find a nice prime around hundred milllion or so? There is lots
>> > of RAM on the machine.
>> 
>> Try it first just to see if it works. A large number is bad for the
>> general case though.
>> 
>> If you want to patch this nicely then a dynamic size would be
>> good. Generate an array of primes and track how full the table is. If it
>> is too empty then shrink the table (next smaller prime in the array). If
>> it is too full then grow the table (next bigger prime in the array).
>> Making each prime about twice as big as the last gives good results.

> Committed a dynamic hash table implementation.  It uses linear
> hashing, which grows the hash table size gradually, without big rehash
> events (which is bad for latency).

Excellent. 

> Testing is welcome.

I've been running it a little bit. Seems to work fine, no immediate
problems at least, I'll let you know if anything turns up later.


-- 
 - Terje
malmedal@usit.uio.no

------------------------------------------------------------------------------
Lotusphere 2011
Register now for Lotusphere 2011 and learn how
to connect the dots, take your collaborative environment
to the next level, and enter the era of Social Business.
http://p.sf.net/sfu/lotusphere-d2d
_______________________________________________
fuse-devel mailing list
fuse-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/fuse-devel
[prev in list] [next in list] [prev in thread] [next in thread]