[prev in list] [next in list] [prev in thread] [next in thread] 

List:       linux-xfs
Subject:    Tuning for tons of small files?
From:       Dave Lloyd <dlloyd () exegy ! com>
Date:       2006-04-19 16:25:08
Message-ID: 44466464.8050505 () exegy ! com
[Download RAW message or body]


I'm trying to tune XFS for faster read and write times when accessing 
tons of small files.  System is 2.6.16 on a 2x2 core opteron.  Options 
that the fs was made with:
-f -d sunit=128,swidth=512,unwritten=1 -l version=2,sunit=128

Mount options:
noatime,largeio,ihashsize=131072,attr2,barrier,allocsize=1073741824

This is across four hardware raid0 filesystems.  Right now the read and 
write performance for larger files (~8gb) just kills ext3, perhaps 
between 2-4x faster in some instances, but ext3 kills xfs for accessing 
lots of small files.

I've seen that tuning the logbufs up may help out, so that's my next 
step.  I was also thinking that configuring the hardware raids as a 16 
disk JBOD and making it a concat md volume may also distribute out the 
I/O as what I believe what is killing us here is seek time (of course, 
this is if md is smart and tells xfs how the dev is laid out so it can 
put the ags at the correct place to round-robin I/O to the different 
phys devs and I'm not so sure that md is that smart).

Any other ideas to tune XFS for better performance when accessing tons 
of small files?

-- 
Dave Lloyd
Test Engineer, Exegy, Inc.
314.450.5342
dlloyd@exegy.com

["dlloyd.vcf" (text/x-vcard)]

[prev in list] [next in list] [prev in thread] [next in thread]