[prev in list] [next in list] [prev in thread] [next in thread] 

List:       lustre-discuss
Subject:    [Lustre-discuss] How efficient is 'lfs find'?
From:       andreas.dilger () oracle ! com (Andreas Dilger)
Date:       2010-07-16 20:01:06
Message-ID: DE358F02-EF55-42A4-8A12-4F9B54E5C645 () oracle ! com
[Download RAW message or body]



On 2010-07-15, at 15:46, "Adesanya, Adeyemi" <yemi at slac.stanford.edu> wrote:
> We are working on coming up with a backup plan for our Lustre filesystem in case we \
> ever lose an OST in the future. I like the idea of backing up the filesystem at the \
> client level and then identifying what files were stored on a missing OST using \
> 'lfs find'.

Note that there us a bug in 'lfs find' that prevents it working when an OST is down, \
because it sometimes tries to get the object sizes from he OSTs. However, it is \
possible to use 'lfs getstripe -q' to do the same thing. 

> There are also some useful 'lfs find' commands that may assist us in \
> migrating/purging old data such as the use of the '--atime' and '--mtime' options. \
> Can someone comment on the efficiency of 'lfs find'? Does it only query the MDT or \
> will it also contact the OSTs?

It depends on your query. If you are asking only for MDS informaion (uid, gid, name, \
type, osts) it only queries the MDS.  If you ask for size it must query the OSTs. For \
timestamos it depends in the query. If the MDS timestamp is enough to decide then no \
OST RPC is sent (e.g. --mtime -1 and the MDS inode has an mtime newer than that), but \
this depends on the exact query and the inode timestamps. 

Note that you may want to look at e2scan, or ne2scan (modified version of e2scan) \
since it does much of thus already. Also of interest would be CEA's RobinHood policy \
manager. 

Cheers, Andreas


[prev in list] [next in list] [prev in thread] [next in thread]