[prev in list] [next in list] [prev in thread] [next in thread] 

List:       lustre-discuss
Subject:    [Lustre-discuss] [Fwd: RE: HSM stage 1]
From:       jc.lafoucriere () cea ! fr (Jacques-Charles Lafoucriere)
Date:       2007-01-31 10:45:44
Message-ID: 45C0D5A3.4020604 () cea ! fr
[Download RAW message or body]

Hi,

CFS and CEA have started working on the Lustre HSM detailed design and 
implementation.
Directory migration is not supported but we will have an option to 
migrate files attibutes (striping, pathname when file is migrated, ....) 
in the backstore system.
So if you loose all the Lustre namespace you will be able to re-create 
the Lustre namespace from the backstore but not the empty directories 
nor the owner/rights of directories, files. These informations could be 
backup with a separate backup tool (optimized for Lustre).
We will have a whole file pre-migration and later, when free space is 
needed, we will purge the file but keep some data on the OST to avoid 
staging if, for example, someone access few bytes at the beginning of 
the file.
Staging will be trigered by the open or by the first read/write (this 
will be a policy parameter).

JC
Weikuan Yu wrote:

> Resent with an account on the list. Sorry for repetitions.
>
> Weikuan
>
>>
>> Hi,
>> It has been sometime since last discussion on Lustre HSM plan. Folks 
>> here at ORNL are interested to get this work going. Attached is a 
>> diagram about our initial idea and plan for stage 1. What shown is 
>> only about staging. But purging/migration should be more or less the 
>> same path from step 2 to step 9. Only difference is that they may not 
>> be in the critical path of client IO. As you may realize, this is 
>> just to show the idea for stage 1 of the Lustre HSM plan.
>>
>> Along with this diagram, I have also a couple of doubts.
>>
>> 1) I know a point was made not to archive striping info. Is Lustre 
>> HSM going to support migration of directories, or just objects of 
>> file data? If yes, does it imply that we need to archive striping 
>> info too?
>>
>> 2) I know the earlier CMOBD is supposed to situated entirely under 
>> OST/MDT. That makes sense for stage 2, i.e. partial file or 
>> per-extent migration. Would it just sufficient to have a caching 
>> module (HSC) under MDS to do per-file migration for stage 1? This 
>> also means that we do not have to save the per-extent migration 
>> status in the metadata, which, if implemented, would just fight for 
>> the limited space with the stripe map descriptors.
>>
>> 3) I put an HSS there as something similar to the current OST, meant 
>> to serve multiple HSOBD in the hope that it will provide flexibility.
>>
>> Please comment on the questions and diagram whether they make sense 
>> or not.
>>
>> We are also curious to know if CFS and CEA have started in this 
>> direction. If so, how could we join hands in some way.
>>
>> Thanks,
>> Weikuan
>
>
>
> ------------------------------------------------------------------------
>
>------------------------------------------------------------------------
>
>_______________________________________________
>Lustre-discuss mailing list
>Lustre-discuss@clusterfs.com
>https://mail.clusterfs.com/mailman/listinfo/lustre-discuss
>  
>

-------------- next part --------------
Skipped content of type multipart/related

[prev in list] [next in list] [prev in thread] [next in thread]