[prev in list] [next in list] [prev in thread] [next in thread] 

List:       lustre-discuss
Subject:    [Lustre-discuss] clarification on mkfs.lustre options
From:       andreas.dilger () oracle ! com (Andreas Dilger)
Date:       2010-07-31 6:31:44
Message-ID: 0E5458AB-2DA3-4BF9-9D92-4A78063F5EA1 () oracle ! com
[Download RAW message or body]

On 2010-07-30, at 13:14, Sebastian Gutierrez <gutseb at cs.stanford.edu> wrote:
> > If you are planning on expanding this at the RAID6 level to be an 8+2 \
> > configuration, you should specify "-E stripe=256,stride=64".  
> 
> Are there any potential negatives here?   I initially used a 6 disk raid 10 but I \
> ended up with wasted space on the filesystem since I could not fit the 1M lustre \
> I/O into the number of active disks cleanly.  Would there be a way to minimize the \
> amount of wasted space if I wanted to stick to raid 1/0?   I assume that aligned \
> I/O is always preferred. 

For RAID-1+0 the alignment is much less important. While there is still some negative \
effect if the 1MB read of write is not aligned (because it will make an extra pair of \
disks active to fill the RPC) this is not nearly so bad as RAID-5/6 where it will \
cause also the parity chunk to be rewritten. 

If you are using a 6-disk RAID-1+0 then it would be OK for example to configure the \
RAID chunksize to be 128kB. While this means that a 1MB IO would handle 3*128kB from \
two pairs of disks and 4*128kB from the third pair of disks (each IO would be \
sequential though).

 It means a given pair of disks would do a bit more work than the others for a given \
RPC, but since the IO is sequential (assuming the request itself is sequential) it \
will not need an extra seek for the last disk and the extra IO is a minimal effort. 

> > 
> > Also, 16TB LUN support is only available with ext4, so if you have 2TB drives you \
> > need to make sure to download the right ldiskfs package.
> 
> I am creating these filesystems with lustre 1.8.3 from the prebuilt RPMs.  Do you \
> mean that there is a different ldiskfs package I should use?  

There should be an ldiskfs-ext4 RPM available for download with 1.8.3 and later (for \
specific vendor kernels). The ext3 code has had a  lot more testing than ext4 so we \
recommend using ext3 unless there is a reason to use ext4 (e.g. > 8TB LUN size).  

> Should i also use the flexible block group options on the OSTs?
> -O flex_bg and -G nr_merged_groups

The flex_bg feature is only available with ext4. We haven't done any testing with \
this feature yet, but in theory it can help. 

Cheers,  Andreas


[prev in list] [next in list] [prev in thread] [next in thread]