[prev in list] [next in list] [prev in thread] [next in thread] 

List:       opensuse
Subject:    Re: [opensuse] Raid5/LVM2/XFS alignment
From:       "Greg Freemyer" <greg.freemyer () gmail ! com>
Date:       2008-01-31 20:15:44
Message-ID: 87f94c370801311215ib72732et5dee4c1fca702a47 () mail ! gmail ! com
[Download RAW message or body]

Resend now that I'm resubscribed

On Jan 31, 2008 2:32 PM, Greg Freemyer <greg.freemyer@gmail.com> wrote:
>
> On Jan 31, 2008 12:48 PM, Ciro Iriarte <cyruspy@gmail.com> wrote:
> >
> > > > Just for the record, dealing with a bug that made the raid hang, found
> > > > a workaround that also gave me performance boost: "echo 4096 >
> > > > /sys/block/md2/md/stripe_cache_size"
> > > >
> > > > Result:
> > > >
> > > > mainwks:~ # dd if=/dev/zero bs=1024k count=1000 of=/datos/test
> > > > 1000+0 records in
> > > > 1000+0 records out
> > > > 1048576000 bytes (1,0 GB) copied, 6,78341 s, 155 MB/s
> > > >
> > > > mainwks:~ # rm /datos/test
> > > >
> > > > mainwks:~ # dd if=/dev/zero bs=1024k count=20000 of=/datos/test
> > > > 20000+0 records in
> > > > 20000+0 records out
> > > > 20971520000 bytes (21 GB) copied, 199,135 s, 105 MB/s
> > > >
> > > > Ciro
> > > >
> > > Ciro,
> > >
> > > 105 MB/s seems strange to me.  I would have expected 75 MB/s or 225MB/ s
> > >
> > > ie. For normal non-full stripe i/o, it should be 75MB/s * 4 / 4.
> > > Where 75MB/sec is what I see for one drive typically, the first 4 is
> > > the number of drives that can be doing parallel i/o and the second 4
> > > is the number of i/o's per write.
> > >
> > > ie. When you do a non-full stripe write, the kernel has to read the
> > > old checksum.  read the old chunk data, recalc the checksum, write the
> > > new chunk data, write the checksum.
> > >
> > > Out of curiosity, on the dd line, do you get better performance if you
> > > set your blocksize to exactly one stripe?  ie. 3x 256KB = 768KB
> > > stripe.   I've read the Linux's raid5 implementation is optimized to
> > > handle full stripe write's.
> > >
> > > ie. Writing 3 chunks produces:  Calc new checksum from all new data,
> > > Write d1, d2, d3, p so to get 3 256KB chunks to the drive, the kernel
> > > ends up invoking 4 256KB writes.
> > >
> > > Or 75 MB/s * 4 * 3 / 4 = 225 MB / sec
> > >
> > > If you have everything optimized, I think you should see the same
> > > performance with a 2-stripe write.  ie. 6x 256KB.  If your
> > > optimization is wrong, you will see a speed improvement because the
> > > alignment between your writes and stripes will be wrong.  With the
> > > bigger write, you will be guaranteed at least one full stripe write.
> > >
> > > Thanks
> > > Greg
> > > --
> > > Greg Freemyer
> >
> > Well, got this now:
> >
> > mainwks:~> dd if=/dev/zero of=/datos/test bs=768k count=10000
> > 10000+0 records in
> > 10000+0 records out
> > 7864320000 bytes (7,9 GB) copied, 63,5425 s, 124 MB/s
> >
> > mainwks:~> dd if=/dev/zero of=/datos/test2 bs=768k count=20000
> > 20000+0 records in
> > 20000+0 records out
> > 15728640000 bytes (16 GB) copied, 141,317 s, 111 MB/s
> >
> > The filesystem is not empty anymore and i had several applications
> > running during this test.
> >
> > Ciro
> >
>
> Ciro,
>
> 125MB/sec is not half bad for write speed, especially if the system
> was busy doing other things.
>
> Out of curiosity do you mind checking your read speed.  ie. dd
> if=large_file of=/dev/null bs=768k
>
> Since you will be using this drive mostly to hold large static files,
> the next question is how fast is a normal cp, vs. a dd.
>
> ie. Since you can optimize the bs with dd, you may want to use dd to
> load your large files onto the drive.
>
>
> Greg
> --
> Greg Freemyer
> Litigation Triage Solutions Specialist
> http://www.linkedin.com/in/gregfreemyer
> First 99 Days Litigation White Paper -
> http://www.norcrossgroup.com/forms/whitepapers/99%20Days%20whitepaper.pdf
>
> The Norcross Group
> The Intersection of Evidence & Technology
> http://www.norcrossgroup.com
>



-- 
Greg Freemyer
Litigation Triage Solutions Specialist
http://www.linkedin.com/in/gregfreemyer
First 99 Days Litigation White Paper -
http://www.norcrossgroup.com/forms/whitepapers/99%20Days%20whitepaper.pdf

The Norcross Group
The Intersection of Evidence & Technology
http://www.norcrossgroup.com
-- 
To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org
For additional commands, e-mail: opensuse+help@opensuse.org

[prev in list] [next in list] [prev in thread] [next in thread]