[prev in list] [next in list] [prev in thread] [next in thread]
List: linux-lvm
Subject: [linux-lvm] Best way to run LVM over multiple SW RAIDs?
From: Daniel Janzon <daniel.janzon () edgeware ! tv>
Date: 2019-10-29 8:47:29
Message-ID: A8A53B0E4B7ACF4BAD5C77D3671AC8976AB85B3B () EX01 ! edgeware ! tv
[Download RAW message or body]
Hello,
I have a server with very high load using four NVMe SSDs and therefore no H=
W RAID. Instead I used SW RAID with the mdadm tool. Using one RAID5 volume =
does not work well since the driver can only utilize one CPU core which spi=
kes at 100% and harms performance. Therefore I created 8 partitions on each=
disk, and 8 RAID5s across the four disks.
Now I want to bring them together with LVM. If I do not use a striped volum=
e I get high performance (in expected magnitude according to disk specs). B=
ut when I use a striped volume, performance drops to a magnitude below. The=
reason I am looking for a striped setup is to ensure that data is spread w=
ell over the drive to guarantee a good worst-case performance. With linear =
allocation rather than striped, if load is directed to files on the first P=
V (a SW RAID) the system is again exposed to the 1-core limitation.
I tried "--stripes 8 --stripesize 512", and would appreciate any ideas of o=
ther things to try. I guess the performance hit can be in the file system a=
s well. I tried XFS and EXT4 with default settings.
Kind Regards,
Daniel
[Attachment #3 (text/html)]
<html dir="ltr">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
<style type="text/css" id="owaParaStyle"></style>
</head>
<body fpstyle="1" ocsi="0">
<div style="direction: ltr;font-family: Tahoma;color: #000000;font-size: 10pt;">
<div>Hello,</div>
<div><br>
</div>
<div>I have a server with very high load using four NVMe SSDs and therefore no HW \
RAID. Instead I used SW RAID with the mdadm tool. Using one RAID5 volume does not \
work well since the driver can only utilize one CPU core which spikes at 100% and \
harms performance. Therefore I created 8 partitions on each disk, and 8 RAID5s \
across the four disks.</div> <div><br>
</div>
<div>Now I want to bring them together with LVM. If I do not use a striped volume I \
get high performance (in expected magnitude according to disk specs). But when I use \
a striped volume, performance drops to a magnitude below. The reason I am looking for \
a striped setup is to ensure that data is spread well over the drive to guarantee a \
good worst-case performance. With linear allocation rather than striped, if load is \
directed to files on the first PV (a SW RAID) the system is again exposed to the \
1-core limitation.<br> </div>
<div><br>
</div>
<div>I tried "--stripes 8 --stripesize 512", and would appreciate any ideas \
of other things to try. I guess the performance hit can be in the file system as \
well. I tried XFS and EXT4 with default settings.</div> <div><br>
</div>
<div>Kind Regards, <br>
</div>
<div>Daniel</div>
<div><br>
</div>
</div>
</body>
</html>
_______________________________________________
linux-lvm mailing list
linux-lvm@redhat.com
https://www.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
--===============3613339777029933741==--
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic