[prev in list] [next in list] [prev in thread] [next in thread]
List: lustre-discuss
Subject: Re: [lustre-discuss] lustre-discuss Digest, Vol 158, Issue 10
From: Kurt Strosahl <strosahl () jlab ! org>
Date: 2019-05-09 20:38:42
Message-ID: BN8PR09MB35702233D2D13D07283B2494A7330 () BN8PR09MB3570 ! namprd09 ! prod ! outlook ! com
[Download RAW message or body]
Presently I'm experimenting with the following:
/etc/zfs/vdev_id.conf
alias ost01d1shasl00 /dev/disk/by-id/dm-uuid-mpath-35000cca26b825d6c
alias ost01d2shasl01 /dev/disk/by-id/dm-uuid-mpath-35000cca26b860178
alias ost01d3shasl02 /dev/disk/by-id/dm-uuid-mpath-35000cca26c1e2cb4
alias ost01d4shasl03 /dev/disk/by-id/dm-uuid-mpath-35000cca2680a8280
and using that to make sure that the disks have persistent names across reboots. I \
still need to test pulling out one of the SAS cables to make sure that the \
multipathing works, and test replacing a disk.
I read on http://wiki.lustre.org/ZFS_OSD_Hardware_Considerations that
"The configuration of the device multi-mapper service is quite complex and can affect \
the performance characteristics of the solution. In some cases, JBODs can exhibit bad \
behavior from using load-balanced IO balancing, when in fact all the requests for a \
disk are expected to arrive from a single interface. For this reason, when working \
with JBODS it is recommended to use the path_grouping_policy that enables \
failover-only capability."
so I presently set my systems to failover mode.
w/r,
Kurt
________________________________
From: lustre-discuss <lustre-discuss-bounces@lists.lustre.org> on behalf of \
lustre-discuss-request@lists.lustre.org \
<lustre-discuss-request@lists.lustre.org>
Sent: Thursday, May 9, 2019 4:21 PM
To: lustre-discuss@lists.lustre.org
Subject: lustre-discuss Digest, Vol 158, Issue 10
Send lustre-discuss mailing list submissions to
lustre-discuss@lists.lustre.org
To subscribe or unsubscribe via the World Wide Web, visit
https://gcc01.safelinks.protection.outlook.com/?url=http%3A%2F%2Flists.lustre. \
org%2Flistinfo.cgi%2Flustre-discuss-lustre.org&data=02%7C01%7Cstrosahl%40jlab.org% \
7Caf7c47376b2d44b69a1708d6d4bbe959%7Cb4d7ee1f4fb34f0690372b5b522042ab%7C1%7C0%7C636930 \
300867938953&sdata=rP9azd3ble3LOVL%2BXUmfghJJsD0HlAGWrk2djNP1Ty4%3D&reserved=0
or, via email, send a message with subject or body 'help' to
lustre-discuss-request@lists.lustre.org
You can reach the person managing the list at
lustre-discuss-owner@lists.lustre.org
When replying, please edit your Subject line so it is more specific
than "Re: Contents of lustre-discuss digest..."
Today's Topics:
1. Enable multipath for existing Lustre OST with ZFS backend
(Tung-Han Hsieh)
----------------------------------------------------------------------
Message: 1
Date: Fri, 10 May 2019 03:25:52 +0800
From: Tung-Han Hsieh <thhsieh@twcp1.phys.ntu.edu.tw>
To: lustre-discuss@lists.lustre.org
Subject: [lustre-discuss] Enable multipath for existing Lustre OST
with ZFS backend
Message-ID: <20190509192551.GA5793@twcp1.phys.ntu.edu.tw>
Content-Type: text/plain; charset=big5
Greetings,
Recently we have a new storage device. It has dual RAID controllers
with two fibre connections to the file server which map the LUM of
the storage to the server:
# lsscsi -g
[5:0:0:0] disk IFT DS 1000 Series 661J /dev/sdb /dev/sg4
[6:0:0:0] disk IFT DS 1000 Series 661J /dev/sdc /dev/sg6
# /lib/udev/scsi_id -g -u /dev/sdb
3600d02310009ff8750249f7e31c5fd86
# /lib/udev/scsi_id -g -u /dev/sdc
3600d02310009ff8750249f7e31c5fd86
So /dev/sdb and /dev/sdc are actually the same LUM of the storage.
We have created the Lustre OST with ZFS backend on /dev/sdb:
# mkfs.lustre --ost --fsname chome --mgsnode=<host> --index=0 \
--backfstype=zfs chome_ost/ost /dev/sdb
It works fine. But soon after that, I was told that I should setup
multipath to take the advantage of dual fibre channel for load
balance and HA. I am wondering whether it is too late or not because
we already have data of Lustre file system running on it.
I read the documents of multipath. It seems that after setting
multipath, both /dev/sdb and /dev/sdc are re-mapped to, say,
/dev/mapper/mpath0. The existing data is probably not affacted.
What we need to do is just to replace the device name /dev/sdb
by /dev/mapper/mpath0 (please correct me if I am wrong). So the
problem seems leading to ZFS. Now my OST pool "chome_ost/ost" was
created on /dev/sdb. Could we replace the pool device name to
/dev/mapper/mpath0 ?
Thanks very much for you suggestions in advance :)
Best Regards,
T.H.Hsieh
------------------------------
Subject: Digest Footer
_______________________________________________
lustre-discuss mailing list
lustre-discuss@lists.lustre.org
https://gcc01.safelinks.protection.outlook.com/?url=http%3A%2F%2Flists.lustre.org%2Fli \
stinfo.cgi%2Flustre-discuss-lustre.org&data=02%7C01%7Cstrosahl%40jlab.org%7Caf7c47 \
376b2d44b69a1708d6d4bbe959%7Cb4d7ee1f4fb34f0690372b5b522042ab%7C1%7C0%7C636930300867938953&sdata=rP9azd3ble3LOVL%2BXUmfghJJsD0HlAGWrk2djNP1Ty4%3D&reserved=0
------------------------------
End of lustre-discuss Digest, Vol 158, Issue 10
***********************************************
[Attachment #3 (text/html)]
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
<style type="text/css" style="display:none;"><!-- P {margin-top:0;margin-bottom:0;} \
--></style> </head>
<body dir="ltr">
<div id="divtagdefaultwrapper" \
style="font-size:12pt;color:#000000;font-family:Calibri,Helvetica,sans-serif;" \
dir="ltr"> <p style="margin-top:0;margin-bottom:0">Presently I'm experimenting with \
the following:</p> <p style="margin-top:0;margin-bottom:0"><br>
</p>
<p style="margin-top:0;margin-bottom:0"></p>
<div>/etc/zfs/vdev_id.conf</div>
<div>alias ost01d1shasl00 /dev/disk/by-id/dm-uuid-mpath-35000cca26b825d6c</div>
<div>alias ost01d2shasl01 /dev/disk/by-id/dm-uuid-mpath-35000cca26b860178</div>
<div>alias ost01d3shasl02 /dev/disk/by-id/dm-uuid-mpath-35000cca26c1e2cb4</div>
<div>alias ost01d4shasl03 /dev/disk/by-id/dm-uuid-mpath-35000cca2680a8280</div>
<br>
<p></p>
<p style="margin-top:0;margin-bottom:0">and using that to make sure that the disks \
have persistent names across reboots. I still need to test pulling out one of \
the SAS cables to make sure that the multipathing works, and test replacing a \
disk.</p> <p style="margin-top:0;margin-bottom:0"><br>
</p>
<p style="margin-top:0;margin-bottom:0"><br>
</p>
I read on <a href="http://wiki.lustre.org/ZFS_OSD_Hardware_Considerations" \
class="OWAAutoLink" id="LPlnk187229">http://wiki.lustre.org/ZFS_OSD_Hardware_Considerations</a> that
<div><br>
</div>
<div>"<span style="color: rgb(34, 34, 34); font-family: sans-serif; font-size: \
14px;">The configuration of the device multi-mapper service is quite complex and can \
affect the performance characteristics of the solution. In some cases, JBODs can \
exhibit bad behavior from using load-balanced IO balancing, when in fact all the \
requests for a disk are expected to arrive from a single interface. For this reason, \
when working with JBODS it is recommended to use the </span><code \
style="font-family: monospace, Courier; background-color: rgb(248, 249, 250); border: \
1px solid rgb(234, 236, 240); border-radius: 2px; padding: 1px 4px; font-size: \
14px;">path_grouping_policy</code><span style="color: rgb(34, 34, 34); font-family: \
sans-serif; font-size: 14px;"> that enables failover-only \
capability."</span></div> <div><font color="#222222" face="sans-serif"><span \
style="font-size: 14px;"><br> </span></font></div>
<div><span style="color: rgb(34, 34, 34); font-family: sans-serif; font-size: \
14px;"></span><font color="#222222" face="sans-serif"><span style="font-size: \
14px;">so I presently set my systems to failover mode.</span></font></div> <div><font \
color="#222222" face="sans-serif"><span style="font-size: 14px;"><br> \
</span></font></div> <div><font color="#222222" face="sans-serif"><span \
style="font-size: 14px;">w/r,</span></font></div> <div><font color="#222222" \
face="sans-serif"><span style="font-size: 14px;">Kurt<br> </span></font><br>
<br>
<br>
<div style="color: rgb(0, 0, 0);">
<hr style="display:inline-block;width:98%" tabindex="-1">
<div id="divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" \
style="font-size:11pt" color="#000000"><b>From:</b> lustre-discuss \
<lustre-discuss-bounces@lists.lustre.org> on behalf of \
lustre-discuss-request@lists.lustre.org \
<lustre-discuss-request@lists.lustre.org><br> <b>Sent:</b> Thursday, May 9, \
2019 4:21 PM<br> <b>To:</b> lustre-discuss@lists.lustre.org<br>
<b>Subject:</b> lustre-discuss Digest, Vol 158, Issue 10</font>
<div> </div>
</div>
<div class="BodyFragment"><font size="2"><span style="font-size:11pt;">
<div class="PlainText">Send lustre-discuss mailing list submissions to<br>
lustre-discuss@lists.lustre.org<br>
<br>
To subscribe or unsubscribe via the World Wide Web, visit<br>
<a \
href="https://gcc01.safelinks.protection.outlook.com/?url=http%3A%2F%2Flists.lustre.or \
g%2Flistinfo.cgi%2Flustre-discuss-lustre.org&amp;data=02%7C01%7Cstrosahl%40jlab.or \
g%7Caf7c47376b2d44b69a1708d6d4bbe959%7Cb4d7ee1f4fb34f0690372b5b522042ab%7C1%7C0%7C6369 \
30300867938953&amp;sdata=rP9azd3ble3LOVL%2BXUmfghJJsD0HlAGWrk2djNP1Ty4%3D&amp;reserved=0" \
id="LPlnk239685" class="OWAAutoLink" previewremoved="true"> \
https://gcc01.safelinks.protection.outlook.com/?url=http%3A%2F%2Flists.lustre.org%2Fli \
stinfo.cgi%2Flustre-discuss-lustre.org&amp;data=02%7C01%7Cstrosahl%40jlab.org%7Caf \
7c47376b2d44b69a1708d6d4bbe959%7Cb4d7ee1f4fb34f0690372b5b522042ab%7C1%7C0%7C6369303008 \
67938953&amp;sdata=rP9azd3ble3LOVL%2BXUmfghJJsD0HlAGWrk2djNP1Ty4%3D&amp;reserved=0</a><br>
or, via email, send a message with subject or body 'help' to<br>
\
lustre-discuss-request@lists.lustre.org<br> <br>
You can reach the person managing the list at<br>
lustre-discuss-owner@lists.lustre.org<br>
<br>
When replying, please edit your Subject line so it is more specific<br>
than "Re: Contents of lustre-discuss digest..."<br>
<br>
<br>
Today's Topics:<br>
<br>
1. Enable multipath for existing Lustre OST with ZFS backend<br>
(Tung-Han Hsieh)<br>
<br>
<br>
----------------------------------------------------------------------<br>
<br>
Message: 1<br>
Date: Fri, 10 May 2019 03:25:52 +0800<br>
From: Tung-Han Hsieh <thhsieh@twcp1.phys.ntu.edu.tw><br>
To: lustre-discuss@lists.lustre.org<br>
Subject: [lustre-discuss] Enable multipath for existing Lustre OST<br>
with ZFS backend<br>
Message-ID: <20190509192551.GA5793@twcp1.phys.ntu.edu.tw><br>
Content-Type: text/plain; charset=big5<br>
<br>
Greetings,<br>
<br>
Recently we have a new storage device. It has dual RAID controllers<br>
with two fibre connections to the file server which map the LUM of<br>
the storage to the server:<br>
<br>
# lsscsi -g<br>
[5:0:0:0] disk IFT \
DS 1000 Series 661J /dev/sdb /dev/sg4<br> \
[6:0:0:0] disk IFT \
DS 1000 Series 661J /dev/sdc /dev/sg6<br> <br>
# /lib/udev/scsi_id -g -u /dev/sdb<br>
3600d02310009ff8750249f7e31c5fd86<br>
<br>
# /lib/udev/scsi_id -g -u /dev/sdc<br>
3600d02310009ff8750249f7e31c5fd86<br>
<br>
So /dev/sdb and /dev/sdc are actually the same LUM of the storage.<br>
<br>
We have created the Lustre OST with ZFS backend on /dev/sdb:<br>
<br>
# mkfs.lustre --ost --fsname chome --mgsnode=<host> --index=0 \<br>
\
--backfstype=zfs chome_ost/ost /dev/sdb<br> <br>
It works fine. But soon after that, I was told that I should setup<br>
multipath to take the advantage of dual fibre channel for load<br>
balance and HA. I am wondering whether it is too late or not because<br>
we already have data of Lustre file system running on it.<br>
<br>
I read the documents of multipath. It seems that after setting<br>
multipath, both /dev/sdb and /dev/sdc are re-mapped to, say,<br>
/dev/mapper/mpath0. The existing data is probably not affacted.<br>
What we need to do is just to replace the device name /dev/sdb<br>
by /dev/mapper/mpath0 (please correct me if I am wrong). So the<br>
problem seems leading to ZFS. Now my OST pool "chome_ost/ost" was<br>
created on /dev/sdb. Could we replace the pool device name to<br>
/dev/mapper/mpath0 ?<br>
<br>
Thanks very much for you suggestions in advance :)<br>
<br>
Best Regards,<br>
<br>
T.H.Hsieh<br>
<br>
<br>
------------------------------<br>
<br>
Subject: Digest Footer<br>
<br>
_______________________________________________<br>
lustre-discuss mailing list<br>
lustre-discuss@lists.lustre.org<br>
<a href="https://gcc01.safelinks.protection.outlook.com/?url=http%3A%2F%2Flists.lustre \
.org%2Flistinfo.cgi%2Flustre-discuss-lustre.org&amp;data=02%7C01%7Cstrosahl%40jlab \
.org%7Caf7c47376b2d44b69a1708d6d4bbe959%7Cb4d7ee1f4fb34f0690372b5b522042ab%7C1%7C0%7C6 \
36930300867938953&amp;sdata=rP9azd3ble3LOVL%2BXUmfghJJsD0HlAGWrk2djNP1Ty4%3D&amp;reserved=0" \
id="LPlnk643673" class="OWAAutoLink" \
previewremoved="true">https://gcc01.safelinks.protection.outlook.com/?url=http%3A%2F%2 \
Flists.lustre.org%2Flistinfo.cgi%2Flustre-discuss-lustre.org&amp;data=02%7C01%7Cst \
rosahl%40jlab.org%7Caf7c47376b2d44b69a1708d6d4bbe959%7Cb4d7ee1f4fb34f0690372b5b522042a \
b%7C1%7C0%7C636930300867938953&amp;sdata=rP9azd3ble3LOVL%2BXUmfghJJsD0HlAGWrk2djNP1Ty4%3D&amp;reserved=0</a><br>
<br>
<br>
------------------------------<br>
<br>
End of lustre-discuss Digest, Vol 158, Issue 10<br>
***********************************************<br>
</div>
</span></font></div>
</div>
</div>
</div>
</body>
</html>
_______________________________________________
lustre-discuss mailing list
lustre-discuss@lists.lustre.org
http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
--===============0705197490688052137==--
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic