[prev in list] [next in list] [prev in thread] [next in thread]
List: libvirt-users
Subject: [libvirt-users] NPIV storage pools do not map to same LUN units across hosts.
From: Nitesh Konkar <niteshkonkar.libvirt () gmail ! com>
Date: 2016-07-15 21:07:15
Message-ID: CA+beMh7DkJzHF_iFg6y4kXbOU4dt8dKTMG2Zo_Xz8TMqSVUROQ () mail ! gmail ! com
[Download RAW message or body]
Link: http://wiki.libvirt.org/page/NPIV_in_libvirt
Topic: Virtual machine configuration change to use vHBA LUN
There is a NPIV storage pool defined on two hosts and pool contains a
total of 8 volumes, allocated from a storage device.
Source:
# virsh vol-list poolvhba0
Name Path
------------------------------------------------------------------------------
unit:0:0:0 /dev/disk/by-id/wwn-0x6005076802818bda3000000000000366
unit:0:0:1 /dev/disk/by-id/wwn-0x6005076802818bda3000000000000367
unit:0:0:2 /dev/disk/by-id/wwn-0x6005076802818bda3000000000000368
unit:0:0:3 /dev/disk/by-id/wwn-0x6005076802818bda3000000000000369
unit:0:0:4 /dev/disk/by-id/wwn-0x6005076802818bda300000000000036a
unit:0:0:5 /dev/disk/by-id/wwn-0x6005076802818bda3000000000000380
unit:0:0:6 /dev/disk/by-id/wwn-0x6005076802818bda3000000000000381
unit:0:0:7 /dev/disk/by-id/wwn-0x6005076802818bda3000000000000382
--------------------------------------------------------------------
Destination:
--------------------------------------------------------------------
# virsh vol-list poolvhba0
Name Path
------------------------------------------------------------------------------
unit:0:0:0 /dev/disk/by-id/wwn-0x6005076802818bda3000000000000380
unit:0:0:1 /dev/disk/by-id/wwn-0x6005076802818bda3000000000000381
unit:0:0:2 /dev/disk/by-id/wwn-0x6005076802818bda3000000000000382
unit:0:0:3 /dev/disk/by-id/wwn-0x6005076802818bda3000000000000367
unit:0:0:4 /dev/disk/by-id/wwn-0x6005076802818bda3000000000000368
unit:0:0:5 /dev/disk/by-id/wwn-0x6005076802818bda3000000000000366
unit:0:0:6 /dev/disk/by-id/wwn-0x6005076802818bda300000000000036a
unit:0:0:7 /dev/disk/by-id/wwn-0x6005076802818bda3000000000000369
--------------------------------------------------------------------
As you can see in the above output,the same set of eight LUNs from the
storage server have been mapped,
but the order that the LUNs are probed on each host is different,
resulting in different unit names
on the two different hosts .
If the the guest XMLs is referencing its storage by "unit" number then is
it safe to migrate such guests because the "unit number" is assigned by the
driver according to the specific way it probes the storage and hence
when you migrate
these guests , it results in different unit names on the destination hosts.
Thus the migrated guest gets mapped to the wrong LUNs and is given the
wrong disks.
The problem is that the LUN numbers on the destination host and source
host do not agree.
Example, LUN 0 on source_host, for example, may be LUN 5 on destination_host.
When the guest is given the wrong disk, it suffers a fatal I/O error. (This is
manifested as fatal I/O errors since the guest has no idea that its disks just
changed out under it.)The migration does not take into account that
the unit numbers do
match on on the source and destination sides.
So, should libvirt make sure that the guest domains reference NPIV
pool volumes by their
globally-unique wwn instead of by "unit" numbers?
The guest XML references its storage by "unit" number.
Eg:-
<disk type='volume' device='lun'>
<driver name='qemu' type='raw' cache='none'/>
<source pool='poolvhba0' volume='unit:0:0:0'/>
<backingStore/>
<target dev='vdb' bus='virtio'/>
<alias name='virtio-disk1'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x05'
function='0x0'/>
</disk>
I am planning to write a patch for it. Any comments on the above
observation/approach would be appreciated.
Thanks,
Nitesh.
[Attachment #3 (text/html)]
<div dir="ltr"><div dir="ltr"><div>Link: <a \
href="http://wiki.libvirt.org/page/NPIV_in_libvirt" \
target="_blank">http://wiki.libvirt.org/page/NPIV_in_libvirt</a> <br>Topic: Virtual \
machine configuration change to use vHBA LUN<br> <br>There is a NPIV storage pool \
defined on two hosts and pool contains a total of 8 volumes, allocated from a \
storage device.<br><br></div><div>Source:<br></div><div><pre><font size="4"><span \
style="font-family:arial,helvetica,sans-serif"># virsh vol-list poolvhba0 Name \
Path
------------------------------</span></font><font size="4"><span \
style="font-family:arial,helvetica,sans-serif">------------------------------</span></font><font \
size="4"><span style="font-family:arial,helvetica,sans-serif">------------------ \
unit:0:0:0 /dev/disk/by-id/wwn-</span></font><font size="4"><span \
style="font-family:arial,helvetica,sans-serif">0x6005076802818bda300000000000</span></font><font \
size="4"><span style="font-family:arial,helvetica,sans-serif">0366 unit:0:0:1 \
/dev/disk/by-id/wwn-</span></font><font size="4"><span \
style="font-family:arial,helvetica,sans-serif">0x6005076802818bda300000000000</span></font><font \
size="4"><span style="font-family:arial,helvetica,sans-serif">0367 unit:0:0:2 \
/dev/disk/by-id/wwn-</span></font><font size="4"><span \
style="font-family:arial,helvetica,sans-serif">0x6005076802818bda300000000000</span></font><font \
size="4"><span style="font-family:arial,helvetica,sans-serif">0368 unit:0:0:3 \
/dev/disk/by-id/wwn-</span></font><font size="4"><span \
style="font-family:arial,helvetica,sans-serif">0x6005076802818bda300000000000</span></font><font \
size="4"><span style="font-family:arial,helvetica,sans-serif">0369 unit:0:0:4 \
/dev/disk/by-id/wwn-</span></font><font size="4"><span \
style="font-family:arial,helvetica,sans-serif">0x6005076802818bda300000000000</span></font><font \
size="4"><span style="font-family:arial,helvetica,sans-serif">036a unit:0:0:5 \
/dev/disk/by-id/wwn-</span></font><font size="4"><span \
style="font-family:arial,helvetica,sans-serif">0x6005076802818bda300000000000</span></font><font \
size="4"><span style="font-family:arial,helvetica,sans-serif">0380 unit:0:0:6 \
/dev/disk/by-id/wwn-</span></font><font size="4"><span \
style="font-family:arial,helvetica,sans-serif">0x6005076802818bda300000000000</span></font><font \
size="4"><span style="font-family:arial,helvetica,sans-serif">0381 unit:0:0:7 \
/dev/disk/by-id/wwn-</span></font><font size="4"><span \
style="font-family:arial,helvetica,sans-serif">0x6005076802818bda300000000000</span></font><font \
size="4"><span style="font-family:arial,helvetica,sans-serif">0382
------------------------------</span></font><font size="4"><span \
style="font-family:arial,helvetica,sans-serif">------------------------------</span></font><font \
size="4"><span style="font-family:arial,helvetica,sans-serif">-------- \
<br>Destination:
------------------------------</span></font><font size="4"><span \
style="font-family:arial,helvetica,sans-serif">------------------------------</span></font><font \
size="4"><span style="font-family:arial,helvetica,sans-serif">-------- # virsh \
vol-list poolvhba0 Name Path
------------------------------</span></font><font size="4"><span \
style="font-family:arial,helvetica,sans-serif">------------------------------</span></font><font \
size="4"><span style="font-family:arial,helvetica,sans-serif">------------------ \
unit:0:0:0 /dev/disk/by-id/wwn-</span></font><font size="4"><span \
style="font-family:arial,helvetica,sans-serif">0x6005076802818bda300000000000</span></font><font \
size="4"><span style="font-family:arial,helvetica,sans-serif">0380 unit:0:0:1 \
/dev/disk/by-id/wwn-</span></font><font size="4"><span \
style="font-family:arial,helvetica,sans-serif">0x6005076802818bda300000000000</span></font><font \
size="4"><span style="font-family:arial,helvetica,sans-serif">0381 unit:0:0:2 \
/dev/disk/by-id/wwn-</span></font><font size="4"><span \
style="font-family:arial,helvetica,sans-serif">0x6005076802818bda300000000000</span></font><font \
size="4"><span style="font-family:arial,helvetica,sans-serif">0382 unit:0:0:3 \
/dev/disk/by-id/wwn-</span></font><font size="4"><span \
style="font-family:arial,helvetica,sans-serif">0x6005076802818bda300000000000</span></font><font \
size="4"><span style="font-family:arial,helvetica,sans-serif">0367 unit:0:0:4 \
/dev/disk/by-id/wwn-</span></font><font size="4"><span \
style="font-family:arial,helvetica,sans-serif">0x6005076802818bda300000000000</span></font><font \
size="4"><span style="font-family:arial,helvetica,sans-serif">0368 unit:0:0:5 \
/dev/disk/by-id/wwn-</span></font><font size="4"><span \
style="font-family:arial,helvetica,sans-serif">0x6005076802818bda300000000000</span></font><font \
size="4"><span style="font-family:arial,helvetica,sans-serif">0366 unit:0:0:6 \
/dev/disk/by-id/wwn-</span></font><font size="4"><span \
style="font-family:arial,helvetica,sans-serif">0x6005076802818bda300000000000</span></font><font \
size="4"><span style="font-family:arial,helvetica,sans-serif">036a unit:0:0:7 \
/dev/disk/by-id/wwn-</span></font><font size="4"><span \
style="font-family:arial,helvetica,sans-serif">0x6005076802818bda300000000000</span></font><font \
size="4"><span style="font-family:arial,helvetica,sans-serif">0369
------------------------------</span></font><font size="4"><span \
style="font-family:arial,helvetica,sans-serif">------------------------------</span></font><font \
size="4"><span style="font-family:arial,helvetica,sans-serif">--------<br><br>As you \
can see in the above output,the same set of eight LUNs from the storage server have \
been mapped,<br>but the order that the LUNs are probed on each host is different, \
resulting in different unit names <br>on the two different hosts \
.<br></span></font></pre></div><pre><font size="4"><span \
style="font-family:arial,helvetica,sans-serif">If the the guest XMLs is referencing \
its storage by "unit" number then is <br>it safe to migrate such guests \
because the "unit number" is assigned by the <br>driver according to the \
specific way it probes the storage and hence when you migrate <br>these guests , it \
results in different unit names on the destination hosts. <br>Thus the migrated \
guest gets mapped to the wrong LUNs and is given the wrong disks.<br>The problem is \
that the LUN numbers on the destination host and source host do not agree. \
<br>Example, LUN 0 on source_host, for example, may be LUN 5 on \
destination_host.<br>When the guest is given the wrong disk, it suffers a fatal I/O \
error. (This is <br>manifested as fatal I/O errors since the guest has no idea that \
its disks just <br>changed out under it.)The migration does not take into account \
that the unit numbers do <br>match on on the source and destination sides.<br><br>So, \
should libvirt make sure that the guest domains reference NPIV pool volumes by \
their<br>globally-unique wwn instead of by "unit" numbers?<br><br>The guest \
XML references its storage by "unit" number.<br><br>Eg:-<br><disk \
type='volume' device='lun'>
<driver name='qemu' type='raw' cache='none'/>
<source pool='poolvhba0' volume='unit:0:0:0'/>
<backingStore/>
<target dev='vdb' bus='virtio'/>
<alias name='virtio-disk1'/>
<address type='pci' domain='0x0000' bus='0x00' \
slot='0x05' function='0x0'/> \
</disk><br><br></span></font></pre><pre><font size="4"><span \
style="font-family:arial,helvetica,sans-serif">I am planning to write a patch for it. \
Any comments on the above observation/approach would be appreciated. \
<br></span></font></pre><pre><font size="4"><span \
style="font-family:arial,helvetica,sans-serif">Thanks,<br></span></font></pre><pre><font \
size="4"><span style="font-family:arial,helvetica,sans-serif">Nitesh. \
</span></font></pre></div></div>
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic