[prev in list] [next in list] [prev in thread] [next in thread]
List: veritas-vx
Subject: Re: [Veritas-vx] VxVm
From: Sengor <sengork () gmail ! com>
Date: 2010-04-02 15:15:01
Message-ID: p2r8417e52e1004020815ge9944fe4xb95878ee9b6c05a2 () mail ! gmail ! com
[Download RAW message or body]
[Attachment #2 (multipart/alternative)]
Guys,
A common misconception, I'm pretty sure PowerPath will failover stuff even
if you're using one of the native OS path devices as the filesystem mount
device. This is explicitly stated in the PP Admin guide and happens
transparently through the driver stack. Question should be which one of the
VxDMP or PP is actually actively doing multipathing, I'm guessing VxDMP goes
into passive mode when it sees PP.
Also there's a script which comes with VCS that'll probe your LUNs and
inquire whether they support any necessary SCSI reservations for the cluster
to operate. From memory it might have been vxfentsthdw.
On Wed, Mar 31, 2010 at 11:04 AM, Stuart Andrews <
Stuart_Andrews@symantec.com> wrote:
> Colin
>
>
>
> OK – I see you are using enclosure naming – and BTW – the /dev/vx/dmp
> /dev/vx/dmp are just the same as disks BUT with the added provision of DMP
> to keep the device online.
>
> How many paths to these devices
>
> # vxdmpadm getsubpaths
>
> or
>
> # vxdisk path
>
>
>
> If a CLARiiON , and if only 2 paths – then set the array iopolicy to
> singleactive – this is most likely – see later for DMP
>
> # vxdmpadm listenclosure all
>
> # vxdmpadm setattr enclosure ENC_name iopolicy=singleactive
>
> If a CLARiiON , and if more than 2 paths – then set the array iopolicy to
> balanced – and DMP does know how to stop IO to the secondary paths. Note –
> this is against EMC recommendations but it works.
>
> # vxdmpadm listenclosure all
>
> # vxdmpadm setattr enclosure ENC_name iopolicy=balanced
>
>
>
> I also notice that the enclosure names are lower case – indicating a V5.x
> release VxVM installed. – are the CLARiiON APMs running ?
>
> # vxdmpadm listapm all
>
> Check the CLARiiON are Active
>
>
>
> If a fencing cluster – then these are local LUNs at the point of
> /dev/vx/dmp names for the devices. And yes SCSI3 keys will be placed on them
> in a fencing cluster
>
> # gabconfig –a
>
> Check if there is Port b membership – if so then yes you have a fencing
> cluster.
>
>
>
> Check also the DMP block switch – it may be that with the iopolicy
> incorrect, and on low IO you did not reach the limit for
>
> # vxdmpadm gettune all
>
> dmp_pathswitch_blks_shift
>
> Now – when busy and IO chunks bigger than path switch level ( and with
> iopolicy incorrect ) a path switch will cause a trespass (check SAN logs)
> AND a block, drain, resume on the DMP path. There will be a failover
> message logged in /etc/vx/dmpevents.log – check here also.
>
>
>
> Stuart
>
>
> ------------------------------
>
> *From:* veritas-vx-bounces@mailman.eng.auburn.edu [mailto:
> veritas-vx-bounces@mailman.eng.auburn.edu] *On Behalf Of *Collin
> *Sent:* Wednesday, 31 March 2010 2:09 AM
> *To:* William Havey
> *Cc:* veritas-vx@mailman.eng.auburn.edu
> *Subject:* Re: [Veritas-vx] VxVm
>
>
>
> Sorry for any confusion...
>
> I've got several powerpath devices from a dead system that I'm mounting
> temporarily on one node in my cluster. I run a devfsadm -Cv and vxdctl
> enable. After that I can see the powerpath devices listed as...
>
> emc_clarrion0_10 auto:none - - online invalid
>
> I modified my /etc/vfstab file to mount the devices..
>
> /dev/vx/dmp/emc_clariion0_10s6 /dev/vx/rdmp/emc_clariion0_10s6 /u10
> ufs 3 yes -
>
> The device mounts and I can access the file system with all my data. When
> the activity starts to increase on these temporary mount points, I see a
> count down on the console that port H has lost connectivity. After the 16
> seconds, the node panics and of course reboots. However, if I mount the
> power path devices using a single path..
>
> /dev/dsk/c1t5006016100600432d10s6 /dev/rdsk/c1t5006016100600432d10s6
> /u10 ufs 3 yes -
>
> I never get the port H losing connectivity.
>
> I want to use the dmp name in case I lose a path to these disk.
>
> Any reason why using the dmp name causes port H to lose connectivity vs.
> using a single path?
>
> Thanks,
> Collin
>
> On Tue, Mar 30, 2010 at 10:48 AM, William Havey <bbhavey@gmail.com> wrote:
>
> The original message states "mount these disks as
> /dev/vx/dmp/<emc_array>_Xs6 ". Perhaps this is normal behavior. Mounts are
> of devices which receive I/O. A "/dev/vx/dmp/..." device entry isn't I/O
> capable.
>
> I think a clearer statement of what Collin intends to do is needed.
>
> Bill
>
>
>
> On Tue, Mar 30, 2010 at 3:01 AM, Dmitry Glushenok <glush@jet.msk.su>
> wrote:
>
> Hello,
>
> Panic string and previous messages usually helps to understand cause..
> Release notes to RP2-RP3 also provides short descriptions of fixed issues
> like "Fixed the cause of a system panic when mutex_panic() was called from
> vol_rwsleep_wrlock()."
>
>
> On 29.03.2010, at 19:02, Collin wrote:
>
> > I've got the following....
> >
> > Solaris 10
> > VxVM 5.0MP3RP1HF12
> >
>
> > I have a number of mount points that are being migrated from
> /dev/dsk/cXtXdXsX to clustered mount points. The problem I'm having is if I
> mount these disk in the /dev/dsk/cXtXdXsX format I run the risk that if
> something were to cause the direct path to go down I would lose the
> databases on these mount points. But when I mount these disks as
> /dev/vx/dmp/<emc_array>_Xs6 my system panics and core dumps.
>
> >
> > Does VxVM have any issues mounting /dev/vx/dmp/<emc_array>_Xs6??
> >
> > Thanks,
> > Collin
> > _______________________________________________
> > Veritas-vx maillist - Veritas-vx@mailman.eng.auburn.edu
> > http://mailman.eng.auburn.edu/mailman/listinfo/veritas-vx
>
> --
> Dmitry Glushenok
> Jet Infosystems
>
>
> _______________________________________________
> Veritas-vx maillist - Veritas-vx@mailman.eng.auburn.edu
> http://mailman.eng.auburn.edu/mailman/listinfo/veritas-vx
>
>
>
>
>
> _______________________________________________
> Veritas-vx maillist - Veritas-vx@mailman.eng.auburn.edu
> http://mailman.eng.auburn.edu/mailman/listinfo/veritas-vx
>
>
--
sengork
[Attachment #5 (text/html)]
<div>Guys,<br></div><div><br></div><div>A common misconception, I'm pretty sure \
PowerPath will failover stuff even if you're using one of the native OS path \
devices as the filesystem mount device. This is explicitly stated in the PP Admin \
guide and happens transparently through the driver stack. Question should be which \
one of the VxDMP or PP is actually actively doing multipathing, I'm guessing \
VxDMP goes into passive mode when it sees PP. </div> <div><br></div><div>Also \
there's a script which comes with VCS that'll probe your LUNs and inquire \
whether they support any necessary SCSI reservations for the cluster to operate. From \
memory it might have been vxfentsthdw.</div> \
<div><br></div><div><br></div><div><br></div><div class="gmail_quote">On Wed, Mar 31, \
2010 at 11:04 AM, Stuart Andrews <span dir="ltr"><<a \
href="mailto:Stuart_Andrews@symantec.com">Stuart_Andrews@symantec.com</a>></span> \
wrote:<br> <blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px \
#ccc solid;padding-left:1ex;">
<div lang="EN-US" link="blue" vlink="blue">
<div>
<p class="MsoNormal"><font size="2" color="navy" face="Arial"><span \
style="font-size:10.0pt;font-family:Arial;color:navy">Colin</span></font></p>
<p class="MsoNormal"><font size="2" color="navy" face="Arial"><span \
style="font-size:10.0pt;font-family:Arial;color:navy"> </span></font></p>
<p class="MsoNormal"><font size="2" color="navy" face="Arial"><span \
style="font-size:10.0pt;font-family:Arial;color:navy">OK – I see you are using \
enclosure naming – and BTW – the /dev/vx/dmp /dev/vx/dmp are just the same as
disks BUT with the added provision of DMP to keep the device \
online.</span></font></p>
<p class="MsoNormal"><font size="2" color="navy" face="Arial"><span \
style="font-size:10.0pt;font-family:Arial;color:navy">How many paths to these \
devices</span></font></p>
<p class="MsoNormal"><font size="2" color="navy" face="Arial"><span \
style="font-size:10.0pt;font-family:Arial;color:navy"># vxdmpadm getsubpaths \
</span></font></p>
<p class="MsoNormal"><font size="2" color="navy" face="Arial"><span \
style="font-size:10.0pt;font-family:Arial;color:navy">or</span></font></p>
<p class="MsoNormal"><font size="2" color="navy" face="Arial"><span \
style="font-size:10.0pt;font-family:Arial;color:navy"># vxdisk path</span></font></p>
<p class="MsoNormal"><font size="2" color="navy" face="Arial"><span \
style="font-size:10.0pt;font-family:Arial;color:navy"> </span></font></p>
<p class="MsoNormal"><font size="2" color="navy" face="Arial"><span \
style="font-size:10.0pt;font-family:Arial;color:navy">If a CLARiiON , and if only 2 \
paths – then set the array iopolicy to singleactive – this is most likely –
see later for DMP </span></font></p>
<p class="MsoNormal"><font size="2" color="navy" face="Arial"><span \
style="font-size:10.0pt;font-family:Arial;color:navy"># vxdmpadm listenclosure \
all</span></font></p>
<p class="MsoNormal"><font size="2" color="navy" face="Arial"><span \
style="font-size:10.0pt;font-family:Arial;color:navy"># vxdmpadm setattr enclosure \
ENC_name iopolicy=singleactive</span></font></p>
<p class="MsoNormal"><font size="2" color="navy" face="Arial"><span \
style="font-size:10.0pt;font-family:Arial;color:navy">If a CLARiiON , and if more \
than 2 paths – then set the array iopolicy to balanced – and DMP does know how to \
stop IO to the secondary paths. Note – this is against EMC recommendations
but it works.</span></font></p>
<p class="MsoNormal"><font size="2" color="navy" face="Arial"><span \
style="font-size:10.0pt;font-family:Arial;color:navy"># vxdmpadm listenclosure \
all</span></font></p>
<p class="MsoNormal"><font size="2" color="navy" face="Arial"><span \
style="font-size:10.0pt;font-family:Arial;color:navy"># vxdmpadm setattr enclosure \
ENC_name iopolicy=balanced</span></font></p>
<p class="MsoNormal"><font size="2" color="navy" face="Arial"><span \
style="font-size:10.0pt;font-family:Arial;color:navy"> </span></font></p>
<p class="MsoNormal"><font size="2" color="navy" face="Arial"><span \
style="font-size:10.0pt;font-family:Arial;color:navy">I also notice that the \
enclosure names are lower case – indicating a V5.x release VxVM installed. – are the
CLARiiON APMs running ?</span></font></p>
<p class="MsoNormal"><font size="2" color="navy" face="Arial"><span \
style="font-size:10.0pt;font-family:Arial;color:navy"># vxdmpadm listapm all \
</span></font></p>
<p class="MsoNormal"><font size="2" color="navy" face="Arial"><span \
style="font-size:10.0pt;font-family:Arial;color:navy">Check the CLARiiON are \
Active</span></font></p>
<p class="MsoNormal"><font size="2" color="navy" face="Arial"><span \
style="font-size:10.0pt;font-family:Arial;color:navy"> </span></font></p>
<p class="MsoNormal"><font size="2" color="navy" face="Arial"><span \
style="font-size:10.0pt;font-family:Arial;color:navy">If a fencing cluster – then \
these are local LUNs at the point of /dev/vx/dmp names for the devices. And yes SCSI3
keys will be placed on them in a fencing cluster </span></font></p>
<p class="MsoNormal"><font size="2" color="navy" face="Arial"><span \
style="font-size:10.0pt;font-family:Arial;color:navy"># gabconfig \
–a</span></font></p>
<p class="MsoNormal"><font size="2" color="navy" face="Arial"><span \
style="font-size:10.0pt;font-family:Arial;color:navy">Check if there is Port b \
membership – if so then yes you have a fencing cluster.</span></font></p>
<p class="MsoNormal"><font size="2" color="navy" face="Arial"><span \
style="font-size:10.0pt;font-family:Arial;color:navy"> </span></font></p>
<p class="MsoNormal"><font size="2" color="navy" face="Arial"><span \
style="font-size:10.0pt;font-family:Arial;color:navy">Check also the DMP block switch \
– it may be that with the iopolicy incorrect, and on low IO you did not reach the \
limit for </span></font></p>
<p class="MsoNormal"><font size="2" color="navy" face="Arial"><span \
style="font-size:10.0pt;font-family:Arial;color:navy"># vxdmpadm gettune all \
</span></font></p>
<p class="MsoNormal"><font size="2" color="navy" face="Arial"><span \
style="font-size:10.0pt;font-family:Arial;color:navy">dmp_pathswitch_blks_shift \
</span></font></p>
<p class="MsoNormal"><font size="2" color="navy" face="Arial"><span \
style="font-size:10.0pt;font-family:Arial;color:navy">Now – when busy and IO chunks \
bigger than path switch level ( and with iopolicy incorrect ) a path switch will \
cause a trespass (check SAN logs) AND a block, drain, resume on the DMP path. There
will be a failover message logged in /etc/vx/dmpevents.log – check here
also.</span></font></p>
<p class="MsoNormal"><font size="2" color="navy" face="Arial"><span \
style="font-size:10.0pt;font-family:Arial;color:navy"> </span></font></p>
<p class="MsoNormal"><font size="2" color="navy" face="Arial"><span \
style="font-size:10.0pt;font-family:Arial;color:navy">Stuart</span></font></p>
<p class="MsoNormal"><font size="2" color="navy" face="Arial"><span \
style="font-size:10.0pt;font-family:Arial;color:navy"> </span></font></p>
<div>
<div class="MsoNormal" align="center" style="text-align:center"><font size="3" \
face="Times New Roman"><span style="font-size:12.0pt">
<hr size="2" width="100%" align="center">
</span></font></div>
<p class="MsoNormal"><b><font size="2" face="Tahoma"><span \
style="font-size:10.0pt;font-family:Tahoma;font-weight:bold">From:</span></font></b><font \
size="2" face="Tahoma"><span style="font-size:10.0pt;font-family:Tahoma"> <a \
href="mailto:veritas-vx-bounces@mailman.eng.auburn.edu" \
target="_blank">veritas-vx-bounces@mailman.eng.auburn.edu</a> [mailto:<a \
href="mailto:veritas-vx-bounces@mailman.eng.auburn.edu" \
target="_blank">veritas-vx-bounces@mailman.eng.auburn.edu</a>] <b><span \
style="font-weight:bold">On Behalf Of </span></b>Collin<br> <b><span \
style="font-weight:bold">Sent:</span></b> Wednesday, 31 March 2010 2:09 AM<br>
<b><span style="font-weight:bold">To:</span></b> William Havey<br>
<b><span style="font-weight:bold">Cc:</span></b>
<a href="mailto:veritas-vx@mailman.eng.auburn.edu" \
target="_blank">veritas-vx@mailman.eng.auburn.edu</a><br> <b><span \
style="font-weight:bold">Subject:</span></b> Re: [Veritas-vx] VxVm</span></font></p>
</div><div><div class="h5">
<p class="MsoNormal"><font size="3" face="Times New Roman"><span \
style="font-size:12.0pt"> </span></font></p>
<p class="MsoNormal" style="margin-bottom:12.0pt"><font size="3" face="Times New \
Roman"><span style="font-size:12.0pt">Sorry for any confusion...<br>
<br>
I've got several powerpath devices from a dead system that I'm mounting
temporarily on one node in my cluster. I run a devfsadm -Cv and vxdctl
enable. After that I can see the powerpath devices listed as...<br>
<br>
emc_clarrion0_10 auto:none
- -
online invalid<br>
<br>
I modified my /etc/vfstab file to mount the devices..<br>
<br>
/dev/vx/dmp/emc_clariion0_10s6
/dev/vx/rdmp/emc_clariion0_10s6 /u10 ufs 3
yes -<br>
<br>
The device mounts and I can access the file system with all my data. When
the activity starts to increase on these temporary mount points, I see a count
down on the console that port H has lost connectivity. After the 16 seconds,
the node panics and of course reboots. However, if I mount the power path
devices using a single path..<br>
<br>
/dev/dsk/c1t5006016100600432d10s6
/dev/rdsk/c1t5006016100600432d10s6 /u10 ufs 3 yes -<br>
<br>
I never get the port H losing connectivity. <br>
<br>
I want to use the dmp name in case I lose a path to these disk.<br>
<br>
Any reason why using the dmp name causes port H to lose connectivity vs. using
a single path?<br>
<br>
Thanks,<br>
Collin</span></font></p>
<div>
<p class="MsoNormal"><font size="3" face="Times New Roman"><span \
style="font-size:12.0pt">On Tue, Mar 30, 2010 at 10:48 AM, William Havey <<a \
href="mailto:bbhavey@gmail.com" target="_blank">bbhavey@gmail.com</a>> \
wrote:</span></font></p>
<p class="MsoNormal"><font size="3" face="Times New Roman"><span \
style="font-size:12.0pt">The original message states "mount \
these disks as
/dev/vx/dmp/<emc_array>_Xs6 ". Perhaps this is normal behavior.
Mounts are of devices which receive I/O. A "/dev/vx/dmp/..." device
entry isn't I/O capable.<br>
<br>
I think a clearer statement of what Collin intends to do is needed.<br>
<br>
Bill</span></font></p>
<div>
<div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><font size="3" face="Times New \
Roman"><span style="font-size:12.0pt"> </span></font></p>
<div>
<p class="MsoNormal"><font size="3" face="Times New Roman"><span \
style="font-size:12.0pt">On Tue, Mar 30, 2010 at 3:01 AM, Dmitry Glushenok <<a \
href="mailto:glush@jet.msk.su" target="_blank">glush@jet.msk.su</a>> \
wrote:</span></font></p>
<p class="MsoNormal"><font size="3" face="Times New Roman"><span \
style="font-size:12.0pt">Hello,<br> <br>
Panic string and previous messages usually helps to understand cause.. Release
notes to RP2-RP3 also provides short descriptions of fixed issues like
"Fixed the cause of a system panic when mutex_panic() was called from
vol_rwsleep_wrlock()."</span></font></p>
<div>
<p class="MsoNormal"><font size="3" face="Times New Roman"><span \
style="font-size:12.0pt"><br> On 29.03.2010, at 19:02, Collin wrote:<br>
<br>
> I've got the following....<br>
><br>
> Solaris 10<br>
> VxVM 5.0MP3RP1HF12<br>
></span></font></p>
</div>
<p class="MsoNormal"><font size="3" face="Times New Roman"><span \
style="font-size:12.0pt">> I have a number of mount points that are being migrated \
from
/dev/dsk/cXtXdXsX to clustered mount points. The problem I'm having is if
I mount these disk in the /dev/dsk/cXtXdXsX format I run the risk that if
something were to cause the direct path to go down I would lose the databases
on these mount points. But when I mount these disks as
/dev/vx/dmp/<emc_array>_Xs6 my system panics and core dumps.</span></font></p>
<div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><font size="3" face="Times New \
Roman"><span style="font-size:12.0pt">><br> > Does VxVM have any issues \
mounting /dev/vx/dmp/<emc_array>_Xs6??<br> ><br>
> Thanks,<br>
> Collin<br>
> _______________________________________________<br>
> Veritas-vx maillist - <a href="mailto:Veritas-vx@mailman.eng.auburn.edu" \
target="_blank">Veritas-vx@mailman.eng.auburn.edu</a><br> > <a \
href="http://mailman.eng.auburn.edu/mailman/listinfo/veritas-vx" \
target="_blank">http://mailman.eng.auburn.edu/mailman/listinfo/veritas-vx</a></span></font></p>
</div>
<p class="MsoNormal"><font size="3" color="#888888" face="Times New Roman"><span \
style="font-size:12.0pt;color:#888888">--<br> Dmitry Glushenok<br>
Jet Infosystems</span></font></p>
<div>
<div>
<p class="MsoNormal"><font size="3" face="Times New Roman"><span \
style="font-size:12.0pt"><br> _______________________________________________<br>
Veritas-vx maillist - <a href="mailto:Veritas-vx@mailman.eng.auburn.edu" \
target="_blank">Veritas-vx@mailman.eng.auburn.edu</a><br> <a \
href="http://mailman.eng.auburn.edu/mailman/listinfo/veritas-vx" \
target="_blank">http://mailman.eng.auburn.edu/mailman/listinfo/veritas-vx</a></span></font></p>
</div>
</div>
</div>
<p class="MsoNormal"><font size="3" face="Times New Roman"><span \
style="font-size:12.0pt"> </span></font></p>
</div>
</div>
</div>
<p class="MsoNormal"><font size="3" face="Times New Roman"><span \
style="font-size:12.0pt"> </span></font></p>
</div></div></div>
</div>
<br>_______________________________________________<br>
Veritas-vx maillist - <a \
href="mailto:Veritas-vx@mailman.eng.auburn.edu">Veritas-vx@mailman.eng.auburn.edu</a><br>
<a href="http://mailman.eng.auburn.edu/mailman/listinfo/veritas-vx" \
target="_blank">http://mailman.eng.auburn.edu/mailman/listinfo/veritas-vx</a><br> \
<br></blockquote></div><br><br clear="all"><br>-- <br>sengork<br>
_______________________________________________
Veritas-vx maillist - Veritas-vx@mailman.eng.auburn.edu
http://mailman.eng.auburn.edu/mailman/listinfo/veritas-vx
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic