[prev in list] [next in list] [prev in thread] [next in thread]
List: gpfsug-discuss
Subject: Re: [gpfsug-discuss] mmapplypolicy slow
From: "Peeples, Heath" <heathp () HPC ! MsState ! Edu>
Date: 2021-08-03 19:11:06
Message-ID: BN0PR01MB7056BCBEC671D0B48D05CF8C9CF09 () BN0PR01MB7056 ! prod ! exchangelabs ! com
[Download RAW message or body]
[Attachment #2 (text/plain)]
Well we want to run it against the snapshot so I think we need the -S option. When \
you run it against the FS filtering by SNAP_ID, you miss any files that have been \
deleted since the snapshot.
Heath
From: gpfsug-discuss-bounces@spectrumscale.org \
<gpfsug-discuss-bounces@spectrumscale.org> On Behalf Of Jan-Frode \
Myklebust
Sent: Tuesday, August 3, 2021 2:09 PM
To: gpfsug main discussion list <gpfsug-discuss@spectrumscale.org>
Subject: Re: [gpfsug-discuss] mmapplypolicy slow
Just guessing, but maybe some «where SNAP_ID('SnapshotName') » is better than -S \
20210104?
?
tir. 3. aug. 2021 kl. 20:59 skrev Peeples, Heath \
<heathp@hpc.msstate.edu<mailto:heathp@hpc.msstate.edu>>: Yes, that advertisement \
makes me smile 😊
I have tried both of those, and they do not seem to make any difference. I have also \
played with the -A and -a parameters with no combination that I can find making it \
any better.
Thanks for the feedback.
From: gpfsug-discuss-bounces@spectrumscale.org<mailto:gpfsug-discuss-bounces@spectrumscale.org> \
<gpfsug-discuss-bounces@spectrumscale.org<mailto:gpfsug-discuss-bounces@spectrumscale.org>> \
On Behalf Of Jan-Frode Myklebust
Sent: Tuesday, August 3, 2021 1:49 PM
To: gpfsug main discussion list \
<gpfsug-discuss@spectrumscale.org<mailto:gpfsug-discuss@spectrumscale.org>>
Subject: Re: [gpfsug-discuss] mmapplypolicy slow
So…. the advertisement ssys we should be able to do 1M files/s…
http://files.gpfsug.org/presentations/2018/USA/SpectrumScalePolicyBP.pdf<http://secure \
-web.cisco.com/1qQpvD9F4NtOuzGs8G_8PtFl-CwL-8JNJO1mx5gEbI7QwDLGI58rp5A1Zj3Q1nQoonzr0pd \
bS8-5R4oc5DLLmaBS6PQJ0HXdquIC5cLIlmasuY6rJfZ_dnC7zPJmBySjicZ0rFC-nrhKEnu-_J-arj4xmzx2V \
xDYJlbreMnsJfs8xW-c4B_Hnf7N9sPPXQ2b6C8jORSfUogYE0Lue9d2OEn-ITxLZdDzB34linSMOE1kYikuG5e \
wgP6p0nhcznywEWQr_aE1zvb_kgT9pW4ldt39RkTfGbH-O72Aa-sBxa9NilSUt8K-5wXh6KkXTvTln/http%3A%2F%2Ffiles.gpfsug.org%2Fpresentations%2F2018%2FUSA%2FSpectrumScalePolicyBP.pdf>
First I would try is if maybe limiting which nodes are used for the processing \
helps. Maybe limit to the NSD-servers (-N nodenames) ?
Also, --choice-algorithm fast might help..?
-jf
tir. 3. aug. 2021 kl. 20:18 skrev Peeples, Heath \
<heathp@hpc.msstate.edu<mailto:heathp@hpc.msstate.edu>>: I am trying to use \
mmapplypolicy to determine what files are modified in a particular snapshop. Below I \
have the policy and mmapplypolicy parameters I am using. It is taking 3-4 hours to \
list these files. There are roughly 176M files. This is running on a DDN Gridscaler \
12K system. Seems to me after seeing others posts we should be able to run this \
much quicker. Any help would be appreciated.
mmapplypolicy fs2 -S 20210104 -I defer -A 175 -a 72 -B 10000 -N all -g \
/fs1/tmp/heath -P /fs2/tmp/snaps/heath.pol -f /fs2/tmp/snaps/heath/output/20210104
RULE 'find_mods' list 'mods'
DIRECTORIES_PLUS
WEIGHT(0)
SHOW( VARCHAR(USER_ID) || ' ' || varchar(FILE_SIZE) || ' ' || \
varchar(MODIFICATION_SNAPID) )
Heath
_______________________________________________
gpfsug-discuss mailing list
gpfsug-discuss at spectrumscale.org<http://secure-web.cisco.com/13y0L5a_-UdTp06HAY-SQn \
bTVm8HkHswo4iplosUAzEBhW3s6Y6LnTuCvikd3KQuk_YPuAuKhDLAzBjCcmelrVS146XsPHgLLLSbAb3bAKY9 \
dZ10sV8_ErYKn0OqyE8gPgh4kjWnswfL7oomTnhYj4D1Q2mY83UI82dAJAX-2n8NaxlfloOFS23DgFMEULXJgy \
-wNH_cLemLqseter9-Rj2yO3jsruIBj3_DcY_Z87ffxo0SQcYazbJvThTeqb5C1o6hepHZkalgmUvQE6K2CAV_sX4cWrNs5GNqPy7xifM9tlbvhwJvwSvZGlTO_da4e/http%3A%2F%2Fspectrumscale.org>
http://gpfsug.org/mailman/listinfo/gpfsug-discuss<http://secure-web.cisco.com/1joeovh \
VrYzawY9kLy658LHgv3dW6Ta7uZzGAlu08q1o21Rw4o615cnGStXQ0NQeAYE2jCpJAzC7DKAzkxD_cgeRLlQkK \
VRPFm8QFIMXwGcESXOwOmKXVVD5fq8H0gBD6VasiBdUmWrijLwK0QFvEduBRSXIqQ-sWoxrR1onYwyJEPV--XM \
h-r1O-92mnjtaUNVuSS2KXElVq8rc2O33St8Ga2qd5QSTT2pUQcbr2IJUM4a2gu0ZrtVQgRBRTZU1ZuoPnFZY8 \
AL4tubpqa6vK_x_dt-W6bkV1XKSre2XQMxCaNb9Bm2_bKJgfhamPC-Rf/http%3A%2F%2Fgpfsug.org%2Fmailman%2Flistinfo%2Fgpfsug-discuss>
_______________________________________________
gpfsug-discuss mailing list
gpfsug-discuss at spectrumscale.org<http://secure-web.cisco.com/1Jxk0mTZJfEGRuwDAfk6WE \
TmP_z5AYx4ZEYSsRf6Iq7v69ncxGgt70_hIM13VkBESWG6_5J1sazT3n6tfxKKmStUZmOj9j2XvYb9AXirO0aU \
5Doiod8DV6Exzl1bAO3nro5-AzRdLy12lqtXiY2AecwMn57_JZN9eTrPPB73pyXJrf2Pb0i5PfDRq6o4o2kC5p \
Z-UgYlYJP2b4Z37vzHDHis8pAUPvc6mKIBdtd_IUbJdvxGfErw3WZIyIdCVNIOyuvVqLDnQcJI9O291mb6-qOXUPiAxrONdm5c0LsU_HkgGbwYXkSRdEUuUtB3n3uM2/http%3A%2F%2Fspectrumscale.org>
http://gpfsug.org/mailman/listinfo/gpfsug-discuss<http://secure-web.cisco.com/1-tpllN \
riTqNKP656g4djWNutzuUwZnTeFHXINw7U_4ltbsDdb04bvj1gxe7NEW3YrW-ANqci01QM9ftyYnaBWrgiTCH3 \
PZYu5n3jFwUhSYE8cZh88leH2VaqiXQtN2yH0KW4fHgiO16MYEz35tyoaWGxVcPSk0ExX4ZuXohXCmMt9RrUuB \
ygXw4_v15mAoDPha-mBUTOMtnoWQG7Hq4jJ8CpD6DGaYMV_E23CbJN5fInLFGYJ9b9tP80C_CKMhts_TuB1Z8l \
6bEYnWj62NqAV-936iq_nLtLoZLKwjZD2-WzENisFTchV_InvVhRgwoL/http%3A%2F%2Fgpfsug.org%2Fmailman%2Flistinfo%2Fgpfsug-discuss>
[Attachment #3 (text/html)]
<html xmlns:v="urn:schemas-microsoft-com:vml" \
xmlns:o="urn:schemas-microsoft-com:office:office" \
xmlns:w="urn:schemas-microsoft-com:office:word" \
xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" \
xmlns="http://www.w3.org/TR/REC-html40"> <head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:"Segoe UI Emoji";
panose-1:2 11 5 2 4 2 4 2 2 3;}
@font-face
{font-family:"Helvetica Neue";}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
span.ph
{mso-style-name:ph;}
span.EmailStyle19
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-family:"Calibri",sans-serif;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-US" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal">Well we want to run it against the snapshot so I think we need \
the -S option. When you run it against the FS filtering by SNAP_ID, you miss \
any files that have been deleted since the snapshot.<o:p></o:p></p> <p \
class="MsoNormal"><o:p> </o:p></p> <p class="MsoNormal">Heath<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><b>From:</b> gpfsug-discuss-bounces@spectrumscale.org \
<gpfsug-discuss-bounces@spectrumscale.org> <b>On Behalf Of </b>Jan-Frode \
Myklebust<br> <b>Sent:</b> Tuesday, August 3, 2021 2:09 PM<br>
<b>To:</b> gpfsug main discussion list <gpfsug-discuss@spectrumscale.org><br>
<b>Subject:</b> Re: [gpfsug-discuss] mmapplypolicy slow<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Just guessing, but maybe some «where <b><span \
style="font-family:"Helvetica \
Neue";letter-spacing:.1pt">SNAP_ID('</span></b><span class="ph"><i><span \
style="font-family:"Helvetica Neue";letter-spacing:.1pt;border:none \
windowtext 1.0pt;padding:0in">SnapshotName</span></i></span><b><span \
style="font-family:"Helvetica Neue";letter-spacing:.1pt">') » is better \
than -S 20210104?</span></b><o:p></o:p></p> </div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal"><b><span style="font-family:"Helvetica \
Neue";letter-spacing:.1pt">?</span></b><o:p></o:p></p> </div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<div>
<p class="MsoNormal">tir. 3. aug. 2021 kl. 20:59 skrev Peeples, Heath <<a \
href="mailto:heathp@hpc.msstate.edu">heathp@hpc.msstate.edu</a>>:<o:p></o:p></p> \
</div> <blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in \
0in 6.0pt;margin-left:4.8pt;margin-right:0in"> <div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Yes, \
that advertisement makes me smile <span style="font-family:"Segoe UI \
Emoji",sans-serif">😊</span><o:p></o:p></p> <p class="MsoNormal" \
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p> <p \
class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">I have \
tried both of those, and they do not seem to make any difference. I have also \
played with the -A and -a parameters with no combination that I can find making it \
any better.<o:p></o:p></p> <p class="MsoNormal" \
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p> <p \
class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Thanks \
for the feedback.<o:p></o:p></p> </div>
</div>
<div>
<div>
<p class="MsoNormal" \
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p> <p \
class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><b>From:</b>
<a href="mailto:gpfsug-discuss-bounces@spectrumscale.org" \
target="_blank">gpfsug-discuss-bounces@spectrumscale.org</a> <<a \
href="mailto:gpfsug-discuss-bounces@spectrumscale.org" \
target="_blank">gpfsug-discuss-bounces@spectrumscale.org</a>> <b>On Behalf Of \
</b>Jan-Frode Myklebust<br> <b>Sent:</b> Tuesday, August 3, 2021 1:49 PM<br>
<b>To:</b> gpfsug main discussion list <<a \
href="mailto:gpfsug-discuss@spectrumscale.org" \
target="_blank">gpfsug-discuss@spectrumscale.org</a>><br> <b>Subject:</b> Re: \
[gpfsug-discuss] mmapplypolicy slow<o:p></o:p></p> <p class="MsoNormal" \
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p> \
<div> <p class="MsoNormal" \
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">So…. the advertisement \
ssys we should be able to do 1M files/s… <o:p></o:p></p> <div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><a \
href="http://secure-web.cisco.com/1qQpvD9F4NtOuzGs8G_8PtFl-CwL-8JNJO1mx5gEbI7QwDLGI58r \
p5A1Zj3Q1nQoonzr0pdbS8-5R4oc5DLLmaBS6PQJ0HXdquIC5cLIlmasuY6rJfZ_dnC7zPJmBySjicZ0rFC-nr \
hKEnu-_J-arj4xmzx2VxDYJlbreMnsJfs8xW-c4B_Hnf7N9sPPXQ2b6C8jORSfUogYE0Lue9d2OEn-ITxLZdDz \
B34linSMOE1kYikuG5ewgP6p0nhcznywEWQr_aE1zvb_kgT9pW4ldt39RkTfGbH-O72Aa-sBxa9NilSUt8K-5w \
Xh6KkXTvTln/http%3A%2F%2Ffiles.gpfsug.org%2Fpresentations%2F2018%2FUSA%2FSpectrumScalePolicyBP.pdf" \
target="_blank">http://files.gpfsug.org/presentations/2018/USA/SpectrumScalePolicyBP.pdf</a><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" \
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p> \
</div> <div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">First \
I would try is if maybe limiting which nodes are used for the processing helps. \
Maybe limit to the NSD-servers (-N nodenames) ?<o:p></o:p></p> </div>
</div>
<div>
<p class="MsoNormal" \
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p> \
</div> <div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Also, \
--choice-algorithm fast might help..?<o:p></o:p></p> </div>
<div>
<p class="MsoNormal" \
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p> \
</div> <div>
<p class="MsoNormal" \
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p> \
</div> <div>
<p class="MsoNormal" \
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p> \
</div> <div>
<p class="MsoNormal" \
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> -jf<o:p></o:p></p> \
</div> <div>
<p class="MsoNormal" \
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p> \
</div> <div>
<p class="MsoNormal" \
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p> \
<div> <div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">tir. \
3. aug. 2021 kl. 20:18 skrev Peeples, Heath <<a \
href="mailto:heathp@hpc.msstate.edu" \
target="_blank">heathp@hpc.msstate.edu</a>>:<o:p></o:p></p> </div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in \
6.0pt;margin-left:4.8pt;margin-top:5.0pt;margin-right:0in;margin-bottom:5.0pt"> <div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span \
style="font-family:"Courier New"">I am trying to use mmapplypolicy to \
determine what files are modified in a particular snapshop. Below I have the \
policy and mmapplypolicy parameters I am using. It is taking 3-4 hours to list \
these files. There are roughly 176M files. This is running on a DDN Gridscaler \
12K system. Seems to me after seeing others posts we should be able to \
run this much quicker. Any help would be appreciated.</span><o:p></o:p></p> <p \
class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span \
style="font-family:"Courier New""> </span><o:p></o:p></p> <p \
class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span \
style="font-family:"Courier New""> </span><o:p></o:p></p> <p \
class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span \
style="font-family:"Courier New"">mmapplypolicy fs2 -S 20210104 -I defer -A \
175 -a 72 -B 10000 -N all -g /fs1/tmp/heath -P /fs2/tmp/snaps/heath.pol -f \
/fs2/tmp/snaps/heath/output/20210104</span><o:p></o:p></p> <p class="MsoNormal" \
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span \
style="font-family:"Courier New""> </span><o:p></o:p></p> <p \
class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span \
style="font-family:"Courier New"">RULE 'find_mods' list \
'mods'</span><o:p></o:p></p> <p class="MsoNormal" \
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span \
style="font-family:"Courier New"">DIRECTORIES_PLUS</span><o:p></o:p></p> <p \
class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span \
style="font-family:"Courier New"">WEIGHT(0)</span><o:p></o:p></p> <p \
class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span \
style="font-family:"Courier New"">SHOW( VARCHAR(USER_ID) || ' ' || \
varchar(FILE_SIZE) || ' ' || varchar(MODIFICATION_SNAPID) )</span><o:p></o:p></p> <p \
class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span \
style="font-family:"Courier New""> </span><o:p></o:p></p> <p \
class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span \
style="font-family:"Courier New""> </span><o:p></o:p></p> <p \
class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span \
style="font-family:"Courier New"">Heath</span><o:p></o:p></p> </div>
</div>
<p class="MsoNormal" \
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">_______________________________________________<br>
gpfsug-discuss mailing list<br>
gpfsug-discuss at <a \
href="http://secure-web.cisco.com/13y0L5a_-UdTp06HAY-SQnbTVm8HkHswo4iplosUAzEBhW3s6Y6L \
nTuCvikd3KQuk_YPuAuKhDLAzBjCcmelrVS146XsPHgLLLSbAb3bAKY9dZ10sV8_ErYKn0OqyE8gPgh4kjWnsw \
fL7oomTnhYj4D1Q2mY83UI82dAJAX-2n8NaxlfloOFS23DgFMEULXJgy-wNH_cLemLqseter9-Rj2yO3jsruIB \
j3_DcY_Z87ffxo0SQcYazbJvThTeqb5C1o6hepHZkalgmUvQE6K2CAV_sX4cWrNs5GNqPy7xifM9tlbvhwJvwSvZGlTO_da4e/http%3A%2F%2Fspectrumscale.org" \
target="_blank"> spectrumscale.org</a><br>
<a href="http://secure-web.cisco.com/1joeovhVrYzawY9kLy658LHgv3dW6Ta7uZzGAlu08q1o21Rw4 \
o615cnGStXQ0NQeAYE2jCpJAzC7DKAzkxD_cgeRLlQkKVRPFm8QFIMXwGcESXOwOmKXVVD5fq8H0gBD6VasiBd \
UmWrijLwK0QFvEduBRSXIqQ-sWoxrR1onYwyJEPV--XMh-r1O-92mnjtaUNVuSS2KXElVq8rc2O33St8Ga2qd5 \
QSTT2pUQcbr2IJUM4a2gu0ZrtVQgRBRTZU1ZuoPnFZY8AL4tubpqa6vK_x_dt-W6bkV1XKSre2XQMxCaNb9Bm2_bKJgfhamPC-Rf/http%3A%2F%2Fgpfsug.org%2Fmailman%2Flistinfo%2Fgpfsug-discuss" \
target="_blank">http://gpfsug.org/mailman/listinfo/gpfsug-discuss</a><o:p></o:p></p> \
</blockquote> </div>
</div>
</div>
</div>
<p class="MsoNormal">_______________________________________________<br>
gpfsug-discuss mailing list<br>
gpfsug-discuss at <a \
href="http://secure-web.cisco.com/1Jxk0mTZJfEGRuwDAfk6WETmP_z5AYx4ZEYSsRf6Iq7v69ncxGgt \
70_hIM13VkBESWG6_5J1sazT3n6tfxKKmStUZmOj9j2XvYb9AXirO0aU5Doiod8DV6Exzl1bAO3nro5-AzRdLy \
12lqtXiY2AecwMn57_JZN9eTrPPB73pyXJrf2Pb0i5PfDRq6o4o2kC5pZ-UgYlYJP2b4Z37vzHDHis8pAUPvc6 \
mKIBdtd_IUbJdvxGfErw3WZIyIdCVNIOyuvVqLDnQcJI9O291mb6-qOXUPiAxrONdm5c0LsU_HkgGbwYXkSRdEUuUtB3n3uM2/http%3A%2F%2Fspectrumscale.org" \
target="_blank"> spectrumscale.org</a><br>
<a href="http://secure-web.cisco.com/1-tpllNriTqNKP656g4djWNutzuUwZnTeFHXINw7U_4ltbsDd \
b04bvj1gxe7NEW3YrW-ANqci01QM9ftyYnaBWrgiTCH3PZYu5n3jFwUhSYE8cZh88leH2VaqiXQtN2yH0KW4fH \
giO16MYEz35tyoaWGxVcPSk0ExX4ZuXohXCmMt9RrUuBygXw4_v15mAoDPha-mBUTOMtnoWQG7Hq4jJ8CpD6DG \
aYMV_E23CbJN5fInLFGYJ9b9tP80C_CKMhts_TuB1Z8l6bEYnWj62NqAV-936iq_nLtLoZLKwjZD2-WzENisFTchV_InvVhRgwoL/http%3A%2F%2Fgpfsug.org%2Fmailman%2Flistinfo%2Fgpfsug-discuss" \
target="_blank">http://gpfsug.org/mailman/listinfo/gpfsug-discuss</a><o:p></o:p></p> \
</blockquote> </div>
</div>
</div>
</body>
</html>
_______________________________________________
gpfsug-discuss mailing list
gpfsug-discuss at spectrumscale.org
http://gpfsug.org/mailman/listinfo/gpfsug-discuss
--===============8201119912697207886==--
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic