[prev in list] [next in list] [prev in thread] [next in thread]
List: gpfsug-discuss
Subject: Re: [gpfsug-discuss] RDMA data from Zimon
From: "Eric Agar" <agar () us ! ibm ! com>
Date: 2018-02-15 17:08:08
Message-ID: OF5FA7CC95.ECC83F78-ON85258235.005E05FB-85258235.005E211B () notes ! na ! collabserv ! com
[Download RAW message or body]
[Attachment #2 (multipart/related)]
[Attachment #4 (multipart/alternative)]
[Attachment #6 (text/plain)]
Kristy,
I experimented a bit with this some months ago and looked at the ZIMon
source code. I came to the conclusion that ZIMon is reporting values
obtained from the IB counters (actually, delta values adjusted for time)
and that yes, for port_xmit_data and port_rcv_data, one would need to
multiply the values by 4 to make sense of them.
To obtain a port_xmit_data value, the ZIMon sensor first looks
for /sys/class/infiniband/<ibdev>/ports/<port>/counters_ext/port_xmit_data_64,
and if that is not found then looks
for /sys/class/infiniband/<ibdev>/ports/<port>/counters/port_xmit_data.
Similarly for other counters/metrics.
Full disclosure: I am not an IB expert nor a ZIMon developer.
I hope this helps.
Eric M. Agar
agar@us.ibm.com
From: Kristy Kallback-Rose <kkr@lbl.gov>
To: gpfsug main discussion list <gpfsug-discuss@spectrumscale.org>
Date: 02/14/2018 08:47 PM
Subject: [gpfsug-discuss] RDMA data from Zimon
Sent by: gpfsug-discuss-bounces@spectrumscale.org
Hi,
Can one of the IBMers tell me if port_xmit_data and port_rcv_data from
Zimon can be interpreted as RDMA Bytes/sec? Ideally, also how this data is
being collected? I'm looking here:
https://www.ibm.com/support/knowledgecenter/en/STXKQY_5.0.0/com.ibm.spectrum.scale.v5r00.doc/bl1hlp_monnetworksmetrics.htm
But then I also look here: https://community.mellanox.com/docs/DOC-2751
and see "Total number of data octets, divided by 4 (lanes), received on all
VLs. This is 64 bit counter." So I wasn't sure if some multiplication by 4
was in order.
Please advise.
Cheers,
Kristy_______________________________________________
gpfsug-discuss mailing list
gpfsug-discuss at spectrumscale.org
https://urldefense.proofpoint.com/v2/url?u=http-3A__gpfsug.org_mailman_listinfo_gpfsug \
-2Ddiscuss&d=DwICAg&c=jf_iaSHvJObTbx-siA1ZOg&r=IbxtjdkPAM2Sbon4Lbbi4w&m=zIRb70L9sx_FvvC9IcWVKLOSOOFnx-hIGfjw0kUN7bw&s=D1g4YTG5WeUiHI3rCPr_kkPxbG9V9E-18UGXBeCvfB8&e=
[Attachment #7 (text/html)]
<html><body><p><font size="2">Kristy,</font><br><br><font size="2">I experimented a \
bit with this some months ago and looked at the ZIMon source code. I came to the \
conclusion that ZIMon is reporting values obtained from the IB counters (actually, \
delta values adjusted for time) and that yes, for port_xmit_data and port_rcv_data, \
one would need to multiply the values by 4 to make sense of them.</font><br><br><font \
size="2">To obtain a port_xmit_data value, the ZIMon sensor first looks for \
/sys/class/infiniband/<ibdev>/ports/<port>/counters_ext/port_xmit_data_64, \
and if that is not found then looks for \
/sys/class/infiniband/<ibdev>/ports/<port>/counters/port_xmit_data. \
Similarly for other counters/metrics.</font><br><br><font size="2">Full disclosure: I \
am not an IB expert nor a ZIMon developer.</font><br><br><font size="2">I hope this \
helps.</font><br><font size="2"><br><br>Eric M. \
Agar<br>agar@us.ibm.com<br></font><br><br><img width="16" height="16" \
src="cid:1__=0ABB08A6DFCD836B8f9e8a93df938690918c0AB@" border="0" alt="Inactive hide \
details for Kristy Kallback-Rose ---02/14/2018 08:47:59 PM---Hi, Can one of the \
IBMers tell me if port_xmit_data"><font size="2" color="#424282">Kristy Kallback-Rose \
---02/14/2018 08:47:59 PM---Hi, Can one of the IBMers tell me if port_xmit_data and \
port_rcv_data from Zimon can be interpreted</font><br><br><font size="2" \
color="#5F5F5F">From: </font><font size="2">Kristy Kallback-Rose \
<kkr@lbl.gov></font><br><font size="2" color="#5F5F5F">To: </font><font \
size="2">gpfsug main discussion list \
<gpfsug-discuss@spectrumscale.org></font><br><font size="2" \
color="#5F5F5F">Date: </font><font size="2">02/14/2018 08:47 \
PM</font><br><font size="2" color="#5F5F5F">Subject: </font><font \
size="2">[gpfsug-discuss] RDMA data from Zimon</font><br><font size="2" \
color="#5F5F5F">Sent by: </font><font \
size="2">gpfsug-discuss-bounces@spectrumscale.org</font><br><hr width="100%" size="2" \
align="left" noshade style="color:#8091A5; "><br><br><br>Hi,<br><br>Can one of the \
IBMers tell me if port_xmit_data and port_rcv_data from Zimon can be interpreted as \
RDMA Bytes/sec? Ideally, also how this data is being collected? I'm looking here: <a \
href="https://www.ibm.com/support/knowledgecenter/en/STXKQY_5.0.0/com.ibm.spectrum.scale.v5r00.doc/bl1hlp_monnetworksmetrics.htm"><u><font \
color="#0000FF">https://www.ibm.com/support/knowledgecenter/en/STXKQY_5.0.0/com.ibm.spectrum.scale.v5r00.doc/bl1hlp_monnetworksmetrics.htm</font></u></a><br><br>But \
then I also look here: <a \
href="https://urldefense.proofpoint.com/v2/url?u=https-3A__community.mellanox.com_docs \
_DOC-2D2751&d=DwMFaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=IbxtjdkPAM2Sbon4Lbbi4w&m=zIRb70L9sx_Fv \
vC9IcWVKLOSOOFnx-hIGfjw0kUN7bw&s=ataLzjZMTIJfHxa5qXIwQdCTq09CQveIxYNnxJQ5pgs&e="><u><font \
color="#0000FF">https://community.mellanox.com/docs/DOC-2751</font></u></a><br><br>and \
see "Total number of data octets, divided by 4 (lanes), received on all VLs. \
This is 64 bit counter." So I wasn't sure if some multiplication by 4 was in \
order.<br><br>Please advise.<br><br>Cheers,<br>Kristy<tt><font \
size="2">_______________________________________________<br>gpfsug-discuss mailing \
list<br>gpfsug-discuss at spectrumscale.org<br></font></tt><tt><font size="2"><a \
href="https://urldefense.proofpoint.com/v2/url?u=http-3A__gpfsug.org_mailman_listinfo_ \
gpfsug-2Ddiscuss&d=DwICAg&c=jf_iaSHvJObTbx-siA1ZOg&r=IbxtjdkPAM2Sbon4Lbbi4w&m=zIRb70L9 \
sx_FvvC9IcWVKLOSOOFnx-hIGfjw0kUN7bw&s=D1g4YTG5WeUiHI3rCPr_kkPxbG9V9E-18UGXBeCvfB8&e="> \
https://urldefense.proofpoint.com/v2/url?u=http-3A__gpfsug.org_mailman_listinfo_gpfsug \
-2Ddiscuss&d=DwICAg&c=jf_iaSHvJObTbx-siA1ZOg&r=IbxtjdkPAM2Sbon4Lbbi4w& \
m=zIRb70L9sx_FvvC9IcWVKLOSOOFnx-hIGfjw0kUN7bw&s=D1g4YTG5WeUiHI3rCPr_kkPxbG9V9E-18UGXBeCvfB8&e=</a></font></tt><tt><font \
size="2"><br></font></tt><br><br><BR> </body></html>
["graycol.gif" (image/gif)]
_______________________________________________
gpfsug-discuss mailing list
gpfsug-discuss at spectrumscale.org
http://gpfsug.org/mailman/listinfo/gpfsug-discuss
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic