[prev in list] [next in list] [prev in thread] [next in thread]
List: ietf
Subject: Re: {Spam?} RE: [bmwg] Opsdir last call review of draft-ietf-bmwg-sdn-controller-benchmark-meth-07
From: bhuvaneswaran.vengainathan () veryxtech ! com
Date: 2018-01-31 8:57:57
Message-ID: 2d305203952bb233f549ea84291ac06f () cloudmail14 ! netcore ! co ! in
[Download RAW message or body]
Thanks Al for formatting Scott's comment.
Hi Scott,
Thank you for reviewing the draft and providing your comments. Please find below our \
response inline to your comments. Best Regards,
Bhuvan
On Tue, Jan 30, 2018 at 08:02 PM, "MORTON, ALFRED C (AL)" wrote:
Hi Scott,
Thanks for your review and insightful comments!
Something modified your original formatting, so
I've tried to restore it below. Authors, please
reply to Scott's comments and add changes to your
working version of the draft.
regards,
Al
doc shepherd
-=-=-=-=-=-=-=-=-=-=-=-
Reviewer: Scott Bradner
Review result: Has Issues
This ID specifies the methodology to be used in testing the performance of SDN
controllers. It is a standard BMWG methodology document and thus, cannot have
any impact, operational or otherwise, on operational networks - see RFC 6815
The following are some comments on the document itself:
section 4.1 - Leaf-Spine used but not defined
[Bhuvan] We will add the definition in the terminology document
section 4.4 - using old software versions seems to add complexity with little
benefit
[Bhuvan] We agree with your observation. We have added this verification to ensure \
backward compatibility. In fact we have given this example (older vs current) for \
better clarity.
section 4.7 - would be useful to say that the variance over the
repeated tests should be reported in the "test reporting" section
- e.g., in section 5.1.1 I would add a row that reports the variance
- same for all tests
[Bhuvan] We will add a row to report variance across all tests
section 5.1.1 - is the topology discovery process slow enough that a 3 second
granularity of the measurement will show differences between systems?
[Bhuvan] We have considered 3 seconds for the worst case scenario
section 5.1.2 - procedure step 2 - what inserts the timestamp in the response message
(R1)?
[Bhuvan] The intent was to find the difference between the transmitted time of \
request and the received time of the corresponding response. I think we need to \
rephrase the text as follows. '2. Record every request transmit (T1) time and the \
corresponding response (R1) received time at the forwarding plane test emulator \
interface (I1) for every successful message exchange'
section 5.1.2 - the ID says "successful messages exchanged" - might it be
useful to report the % of unsuccessful exchanges? [Bhuvan] We will report the % of \
unsuccessful exchanges as well section 5.1.6 - the title & objective description do \
not match
- the title says "rate" but the objective is a count
- the test seems to try to get the rate section 5.1.7
- same issue as section 5.1.6 [Bhuvan] We will reword the objective section to \
reflect the rate. section 5.2.3 - procedure step 1
- this reads as if the addresses are unique but unchanging
- if that is not what is meant then it should specifically say that
the addresses are changing [Bhuvan] The intent is to generate unique src/dst address \
combinations to populate the forwarding table and measure its capacity. Could you \
please clarify what you mean by unchanging in Step 1. ? section 5.3.1 - it might be \
useful to establish specific "invalid messages" since I could imagine that the \
devices could handle different types in different ways that could have an impact on \
speed of handling [Bhuvan] We have clarified the invalid message in the Note \
section. Do we need to add the info in the test report as well?
section 5.3.2 - "a huge number" is somewhat undefined
do you mean to say TCP SYN messages rather than TCP SYNC messages?
[Bhuvan] Yes. We will address this comment in the revised draft.
section 6 - RFC 2026 is referenced in the introduction but not listed in the
references
[Bhuvan] We will include RFC 2026 in the reference
section 9 - I have now retired so no affiliation should be listed
[Bhuvan] We will remove the affiliation
-----Original Message-----
From: bmwg [mailto:bmwg-bounces@ietf.org (mailto:bmwg-bounces@ietf.org)] On Behalf Of \
Scott Bradner
Sent: Monday, January 29, 2018 9:17 PM
To: ops-dir@ietf.org (mailto:ops-dir@ietf.org)
Cc: ietf@ietf.org (mailto:ietf@ietf.org); bmwg@ietf.org (mailto:bmwg@ietf.org); \
draft-ietf-bmwg-sdn-controller- benchmark-meth.all@ietf.org \
(mailto:benchmark-meth.all@ietf.org)
Subject: [bmwg] Opsdir last call review of draft-ietf-bmwg-sdn-
controller-benchmark-meth-07
Reviewer: Scott Bradner
Review result: Has Issues
This ID specifies the methodology to be used in testing the performance
of SDN
controllers. It is a standard BMWG methodology document and thus, cannot
have
any impact, operational or otherwise, on operational networks - see RFC
6815
follow are some comments on the document itself
section 4.1 - Leaf-Spine used but not defined
section 4.4 - using old software versions seems to add complexity with
little
benefit section 4.7 - would be useful to say that the variance over the
repeated tests should be reported in the "test reporting" section -
e.g., in
section 5.1.1 I would add a row that reports the variance - same for all
tests
section 5.1.1 - is the topology discovery process slow enough that a 3
second
granularity of the measurement will show differences between systems?
section
5.1.2 - procedure step 2 - what inserts the timestamp in the response
message
(R1)? section 5.1.2 - the ID says "successful messages exchanged" -
might it be
useful to report the % of unsuccessful exchanges? section 5.1.6 - the
title &
objective description do not match - the title says "rate" but the
objective is
a count - the test seems to try to get the rate section 5.1.7 - same
issue as
section 5.1.6 section 5.2.3 - procedure step 1 - this reads as if the
addresses
are unique but unchanging - if that is not what is meant then it should
specifically say that the addresses are changing section 5.3.1 - it
might be
useful to establish specific "invalid messages" since I could imagine
that the
devices could handle different types in different ways that could have
an
impact on speed of handling section 5.3.2 - "a huge number" is somewhat
undefined
do you mean to say TCP SYN messages rather than TCP SYNC
messages?
section 6 - RFC 2026 is referenced in the introduction but not listed in
the
references section 9 - I have now retired so no affiliation should be
listed
_______________________________________________
bmwg mailing list
bmwg@ietf.org (mailto:bmwg@ietf.org)
https://urldefense.proofpoint.com/v2/url?u=https \
(https://urldefense.proofpoint.com/v2/url?u=https)- \
3A__www.ietf.org_mailman_listinfo_bmwg&d=DwICAg&c=LFYZ- \
o9_HUMeMTSQicvjIg&r=OfsSu8kTIltVyD1oL72cBw&m=Oip5- \
2t1wtJ1ZpUIlQWQcvgNAVgydAS4jCoQi_LL50Q&s=WP7X4VUxjR59g9gvjUFj_8zTQrz- \
iKc2jY9FOaPTlZY&e=
DISCLAIMER: Privileged and/or Confidential information may be
contained in this message. If you are not the addressee of this message,
you may not copy, use or deliver this message to anyone. In such
event,you should destroy the message and kindly notify the sender by
reply e-mail.
It is understood that opinions or conclusions that do not relate to the
official business of the company are neither given nor endorsed by the
company.
[Attachment #3 (text/html)]
<!DOCTYPE html><html><head><meta http-equiv="Content-Type" content="text/html; \
charset=utf-8" /></head><body><div data-crea="font-wrapper" style="font-family: \
Calibri; font-size: 16px; direction: ltr"><div style="font-family: Calibri; \
font-size: 16px"></div><div>Thanks Al for formatting Scott's \
comment.</div><div><br></div><div>Hi Scott,</div><div><br></div><div>Thank you for \
reviewing the draft and providing your comments. Please find below our response \
inline to your comments. </div><div><br><div></div>Best \
Regards,</div><div>Bhuvan<br><br><div data-anchor="reply-title">On Tue, Jan 30, 2018 \
at 08:02 PM, "MORTON, ALFRED C (AL)" <acmorton@att.com> \
wrote:</div><blockquote><div>Hi Scott,<br>Thanks for your review and insightful \
comments!<br><br>Something modified your original formatting, so<br>I've tried to \
restore it below. Authors, please <br>reply to Scott's comments and add changes to \
your <br>working version of the draft.<br><br>regards,<br>Al<br>doc \
shepherd<br><br>-=-=-=-=-=-=-=-=-=-=-=-<br><br>Reviewer: Scott Bradner<br>Review \
result: Has Issues<br><br>This ID specifies the methodology to be used in testing the \
performance of SDN<br>controllers. It is a standard BMWG methodology document and \
thus, cannot have<br>any impact, operational or otherwise, on operational networks - \
see RFC 6815<br><br>The following are some comments on the document \
itself:<br><br>section 4.1 - Leaf-Spine used but not defined<br>[Bhuvan] We will add \
the definition in the terminology document</div></blockquote><span> \
</span><br><blockquote><div>section 4.4 - using old software versions seems to add \
complexity with little<br>benefit <br>[Bhuvan] We agree with your observation. We \
have added this verification to ensure backward compatibility. In fact we have given \
this example (older vs current) for better clarity.</div></blockquote><span> \
</span><br><blockquote><div>section 4.7 - would be useful to say that the variance \
over the<br>repeated tests should be reported in the "test reporting" section <br>- \
e.g., in section 5.1.1 I would add a row that reports the variance <br>- same for all \
tests<br>[Bhuvan] We will add a row to report variance across all \
tests</div></blockquote><span> </span><br><blockquote><div>section 5.1.1 - is the \
topology discovery process slow enough that a 3 second<br>granularity of the \
measurement will show differences between systems? <br>[Bhuvan] We have considered 3 \
seconds for the worst case scenario</div></blockquote><span> \
</span><br><blockquote><div>section 5.1.2 - procedure step 2 - what inserts the \
timestamp in the response message<br>(R1)? <br>[Bhuvan] The intent was to find the \
difference between the transmitted time of request and the received time of the \
corresponding response. I think we need to rephrase the text as follows. <span \
style="font-family: monospace, serif; font-size: 13.3333px; white-space: pre-wrap;"> \
'2. Record every request transmit (T1) time and the</span><span style="font-family: \
monospace, serif; font-size: 13.3333px; white-space: pre-wrap;"> corresponding \
response (R1) received time at the</span><span style="font-family: monospace, serif; \
font-size: 13.3333px; white-space: pre-wrap;"> forwarding plane test emulator \
interface (I1) for every</span><span style="font-family: monospace, serif; font-size: \
13.3333px; white-space: pre-wrap;"> successful message \
exchange'</span></div></blockquote><span> </span><br><blockquote><div>section 5.1.2 \
- the ID says "successful messages exchanged" - might it be<br>useful to report the % \
of unsuccessful exchanges? </div></blockquote><span>[Bhuvan] We will report the % of \
unsuccessful exchanges as well </span><br><blockquote><div><br>section 5.1.6 - the \
title & objective description do not match <br>- the title says "rate" but the \
objective is a count <br>- the test seems to try to get the rate section 5.1.7 <br>- \
same issue as section 5.1.6 </div></blockquote><span>[Bhuvan] We will reword the \
objective section to reflect the rate. </span><br><blockquote><div><br>section 5.2.3 \
- procedure step 1 <br>- this reads as if the addresses are unique but unchanging \
<br>- if that is not what is meant then it should specifically say that <br>the \
addresses are changing </div></blockquote><span>[Bhuvan] The intent is to generate \
unique src/dst address combinations to populate the forwarding table and measure its \
capacity. Could you please clarify what you mean by unchanging in Step 1. \
?</span><br><blockquote><div><br>section 5.3.1 - it might be useful to establish \
specific "invalid messages" <br>since I could imagine that the devices could handle \
different types in <br>different ways that could have an impact on speed of handling \
<br>[Bhuvan] We have clarified the invalid message in the Note section. Do we need to \
add the info in the test report as well?</div></blockquote><span> \
</span><br><blockquote><div>section 5.3.2 - "a huge number" is somewhat \
undefined<br>do you mean to say TCP SYN messages rather than TCP SYNC \
messages?<br>[Bhuvan] Yes. We will address this comment in the revised \
draft.</div></blockquote><span> </span><br><blockquote><div>section 6 - RFC 2026 is \
referenced in the introduction but not listed in the<br>references <br>[Bhuvan] We \
will include RFC 2026 in the reference </div></blockquote><span> \
</span><br><blockquote><div>section 9 - I have now retired so no affiliation should \
be listed<br>[Bhuvan] We will remove the affiliation</div></blockquote><span> \
</span><br><blockquote><div><blockquote>-----Original Message-----<br>From: bmwg \
[mailto:<a target="_blank" href="mailto:bmwg-bounces@ietf.org" style="cursor: \
inherit;">bmwg-bounces@ietf.org</a>] On Behalf Of Scott Bradner<br>Sent: Monday, \
January 29, 2018 9:17 PM<br>To: <a target="_blank" href="mailto:ops-dir@ietf.org" \
style="cursor: inherit;">ops-dir@ietf.org</a><br>Cc: <a target="_blank" \
href="mailto:ietf@ietf.org" style="cursor: inherit;">ietf@ietf.org</a>; <a \
target="_blank" href="mailto:bmwg@ietf.org" style="cursor: \
inherit;">bmwg@ietf.org</a>; draft-ietf-bmwg-sdn-controller-<br><a target="_blank" \
href="mailto:benchmark-meth.all@ietf.org" style="cursor: \
inherit;">benchmark-meth.all@ietf.org</a><br>Subject: [bmwg] Opsdir last call review \
of draft-ietf-bmwg-sdn-<br>controller-benchmark-meth-07<br><br>Reviewer: Scott \
Bradner<br>Review result: Has Issues<br><br>This ID specifies the methodology to be \
used in testing the performance<br>of SDN<br>controllers. It is a standard BMWG \
methodology document and thus, cannot<br>have<br>any impact, operational or \
otherwise, on operational networks - see RFC<br>6815<br><br>follow are some comments \
on the document itself<br><br>section 4.1 - Leaf-Spine used but not \
defined<br>section 4.4 - using old software versions seems to add complexity \
with<br>little<br>benefit section 4.7 - would be useful to say that the variance over \
the<br>repeated tests should be reported in the "test reporting" section -<br>e.g., \
in<br>section 5.1.1 I would add a row that reports the variance - same for \
all<br>tests<br>section 5.1.1 - is the topology discovery process slow enough that a \
3<br>second<br>granularity of the measurement will show differences between \
systems?<br>section<br>5.1.2 - procedure step 2 - what inserts the timestamp in the \
response<br>message<br>(R1)? section 5.1.2 - the ID says "successful messages \
exchanged" -<br>might it be<br>useful to report the % of unsuccessful exchanges? \
section 5.1.6 - the<br>title &<br>objective description do not match - the title \
says "rate" but the<br>objective is<br>a count - the test seems to try to get the \
rate section 5.1.7 - same<br>issue as<br>section 5.1.6 section 5.2.3 - procedure step \
1 - this reads as if the<br>addresses<br>are unique but unchanging - if that is not \
what is meant then it should<br>specifically say that the addresses are changing \
section 5.3.1 - it<br>might be<br>useful to establish specific "invalid messages" \
since I could imagine<br>that the<br>devices could handle different types in \
different ways that could have<br>an<br>impact on speed of handling section 5.3.2 - \
"a huge number" is somewhat<br>undefined<br>do you mean to say TCP SYN messages \
rather than TCP SYNC<br>messages?<br>section 6 - RFC 2026 is referenced in the \
introduction but not listed in<br>the<br>references section 9 - I have now retired so \
no affiliation should \
be<br>listed<br><br>_______________________________________________<br>bmwg mailing \
list<br><a target="_blank" href="mailto:bmwg@ietf.org" style="cursor: \
inherit;">bmwg@ietf.org</a><br><a target="_blank" \
href="https://urldefense.proofpoint.com/v2/url?u=https" style="cursor: \
inherit;">https://urldefense.proofpoint.com/v2/url?u=https</a>-<br>3A__www.ietf.org_ma \
ilman_listinfo_bmwg&d=DwICAg&c=LFYZ-<br>o9_HUMeMTSQicvjIg&r=OfsSu8kTIltVyD \
1oL72cBw&m=Oip5-<br>2t1wtJ1ZpUIlQWQcvgNAVgydAS4jCoQi_LL50Q&s=WP7X4VUxjR59g9gvj \
UFj_8zTQrz-<br>iKc2jY9FOaPTlZY&e=</blockquote></div></blockquote></div></div>DISCLAIMER: \
Privileged and/or Confidential information may be contained in this message. If you \
are not the addressee of this message, you may not copy, use or deliver this message \
to anyone. In such event,you should destroy the message and kindly notify the sender \
by reply e-mail.
It is understood that opinions or conclusions that do not relate to the
official business of the company are neither given nor endorsed by the
company.
</body></html>
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic