[prev in list] [next in list] [prev in thread] [next in thread]
List: bro
Subject: [Zeek] Re: Questions about NICs, PF_RING, and AF_Packet
From: Steve Edgar <se10 () cornell ! edu>
Date: 2021-03-24 20:31:18
Message-ID: 187679A7-557A-4301-A62C-443A0DC71277 () cornell ! edu
[Download RAW message or body]
[Attachment #2 (text/plain)]
Big thanks for the information. This is very helpful. (The Suricata Extreme \
Performance Tuning documents are great work!)
I have a question about …
> On Tue, Mar 23, 2021 at 6:37 PM Michał Purzyński <michalpurzynski1@gmail.com> \
> wrote: AF_Packet is as zero-copy as PF_RING is. All of the PF_Ring documentation is \
> outdated by like 10+ years.
Is the above comparison for AF_Packet and Vanilla PF_RING, or AF_Packet and PF_RING \
ZC (Zero Copy)?
From: Mike Dopheide <dopheide@gmail.com>
Date: Tuesday, March 23, 2021 at 8:03 PM
To: Michał Purzyński <michalpurzynski1@gmail.com>
Cc: Steve Edgar <se10@cornell.edu>, "zeek@lists.zeek.org" <zeek@lists.zeek.org>
Subject: Re: [Zeek] Re: Questions about NICs, PF_RING, and AF_Packet
Attached is a systemd service file that may help with that hash command (and a bunch \
of other stuff) if you're using Intel XL710s. Credit goes to Vlad.
-Dop
On Tue, Mar 23, 2021 at 6:37 PM Michał Purzyński \
<michalpurzynski1@gmail.com<mailto:michalpurzynski1@gmail.com>> wrote: Hey Steve, \
answers inline.
On Tue, Mar 23, 2021 at 10:08 AM Steve Edgar \
<se10@cornell.edu<mailto:se10@cornell.edu>> wrote:
I am new to Zeek and will be setting up a Zeek system which will use a 10G NIC. I am \
not sure what NIC/driver configuration to use, and have some questions about PF_RING \
and AF_Packet. At …
https://docs.zeek.org/en/current/cluster-setup.html#using-pf-ring
… it looks like PF_RING, also known as "Vanilla PF_RING" …
https://www.ntop.org/guides/pf_ring/vanilla.html#vanilla-pf-ring
… makes it possible to assign worker processes to CPU cores by using "packet \
clustering" …
Indeed it is a best performance practice to assign cores to worker nodes and that's \
supported by both AF_Packet and PF_RING
https://www.ntop.org/guides/pf_ring/vanilla.html#packet-clustering
Is this essentially implementing symmetric Receive Side Scaling?
Packet clustering there means they make it possible to distribute flows among many \
processes to process them, so kind of like symmetric RSS indeed. Supported by both \
PF_RING and AF_Packet.
If so, can Vanilla PF_RING take advantage of a NIC which does symmetric hashing in \
hardware? As far as I know PF_Ring does not use hardware RSS hash (nothing does, \
AF_Packet doesn't either). For AF_Packet read on.
The Zeek docs reference PF_RING+DNA …
https://docs.zeek.org/en/current/cluster-setup.html#using-pf-ring
… although from looking at the ntop site, DNA/Libzero was replaced some time ago \
with PF_RING ZC (Zero Copy) …
https://www.ntop.org/guides/pf_ring/zc.html
Does Zeek support PF_RING ZC?
If so, in Zeek's node.cfg, how does one know what options to use for …
interface=
lb_method=
It looks like the AF_Packet plugin …
https://github.com/J-Gras/zeek-af_packet-plugin
… does what Vanilla PF_RING does, in that it allows Zeek to have multiple worker \
processes which use different CPU cores. Can AF_Packet take advantage of a NIC which \
does symmetric hashing in hardware? Intel made it so confusing. Let's disect
- there is a way to make hashing symmetric by flipping a bit in hardware. No software \
but some Intel experiments uses that
- there is a way to use make RSS symmetric with the right hashing key that's easily \
set with ethtools. This would be your hardware symmetric RSS hashing. You need a \
couple of those ethtool commands to account for fragmented packets, etc.
I'll dig them out tomorrow.
It looks like AF_Packet does not provide a "Zero Copy" type of functionality, found \
in PF_RING ZC. Is that correct? AF_Packet is as zero-copy as PF_RING is. All of the \
PF_Ring documentation is outdated by like 10+ years.
I know this is a lot of questions. Any guidance is appreciated.
I'm sure there will be more and please keep sending them meantime read what Peter \
Manev and the (not-so-humble) myself wrote about Suricata - applies to Zeek as well
https://github.com/pevma/SEPTun
https://github.com/pevma/SEPTun-Mark-II
-- Steve.
--
zeek mailing list -- zeek@lists.zeek.org<mailto:zeek@lists.zeek.org>
To unsubscribe send an email to \
zeek-leave@lists.zeek.org<mailto:zeek-leave@lists.zeek.org>
--
zeek mailing list -- zeek@lists.zeek.org<mailto:zeek@lists.zeek.org>
To unsubscribe send an email to \
zeek-leave@lists.zeek.org<mailto:zeek-leave@lists.zeek.org>
[Attachment #3 (text/html)]
<html xmlns:o="urn:schemas-microsoft-com:office:office" \
xmlns:w="urn:schemas-microsoft-com:office:word" \
xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" \
xmlns="http://www.w3.org/TR/REC-html40"> <head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
span.EmailStyle19
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style>
</head>
<body lang="EN-US" link="blue" vlink="purple" style="word-wrap:break-word">
<div class="WordSection1">
<p class="MsoNormal">Big thanks for the information. This is very \
helpful. (The Suricata Extreme Performance Tuning documents are great \
work!)<o:p></o:p></p> <p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">I have a question about …<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">> On Tue, Mar 23, 2021 at 6:37 PM Michał Purzyński \
<michalpurzynski1@gmail.com> wrote:<o:p></o:p></p> <p class="MsoNormal">> \
AF_Packet is as zero-copy as PF_RING is. All of the PF_Ring \
documentation is outdated by like 10+ years.<o:p></o:p></p> <p \
class="MsoNormal"><o:p> </o:p></p> <p class="MsoNormal">Is the above comparison \
for AF_Packet and Vanilla PF_RING, or AF_Packet and PF_RING ZC (Zero \
Copy)?<o:p></o:p></p> <p class="MsoNormal"><o:p> </o:p></p>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span style="font-size:12.0pt;color:black">From: \
</span></b><span style="font-size:12.0pt;color:black">Mike Dopheide \
<dopheide@gmail.com><br> <b>Date: </b>Tuesday, March 23, 2021 at 8:03 PM<br>
<b>To: </b>Michał Purzyński <michalpurzynski1@gmail.com><br>
<b>Cc: </b>Steve Edgar <se10@cornell.edu>, "zeek@lists.zeek.org" \
<zeek@lists.zeek.org><br> <b>Subject: </b>Re: [Zeek] Re: Questions about NICs, \
PF_RING, and AF_Packet<o:p></o:p></span></p> </div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<div>
<p class="MsoNormal">Attached is a systemd service file that may help with that hash \
command (and a bunch of other stuff) if you're using Intel XL710s. Credit goes \
to Vlad.<o:p></o:p></p> </div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">-Dop<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<div>
<p class="MsoNormal">On Tue, Mar 23, 2021 at 6:37 PM Michał Purzyński <<a \
href="mailto:michalpurzynski1@gmail.com">michalpurzynski1@gmail.com</a>> \
wrote:<o:p></o:p></p> </div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in \
6.0pt;margin-left:4.8pt;margin-right:0in"> <div>
<div>
<p class="MsoNormal">Hey Steve, answers inline.<o:p></o:p></p>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<div>
<p class="MsoNormal">On Tue, Mar 23, 2021 at 10:08 AM Steve Edgar <<a \
href="mailto:se10@cornell.edu" target="_blank">se10@cornell.edu</a>> \
wrote:<o:p></o:p></p> </div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in \
6.0pt;margin-left:4.8pt;margin-right:0in"> <div>
<div>
<p>I am new to Zeek and will be setting up a Zeek system which will use a 10G \
NIC. I am not sure what NIC/driver configuration to use, and have some \
questions about PF_RING and AF_Packet. At …<o:p></o:p></p> <p><a \
href="https://docs.zeek.org/en/current/cluster-setup.html#using-pf-ring" \
target="_blank">https://docs.zeek.org/en/current/cluster-setup.html#using-pf-ring</a><o:p></o:p></p>
<p>… it looks like PF_RING, also known as "Vanilla PF_RING" \
…<o:p></o:p></p> <p><a \
href="https://www.ntop.org/guides/pf_ring/vanilla.html#vanilla-pf-ring" \
target="_blank">https://www.ntop.org/guides/pf_ring/vanilla.html#vanilla-pf-ring</a><o:p></o:p></p>
<p>… makes it possible to assign worker processes to CPU cores by using \
"packet clustering" …<o:p></o:p></p> </div>
</div>
</blockquote>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"> Indeed it is a best performance practice to assign cores \
to worker nodes and that's supported by both AF_Packet and PF_RING<o:p></o:p></p> \
</div> <blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in \
0in 6.0pt;margin-left:4.8pt;margin-right:0in"> <div>
<div>
<p> <o:p></o:p></p>
<p><a href="https://www.ntop.org/guides/pf_ring/vanilla.html#packet-clustering" \
target="_blank">https://www.ntop.org/guides/pf_ring/vanilla.html#packet-clustering</a><o:p></o:p></p>
<p> <o:p></o:p></p>
<p>Is this essentially implementing symmetric Receive Side Scaling? \
<o:p></o:p></p> </div>
</div>
</blockquote>
<div>
<p class="MsoNormal">Packet clustering there means they make it possible to \
distribute flows among many processes to process them, so kind of like symmetric RSS \
indeed. Supported by both PF_RING and AF_Packet.<o:p></o:p></p> </div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in \
6.0pt;margin-left:4.8pt;margin-right:0in"> <div>
<div>
<p> <o:p></o:p></p>
<p>If so, can Vanilla PF_RING take advantage of a NIC which does symmetric hashing in \
hardware?<o:p></o:p></p> </div>
</div>
</blockquote>
<div>
<p class="MsoNormal">As far as I know PF_Ring does not use hardware RSS hash (nothing \
does, AF_Packet doesn't either). For AF_Packet read on. <o:p></o:p></p>
</div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in \
6.0pt;margin-left:4.8pt;margin-right:0in"> <div>
<div>
<p> <o:p></o:p></p>
<p>The Zeek docs reference PF_RING+DNA …<o:p></o:p></p>
<p> <o:p></o:p></p>
<p><a href="https://docs.zeek.org/en/current/cluster-setup.html#using-pf-ring" \
target="_blank">https://docs.zeek.org/en/current/cluster-setup.html#using-pf-ring</a><o:p></o:p></p>
<p> <o:p></o:p></p>
<p>… although from looking at the ntop site, DNA/Libzero was replaced some time ago \
with PF_RING ZC (Zero Copy) …<o:p></o:p></p> <p> <o:p></o:p></p>
<p><a href="https://www.ntop.org/guides/pf_ring/zc.html" \
target="_blank">https://www.ntop.org/guides/pf_ring/zc.html</a><o:p></o:p></p> \
<p> <o:p></o:p></p> <p>Does Zeek support PF_RING ZC? <o:p></o:p></p>
<p> <o:p></o:p></p>
<p>If so, in Zeek's node.cfg, how does one know what options to use for \
…<o:p></o:p></p> <p> <o:p></o:p></p>
<p>interface=<o:p></o:p></p>
<p>lb_method=<o:p></o:p></p>
<p> <o:p></o:p></p>
<p>It looks like the AF_Packet plugin …<o:p></o:p></p>
<p> <o:p></o:p></p>
<p><a href="https://github.com/J-Gras/zeek-af_packet-plugin" \
target="_blank">https://github.com/J-Gras/zeek-af_packet-plugin</a><o:p></o:p></p> \
<p> <o:p></o:p></p> <p>… does what Vanilla PF_RING does, in that it allows \
Zeek to have multiple worker processes which use different CPU cores. Can \
AF_Packet take advantage of a NIC which does symmetric hashing in \
hardware?<o:p></o:p></p> </div>
</div>
</blockquote>
<div>
<p class="MsoNormal">Intel made it so confusing. Let's disect<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">- there is a way to make hashing symmetric by flipping a bit in \
hardware. No software but some Intel experiments uses that<o:p></o:p></p> </div>
<div>
<p class="MsoNormal">- there is a way to use make RSS symmetric with the right \
hashing key that's easily set with ethtools. This would be your hardware symmetric \
RSS hashing.<o:p></o:p></p> </div>
<div>
<p class="MsoNormal">You need a couple of those ethtool commands to account for \
fragmented packets, etc.<o:p></o:p></p> </div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">I'll dig them out tomorrow.<o:p></o:p></p>
</div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in \
6.0pt;margin-left:4.8pt;margin-right:0in"> <div>
<div>
<p> <o:p></o:p></p>
<p>It looks like AF_Packet does not provide a "Zero Copy" type of \
functionality, found in PF_RING ZC. Is that correct?<o:p></o:p></p> </div>
</div>
</blockquote>
<div>
<p class="MsoNormal">AF_Packet is as zero-copy as PF_RING is. All of the PF_Ring \
documentation is outdated by like 10+ years. <o:p></o:p></p>
</div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in \
6.0pt;margin-left:4.8pt;margin-right:0in"> <div>
<div>
<p> <o:p></o:p></p>
<p>I know this is a lot of questions. Any guidance is \
appreciated.<o:p></o:p></p> </div>
</div>
</blockquote>
<div>
<p class="MsoNormal">I'm sure there will be more and please keep sending them \
meantime read what Peter Manev and the (not-so-humble) myself wrote about Suricata - \
applies to Zeek as well<o:p></o:p></p> </div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal"><a href="https://github.com/pevma/SEPTun" \
target="_blank">https://github.com/pevma/SEPTun</a><o:p></o:p></p> </div>
<div>
<p class="MsoNormal"><a href="https://github.com/pevma/SEPTun-Mark-II" \
target="_blank">https://github.com/pevma/SEPTun-Mark-II</a><o:p></o:p></p> </div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in \
6.0pt;margin-left:4.8pt;margin-right:0in"> <div>
<div>
<p> <o:p></o:p></p>
<p>-- Steve.<o:p></o:p></p>
<p> <o:p></o:p></p>
</div>
</div>
<p class="MsoNormal"><br>
--<br>
zeek mailing list -- <a href="mailto:zeek@lists.zeek.org" \
target="_blank">zeek@lists.zeek.org</a><br> To unsubscribe send an email to <a \
href="mailto:zeek-leave@lists.zeek.org" target="_blank"> \
zeek-leave@lists.zeek.org</a><o:p></o:p></p> </blockquote>
</div>
</div>
<p class="MsoNormal"><br>
--<br>
zeek mailing list -- <a href="mailto:zeek@lists.zeek.org" \
target="_blank">zeek@lists.zeek.org</a><br> To unsubscribe send an email to <a \
href="mailto:zeek-leave@lists.zeek.org" target="_blank"> \
zeek-leave@lists.zeek.org</a><o:p></o:p></p> </blockquote>
</div>
</div>
</body>
</html>
--
zeek mailing list -- zeek@lists.zeek.org
To unsubscribe send an email to zeek-leave@lists.zeek.org
--===============1712679533495696034==--
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic