[prev in list] [next in list] [prev in thread] [next in thread]
List: bro
Subject: [Bro] Bro Cluster on the Bivio Platform
From: "Joel Ebrahimi" <jebrahimi () bivio ! net>
Date: 2010-11-11 18:12:25
Message-ID: D83235F0F3C86D4D889D8B9A0DA8C6D70593FC86 () corpexc01 ! corp ! networkrobots ! com
[Download RAW message or body]
[Attachment #2 (multipart/alternative)]
I recently built and tested Bro Cluster for the Bivio Platform for some
of our customers and wanted to share the information.
The cluster version of Bro is a very native fit for the Bivio
architecture. The internals of the Bivio platform on a single Bivio 7562
can be thought of as a load balancer and 12 separate Linux systems (this
can scale to 48 systems in a single logical unit). The Linux systems
have their own communication plane within the Bivio system that is
separate from the packet acquisition path and can use this to talk to
the workers, proxies, and the manager. The shared file system also
allows for easy setup. Below are the steps I used to setup the system
and Bro Cluster. I also attached my node.cfg for a Bivio 7562, this file
can be edited so that it reflects the number of cpu cores that will be
running systems for Bro.
Installation
--------------------------
1. Unzip Bro
tar -zxvf bro-1.5-release.tar.gz
2. Change into the Bro directory
cd bro-1.5.1/
3. Configure Bro with desired options
./configure --disable-select-loop --enable-cluster
4. Build Bro
make
5. Install Bro with Broctl
make install-broctl
System Configuration
---------------------------
1. Turn off strict key checking to avoid key prompts when logging into
Bro worker cpus
vi /etc/ssh/ssh_config
add
StrictHostKeyChecking no
2. Generate public/private key
ssh-keygen -t rsa -f /root/.ssh/id_rsa
hit return twice for a blank passphrase
3. Add it to the authorized keys
cat /root/.ssh/id_rsa.pub >> /root/.ssh/authorized_keys2
4. Setup your manager,proxy, and worker nodes in your node.cfg
configuration file
# $Id: node.cfg,v 1.1 2010/11/05 19:49:46 jebrahimi Exp $
#
# Node configuration
#
[manager]
type=manager
host=CPU-X
[proxy-1]
type=proxy
host=CPU-X
[worker-1]
type=worker
host=CPU-1c0
interface=default
[worker-2]
type=worker
host=CPU-1c1
interface=default
[worker-3]
type=worker
host=CPU-2c0
interface=default
[worker-4]
type=worker
host=CPU-2c1
interface=default
[worker-5]
type=worker
host=CPU-3c0
interface=default
[worker-6]
type=worker
host=CPU-3c1
interface=default
[worker-7]
type=worker
host=CPU-4c0
interface=default
[worker-8]
type=worker
host=CPU-4c1
interface=default
[worker-9]
type=worker
host=CPU-5c0
interface=default
[worker-10]
type=worker
host=CPU-5c1
interface=default
[worker-11]
type=worker
host=CPU-6c0
interface=default
[worker-12]
type=worker
host=CPU-6c1
interface=default
5. Edit your networks.cfg and broctl.cfg in /usr/local/bro/etc/
6. You will need to add the Bro binaries to you Path
export PATH="$PATH:/usr/local/bro/bin"
7. Install workers and proxies
broctl install
Running Bro
-----------------
1. Since we are running Bro through the cluster shell
and not Bivios nrsp we will need to force on load sharing to the APC
CPUs
nrsp loadshare all on
2. Add the crontab entry for some required Bro tasks, enter cron
crontab -e
then add
0-59/5 * * * * /usr/local/bro/bin/broctl cron
3. Start Bro
broctl start
// Joel
Joel Ebrahimi
Solutions Architect
Bivio Networks Inc.
[Attachment #5 (text/html)]
<html xmlns:v="urn:schemas-microsoft-com:vml" \
xmlns:o="urn:schemas-microsoft-com:office:office" \
xmlns:w="urn:schemas-microsoft-com:office:word" \
xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" \
xmlns="http://www.w3.org/TR/REC-html40"><head><meta http-equiv=Content-Type \
content="text/html; charset=us-ascii"><meta name=Generator content="Microsoft Word 12 \
(filtered medium)"><style><!-- /* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri","sans-serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
span.EmailStyle17
{mso-style-type:personal-compose;
font-family:"Calibri","sans-serif";
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]--></head><body lang=EN-US link=blue vlink=purple><div \
class=WordSection1><p class=MsoNormal>I recently built and tested Bro Cluster for the \
Bivio Platform for some of our customers and wanted to share the \
information.<o:p></o:p></p><p class=MsoNormal><o:p> </o:p></p><p \
class=MsoNormal>The cluster version of Bro is a very native fit for the Bivio \
architecture. The internals of the Bivio platform on a single Bivio 7562 can be \
thought of as a load balancer and 12 separate Linux systems (this can scale to 48 \
systems in a single logical unit). The Linux systems have their own communication \
plane within the Bivio system that is separate from the packet acquisition path and \
can use this to talk to the workers, proxies, and the manager. The shared \
file system also allows for easy setup. Below are the steps I used to setup the \
system and Bro Cluster. I also attached my node.cfg for a Bivio 7562, this file can \
be edited so that it reflects the number of cpu cores that will be running systems \
for Bro. <o:p></o:p></p><p class=MsoNormal><o:p> </o:p></p><p \
class=MsoNormal>Installation<o:p></o:p></p><p \
class=MsoNormal>--------------------------<o:p></o:p></p><p \
class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>1. Unzip Bro<o:p></o:p></p><p \
class=MsoNormal><o:p> </o:p></p><p class=MsoNormal> tar -zxvf \
bro-1.5-release.tar.gz<o:p></o:p></p><p class=MsoNormal><o:p> </o:p></p><p \
class=MsoNormal>2. Change into the Bro directory<o:p></o:p></p><p \
class=MsoNormal><o:p> </o:p></p><p class=MsoNormal> cd \
bro-1.5.1/<o:p></o:p></p><p class=MsoNormal><o:p> </o:p></p><p \
class=MsoNormal>3. Configure Bro with desired options<o:p></o:p></p><p \
class=MsoNormal><o:p> </o:p></p><p class=MsoNormal> \
./configure --disable-select-loop --enable-cluster<o:p></o:p></p><p \
class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>4. Build Bro<o:p></o:p></p><p \
class=MsoNormal><o:p> </o:p></p><p class=MsoNormal> \
make<o:p></o:p></p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>5. \
Install Bro with Broctl<o:p></o:p></p><p class=MsoNormal><o:p> </o:p></p><p \
class=MsoNormal> make install-broctl<o:p></o:p></p><p \
class=MsoNormal><o:p> </o:p></p><p class=MsoNormal><o:p> </o:p></p><p \
class=MsoNormal>System Configuration<o:p></o:p></p><p \
class=MsoNormal>---------------------------<o:p></o:p></p><p \
class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>1. Turn off strict key \
checking to avoid key prompts when logging into Bro worker cpus<o:p></o:p></p><p \
class=MsoNormal><o:p> </o:p></p><p class=MsoNormal> vi \
/etc/ssh/ssh_config<o:p></o:p></p><p class=MsoNormal><o:p> </o:p></p><p \
class=MsoNormal>add<o:p></o:p></p><p class=MsoNormal><o:p> </o:p></p><p \
class=MsoNormal> StrictHostKeyChecking \
no<o:p></o:p></p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>2. \
Generate public/private key<o:p></o:p></p><p class=MsoNormal><o:p> </o:p></p><p \
class=MsoNormal> ssh-keygen -t rsa -f /root/.ssh/id_rsa<o:p></o:p></p><p \
class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>hit return twice for a blank \
passphrase <o:p></o:p></p><p class=MsoNormal><o:p> </o:p></p><p \
class=MsoNormal>3. Add it to the authorized keys<o:p></o:p></p><p \
class=MsoNormal><o:p> </o:p></p><p class=MsoNormal> cat \
/root/.ssh/id_rsa.pub >> /root/.ssh/authorized_keys2<o:p></o:p></p><p \
class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>4. Setup your manager,proxy, \
and worker nodes in your node.cfg configuration file<o:p></o:p></p><p \
class=MsoNormal><o:p> </o:p></p><p class=MsoNormal># $Id: node.cfg,v 1.1 \
2010/11/05 19:49:46 jebrahimi Exp $<o:p></o:p></p><p \
class=MsoNormal>#<o:p></o:p></p><p class=MsoNormal># Node \
configuration<o:p></o:p></p><p class=MsoNormal>#<o:p></o:p></p><p \
class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>[manager]<o:p></o:p></p><p \
class=MsoNormal>type=manager<o:p></o:p></p><p \
class=MsoNormal>host=CPU-X<o:p></o:p></p><p class=MsoNormal><o:p> </o:p></p><p \
class=MsoNormal>[proxy-1]<o:p></o:p></p><p \
class=MsoNormal>type=proxy<o:p></o:p></p><p \
class=MsoNormal>host=CPU-X<o:p></o:p></p><p class=MsoNormal><o:p> </o:p></p><p \
class=MsoNormal>[worker-1]<o:p></o:p></p><p \
class=MsoNormal>type=worker<o:p></o:p></p><p \
class=MsoNormal>host=CPU-1c0<o:p></o:p></p><p \
class=MsoNormal>interface=default<o:p></o:p></p><p \
class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>[worker-2]<o:p></o:p></p><p \
class=MsoNormal>type=worker<o:p></o:p></p><p \
class=MsoNormal>host=CPU-1c1<o:p></o:p></p><p \
class=MsoNormal>interface=default<o:p></o:p></p><p \
class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>[worker-3]<o:p></o:p></p><p \
class=MsoNormal>type=worker<o:p></o:p></p><p \
class=MsoNormal>host=CPU-2c0<o:p></o:p></p><p \
class=MsoNormal>interface=default<o:p></o:p></p><p \
class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>[worker-4]<o:p></o:p></p><p \
class=MsoNormal>type=worker<o:p></o:p></p><p \
class=MsoNormal>host=CPU-2c1<o:p></o:p></p><p \
class=MsoNormal>interface=default<o:p></o:p></p><p \
class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>[worker-5]<o:p></o:p></p><p \
class=MsoNormal>type=worker<o:p></o:p></p><p \
class=MsoNormal>host=CPU-3c0<o:p></o:p></p><p \
class=MsoNormal>interface=default<o:p></o:p></p><p \
class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>[worker-6]<o:p></o:p></p><p \
class=MsoNormal>type=worker<o:p></o:p></p><p \
class=MsoNormal>host=CPU-3c1<o:p></o:p></p><p \
class=MsoNormal>interface=default<o:p></o:p></p><p \
class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>[worker-7]<o:p></o:p></p><p \
class=MsoNormal>type=worker<o:p></o:p></p><p \
class=MsoNormal>host=CPU-4c0<o:p></o:p></p><p \
class=MsoNormal>interface=default<o:p></o:p></p><p \
class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>[worker-8]<o:p></o:p></p><p \
class=MsoNormal>type=worker<o:p></o:p></p><p \
class=MsoNormal>host=CPU-4c1<o:p></o:p></p><p \
class=MsoNormal>interface=default<o:p></o:p></p><p \
class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>[worker-9]<o:p></o:p></p><p \
class=MsoNormal>type=worker<o:p></o:p></p><p \
class=MsoNormal>host=CPU-5c0<o:p></o:p></p><p \
class=MsoNormal>interface=default<o:p></o:p></p><p \
class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>[worker-10]<o:p></o:p></p><p \
class=MsoNormal>type=worker<o:p></o:p></p><p \
class=MsoNormal>host=CPU-5c1<o:p></o:p></p><p \
class=MsoNormal>interface=default<o:p></o:p></p><p \
class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>[worker-11]<o:p></o:p></p><p \
class=MsoNormal>type=worker<o:p></o:p></p><p \
class=MsoNormal>host=CPU-6c0<o:p></o:p></p><p \
class=MsoNormal>interface=default<o:p></o:p></p><p \
class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>[worker-12]<o:p></o:p></p><p \
class=MsoNormal>type=worker<o:p></o:p></p><p \
class=MsoNormal>host=CPU-6c1<o:p></o:p></p><p \
class=MsoNormal>interface=default<o:p></o:p></p><p \
class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>5. Edit your networks.cfg and \
broctl.cfg in /usr/local/bro/etc/<o:p></o:p></p><p \
class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>6. You will need to add the \
Bro binaries to you Path<o:p></o:p></p><p class=MsoNormal><o:p> </o:p></p><p \
class=MsoNormal> export \
PATH="$PATH:/usr/local/bro/bin"<o:p></o:p></p><p \
class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>7. Install workers and \
proxies<o:p></o:p></p><p class=MsoNormal><o:p> </o:p></p><p \
class=MsoNormal> broctl install<o:p></o:p></p><p \
class=MsoNormal><o:p> </o:p></p><p class=MsoNormal><o:p> </o:p></p><p \
class=MsoNormal>Running Bro<o:p></o:p></p><p \
class=MsoNormal>-----------------<o:p></o:p></p><p class=MsoNormal>1. Since we are \
running Bro through the cluster shell <o:p></o:p></p><p class=MsoNormal>and not \
Bivios nrsp we will need to force on load sharing to the APC CPUs<o:p></o:p></p><p \
class=MsoNormal><o:p> </o:p></p><p class=MsoNormal> nrsp \
loadshare all on<o:p></o:p></p><p class=MsoNormal><o:p> </o:p></p><p \
class=MsoNormal>2. Add the crontab entry for some required Bro tasks, enter \
cron<o:p></o:p></p><p class=MsoNormal><o:p> </o:p></p><p \
class=MsoNormal> crontab -e<o:p></o:p></p><p \
class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>then add<o:p></o:p></p><p \
class=MsoNormal><o:p> </o:p></p><p class=MsoNormal> 0-59/5 * * \
* * /usr/local/bro/bin/broctl cron<o:p></o:p></p><p \
class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>3. Start Bro<o:p></o:p></p><p \
class=MsoNormal><o:p> </o:p></p><p class=MsoNormal> broctl \
start<o:p></o:p></p><p class=MsoNormal><o:p> </o:p></p><p \
class=MsoNormal><o:p> </o:p></p><p class=MsoNormal><o:p> </o:p></p><p \
class=MsoNormal>// Joel <o:p></o:p></p><p class=MsoNormal><o:p> </o:p></p><p \
class=MsoNormal>Joel Ebrahimi<o:p></o:p></p><p class=MsoNormal>Solutions \
Architect<o:p></o:p></p><p class=MsoNormal>Bivio Networks Inc.<o:p></o:p></p><p \
class=MsoNormal><o:p> </o:p></p></div></body></html>
["node.cfg" (application/octet-stream)]
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic