'[Snort-devel] Re: [Snort-users] Stream4 and other stuff'

[prev in list] [next in list] [prev in thread] [next in thread] 

List:       snort-devel
Subject:    [Snort-devel] Re: [Snort-users] Stream4 and other stuff
From:       Martin Roesch <roesch () sourcefire ! com>
Date:       2001-06-29 16:34:57
[Download RAW message or body]

Good catch, thanks.

     -Marty

"Mayers, Philip J" wrote:
> 
> On a slightly related note, since I'm running with "preprocessor stream4:
> keepstats"
> 
> diff -r1.13 spp_stream4.c
> 470c470
> <        free(toks[i]);
> ---
> >        free(toks[num_toks]);
> 
> I think that's right, otherwise supplying any argumens will break (note -
> the indentation above may be off...
> 
> Regards,
> Phil
> 
> +----------------------------------+
> | Phil Mayers, Network Support     |
> | Centre for Computing Services    |
> | Imperial College                 |
> +----------------------------------+
> 
> -----Original Message-----
> From: Martin Roesch [mailto:roesch@sourcefire.com]
> Sent: 29 June 2001 14:29
> To: Mayers, Philip J
> Cc: snort-users; snort-dev
> Subject: Re: [Snort-users] Stream4 and other stuff
> 
> I've had another report of segfaults on RH 7, is this build 30 that's
> crashing?  Can you give me a traceback (read the BUGS file) so I can see
> where the problem is?
> 
>      -Marty
> 
> "Mayers, Philip J" wrote:
> >
> > >From the CVS of an hour or two ago, I get a Segmentation Fault on startup
> -
> > the system is an intel RedHat 7.0 box. Commenting out the stream4
> > preprocessor lines allows snort to run OK.
> >
> > (Note - we do have a rather high-traffic site, currently doing 45megabits,
> > although we skip all port 80 traffic.)
> >
> > Regards,
> > Phil
> >
> > +----------------------------------+
> > | Phil Mayers, Network Support     |
> > | Centre for Computing Services    |
> > | Imperial College                 |
> > +----------------------------------+
> >
> > -----Original Message-----
> > From: Martin Roesch [mailto:roesch@sourcefire.com]
> > Sent: 28 June 2001 18:01
> > To: snort-users; snort-dev
> > Subject: [Snort-users] Stream4 and other stuff
> >
> > Hi everyone,
> >      Sorry I've been quiet lately, I'm busier than a one legged man at
> > an ass-kicking contest lately and I haven't been very good about
> > answering my email, but I have a good reason.
> >      Those of you who have been tracking CVS lately may have noticed a
> > new module go into the mix within the last week: spp_stream4.  I just
> > checked what I believe to be stable version of the new plugin into CVS,
> > so it's available and turned on by default in version 1.8-beta8 (build
> > 29).  Stream4 is an entirely new preprocessor that preforms two
> > functions:
> >
> > 1) Stateful inspection of TCP sessions
> > 2) TCP stream reassembly
> >
> > I implemented stream4 out of the desire to have more robust stream
> > reassembly capabilities and the desire to defeat the latest "stateless
> > attacks" that have been coming out against Snort (c.f. stick and snot).
> > Stream4 is written with the intent to let Snort be able to handle
> > performing stream reassembly for "enterprise class" users, people who
> > need to track and reassemble more than 256 streams simultaneously.  I've
> > optimized the code fairly extensively to be robust, stable, and fast.
> > The testing and calculations I've performed lead me to be fairly
> > confident that stream4 can provide full stream reassembly for several
> > thousand simultaneous connections and stateful inspection for upwards of
> > 64,000 simultaneous sessions.
> >
> > Stream4 is a large and complex piece of code (almost 2000 lines) and
> > there are a lot of options associated with its runtime configuration, so
> > I'll go over them here.
> >
> > preprocessor stream4: [noinspect], [keepstats], [timeout <seconds>],
> > [memcap <bytes>], [noalerts]
> >
> > noinspect - disable stateful inspection
> >
> > keepstats - record session summary information in <logdir>/session.log
> >
> > timeout <seconds> - amount of time to keep an inactive stream in the
> > state table, sessions that are flushed will automatically be picked up
> > again if more activity is seen, default is 30 seconds
> >
> > memcap <bytes> - number of bytes to set the memory cap at, if this limit
> > is exceeded stream4 will aggressively prune inactive sessions, default
> > is 8MB
> >
> > noalerts - turns off alerts for stream events of note, such as evasive
> > RST packets, data on the SYN packet, and out of window sequence numbers
> >
> > preprocessor stream4_reassemble: [clientonly], [serveronly], [noalerts],
> > [ports <portlist>]
> >
> > clientonly - provide reassembly for the client side of a connection only
> >
> > serveronly - provide reassembly for the server side of a connection only
> >
> > noalerts - don't alert on events that may be insertion or evasion
> > attacks
> >
> > ports <portlist> - a whitespace separated lit of ports to perform
> > reassembly for, "all" provides reassembly for all ports, "default"
> > provides reassembly for ports 21 23 25 53 80 110 111 143 and 513
> >
> > Just setting the stream4 and stream4_reassemble directives without
> > arguments in the snort.conf file will set them up in their default
> > configuration
> >
> > [*] stream4 defaults:
> > Session timeout: 30 seconds
> > Session memory cap: 8388608 bytes
> > Stateful Inspection: ACTIVE
> > Stream Reassembly: INACTIVE
> > Stream Stats: INACTIVE
> > State Alerts: ACTIVE
> >
> > [*] stream4_reassemble defaults:
> > Reassemble client: ACTIVE
> > Reassemble server: INACTIVE
> > Reassemble ports: 21 23 25 53 80 143 110 111 513
> > Reassembly alerts: ACTIVE
> >
> > There is a new command line switch that is used in concert with the
> > stream4 code, "-z".  The -z switch can take one of two arguments: "est"
> > and "all".  The "all" argument is the default if you don't specify
> > anything and tells Snort to alert normally.  If the -z switch is
> > specified with the "est" argument, Snort will only alert (for TCP
> > traffic) on streams that have been established via a three way handshake
> > or streams where cooperative bidirectional activity has been observed
> > (i.e. where some traffic went one way and something other than a RST or
> > FIN was seen going back to the originator).  With "-z est" turned on,
> > Snort completely ignores TCP-based stick/snot "attacks".
> >
> > Besides that, there is another new command line switch, "-k".  The -k
> > switch allows you to modify the checksum tests that Snort performs in
> > the decoder stage.  It can take the following arguments:
> >
> > "noip" - turn off IP checksum verification
> >
> > "notcp" - turn off TCP checksum verification
> >
> > "noudp" - turn off UDP checksums
> >
> > "noicmp" - turn off ICMP checksums
> >
> > "none" - turn off all checksums
> >
> > Ths is done as a mechanism to let people have control over a subsystem
> > in Snort that can eat a lot of CPU cycles if you're not careful.  For
> > example, in many networks today, packets with bad IP checksums never
> > make it past the router or switch, and so IP checksum verification is a
> > waste of time for Snort on these networks.  Allowing people to turn that
> > subsystem off (or tune it) lets them get better performance without
> > having to make code tweaks.
> >
> > I've also added some new "stuff" to the output stage of Snort recently.
> > If you've noticed lately, there's some new data in the alerts that looks
> > something like "[1:523:1]" just before the alert message.  This is the
> > new Snort unique rule ID information that we've recently integrated into
> > the rule set that will allow much better correlation of Snort rules to
> > other vulnerability databases (CVE, Bugtraq, etc) and provide much
> > easier machine processing of Snort output as well.
> >
> > The fields in the block are [generator:sid:revision].  Generator is the
> > piece of Snort that generated the alert.  The generators.h file contains
> > the list of generators and the SIDs that they can generate, except for
> > the primary detection engine (generator 1).  The SID is the "Snort ID"
> > of the rule, a unique rule ID assigned to the rule by myself.  The
> > partitioning of the SID space is as follows:
> >
> > 0-100:       Reserved for Marty
> > 101-999,999: Reserved for "official" Snort rules
> > 1,000,000+:  User land SID space
> >
> > Each there is a full set of SIDs available per generator, and the SID
> > space is a 32-bit unsigned integer inside Snort, so the values can go up
> > to 4 billion or so.
> >
> > The last number in the block is the revision number of the rule.  As
> > rules change and their messages change (today's "SYN FIN scan" can
> > become tomorrow's "SCAN - SYN FIN") we want to be able to track the
> > revision history of the rules so that people who are doing machine
> > processing of Snort rules (like ARIS, incidents.org, etc) can
> > effectively perform their own back-end tasks.  The revision number
> > updating depends on the ruleset maintainer, but in general anytime you
> > tweak a rule, you should update the revision number.  If you modify a
> > rule that ships with Snort, please set the revision number to "0" so
> > that any machine processors will know you're running modified standard
> > rules.
> >
> > Snort will also print out the transport protocol designation in
> > fast/syslog alerts now as well.
> >
> > In output land, we have a new output plugin called spo_unified.  This
> > plugin is used for both alerting and logging, and writes alerts and logs
> > to two separate binary formatted files.  The alert file contains the
> > following information of interest:
> >
> > Event data (generator, sid, rev, classification, priority,
> > event_reference)
> > timestamp
> > source IP
> > dest IP
> > source port
> > dest port
> > protocol
> > tcp flags (if any)
> >
> > The log file contains the event data, flags that indicate the nature of
> > the stored packet (reassembled frag, etc), and the raw binary packet
> > (like in the log_tcpdump plugin).
> >
> > Using this system we will be able to write Snort output to high speed
> > binary spool files and then use a second process to reformat the data
> > for consumption by the user.  I'm currently writing a program to perform
> > just that task called 'barnyard'.  Barnyard can read the binary file
> > formats of the unified alert and log spool files and present that data
> > to output plugins, which can then format the data for output to whatever
> > reporting and storage system is desired (database, XML, pcap, syslog,
> > etc).  Once this program is ready I will go into more detail on using
> > the integrated system to enable higer performance implementations of
> > Snort.
> >
> > That's it for right now.  If you're feeling sufficiently motivated,
> > please check out Snort version 1.8-beta8 (build 29) from CVS and test it
> > out for performance, memory leaks, and stability.  If all goes well,
> > this is the codebase that 1.8 will be based on.
> >
> >      -Marty
> >
> > --
> > Martin Roesch
> > roesch@sourcefire.com
> > http://www.sourcefire.com - http://www.snort.org
> >
> > _______________________________________________
> > Snort-users mailing list
> > Snort-users@lists.sourceforge.net
> > Go to this URL to change user options or unsubscribe:
> > http://lists.sourceforge.net/lists/listinfo/snort-users
> > Snort-users list archive:
> > http://www.geocrawler.com/redir-sf.php3?list=snort-users
> >
> > _______________________________________________
> > Snort-users mailing list
> > Snort-users@lists.sourceforge.net
> > Go to this URL to change user options or unsubscribe:
> > http://lists.sourceforge.net/lists/listinfo/snort-users
> > Snort-users list archive:
> > http://www.geocrawler.com/redir-sf.php3?list=snort-users
> 
> --
> Martin Roesch
> roesch@sourcefire.com
> http://www.sourcefire.com - http://www.snort.org
> 
> _______________________________________________
> Snort-users mailing list
> Snort-users@lists.sourceforge.net
> Go to this URL to change user options or unsubscribe:
> http://lists.sourceforge.net/lists/listinfo/snort-users
> Snort-users list archive:
> http://www.geocrawler.com/redir-sf.php3?list=snort-users

--
Martin Roesch
roesch@sourcefire.com
http://www.sourcefire.com - http://www.snort.org

_______________________________________________
Snort-devel mailing list
Snort-devel@lists.sourceforge.net
http://lists.sourceforge.net/lists/listinfo/snort-devel

[prev in list] [next in list] [prev in thread] [next in thread]
Configure | About | News | Add a list | Sponsored by KoreLogic