[prev in list] [next in list] [prev in thread] [next in thread]
List: snort-devel
Subject: Re: [Snort-devel] Snort Segfault
From: Patrick Mullen <pmullen () sourcefire ! com>
Date: 2014-10-07 14:41:50
Message-ID: CAMhPpEU5d4hqHr3sV7Hf992C5UnZbd0QVfaDHAJC2FrxTERRKA () mail ! gmail ! com
[Download RAW message or body]
[Attachment #2 (multipart/alternative)]
Peter,
Thank you for the report. I believe I have identified and corrected the
issue. It should be released tomorrow (08 October 2014), so let me know if
you continue to have problems after the update.
In the mean time, you can disable detection for gid 3, sid 13667 and the
crashes should go away.
Thanks,
~Patrick
On Mon, Oct 6, 2014 at 8:48 PM, Peter Fyon <peter.fyon@gmail.com> wrote:
> Looks like there's an issue with the precompiled protocol-dns.so rules
> under ubuntu 14.04. Probably not a snort-devel related problem.
>
> Peter
> On Oct 6, 2014 5:10 PM, "Peter Fyon" <peter.fyon@gmail.com> wrote:
>
>> Hey list,
>>
>> I recently found my snort setup was segfaulting randomly (although, until
>> I ran it through strace, I didn't see the segfault message come out
>> anywhere).
>>
>> My snort box is sitting on a SPAN port with a single interface using the
>> nfq DAQ. It would run through an unknown number of packets and die. I'm
>> going to leave all the debug stuff at the bottom of this email for clarity,
>> but is this a netfilter issue or something to do with snort, or am I seeing
>> some malformed packets (I see a dns rule near the top of the backtrace)
>> causing the crash?
>>
>>
>> gdb output from a crash:
>>
>> <snip>
>> Commencing packet processing (pid=12856)
>> Decoding Raw IP4
>>
>> Program received signal SIGSEGV, Segmentation fault.
>> 0xb7d9812b in checksum () from
>> /usr/lib/i386-linux-gnu/libnetfilter_queue.so.1
>> (gdb) bt
>> #0 0xb7d9812b in checksum () from
>> /usr/lib/i386-linux-gnu/libnetfilter_queue.so.1
>> #1 0xb7387195 in rule13667eval (p=0x875c7a0 <s_packet>) at
>> protocol-dns_kb945553-dns-cache-poison.c:313
>> #2 0xb7430e8f in CheckRule (p=0x875c7a0 <s_packet>, r=0xb73ae620
>> <rule13667>) at sf_snort_detection_engine.c:148
>> #3 0x080b713d in DynamicCheck (option_data=0xc884548, p=0x875c7a0
>> <s_packet>) at sp_dynamic.c:261
>> #4 0x0809ffec in detection_option_node_evaluate (node=0xc8cf770,
>> eval_data=eval_data@entry=0xbffff210) at detection_options.c:1140
>> #5 0x0808b30f in detection_option_tree_evaluate (root=0xc8ac790,
>> eval_data=eval_data@entry=0xbffff210) at fpdetect.c:580
>> #6 0x0808b778 in fpEvalHeaderSW (port_group=0xc8ac710, p=0x875c7a0
>> <s_packet>, check_ports=<optimized out>, ip_rule=0 '\000', omd=0x8ecc628)
>> at fpdetect.c:1341
>> #7 0x0808d0f5 in fpEvalHeaderUdp (omd=0x8ecc628, p=0x875c7a0 <s_packet>)
>> at fpdetect.c:1458
>> #8 fpEvalPacket (p=p@entry=0x875c7a0 <s_packet>) at fpdetect.c:1708
>> #9 0x08084b52 in Detect (p=p@entry=0x875c7a0 <s_packet>) at detect.c:523
>> #10 0x080852a9 in Preprocess (p=p@entry=0x875c7a0 <s_packet>) at
>> detect.c:247
>> #11 0x08078f28 in ProcessPacket (p=p@entry=0x875c7a0 <s_packet>,
>> pkthdr=pkthdr@entry=0xbffff410, pkt=pkt@entry=0x1381d460 "E", ft=ft@entry=0x0)
>> at snort.c:1867
>> #12 0x0807b0fc in PacketCallback (user=0x0, pkthdr=0xbffff410,
>> pkt=0x1381d460 "E") at snort.c:1704
>> #13 0x0812f197 in daq_nfq_callback (qh=0x120e2700, nfmsg=0x1381d420,
>> nfad=0xbffff47c, data=0xc8b3f40) at daq_nfq.c:455
>> #14 0xb7d967e3 in ?? () from
>> /usr/lib/i386-linux-gnu/libnetfilter_queue.so.1
>> #15 0xb7b37484 in nfnl_handle_packet () from
>> /usr/lib/i386-linux-gnu/libnfnetlink.so.0
>> #16 0xb7d96ded in nfq_handle_packet () from
>> /usr/lib/i386-linux-gnu/libnetfilter_queue.so.1
>> #17 0x0812f01d in nfq_daq_acquire (handle=0xc8b3f40, c=0,
>> callback=0x807afc0 <PacketCallback>, metaback=0x0, user=0x0) at
>> daq_nfq.c:530
>> #18 0x0809818b in DAQ_Acquire (max=max@entry=0, callback=callback@entry=0x807afc0
>> <PacketCallback>, user=user@entry=0x0) at sfdaq.c:540
>> #19 0x0807cea1 in PacketLoop () at snort.c:3210
>> #20 SnortMain (argc=7, argv=argv@entry=0xbffff7c4) at snort.c:907
>> #21 0x0804c0b6 in main (argc=7, argv=0xbffff7c4) at snort.c:807
>> (gdb)
>>
>>
>>
>> It looked like it ran only through a couple hundred packets before
>> crashing:
>>
>> root@server:/proc/net/netfilter# cat nfnetlink_queue
>> 0 12856 29 2 65531 0 0 58 1
>>
>> (a previous run it processed at least 152, I wasn't as diligent cat'ing
>> nfnetlink_queue this time)
>>
>> Output from a crash through strace:
>>
>>
>> select(4, [3], NULL, NULL, {1, 0}) = 1 (in [3], left {0, 999984})
>> recv(3,
>> "\223\0\0\0\0\3\0\0\0\0\0\0\0\0\0\0\2\0\0\0\v\0\1\0\0\0\0\234\10\0\0\0"...,
>> 66047, 0) = 147
>> sendmsg(3, {msg_name(12)={sa_family=AF_NETLINK, pid=0, groups=00000000},
>> msg_iov(1)=[{"
>> \0\0\0\1\3\1\0\0\0\0\0\0\0\0\0\0\0\0\0\f\0\2\0\0\0\0\1\0\0\0\234", 32}],
>> msg_controllen=0, msg_flags=0}, 0) = 32
>> select(4, [3], NULL, NULL, {1, 0}) = 1 (in [3], left {0, 983779})
>> recv(3,
>> "\372\0\0\0\0\3\0\0\0\0\0\0\0\0\0\0\2\0\0\0\v\0\1\0\0\0\0\235\10\0\0\0"...,
>> 66047, 0) = 250
>> --- SIGSEGV {si_signo=SIGSEGV, si_code=SEGV_MAPERR, si_addr=0x8c} ---
>> rt_sigaction(SIGSEGV, {SIG_DFL, [], 0}, {0x807de00, [], 0}, 8) = 0
>> tgkill(12422, 12422, SIGSEGV) = 0
>> sigreturn() (mask []) = 13667
>> --- SIGSEGV {si_signo=SIGSEGV, si_code=SI_TKILL, si_pid=12422, si_uid=0}
>> ---
>> +++ killed by SIGSEGV (core dumped) +++
>>
>>
>> Hope someone can help, it's super annoying having snort crash all the
>> time.
>>
>> Peter
>>
>>
>
> ------------------------------------------------------------------------------
> Meet PCI DSS 3.0 Compliance Requirements with EventLog Analyzer
> Achieve PCI DSS 3.0 Compliant Status with Out-of-the-box PCI DSS Reports
> Are you Audit-Ready for PCI DSS 3.0 Compliance? Download White paper
> Comply to PCI DSS 3.0 Requirement 10 and 11.5 with EventLog Analyzer
>
> http://pubads.g.doubleclick.net/gampad/clk?id=154622311&iu=/4140/ostg.clktrk
> _______________________________________________
> Snort-devel mailing list
> Snort-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/snort-devel
> Archive:
> http://sourceforge.net/mailarchive/forum.php?forum_name=snort-devel
>
> Please visit http://blog.snort.org for the latest news about Snort!
>
--
Patrick Mullen
Response Research Manager
Sourcefire VRT
[Attachment #5 (text/html)]
<div dir="ltr">Peter,<div><br></div><div>Thank you for the report. I believe I have \
identified and corrected the issue. It should be released tomorrow (08 October \
2014), so let me know if you continue to have problems after the \
update.</div><div><br></div><div>In the mean time, you can disable detection for gid \
3, sid 13667 and the crashes should go \
away.</div><div><br></div><div><br></div><div>Thanks,</div><div><br></div><div>~Patrick</div><div><br></div></div><div \
class="gmail_extra"><br><div class="gmail_quote">On Mon, Oct 6, 2014 at 8:48 PM, \
Peter Fyon <span dir="ltr"><<a href="mailto:peter.fyon@gmail.com" \
target="_blank">peter.fyon@gmail.com</a>></span> wrote:<br><blockquote \
class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc \
solid;padding-left:1ex"><p dir="ltr">Looks like there's an issue with the \
precompiled protocol-dns.so rules under ubuntu 14.04. Probably not a snort-devel \
related problem. </p><span class="HOEnZb"><font color="#888888"> <p \
dir="ltr">Peter</p></font></span><div class="HOEnZb"><div class="h5"> <div \
class="gmail_quote">On Oct 6, 2014 5:10 PM, "Peter Fyon" <<a \
href="mailto:peter.fyon@gmail.com" target="_blank">peter.fyon@gmail.com</a>> \
wrote:<br type="attribution"><blockquote class="gmail_quote" style="margin:0 0 0 \
.8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div>Hey \
list,</div><div><br></div><div>I recently found my snort setup was segfaulting \
randomly (although, until I ran it through strace, I didn't see the segfault \
message come out anywhere).</div><div><br></div><div>My snort box is sitting on a \
SPAN port with a single interface using the nfq DAQ. It would run through an unknown \
number of packets and die. I'm going to leave all the debug stuff at the bottom \
of this email for clarity, but is this a netfilter issue or something to do with \
snort, or am I seeing some malformed packets (I see a dns rule near the top of the \
backtrace) causing the crash?</div><div><br></div><div><br></div><div>gdb output from \
a crash:</div><div><br></div><div><snip></div><div>Commencing packet processing \
(pid=12856)</div><div>Decoding Raw IP4</div><div><br></div><div>Program received \
signal SIGSEGV, Segmentation fault.</div><div>0xb7d9812b in checksum () from \
/usr/lib/i386-linux-gnu/libnetfilter_queue.so.1</div><div>(gdb) bt</div><div>#0 \
0xb7d9812b in checksum () from \
/usr/lib/i386-linux-gnu/libnetfilter_queue.so.1</div><div>#1 0xb7387195 in \
rule13667eval (p=0x875c7a0 <s_packet>) at \
protocol-dns_kb945553-dns-cache-poison.c:313</div><div>#2 0xb7430e8f in CheckRule \
(p=0x875c7a0 <s_packet>, r=0xb73ae620 <rule13667>) at \
sf_snort_detection_engine.c:148</div><div>#3 0x080b713d in DynamicCheck \
(option_data=0xc884548, p=0x875c7a0 <s_packet>) at \
sp_dynamic.c:261</div><div>#4 0x0809ffec in detection_option_node_evaluate \
(node=0xc8cf770, eval_data=eval_data@entry=0xbffff210) at \
detection_options.c:1140</div><div>#5 0x0808b30f in detection_option_tree_evaluate \
(root=0xc8ac790, eval_data=eval_data@entry=0xbffff210) at fpdetect.c:580</div><div>#6 \
0x0808b778 in fpEvalHeaderSW (port_group=0xc8ac710, p=0x875c7a0 <s_packet>, \
check_ports=<optimized out>, ip_rule=0 '\000', omd=0x8ecc628) at \
fpdetect.c:1341</div><div>#7 0x0808d0f5 in fpEvalHeaderUdp (omd=0x8ecc628, \
p=0x875c7a0 <s_packet>) at fpdetect.c:1458</div><div>#8 fpEvalPacket \
(p=p@entry=0x875c7a0 <s_packet>) at fpdetect.c:1708</div><div>#9 0x08084b52 \
in Detect (p=p@entry=0x875c7a0 <s_packet>) at detect.c:523</div><div>#10 \
0x080852a9 in Preprocess (p=p@entry=0x875c7a0 <s_packet>) at \
detect.c:247</div><div>#11 0x08078f28 in ProcessPacket (p=p@entry=0x875c7a0 \
<s_packet>, pkthdr=pkthdr@entry=0xbffff410, pkt=pkt@entry=0x1381d460 \
"E", ft=ft@entry=0x0) at snort.c:1867</div><div>#12 0x0807b0fc in \
PacketCallback (user=0x0, pkthdr=0xbffff410, pkt=0x1381d460 "E") at \
snort.c:1704</div><div>#13 0x0812f197 in daq_nfq_callback (qh=0x120e2700, \
nfmsg=0x1381d420, nfad=0xbffff47c, data=0xc8b3f40) at daq_nfq.c:455</div><div>#14 \
0xb7d967e3 in ?? () from \
/usr/lib/i386-linux-gnu/libnetfilter_queue.so.1</div><div>#15 0xb7b37484 in \
nfnl_handle_packet () from /usr/lib/i386-linux-gnu/libnfnetlink.so.0</div><div>#16 \
0xb7d96ded in nfq_handle_packet () from \
/usr/lib/i386-linux-gnu/libnetfilter_queue.so.1</div><div>#17 0x0812f01d in \
nfq_daq_acquire (handle=0xc8b3f40, c=0, callback=0x807afc0 <PacketCallback>, \
metaback=0x0, user=0x0) at daq_nfq.c:530</div><div>#18 0x0809818b in DAQ_Acquire \
(max=max@entry=0, callback=callback@entry=0x807afc0 <PacketCallback>, \
user=user@entry=0x0) at sfdaq.c:540</div><div>#19 0x0807cea1 in PacketLoop () at \
snort.c:3210</div><div>#20 SnortMain (argc=7, argv=argv@entry=0xbffff7c4) at \
snort.c:907</div><div>#21 0x0804c0b6 in main (argc=7, argv=0xbffff7c4) at \
snort.c:807</div><div>(gdb)</div><div><br></div><div><br></div><div><br></div><div>It \
looked like it ran only through a couple hundred packets before \
crashing:</div><div><br></div><div><div>root@server:/proc/net/netfilter# cat \
nfnetlink_queue </div><div> 0 12856 29 2 65531 0 0 \
58 1</div></div><div><br></div><div>(a previous run it processed at least 152, I \
wasn't as diligent cat'ing nfnetlink_queue this \
time)</div><div><br></div><div>Output from a crash through strace: \
</div><div><br></div><div><br></div><div><div>select(4, [3], NULL, NULL, {1, 0}) \
= 1 (in [3], left {0, 999984})</div><div>recv(3, \
"\223\0\0\0\0\3\0\0\0\0\0\0\0\0\0\0\2\0\0\0\v\0\1\0\0\0\0\234\10\0\0\0"..., \
66047, 0) = 147</div><div>sendmsg(3, {msg_name(12)={sa_family=AF_NETLINK, pid=0, \
groups=00000000}, msg_iov(1)=[{" \
\0\0\0\1\3\1\0\0\0\0\0\0\0\0\0\0\0\0\0\f\0\2\0\0\0\0\1\0\0\0\234", 32}], \
msg_controllen=0, msg_flags=0}, 0) = 32</div><div>select(4, [3], NULL, NULL, {1, 0}) \
= 1 (in [3], left {0, 983779})</div><div>recv(3, \
"\372\0\0\0\0\3\0\0\0\0\0\0\0\0\0\0\2\0\0\0\v\0\1\0\0\0\0\235\10\0\0\0"..., \
66047, 0) = 250</div><div>--- SIGSEGV {si_signo=SIGSEGV, si_code=SEGV_MAPERR, \
si_addr=0x8c} ---</div><div>rt_sigaction(SIGSEGV, {SIG_DFL, [], 0}, {0x807de00, [], \
0}, 8) = 0</div><div>tgkill(12422, 12422, SIGSEGV) = \
0</div><div>sigreturn() (mask []) = 13667</div><div>--- \
SIGSEGV {si_signo=SIGSEGV, si_code=SI_TKILL, si_pid=12422, si_uid=0} \
---</div><div>+++ killed by SIGSEGV (core dumped) \
+++</div></div><div><br></div><div><br></div><div>Hope someone can help, it's \
super annoying having snort crash all the \
time.</div><div><br></div><div>Peter</div><div><br></div></div> </blockquote></div>
</div></div><br>------------------------------------------------------------------------------<br>
Meet PCI DSS 3.0 Compliance Requirements with EventLog Analyzer<br>
Achieve PCI DSS 3.0 Compliant Status with Out-of-the-box PCI DSS Reports<br>
Are you Audit-Ready for PCI DSS 3.0 Compliance? Download White paper<br>
Comply to PCI DSS 3.0 Requirement 10 and 11.5 with EventLog Analyzer<br>
<a href="http://pubads.g.doubleclick.net/gampad/clk?id=154622311&iu=/4140/ostg.clktrk" \
target="_blank">http://pubads.g.doubleclick.net/gampad/clk?id=154622311&iu=/4140/ostg.clktrk</a><br>_______________________________________________<br>
Snort-devel mailing list<br>
<a href="mailto:Snort-devel@lists.sourceforge.net">Snort-devel@lists.sourceforge.net</a><br>
<a href="https://lists.sourceforge.net/lists/listinfo/snort-devel" \
target="_blank">https://lists.sourceforge.net/lists/listinfo/snort-devel</a><br> \
Archive:<br> <a href="http://sourceforge.net/mailarchive/forum.php?forum_name=snort-devel" \
target="_blank">http://sourceforge.net/mailarchive/forum.php?forum_name=snort-devel</a><br>
<br>
Please visit <a href="http://blog.snort.org" \
target="_blank">http://blog.snort.org</a> for the latest news about \
Snort!<br></blockquote></div><br><br clear="all"><div><br></div>-- <br>Patrick \
Mullen<br>Response Research Manager<br>Sourcefire VRT </div>
------------------------------------------------------------------------------
Meet PCI DSS 3.0 Compliance Requirements with EventLog Analyzer
Achieve PCI DSS 3.0 Compliant Status with Out-of-the-box PCI DSS Reports
Are you Audit-Ready for PCI DSS 3.0 Compliance? Download White paper
Comply to PCI DSS 3.0 Requirement 10 and 11.5 with EventLog Analyzer
http://pubads.g.doubleclick.net/gampad/clk?id=154622311&iu=/4140/ostg.clktrk
_______________________________________________
Snort-devel mailing list
Snort-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/snort-devel
Archive:
http://sourceforge.net/mailarchive/forum.php?forum_name=snort-devel
Please visit http://blog.snort.org for the latest news about Snort!
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic