[prev in list] [next in list] [prev in thread] [next in thread]
List: oisf-devel
Subject: Re: [Oisf-devel] how to setup the suricata for extracting the files from ftp protocol and save into
From: zhangqs <zhangqs () act ! buaa ! edu ! cn>
Date: 2018-03-23 4:25:17
Message-ID: 80cf9c27-4b3d-1567-6560-5f0afb753073 () act ! buaa ! edu ! cn
[Download RAW message or body]
[Attachment #2 (multipart/alternative)]
Thanks Victor, I found the code:
TmEcode OutputLoggerLog(ThreadVars *tv,Packet *p,void *thread_data)
{
LoggerThreadStore *thread_store = (LoggerThreadStore *)thread_data;
RootLogger *logger =TAILQ_FIRST(&RootLoggers);
LoggerThreadStoreNode *thread_store_node =TAILQ_FIRST(thread_store);
while (logger && thread_store_node) {
if (logger->LogFunc != NULL) {
logger->LogFunc(tv, p, thread_store_node->thread_data);
}
logger =TAILQ_NEXT(logger, entries);
thread_store_node =TAILQ_NEXT(thread_store_node, entries);
}
return TM_ECODE_OK;
}
> -D
在 2018年03月21日 20:27, Victor Julien 写道:
> On 21-03-18 12:03, zhangqs wrote:
> > Thanks Victor, but i still confuse about how the data write into the
> > disk after FTP parse, where the app-layer-ftp call the
> > logFilestoreLogger? I only find the below relations:
> >
> > LogFilestoreLogger--->LogFilestoreRegister--->OutputRegisterLoggers--->TmModuleLoggerRegister-->RegisterAllModules-->PostConfLoadedSetup-->Main()
> >
> The path is indirect:
>
> If you look at flow-worker.c:FlowWorker you can see that each packet
> goes through the same steps:
>
> 1. flow handle
> 2. tcp tracking/reassembly and app-layer (this includes FTP)
> 3. detect
> 4. outputs by a call to OutputLoggerLog.
>
> The OutputLoggerLog then runs all output modules that are enabled by the
> config.
>
> Cheers,
> Victor
>
>
> >
> > Best regards,
> >
> > Kris
> >
> >
> > 在 2018年03月21日 05:12, Victor Julien 写道:
> > > On 19-03-18 10:34, zhangqs wrote:
> > > > Hi guys,
> > > >
> > > > I have been struggling a few days to the function file extraction, the
> > > > reference doc is:
> > > > http://suricata.readthedocs.io/en/latest/file-extraction/file-extraction.html?highlight=ftp.
> > > >
> > > > The protocol that I want to use is FTP.
> > > > 1) Suricata version is latest that cloned from github.
> > > > 2) I setup the suricata.yaml: file-store.enabled: yes
> > > > 3) I create a rule file hello.rules, its content is:
> > > >
> > > > alert http any any -> any any (msg:"FILE store all"; filestore;
> > > > sid:1; rev:1;)
> > > >
> > > > 4) ./configure --prefix=/usr/ --sysconfdir=/etc/ --localstatedir=/var/
> > > > 5) make && make install
> > > >
> > > > My testing pcap is in the attachment. but I cannot find the
> > > > file(Music.mp3) was extracted and saved into the disk
> > > > (/var/log/suricata/files/).
> > > > Has anybody ever been successful about extraction FTP file into disk?
> > > >
> > > > And then I read the code, and cannot find which code is responsible for
> > > > saving file into the disk?
> > > > I guess the process is:
> > > >
> > > > FTPDataParseRequest-->FTPDataParse-->FileOpenFile|FileAppendData-->StreamingBuffer
> > > >
> > > >
> > > > but the data is still in memory, where is save the StreamingBuffer into
> > > > the disk?
> > > It's stored by the filestore output module. This is defined in
> > > src/log-filestore.c where the main logging function is LogFilestoreLogger
> > >
> > > The API this runs on top of is in output-filedata.c: OutputFiledataLog
> > >
>
[Attachment #5 (text/html)]
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body text="#000000" bgcolor="#FFFFFF">
<p>Thanks Victor, I found the code:</p>
<pre style="background-color:#ffffff;color:#000000;font-family:'DejaVu Sans \
Mono';font-size:11.3pt;"><span style="color:#371f80;">TmEcode \
</span>OutputLoggerLog(<span style="color:#371f80;">ThreadVars </span>*tv, <span \
style="color:#371f80;">Packet </span>*p, <span \
style="color:#000080;font-weight:bold;">void </span>*thread_data) {
<span style="color:#371f80;">LoggerThreadStore </span>*thread_store = (<span \
style="color:#371f80;">LoggerThreadStore </span>*)thread_data; <span \
style="color:#371f80;">RootLogger </span>*logger = <span \
style="color:#1f542e;font-weight:bold;">TAILQ_FIRST</span>(&RootLoggers); <span \
style="color:#371f80;">LoggerThreadStoreNode </span>*thread_store_node = <span \
style="color:#1f542e;font-weight:bold;">TAILQ_FIRST</span>(thread_store); <span \
style="color:#000080;font-weight:bold;">while </span>(logger && \
thread_store_node) {
<span style="color:#000080;font-weight:bold;">if </span>(logger-><span \
style="color:#660e7a;">LogFunc </span>!= NULL) {
logger-><span style="color:#660e7a;">LogFunc</span>(tv, p, \
thread_store_node-><span style="color:#660e7a;">thread_data</span>); }
logger = <span \
style="color:#1f542e;font-weight:bold;">TAILQ_NEXT</span>(logger, \
entries);
thread_store_node = <span \
style="color:#1f542e;font-weight:bold;">TAILQ_NEXT</span>(thread_store_node, \
entries); }
<span style="color:#000080;font-weight:bold;">return </span><span \
style="color:#660e7a;font-weight:bold;font-style:italic;">TM_ECODE_OK</span>; }
<span class="moz-smiley-s5"><span>:-D</span></span>
</pre>
<br>
<div class="moz-cite-prefix">在 2018年03月21日 20:27, Victor Julien \
写道:<br> </div>
<blockquote type="cite"
cite="mid:3aa4dc44-a5d7-5760-fb30-0a183aac7838@inliniac.net">
<pre wrap="">On 21-03-18 12:03, zhangqs wrote:
</pre>
<blockquote type="cite">
<pre wrap="">Thanks Victor, but i still confuse about how the data write into \
the disk after FTP parse, where the app-layer-ftp call the
logFilestoreLogger? I only find the below relations:
LogFilestoreLogger--->LogFilestoreRegister--->OutputRegisterLoggers--->TmModuleLoggerRegister-->RegisterAllModules-->PostConfLoadedSetup-->Main()
</pre>
</blockquote>
<pre wrap="">
The path is indirect:
If you look at flow-worker.c:FlowWorker you can see that each packet
goes through the same steps:
1. flow handle
2. tcp tracking/reassembly and app-layer (this includes FTP)
3. detect
4. outputs by a call to OutputLoggerLog.
The OutputLoggerLog then runs all output modules that are enabled by the
config.
Cheers,
Victor
</pre>
<blockquote type="cite">
<pre wrap="">
Best regards,
Kris
在 2018年03月21日 05:12, Victor Julien 写道:
</pre>
<blockquote type="cite">
<pre wrap="">On 19-03-18 10:34, zhangqs wrote:
</pre>
<blockquote type="cite">
<pre wrap="">Hi guys,
I have been struggling a few days to the function file extraction, the
reference doc is:
<a class="moz-txt-link-freetext" \
href="http://suricata.readthedocs.io/en/latest/file-extraction/file-extraction.html?hi \
ghlight=ftp">http://suricata.readthedocs.io/en/latest/file-extraction/file-extraction.html?highlight=ftp</a>.
The protocol that I want to use is FTP.
1) Suricata version is latest that cloned from github.
2) I setup the suricata.yaml: file-store.enabled: yes
3) I create a rule file hello.rules, its content is:
alert http any any -> any any (msg:"FILE store all"; filestore;
sid:1; rev:1;)
4) ./configure --prefix=/usr/ --sysconfdir=/etc/ --localstatedir=/var/
5) make && make install
My testing pcap is in the attachment. but I cannot find the
file(Music.mp3) was extracted and saved into the disk
(/var/log/suricata/files/).
Has anybody ever been successful about extraction FTP file into disk?
And then I read the code, and cannot find which code is responsible for
saving file into the disk?
I guess the process is:
FTPDataParseRequest-->FTPDataParse-->FileOpenFile|FileAppendData-->StreamingBuffer
but the data is still in memory, where is save the StreamingBuffer into
the disk?
</pre>
</blockquote>
<pre wrap="">It's stored by the filestore output module. This is defined in
src/log-filestore.c where the main logging function is LogFilestoreLogger
The API this runs on top of is in output-filedata.c: OutputFiledataLog
</pre>
</blockquote>
<pre wrap="">
</pre>
</blockquote>
<pre wrap="">
</pre>
</blockquote>
<br>
</body>
</html>
[Attachment #6 (text/plain)]
_______________________________________________
Suricata IDS Devel mailing list: oisf-devel@openinfosecfoundation.org
Site: http://suricata-ids.org | Participate: http://suricata-ids.org/participate/
List: https://lists.openinfosecfoundation.org/mailman/listinfo/oisf-devel
Redmine: https://redmine.openinfosecfoundation.org/
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic