[prev in list] [next in list] [prev in thread] [next in thread]
List: syslog-ng
Subject: Re: [syslog-ng] syslog-ng parsing Error
From: Scheidler, Balázs <balazs.scheidler () balabit ! com>
Date: 2018-06-25 13:53:59
Message-ID: CANWQT2OzbE3co3Riot1ebmJVjgR4dr_8jxMuXWPLJq7MSxb4-w () mail ! gmail ! com
[Download RAW message or body]
[Attachment #2 (multipart/alternative)]
Hi,
* syslog(transport(udp)) expects RFC5424 format, and in versions past 3.3,
it accepts both RFC5424 and RFC3164.
* network(transport(udp)) expects RFC3164, can be enabled to accept RFC5424
by adding flags(syslog-protocol) to the options
Also the two differ in framing of TCP and TLS transports, but are the same
in UDP.
--
Bazsi
On Fri, Jun 22, 2018 at 10:29 PM, David Campeau <David.Campeau@tn.gov>
wrote:
> Looks like messaged are being properly filtered now. I substituted
> "syslog" with "network", and the parsing errors went away. However, I'm
> not sure of the implications of this change? Network() source options vs.
> syslog() source options.
>
>
>
> source s_network {
>
> ## syslog(transport("udp") port(528));
>
> network(transport("udp") port(528));
>
>
>
>
>
>
>
> *From:* syslog-ng [mailto:syslog-ng-bounces@lists.balabit.hu] *On Behalf
> Of *David Campeau
> *Sent:* Friday, June 22, 2018 3:04 PM
>
> *To:* Syslog-ng users' and developers' mailing list
> *Subject:* Re: [syslog-ng] syslog-ng parsing Error
>
>
>
> Thank you for the response.
>
>
>
> This is how the source is set up and is listening. It is expecting UDP on
> port 528. You mentioned syslog(), but does my example need to be tweaked
> in some way?
>
>
>
> source s_network {
>
> syslog(transport("udp") port(528));
>
>
>
>
>
> Best Regards,
>
>
>
>
>
> *From:* syslog-ng [mailto:syslog-ng-bounces@lists.balabit.hu] *On Behalf
> Of *Scheidler, Balázs
> *Sent:* Friday, June 22, 2018 12:44 AM
> *To:* Syslog-ng users' and developers' mailing list
> *Subject:* Re: [syslog-ng] syslog-ng parsing Error
>
>
>
>
>
>
>
> On Jun 21, 2018 18:11, "David Campeau" <David.Campeau@tn.gov> wrote:
>
> Hello,
>
>
>
> I have a syslog source node sending syslogs, and they are being generated
> via a python script, and is using Python Rfc5426SysLogHandler. So, these
> syslog messages should be RFC compliant. However, syslog-ng does prepend
> an error message before sending it on to be put into storage.
>
>
>
> Example error message from syslog-ng = <43>Jun 21 10:27:38 *syslog-ng-Server
> syslog-ng[2559]: **Error processing log message:* xxxxx timestamp,
> source hostname and payload follows.
>
>
>
> I've done some googling, but haven't been able to find out what error 2559
> means.
>
>
>
> 2559 is the pid of the syslog-ng process.
>
>
>
> Any thoughts of what I might do to determine what syslog-ng isn't liking
> about the syslog it is receiving? I need to relay this information to a
> developer so they can make adjustments to the python script.
>
> After the colon the original message is reproduced verbatim, but as far as
> I understand you changed that so judging why parsing failed is not possible.
>
>
>
> One usual suspect is that you are using legacy bsd style source, wheras
> your message is in the 5424 format.
>
>
>
> Using the syslog() source instead of tcp/udp can help.
>
>
>
> Hope this helps.
>
>
>
> Best regards,
>
>
>
> David
>
>
>
>
>
>
>
>
>
>
>
>
> ____________________________________________________________
> __________________
> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> Documentation: http://www.balabit.com/support/documentation/?
> product=syslog-ng
> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>
>
>
> ____________________________________________________________
> __________________
> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> Documentation: http://www.balabit.com/support/documentation/?
> product=syslog-ng
> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>
>
>
[Attachment #5 (text/html)]
<div dir="ltr"><div>Hi,</div><div><br></div><div>* syslog(transport(udp)) expects \
RFC5424 format, and in versions past 3.3, it accepts both RFC5424 and \
RFC3164.</div><div></div><div>* network(transport(udp)) expects RFC3164, can be \
enabled to accept RFC5424 by adding flags(syslog-protocol) to the \
options<br></div><div><br></div><div>Also the two differ in framing of TCP and TLS \
transports, but are the same in UDP.<br></div></div><div class="gmail_extra"><br \
clear="all"><div><div class="gmail_signature" data-smartmail="gmail_signature"><div \
dir="ltr">-- <br>Bazsi<br></div></div></div> <br><div class="gmail_quote">On Fri, Jun \
22, 2018 at 10:29 PM, David Campeau <span dir="ltr"><<a \
href="mailto:David.Campeau@tn.gov" \
target="_blank">David.Campeau@tn.gov</a>></span> wrote:<br><blockquote \
class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc \
solid;padding-left:1ex">
<div link="blue" vlink="purple" lang="EN-US">
<div class="m_8255836174536426610WordSection1">
<p class="MsoNormal"><span \
style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Looks \
like messaged are being properly filtered now. I substituted "syslog" with \
"network", and the parsing errors went away. However, I'm not sure of the \
implications of this change? Network() source options vs. syslog() source options. \
<u></u><u></u></span></p> <p class="MsoNormal"><span \
style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> \
<u></u></span></p> <p class="MsoNormal"><span \
style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"> \
source s_network {<u></u><u></u></span></p> <p class="MsoNormal"><span \
style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">## \
syslog(transport("udp") port(528));<u></u><u></u></span></p> <p \
class="MsoNormal"><span \
style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"> \
network(transport("udp") port(528));<u></u><u></u></span></p> <p \
class="MsoNormal"><span \
style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> \
<u></u></span></p> <p class="MsoNormal"><span \
style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> \
<u></u></span></p> <p class="MsoNormal"><span \
style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> \
<u></u></span></p> <div>
<div style="border:none;border-top:solid #b5c4df 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span \
style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">From:</span></b><span \
style="font-size:10.0pt;font-family:"Tahoma","sans-serif""> \
syslog-ng [mailto:<a href="mailto:syslog-ng-bounces@lists.balabit.hu" \
target="_blank">syslog-ng-bounces@<wbr>lists.balabit.hu</a>] <b>On Behalf Of \
</b>David Campeau<br> <b>Sent:</b> Friday, June 22, 2018 3:04 PM</span></p><div><div \
class="h5"><br> <b>To:</b> Syslog-ng users' and developers' mailing list<br>
<b>Subject:</b> Re: [syslog-ng] syslog-ng parsing \
Error<u></u><u></u></div></div><p></p> </div>
</div><div><div class="h5">
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal"><span \
style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Thank \
you for the response.<u></u><u></u></span></p> <p class="MsoNormal"><span \
style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> \
<u></u></span></p> <p class="MsoNormal"><span \
style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">This \
is how the source is set up and is listening. It is expecting UDP on port 528. \
You mentioned </span>syslog(), but does my example need to be tweaked in some \
way?<span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u><u></u></span></p>
<p class="MsoNormal"><span \
style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> \
<u></u></span></p> <p class="MsoNormal"><span \
style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"> \
source s_network {<u></u><u></u></span></p> <p class="MsoNormal"><span \
style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"> \
syslog(transport("udp") port(528));<u></u><u></u></span></p> <p \
class="MsoNormal"><span \
style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> \
<u></u></span></p> <p class="MsoNormal"><span \
style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> \
<u></u></span></p> <p class="MsoNormal"><span \
style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Best \
Regards,<u></u><u></u></span></p> <p class="MsoNormal"><span \
style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> \
<u></u></span></p> <p class="MsoNormal"><span \
style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> \
<u></u></span></p> <p class="MsoNormal"><b><span \
style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">From:</span></b><span \
style="font-size:10.0pt;font-family:"Tahoma","sans-serif""> \
syslog-ng [mailto:<a href="mailto:syslog-ng-bounces@lists.balabit.hu" \
target="_blank">syslog-ng-bounces@<wbr>lists.balabit.hu</a>] <b>On Behalf Of \
</b>Scheidler, Balázs<br> <b>Sent:</b> Friday, June 22, 2018 12:44 AM<br>
<b>To:</b> Syslog-ng users' and developers' mailing list<br>
<b>Subject:</b> Re: [syslog-ng] syslog-ng parsing Error<u></u><u></u></span></p>
<p class="MsoNormal"><u></u> <u></u></p>
<div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
<div>
<p class="MsoNormal">On Jun 21, 2018 18:11, "David Campeau" <<a \
href="mailto:David.Campeau@tn.gov" target="_blank">David.Campeau@tn.gov</a>> \
wrote:<u></u><u></u></p> <div>
<div>
<p class="MsoNormal">Hello,<u></u><u></u></p>
<p class="MsoNormal"> <u></u><u></u></p>
<p class="MsoNormal">I have a syslog source node sending syslogs, and they are being \
generated via a python script, and is using Python <span \
style="color:black">Rfc5426SysLogHandler. So, these syslog messages should be RFC \
compliant. However, syslog-ng does prepend an error message before sending it on to \
be put into storage. </span><u></u><u></u></p>
<p class="MsoNormal"><span style="color:black"> </span><u></u><u></u></p>
<p class="MsoNormal"><span style="color:black">Example error message from syslog-ng = \
</span><span style="font-size:9.0pt;font-family:"Courier \
New";color:black;background:white"><43>Jun 21 10:27:38 </span><b><span \
style="font-size:9.0pt;font-family:"Courier \
New";color:#953735;background:white">syslog-ng-<wbr>Server syslog-ng[2559]: \
</span></b><b><span style="font-size:10.0pt;font-family:"Courier \
New";color:#953735;background:white">Error <wbr>processing log \
message:</span></b><span style="font-size:9.0pt;font-family:"Courier \
New";color:#953735;background:white"> </span><span \
style="font-size:9.0pt;font-family:"Courier \
New";color:black;background:white">xxxxx timestamp, source hostname and payload \
follows. </span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:9.0pt;font-family:"Courier \
New";color:black;background:white"> </span><u></u><u></u></p> <p \
class="MsoNormal"><span style="color:black;background:white">I've done some googling, \
but haven't been able to find out what error 2559 means.</span><u></u><u></u></p> <p \
class="MsoNormal"><span style="color:black;background:white"> \
</span><u></u><u></u></p> </div>
</div>
</div>
</div>
</div>
<div>
<p class="MsoNormal">2559 is the pid of the syslog-ng process.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<div>
<div>
<blockquote style="border:none;border-left:solid #cccccc 1.0pt;padding:0in 0in 0in \
6.0pt;margin-left:4.8pt;margin-top:5.0pt;margin-right:0in;margin-bottom:5.0pt"> <div>
<div>
<p class="MsoNormal"><span style="color:black;background:white">Any thoughts of what \
I might do to determine what syslog-ng isn't liking about the syslog it is receiving? \
I need to relay this information to a developer so they can make adjustments to the \
python script.</span><u></u><u></u></p> </div>
</div>
</blockquote>
</div>
</div>
</div>
<div>
<p class="MsoNormal">After the colon the original message is reproduced verbatim, but \
as far as I understand you changed that so judging why parsing failed is not \
possible.<u></u><u></u></p> </div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">One usual suspect is that you are using legacy bsd style source, \
wheras your message is in the 5424 format.<u></u><u></u></p> </div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">Using the syslog() source instead of tcp/udp can \
help.<u></u><u></u></p> </div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">Hope this helps.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<div>
<div>
<blockquote style="border:none;border-left:solid #cccccc 1.0pt;padding:0in 0in 0in \
6.0pt;margin-left:4.8pt;margin-top:5.0pt;margin-right:0in;margin-bottom:5.0pt"> <div>
<div>
<p class="MsoNormal"><span style="color:black;background:white">Best \
regards,</span><u></u><u></u></p> <p class="MsoNormal"><span \
style="color:black;background:white"> </span><span \
style="color:#888888"><u></u><u></u></span></p> <p class="MsoNormal"><span \
style="color:black;background:white">David</span><span \
style="color:#888888"><u></u><u></u></span></p> <p class="MsoNormal"><span \
style="font-size:9.0pt;font-family:"Courier \
New";color:black;background:white"> </span><span \
style="color:#888888"><u></u><u></u></span></p> <p class="MsoNormal"><span \
style="font-size:9.0pt;font-family:"Courier \
New";color:black;background:white"> </span><span \
style="color:#888888"><u></u><u></u></span></p> <p class="MsoNormal"><span \
style="color:black"> </span><span style="color:#888888"><u></u><u></u></span></p> <p \
class="MsoNormal"><span style="color:black"> </span><span \
style="color:#888888"><u></u><u></u></span></p> <p class="MsoNormal"><span \
style="color:#888888"> <u></u><u></u></span></p> </div>
</div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><br>
______________________________<wbr>______________________________<wbr>__________________<br>
Member info: <a href="https://lists.balabit.hu/mailman/listinfo/syslog-ng" \
target="_blank"> https://lists.balabit.hu/<wbr>mailman/listinfo/syslog-ng</a><br>
Documentation: <a href="http://www.balabit.com/support/documentation/?product=syslog-ng" \
target="_blank"> http://www.balabit.com/<wbr>support/documentation/?<wbr>product=syslog-ng</a><br>
FAQ: <a href="http://www.balabit.com/wiki/syslog-ng-faq" \
target="_blank">http://www.balabit.com/wiki/<wbr>syslog-ng-faq</a><u></u><u></u></p> \
</blockquote> </div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
</div>
</div>
</div></div></div>
</div>
<br>______________________________<wbr>______________________________<wbr>__________________<br>
Member info: <a href="https://lists.balabit.hu/mailman/listinfo/syslog-ng" \
rel="noreferrer" target="_blank">https://lists.balabit.hu/<wbr>mailman/listinfo/syslog-ng</a><br>
Documentation: <a href="http://www.balabit.com/support/documentation/?product=syslog-ng" \
rel="noreferrer" target="_blank">http://www.balabit.com/<wbr>support/documentation/?<wbr>product=syslog-ng</a><br>
FAQ: <a href="http://www.balabit.com/wiki/syslog-ng-faq" rel="noreferrer" \
target="_blank">http://www.balabit.com/wiki/<wbr>syslog-ng-faq</a><br> <br>
<br></blockquote></div><br></div>
[Attachment #6 (text/plain)]
______________________________________________________________________________
Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
FAQ: http://www.balabit.com/wiki/syslog-ng-faq
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic