[prev in list] [next in list] [prev in thread] [next in thread]
List: syslog-ng
Subject: Re: [syslog-ng] disk-buffer in elasticsearch2 destination loses messages if docker container is kill
From: Jose Angel Santiago <jasantiago () stratio ! com>
Date: 2018-09-25 6:48:17
Message-ID: CAFP3DQ2TQJcAHGL1Mp8vHj_pgy5WTQgeDCWaqLLLiO9UsOax9Q () mail ! gmail ! com
[Download RAW message or body]
[Attachment #2 (multipart/alternative)]
Nevermind, missing messages were not processed because of a bad syslog-ng
wildcard-file source configuration.
Thank you for your support.
2018-09-24 11:30 GMT+02:00 Jose Angel Santiago <jasantiago@stratio.com>:
> Hi,
>
> I've been playing with different values of log-iw-size & mem-buf-size and
> I got to avoid loss of messages :-)
>
> Bad news is that I'm afraid I've found another case of messages loss, I'll
> create a new thread since it's a completely different scenary.
>
> Thanks.
>
> 2018-09-19 18:36 GMT+02:00 Budai, László <laszlo.budai@oneidentity.com>:
>
>> Hi,
>>
>> ... checking your config I found an interesting thing...
>> You set 'bad' values in your config for mem-buf-size and for
>> disk-buf-size :) (it's not your fault... this is something we should
>> change, this should not be so complicated).
>>
>> If you have one source AND disk-buf-size is as big as it is able to store
>> a log-iw-size number of messages then when a message is received, syslog-ng
>> writes it into the diskbuffer and checks the free size of diskbuf.
>> When free_size < mem-buf-size, then the msg won't be acked, which means
>> that the window size of the source is not incremented back, and when window
>> size reach the 0, it won't receive any logs until a msg in the queue is not
>> ACKd back.
>>
>> The question is now what are the defaults?
>> log-iw-size: 100
>> mem-buf-size: 16M
>>
>> This means that -as you set 16M for diskbuf size- the diskbuffer can
>> contain max. 100 messages, as it won't be acked when inserted into the
>> queue (only when the destination is alive...).
>>
>> I'd increase the diskbuf size in your case: set it to a huge number, if
>> you have enough storage. You can play with setting small mem-buf-size, but
>> that should able to store log-iw-size messages (in bytes, so log-iw-size *
>> mean(msg-size-in-bytes) or something similar).
>>
>> L.
>>
>> On Wed, Sep 19, 2018 at 3:50 PM, Jose Angel Santiago <
>> jasantiago@stratio.com> wrote:
>>
>>> Hi,
>>>
>>> You're right, what I saw on my tcpdump file was TCP ACK.
>>>
>>> I guess I lose so many messages because the TCP buffer is getting bigger
>>> and bigger since syslog-relay queue_length from disk_buffer reaches its
>>> limit. In fact, when this queue is not full (i.e with very low msg/sec
>>> ratio) I lose from 0-5 messages according to my tests.
>>>
>>> Which parameter is the one which would match the queue_length of the
>>> disk_buffer? It seems that it should be *mem-buf-length()* but I'm
>>> using reliable=yes so I shouldn't use that parameter, and I'd like to test
>>> my scenary being able to queue more than 99 messages on disk_buffer.
>>>
>>> Thanks.
>>>
>>> 2018-09-19 13:46 GMT+02:00 Budai, László <laszlo.budai@oneidentity.com>:
>>>
>>>> Hi,
>>>>
>>>> what do you mean under 'syslog-relay returns ACK to syslog-ng agent' ?
>>>> I guess this is TCP level ACK.
>>>>
>>>> And TCP level ACK is not the same as what syslog-ng uses for
>>>> controlling the window size of the LogSources.
>>>>
>>>> It is possible that you have pending data in TCP buffers (maintained by
>>>> the kernel) that is not read by syslog-ng, but they are already ACKd by TCP
>>>> itself...
>>>>
>>>> (Solving this kind of situation requires application level ACK, which
>>>> syslog-ng has only in the commercial edition.)
>>>>
>>>> L.
>>>>
>>>>
>>>> On Wed, Sep 19, 2018 at 9:23 AM, Jose Angel Santiago <
>>>> jasantiago@stratio.com> wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> I've realized that the queue_length of the disk_buffer never goes
>>>>> beyond 99, no matter the ratio message/sec logger process is writing or the
>>>>> value of mem-buf-size or disk-buf-size.
>>>>>
>>>>> Which parameter is the one which would match the queue_length of the
>>>>> disk_buffer? It seems that it should be *mem-buf-length()* but I'm
>>>>> using reliable=yes so I shouldn't use that parameter, and I'd like to test
>>>>> my scenary being able to queue more than 99 messages on disk_buffer.
>>>>>
>>>>> Regards.
>>>>>
>>>>> 2018-09-17 14:33 GMT+02:00 Jose Angel Santiago <jasantiago@stratio.com
>>>>> >:
>>>>>
>>>>>> 3.17.2
>>>>>>
>>>>>> Thanks in advance.
>>>>>>
>>>>>> 2018-09-17 14:31 GMT+02:00 Budai, László <
>>>>>> laszlo.budai@oneidentity.com>:
>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>> I'll try to reproduce this issue in my test environment.
>>>>>>> What syslog-ng version do you have?
>>>>>>>
>>>>>>> L.
>>>>>>>
>>>>>>> On Mon, Sep 17, 2018 at 2:04 PM, Jose Angel Santiago <
>>>>>>> jasantiago@stratio.com> wrote:
>>>>>>>
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>> After repeating the test without TLS between syslog-agent &
>>>>>>>> syslog-relay, with 2229 messages logged with a 20 msgs/sec logging ratio,
>>>>>>>> in debug mode and with two tcpdumps running, these are the conclussions:
>>>>>>>>
>>>>>>>> - Syslog-agent sends every single message (checked with tcpdump
>>>>>>>> file and syslog-agent log file)
>>>>>>>> - Syslog-relay returns ACK for every single message, but sometimes
>>>>>>>> the package contains more than one message (checked with tcpdump file)
>>>>>>>> - Checking the syslog-relay log after killing & starting the docker
>>>>>>>> container, I found the trace 'Reliable disk-buffer state loaded;
>>>>>>>> filename='/syslog-ng-00000.rqf', queue_length='99', size='173844'
>>>>>>>> and the queued messages (from msg 236 to msg 334 are processed). The next
>>>>>>>> message processed is the n 440. and it comes from the active network
>>>>>>>> destination again.
>>>>>>>> - Once all logs have been processed, in elasticsearch (syslog-relay
>>>>>>>> destination) I can find messages from 1 to 334, and messages from 440 to
>>>>>>>> 2229.
>>>>>>>>
>>>>>>>> How is it possible that syslog-relay returns ACK for messages 335
>>>>>>>> to 439 to syslog-agent, but they are not in the queue, nor in the .rqf
>>>>>>>> file? Where did these messages go? I know they are sent before starting to
>>>>>>>> fill syslog-agent disk-buffer file (checked with rqf file from syslog-agent
>>>>>>>> when I kill the syslog-relay container). It seems that there's some kind of
>>>>>>>> race condition that makes those messages are not processed/queued while
>>>>>>>> syslog-relay is being killed.
>>>>>>>>
>>>>>>>> Mi procedure consists of:
>>>>>>>>
>>>>>>>> . Start tcpdump on both syslog-agent & syslog-relay hosts
>>>>>>>> - Start logger process
>>>>>>>> - Once logs are being inserted in elasticsearch, I kill the
>>>>>>>> syslog-relay docker container, wait about 10 seconds, and run a new
>>>>>>>> container (using a mapped volume where .rqf file and .persist file are)
>>>>>>>> - Once logs are being inserted in elasticsearch again, I stop the
>>>>>>>> logger process.
>>>>>>>> - I wait until no log is left to be processed
>>>>>>>>
>>>>>>>> Would you test anything else? I'm running out of ideas.
>>>>>>>>
>>>>>>>> PD: Same test with logger process writing 5 msgs/sec ratio produces
>>>>>>>> from 0 to 5 lost messages.
>>>>>>>>
>>>>>>>> Regards.
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> 2018-09-14 14:05 GMT+02:00 Jose Angel Santiago <
>>>>>>>> jasantiago@stratio.com>:
>>>>>>>>
>>>>>>>>> Hi,
>>>>>>>>>
>>>>>>>>> I've got more accurate information about where are my lost
>>>>>>>>> messages.
>>>>>>>>>
>>>>>>>>> Now I'm using disk-buffer in syslog-agent and syslog-relay, and
>>>>>>>>> I've checked that lost messages are the ones sent by the syslog-agent when
>>>>>>>>> the syslog-relay docker container is being killed. I can see those messages
>>>>>>>>> on syslog-agent log (I've got both agent & relay in debug mode) with its
>>>>>>>>> corresponding "Outgoing message" line, but those messages never reach the
>>>>>>>>> relay.
>>>>>>>>>
>>>>>>>>> Could it be that the relay docker container still returns ACK to
>>>>>>>>> the agent (the agent resolves relay fqdn with a custom DNS) while syslog-ng
>>>>>>>>> process within the container is being stopped? I'm about to test again
>>>>>>>>> using tcpdump to confirm this theory,
>>>>>>>>>
>>>>>>>>> BTW, disk-buffers works ok, sometimes I get some duplicated
>>>>>>>>> messages when restarting the relay but that's not a problem for me. Forget
>>>>>>>>> about my .persist file re-creation theory, it doesn't happen.
>>>>>>>>>
>>>>>>>>> Regards.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> 2018-09-13 18:07 GMT+02:00 Péter, Kókai <
>>>>>>>>> peter.kokai@oneidentity.com>:
>>>>>>>>>
>>>>>>>>>> Hello,
>>>>>>>>>>
>>>>>>>>>> It would not make sense to replace the persist file after
>>>>>>>>>> restart, so it is not something that syslog-ng does. Only if that file is
>>>>>>>>>> corrupted, in that case at startup there should be a log about it, have you
>>>>>>>>>> checked the syslog-ng logs ? (it would be better to enable debug and/or
>>>>>>>>>> verbose logs, and if possible share it with us.)
>>>>>>>>>>
>>>>>>>>>> Could you reproduce the same behavior without docker (if
>>>>>>>>>> possible) ?
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Best Regards,
>>>>>>>>>> Peter Kokai
>>>>>>>>>>
>>>>>>>>>> On Thu, Sep 13, 2018 at 4:44 PM Jose Angel Santiago <
>>>>>>>>>> jasantiago@stratio.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> Hi,
>>>>>>>>>>>
>>>>>>>>>>> I guess I know what is happening, when I start from scratch the
>>>>>>>>>>> docker container, even I provide a persist file and a buffer file within
>>>>>>>>>>> the mapped volume, syslog-ng recreates them so all messages in buffer file
>>>>>>>>>>> which were not processed by the relay are lost.
>>>>>>>>>>>
>>>>>>>>>>> Is there any way to tell syslog-ng to use an already existing
>>>>>>>>>>> .persist file so it doesn't recreate the .rqf file?
>>>>>>>>>>>
>>>>>>>>>>> Regards.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> 2018-09-13 16:23 GMT+02:00 Budai, László <
>>>>>>>>>>> laszlo.budai@oneidentity.com>:
>>>>>>>>>>>
>>>>>>>>>>>> Hi,
>>>>>>>>>>>>
>>>>>>>>>>>> one problem could be if the flush-limit would be greater than
>>>>>>>>>>>> 1... in that case syslog-ng would use a HttpBulkMessageProcessor.
>>>>>>>>>>>> In this case syslog-ng pass the message to the
>>>>>>>>>>>> HttpBulkMessageProcessor and sends back a positive ACK to the LogSource (so
>>>>>>>>>>>> the message is removed from the diskbuffer), and if the dockerimage is
>>>>>>>>>>>> killed, all the messages stored in the HttpBulkMessageProcessor are lost.
>>>>>>>>>>>> But in your case syslog-ng should use the
>>>>>>>>>>>> HttpSingleMessageProcessor... which means that the messages are sent
>>>>>>>>>>>> one-by-one...
>>>>>>>>>>>> Could you check the diskbuffer with the dqtool?
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> L.
>>>>>>>>>>>>
>>>>>>>>>>>> On Thu, Sep 13, 2018 at 3:50 PM, Jose Angel Santiago <
>>>>>>>>>>>> jasantiago@stratio.com> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>
>>>>>>>>>>>>> I'm running syslog-ng (with an elasticsearch2 destination
>>>>>>>>>>>>> configured) within a docker container, and I'm trying to avoid loss of
>>>>>>>>>>>>> messages if I kill the docker container and I start it again.
>>>>>>>>>>>>>
>>>>>>>>>>>>> This is my scenary:
>>>>>>>>>>>>>
>>>>>>>>>>>>> - A service which produces 20 lines of log per second
>>>>>>>>>>>>> - A sislog-ng instance reading from a wildcard-file source
>>>>>>>>>>>>> (but actually it only reads logs from the above service, let's call it
>>>>>>>>>>>>> syslog-agent), which sends all logs to another syslog-ng instance (the one
>>>>>>>>>>>>> running in a docker container, let's call it syslog-relay) though a network
>>>>>>>>>>>>> destination.
>>>>>>>>>>>>> - The syslog-relay sends messages to an elasticsearch
>>>>>>>>>>>>> instance, with following configuration:
>>>>>>>>>>>>>
>>>>>>>>>>>>> options {
>>>>>>>>>>>>> chain-hostnames(no);
>>>>>>>>>>>>> use-dns(no);
>>>>>>>>>>>>> keep-hostname(yes);
>>>>>>>>>>>>> owner("syslog-ng");
>>>>>>>>>>>>> group("stratio");
>>>>>>>>>>>>> perm(0640);
>>>>>>>>>>>>> time-reap(30);
>>>>>>>>>>>>> mark-freq(10);
>>>>>>>>>>>>> stats-freq(0);
>>>>>>>>>>>>> bad-hostname("^gconfd$");
>>>>>>>>>>>>> flush-lines(100);
>>>>>>>>>>>>> log-fifo-size(1000);
>>>>>>>>>>>>> };
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> *destination d_elastic_default_0 { elasticsearch2(
>>>>>>>>>>>>> cluster("myelastic") cluster-url("https://myelastic.logs:9200
>>>>>>>>>>>>> <https://myelastic.logs:9200>") client_mode("https")
>>>>>>>>>>>>> index("default") type("log") flush-limit(1)
>>>>>>>>>>>>> disk-buffer( mem-buf-size(16M)
>>>>>>>>>>>>> disk-buf-size(16M) reliable(yes)
>>>>>>>>>>>>> dir("/syslog-ng/log") ) http-auth-type("clientcert")
>>>>>>>>>>>>> java-keystore-filepath("/etc/syslog-ng/certificates/syslog-relay.jks")
>>>>>>>>>>>>> java-keystore-password("XXXXXX")
>>>>>>>>>>>>> java-truststore-filepath("/etc/syslog-ng/certificates/ca-bundle.jks")
>>>>>>>>>>>>> java-truststore-password("XXXXXXXXXX") );};*
>>>>>>>>>>>>>
>>>>>>>>>>>>> - The dir "/syslog-ng/log" is mapped to a path "/tmp/buffer"
>>>>>>>>>>>>> from the host where the docker container is running, so when I kill the
>>>>>>>>>>>>> docker container, the buffer file is not lost.
>>>>>>>>>>>>> - I've set flush-limit to 1 because I thought that I may lost
>>>>>>>>>>>>> 1 message only as much.
>>>>>>>>>>>>>
>>>>>>>>>>>>> This architecture is working fine (flush-limit=1 makes very
>>>>>>>>>>>>> slow, but for this test is ok), but if I kill the syslog-relay docker
>>>>>>>>>>>>> container, wait 5 to 10 seconds and start it again from scratch, I can see
>>>>>>>>>>>>> that several hundreds of logs are missing in elasticsearch. I check it by
>>>>>>>>>>>>> stopping the logger service and letting syslog-ng agent & relay to finish
>>>>>>>>>>>>> the process enqueued messages.
>>>>>>>>>>>>>
>>>>>>>>>>>>> I can see in the syslog-agent stats that all logs messages
>>>>>>>>>>>>> have been processed, so it seems the problem is on the syslog-relay.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Is this behaviour expected? If so, how can I protect against
>>>>>>>>>>>>> loss of messages in case of a syslog-relay docker container unexpected kill?
>>>>>>>>>>>>>
>>>>>>>>>>>>> Thanks in advance.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> --
>>>>>>>>>>>>>
>>>>>>>>>>>>> | Jose Angel Santiago
>>>>>>>>>>>>>
>>>>>>>>>>>>> [image: Logo_signature2.png] <http://www.stratio.com/>
>>>>>>>>>>>>>
>>>>>>>>>>>>> Vía de las dos Castillas, 33, Ática 4, 3 Planta
>>>>>>>>>>>>>
>>>>>>>>>>>>> 28224 Pozuelo de Alarcón, Madrid, Spain
>>>>>>>>>>>>>
>>>>>>>>>>>>> +34 918 286 473 <+34%20918%2028%2064%2073> | www.stratio.com
>>>>>>>>>>>>> <https://twitter.com/stratiobd>
>>>>>>>>>>>>> <https://www.linkedin.com/company/stratiobd>
>>>>>>>>>>>>> <https://www.youtube.com/c/StratioBD>
>>>>>>>>>>>>>
>>>>>>>>>>>>> ____________________________________________________________
>>>>>>>>>>>>> __________________
>>>>>>>>>>>>> Member info: https://lists.balabit.hu/mailm
>>>>>>>>>>>>> an/listinfo/syslog-ng
>>>>>>>>>>>>> Documentation: http://www.balabit.com/support
>>>>>>>>>>>>> /documentation/?product=syslog-ng
>>>>>>>>>>>>> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> ____________________________________________________________
>>>>>>>>>>>> __________________
>>>>>>>>>>>> Member info: https://lists.balabit.hu/mailm
>>>>>>>>>>>> an/listinfo/syslog-ng
>>>>>>>>>>>> Documentation: http://www.balabit.com/support
>>>>>>>>>>>> /documentation/?product=syslog-ng
>>>>>>>>>>>> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>>
>>>>>>>>>>> | Jose Angel Santiago
>>>>>>>>>>>
>>>>>>>>>>> [image: Logo_signature2.png] <http://www.stratio.com/>
>>>>>>>>>>>
>>>>>>>>>>> Vía de las dos Castillas, 33, Ática 4, 3 Planta
>>>>>>>>>>>
>>>>>>>>>>> 28224 Pozuelo de Alarcón, Madrid, Spain
>>>>>>>>>>>
>>>>>>>>>>> +34 918 286 473 <+34%20918%2028%2064%2073> | www.stratio.com
>>>>>>>>>>> <https://twitter.com/stratiobd>
>>>>>>>>>>> <https://www.linkedin.com/company/stratiobd>
>>>>>>>>>>> <https://www.youtube.com/c/StratioBD>
>>>>>>>>>>> ____________________________________________________________
>>>>>>>>>>> __________________
>>>>>>>>>>> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
>>>>>>>>>>> Documentation: http://www.balabit.com/support
>>>>>>>>>>> /documentation/?product=syslog-ng
>>>>>>>>>>> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>> ____________________________________________________________
>>>>>>>>>> __________________
>>>>>>>>>> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
>>>>>>>>>> Documentation: http://www.balabit.com/support
>>>>>>>>>> /documentation/?product=syslog-ng
>>>>>>>>>> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>>
>>>>>>>>> | Jose Angel Santiago
>>>>>>>>>
>>>>>>>>> [image: Logo_signature2.png] <http://www.stratio.com/>
>>>>>>>>>
>>>>>>>>> Vía de las dos Castillas, 33, Ática 4, 3 Planta
>>>>>>>>>
>>>>>>>>> 28224 Pozuelo de Alarcón, Madrid, Spain
>>>>>>>>>
>>>>>>>>> +34 918 286 473 | www.stratio.com
>>>>>>>>> <https://twitter.com/stratiobd>
>>>>>>>>> <https://www.linkedin.com/company/stratiobd>
>>>>>>>>> <https://www.youtube.com/c/StratioBD>
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>>
>>>>>>>> | Jose Angel Santiago
>>>>>>>>
>>>>>>>> [image: Logo_signature2.png] <http://www.stratio.com/>
>>>>>>>>
>>>>>>>> Vía de las dos Castillas, 33, Ática 4, 3 Planta
>>>>>>>>
>>>>>>>> 28224 Pozuelo de Alarcón, Madrid, Spain
>>>>>>>>
>>>>>>>> +34 918 286 473 | www.stratio.com
>>>>>>>> <https://twitter.com/stratiobd>
>>>>>>>> <https://www.linkedin.com/company/stratiobd>
>>>>>>>> <https://www.youtube.com/c/StratioBD>
>>>>>>>>
>>>>>>>> ____________________________________________________________
>>>>>>>> __________________
>>>>>>>> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
>>>>>>>> Documentation: http://www.balabit.com/support
>>>>>>>> /documentation/?product=syslog-ng
>>>>>>>> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>> ____________________________________________________________
>>>>>>> __________________
>>>>>>> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
>>>>>>> Documentation: http://www.balabit.com/support
>>>>>>> /documentation/?product=syslog-ng
>>>>>>> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>>
>>>>>> | Jose Angel Santiago
>>>>>>
>>>>>> [image: Logo_signature2.png] <http://www.stratio.com/>
>>>>>>
>>>>>> Vía de las dos Castillas, 33, Ática 4, 3 Planta
>>>>>>
>>>>>> 28224 Pozuelo de Alarcón, Madrid, Spain
>>>>>>
>>>>>> +34 918 286 473 | www.stratio.com
>>>>>> <https://twitter.com/stratiobd>
>>>>>> <https://www.linkedin.com/company/stratiobd>
>>>>>> <https://www.youtube.com/c/StratioBD>
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>>
>>>>> | Jose Angel Santiago
>>>>>
>>>>> [image: Logo_signature2.png] <http://www.stratio.com/>
>>>>>
>>>>> Vía de las dos Castillas, 33, Ática 4, 3 Planta
>>>>>
>>>>> 28224 Pozuelo de Alarcón, Madrid, Spain
>>>>>
>>>>> +34 918 286 473 | www.stratio.com
>>>>> <https://twitter.com/stratiobd>
>>>>> <https://www.linkedin.com/company/stratiobd>
>>>>> <https://www.youtube.com/c/StratioBD>
>>>>>
>>>>> ____________________________________________________________
>>>>> __________________
>>>>> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
>>>>> Documentation: http://www.balabit.com/support
>>>>> /documentation/?product=syslog-ng
>>>>> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>>>>>
>>>>>
>>>>>
>>>>
>>>> ____________________________________________________________
>>>> __________________
>>>> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
>>>> Documentation: http://www.balabit.com/support
>>>> /documentation/?product=syslog-ng
>>>> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>>>>
>>>>
>>>>
>>>
>>>
>>> --
>>>
>>> | Jose Angel Santiago
>>>
>>> [image: Logo_signature2.png] <http://www.stratio.com/>
>>>
>>> Vía de las dos Castillas, 33, Ática 4, 3 Planta
>>>
>>> 28224 Pozuelo de Alarcón, Madrid, Spain
>>>
>>> +34 918 286 473 | www.stratio.com
>>> <https://twitter.com/stratiobd>
>>> <https://www.linkedin.com/company/stratiobd>
>>> <https://www.youtube.com/c/StratioBD>
>>>
>>> ____________________________________________________________
>>> __________________
>>> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
>>> Documentation: http://www.balabit.com/support
>>> /documentation/?product=syslog-ng
>>> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>>>
>>>
>>>
>>
>> ____________________________________________________________
>> __________________
>> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
>> Documentation: http://www.balabit.com/support/documentation/?product=
>> syslog-ng
>> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>>
>>
>>
>
>
> --
>
> | Jose Angel Santiago
>
> [image: Logo_signature2.png] <http://www.stratio.com/>
>
> Vía de las dos Castillas, 33, Ática 4, 3 Planta
>
> 28224 Pozuelo de Alarcón, Madrid, Spain
>
> +34 918 286 473 | www.stratio.com
> <https://twitter.com/stratiobd>
> <https://www.linkedin.com/company/stratiobd>
> <https://www.youtube.com/c/StratioBD>
>
--
| Jose Angel Santiago
[image: Logo_signature2.png] <http://www.stratio.com/>
Vía de las dos Castillas, 33, Ática 4, 3 Planta
28224 Pozuelo de Alarcón, Madrid, Spain
+34 918 286 473 | www.stratio.com
<https://twitter.com/stratiobd> <https://www.linkedin.com/company/stratiobd>
<https://www.youtube.com/c/StratioBD>
[Attachment #5 (text/html)]
<div dir="ltr"><div>Nevermind, missing messages were not processed because of a bad \
syslog-ng wildcard-file source configuration.</div><div><br></div><div>Thank you for \
your support.<br></div></div><div class="gmail_extra"><br><div \
class="gmail_quote">2018-09-24 11:30 GMT+02:00 Jose Angel Santiago <span \
dir="ltr"><<a href="mailto:jasantiago@stratio.com" \
target="_blank">jasantiago@stratio.com</a>></span>:<br><blockquote \
class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc \
solid;padding-left:1ex"><div dir="ltr"><div>Hi,</div><div><br></div><div>I've \
been playing with different values of log-iw-size & mem-buf-size and I got to \
avoid loss of messages :-)</div><div><br></div><div>Bad news is that I'm afraid \
I've found another case of messages loss, I'll create a new thread since \
it's a completely different \
scenary.</div><div><br></div><div>Thanks.<br></div></div><div class="HOEnZb"><div \
class="h5"><div class="gmail_extra"><br><div class="gmail_quote">2018-09-19 18:36 \
GMT+02:00 Budai, László <span dir="ltr"><<a \
href="mailto:laszlo.budai@oneidentity.com" \
target="_blank">laszlo.budai@oneidentity.com</a>></span><wbr>:<br><blockquote \
class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc \
solid;padding-left:1ex"><div dir="ltr"><div>Hi,</div><div><br></div><div>... checking \
your config I found an interesting thing...</div><div>You set 'bad' values in \
your config for mem-buf-size and for disk-buf-size :) (it's not your fault... \
this is something we should change, this should not be so \
complicated).</div><div><br></div><div>If you have one source AND disk-buf-size is as \
big as it is able to store a log-iw-size number of messages then when a message is \
received, syslog-ng writes it into the diskbuffer and checks the free size of \
diskbuf.</div><div>When free_size < mem-buf-size, then the msg won't be acked, \
which means that the window size of the source is not incremented back, and when \
window size reach the 0, it won't receive any logs until a msg in the queue is \
not ACKd back.</div><div><br></div><div>The question is now what are the \
defaults?</div><div>log-iw-size: 100</div><div>mem-buf-size: \
16M</div><div><br></div><div>This means that -as you set 16M for diskbuf size- the \
diskbuffer can contain max. 100 messages, as it won't be acked when inserted into \
the queue (only when the destination is \
alive...).<br></div><div><br></div><div>I'd increase the diskbuf size in your \
case: set it to a huge number, if you have enough storage. You can play with setting \
small mem-buf-size, but that should able to store log-iw-size messages (in bytes, so \
log-iw-size * mean(msg-size-in-bytes) or something similar).<span \
class="m_3881913468889692784HOEnZb"><font \
color="#888888"><br></font></span></div><span \
class="m_3881913468889692784HOEnZb"><font \
color="#888888"><div><br></div><div>L.<br></div></font></span></div><div \
class="m_3881913468889692784HOEnZb"><div class="m_3881913468889692784h5"><div \
class="gmail_extra"><br><div class="gmail_quote">On Wed, Sep 19, 2018 at 3:50 PM, \
Jose Angel Santiago <span dir="ltr"><<a href="mailto:jasantiago@stratio.com" \
target="_blank">jasantiago@stratio.com</a>></span> wrote:<br><blockquote \
class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc \
solid;padding-left:1ex"><div dir="ltr"><div>Hi,</div><div><br></div><div>You're \
right, what I saw on my tcpdump file was TCP ACK.</div><div><br></div><div>I guess I \
lose so many messages because the TCP buffer is getting bigger and bigger since \
syslog-relay queue_length from disk_buffer reaches its limit. In fact, when this \
queue is not full (i.e with very low msg/sec ratio) I lose from 0-5 messages \
according to my tests.<br></div><div><br></div><div>Which parameter is the one which \
would match the queue_length of the disk_buffer? It seems that it should be \
<i>mem-buf-length()</i> but I'm using reliable=yes so I shouldn't use that \
parameter, and I'd like to test my scenary being able to queue more than 99 \
messages on disk_buffer.</div><div><br></div><div>Thanks.<br></div></div><div \
class="gmail_extra"><br><div class="gmail_quote">2018-09-19 13:46 GMT+02:00 Budai, \
László <span dir="ltr"><<a href="mailto:laszlo.budai@oneidentity.com" \
target="_blank">laszlo.budai@oneidentity.com</a>></span><wbr>:<br><blockquote \
class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc \
solid;padding-left:1ex"><div dir="ltr"><div>Hi,</div><div><br></div><div>what do you \
mean under 'syslog-relay returns ACK to syslog-ng agent' ?</div><div>I guess \
this is TCP level ACK.<br></div><div><br></div><div>And TCP level ACK is not the same \
as what syslog-ng uses for controlling the window size of the \
LogSources.</div><div><br></div><div>It is possible that you have pending data in TCP \
buffers (maintained by the kernel) that is not read by syslog-ng, but they are \
already ACKd by TCP itself...</div><div><br></div><div>(Solving this kind of \
situation requires application level ACK, which syslog-ng has only in the commercial \
edition.)<span class="m_3881913468889692784m_-1956735470216293634m_5907621731121508656HOEnZb"><font \
color="#888888"><br></font></span></div><span \
class="m_3881913468889692784m_-1956735470216293634m_5907621731121508656HOEnZb"><font \
color="#888888"><div><br></div>L.</font></span><div><div \
class="m_3881913468889692784m_-1956735470216293634m_5907621731121508656h5"><br><div><div \
class="gmail_extra"><br><div class="gmail_quote">On Wed, Sep 19, 2018 at 9:23 AM, \
Jose Angel Santiago <span dir="ltr"><<a href="mailto:jasantiago@stratio.com" \
target="_blank">jasantiago@stratio.com</a>></span> wrote:<br><blockquote \
class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc \
solid;padding-left:1ex"><div dir="ltr"><div>Hi,</div><div><br></div><div>I've \
realized that the queue_length of the disk_buffer never goes beyond 99, no matter the \
ratio message/sec logger process is writing or the value of mem-buf-size or \
disk-buf-size.</div><div><br></div><div>Which parameter is the one which would match \
the queue_length of the disk_buffer? It seems that it should be \
<i>mem-buf-length()</i> but I'm using reliable=yes so I shouldn't use that \
parameter, and I'd like to test my scenary being able to queue more than 99 \
messages on disk_buffer.<br></div><div><br></div><div>Regards.<br></div></div><div \
class="gmail_extra"><br><div class="gmail_quote">2018-09-17 14:33 GMT+02:00 Jose \
Angel Santiago <span dir="ltr"><<a href="mailto:jasantiago@stratio.com" \
target="_blank">jasantiago@stratio.com</a>></span>:<br><blockquote \
class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc \
solid;padding-left:1ex"><div dir="ltr"><div>3.17.2</div><div><br></div><div>Thanks in \
advance.<br></div></div><div \
class="m_3881913468889692784m_-1956735470216293634m_5907621731121508656m_-7285963992014489531m_1591842896039309780m_-3601329726171256577HOEnZb"><div \
class="m_3881913468889692784m_-1956735470216293634m_5907621731121508656m_-7285963992014489531m_1591842896039309780m_-3601329726171256577h5"><div \
class="gmail_extra"><br><div class="gmail_quote">2018-09-17 14:31 GMT+02:00 Budai, \
László <span dir="ltr"><<a href="mailto:laszlo.budai@oneidentity.com" \
target="_blank">laszlo.budai@oneidentity.com</a>></span><wbr>:<br><blockquote \
class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc \
solid;padding-left:1ex"><div dir="ltr"><div>Hi,</div><div><br></div><div>I'll try \
to reproduce this issue in my test environment.<br></div><div>What syslog-ng version \
do you have?<span class="m_3881913468889692784m_-1956735470216293634m_5907621731121508 \
656m_-7285963992014489531m_1591842896039309780m_-3601329726171256577m_5267951684435205629HOEnZb"><font \
color="#888888"><br></font></span></div><span \
class="m_3881913468889692784m_-1956735470216293634m_5907621731121508656m_-728596399201 \
4489531m_1591842896039309780m_-3601329726171256577m_5267951684435205629HOEnZb"><font \
color="#888888"><div><br></div><div>L.<br></div></font></span><div><div \
class="m_3881913468889692784m_-1956735470216293634m_5907621731121508656m_-7285963992014489531m_1591842896039309780m_-3601329726171256577m_5267951684435205629h5"><div \
class="gmail_extra"><br><div class="gmail_quote">On Mon, Sep 17, 2018 at 2:04 PM, \
Jose Angel Santiago <span dir="ltr"><<a href="mailto:jasantiago@stratio.com" \
target="_blank">jasantiago@stratio.com</a>></span> wrote:<br><blockquote \
class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc \
solid;padding-left:1ex"><div dir="ltr"><div \
dir="ltr"><div>Hi,</div><div><br></div><div>After repeating the test without TLS \
between syslog-agent & syslog-relay, with 2229 messages logged with a 20 msgs/sec \
logging ratio, in debug mode and with two tcpdumps running, these are the \
conclussions:</div><div></div><div><br></div><div>- Syslog-agent sends every single \
message (checked with tcpdump file and syslog-agent log file)<br></div><div>- \
Syslog-relay returns ACK for every single message, but sometimes the package contains \
more than one message (checked with tcpdump file)<br></div><div>- Checking the \
syslog-relay log after killing & starting the docker container, I found the trace \
'Reliable disk-buffer state loaded; filename='/syslog-ng-00000.rqf<wbr>', \
queue_length='99', size='173844' and the queued messages (from msg \
236 to msg 334 are processed). The next message processed is the n 440. and it \
comes from the active network destination again.</div><div>- Once all logs have been \
processed, in elasticsearch (syslog-relay destination) I can find messages from 1 to \
334, and messages from 440 to 2229.<br></div><div><br></div><div>How is it possible \
that syslog-relay returns ACK for messages 335 to 439 to syslog-agent, but they are \
not in the queue, nor in the .rqf file? Where did these messages go? I know they are \
sent before starting to fill syslog-agent disk-buffer file (checked with rqf file \
from syslog-agent when I kill the syslog-relay container). It seems that there's \
some kind of race condition that makes those messages are not processed/queued \
while syslog-relay is being killed.<br></div><div><br></div><div>Mi procedure \
consists of:</div><div><br></div><div>. Start tcpdump on both syslog-agent & \
syslog-relay hosts<br></div><div>- Start logger process</div><div>- Once logs are \
being inserted in elasticsearch, I kill the syslog-relay docker container, wait about \
10 seconds, and run a new container (using a mapped volume where .rqf file and \
.persist file are)</div><div>- Once logs are being inserted in elasticsearch again, I \
stop the logger process.</div><div>- I wait until no log is left to be \
processed</div><div><br></div><div>Would you test anything else? I'm running out \
of ideas.</div><div><br></div><div>PD: Same test with logger process writing 5 \
msgs/sec ratio produces from 0 to 5 lost \
messages.<br></div><div><br></div><div>Regards.<br></div><div><br></div><div><br></div><div><br></div></div></div><div \
class="gmail_extra"><br><div class="gmail_quote">2018-09-14 14:05 GMT+02:00 Jose \
Angel Santiago <span dir="ltr"><<a href="mailto:jasantiago@stratio.com" \
target="_blank">jasantiago@stratio.com</a>></span>:<br><blockquote \
class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc \
solid;padding-left:1ex"><div dir="ltr"><div \
dir="ltr"><div>Hi,</div><div><br></div><div>I've got more accurate information \
about where are my lost messages.</div><div><br></div><div>Now I'm using \
disk-buffer in syslog-agent and syslog-relay, and I've checked that lost messages \
are the ones sent by the syslog-agent when the syslog-relay docker container is being \
killed. I can see those messages on syslog-agent log (I've got both agent & \
relay in debug mode) with its corresponding "Outgoing message" line, but \
those messages never reach the relay.</div><div><br></div><div>Could it be that the \
relay docker container still returns ACK to the agent (the agent resolves relay fqdn \
with a custom DNS) while syslog-ng process within the container is being stopped? \
I'm about to test again using tcpdump to confirm this \
theory,<br></div><div><br></div><div>BTW, disk-buffers works ok, sometimes I get some \
duplicated messages when restarting the relay but that's not a problem for me. \
Forget about my .persist file re-creation theory, it doesn't \
happen.<br></div><div><br></div><div>Regards.<br></div><div><br></div><div><br></div></div></div><div \
class="m_3881913468889692784m_-1956735470216293634m_5907621731121508656m_-728596399201 \
4489531m_1591842896039309780m_-3601329726171256577m_5267951684435205629m_-4968185276606363308m_5899973325952193668HOEnZb"><div \
class="m_3881913468889692784m_-1956735470216293634m_5907621731121508656m_-728596399201 \
4489531m_1591842896039309780m_-3601329726171256577m_5267951684435205629m_-4968185276606363308m_5899973325952193668h5"><div \
class="gmail_extra"><br><div class="gmail_quote">2018-09-13 18:07 GMT+02:00 Péter, \
Kókai <span dir="ltr"><<a href="mailto:peter.kokai@oneidentity.com" \
target="_blank">peter.kokai@oneidentity.com</a>></span>:<br><blockquote \
class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc \
solid;padding-left:1ex"><div dir="ltr">Hello,<div><br></div><div>It would not make \
sense to replace the persist file after restart, so it is not something that \
syslog-ng does. Only if that file is corrupted, in that case at startup there should \
be a log about it, have you checked the syslog-ng logs ? (it would be better to \
enable debug and/or verbose logs, and if possible share it with \
us.)</div><div><br></div><div>Could you reproduce the same behavior without docker \
(if possible) ?</div><div><br></div><div><br></div><div><br></div><div>Best \
Regards,</div><div>Peter Kokai</div><div><div \
class="m_3881913468889692784m_-1956735470216293634m_5907621731121508656m_-728596399201 \
4489531m_1591842896039309780m_-3601329726171256577m_5267951684435205629m_-4968185276606363308m_5899973325952193668m_-2103616535007880284h5"><div \
dir="ltr"><br><div class="gmail_quote"><div dir="ltr">On Thu, Sep 13, 2018 at 4:44 PM \
Jose Angel Santiago <<a href="mailto:jasantiago@stratio.com" \
target="_blank">jasantiago@stratio.com</a>> wrote:<br></div><blockquote \
class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc \
solid;padding-left:1ex"><div dir="ltr"><div>Hi,</div><div><br></div><div>I guess I \
know what is happening, when I start from scratch the docker container, even I \
provide a persist file and a buffer file within the mapped volume, syslog-ng \
recreates them so all messages in buffer file which were not processed by the relay \
are lost.</div><div><br></div><div>Is there any way to tell syslog-ng to use an \
already existing .persist file so it doesn't recreate the .rqf \
file?</div><div><br></div><div>Regards.<br></div><div><br></div><div><br></div></div><div \
class="gmail_extra"><br><div class="gmail_quote">2018-09-13 16:23 GMT+02:00 Budai, \
László <span dir="ltr"><<a href="mailto:laszlo.budai@oneidentity.com" \
target="_blank">laszlo.budai@oneidentity.com</a>></span><wbr>:<br><blockquote \
class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc \
solid;padding-left:1ex"><div dir="ltr"><div>Hi,</div><div><br></div><div>one problem \
could be if the flush-limit would be greater than 1... in that case syslog-ng would \
use a HttpBulkMessageProcessor.</div><div>In this case syslog-ng pass the message to \
the HttpBulkMessageProcessor and sends back a positive ACK to the LogSource (so the \
message is removed from the diskbuffer), and if the dockerimage is killed, all the \
messages stored in the HttpBulkMessageProcessor are lost.</div><div>But in your case \
syslog-ng should use the HttpSingleMessageProcessor... which means that the messages \
are sent one-by-one... <br></div><div>Could you check the diskbuffer with the \
dqtool?</div><span class="m_3881913468889692784m_-1956735470216293634m_590762173112150 \
</div></div></div></div>
<br></div></div><span>______________________________<wbr>______________________________<wbr>__________________<br>
Member info: <a href="https://lists.balabit.hu/mailman/listinfo/syslog-ng" \
rel="noreferrer" target="_blank">https://lists.balabit.hu/mailm<wbr>an/listinfo/syslog-ng</a><br>
Documentation: <a href="http://www.balabit.com/support/documentation/?product=syslog-ng" \
rel="noreferrer" target="_blank">http://www.balabit.com/support<wbr>/documentation/?product=syslog<wbr>-ng</a><br>
FAQ: <a href="http://www.balabit.com/wiki/syslog-ng-faq" rel="noreferrer" \
target="_blank">http://www.balabit.com/wiki/sy<wbr>slog-ng-faq</a><br> <br>
<br></span></blockquote></div><br></div></div>
<br>______________________________<wbr>______________________________<wbr>__________________<br>
Member info: <a href="https://lists.balabit.hu/mailman/listinfo/syslog-ng" \
rel="noreferrer" target="_blank">https://lists.balabit.hu/mailm<wbr>an/listinfo/syslog-ng</a><br>
Documentation: <a href="http://www.balabit.com/support/documentation/?product=syslog-ng" \
rel="noreferrer" target="_blank">http://www.balabit.com/support<wbr>/documentation/?product=syslog<wbr>-ng</a><br>
FAQ: <a href="http://www.balabit.com/wiki/syslog-ng-faq" rel="noreferrer" \
target="_blank">http://www.balabit.com/wiki/sy<wbr>slog-ng-faq</a><br> <br>
<br></blockquote></div><br><br clear="all"><br>-- <br><div \
class="m_3881913468889692784m_-1956735470216293634m_5907621731121508656m_-728596399201 \
4489531m_1591842896039309780m_-3601329726171256577m_5267951684435205629m_-496818527660 \
6363308m_5899973325952193668m_-2103616535007880284m_8502744244687559657m_-28140536971310282m_-1370184941503478701m_3567492753405886787gmail_signature" \
data-smartmail="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div \
dir="ltr"><div dir="ltr"><div dir="ltr"><br><div><div \
style="color:rgb(136,136,136);font-size:12.8000001907349px"><div dir="ltr"><div \
dir="ltr"><div style="color:rgb(0,0,0);font-family:'Times New \
Roman';font-size:12.7272720336914px"><span \
style="font-size:9pt;font-family:Tahoma;color:rgb(217,217,217);font-weight:700;vertical-align:baseline;white-space:pre-wrap">|</span><span \
style="font-size:9pt;font-family:Tahoma;color:rgb(102,102,102);vertical-align:baseline;white-space:pre-wrap"> \
Jose Angel Santiago</span><br></div><div \
style="color:rgb(0,0,0);font-family:'Times New \
Roman';font-size:12.7272720336914px"><div \
style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.7272720336914px"><span><br><p \
dir="ltr" style="line-height:1.656;margin-top:0pt;margin-bottom:0pt"><a \
href="http://www.stratio.com/" target="_blank"><span \
style="font-size:10pt;font-family:Arial;color:rgb(17,85,204);background-color:transparent;vertical-align:baseline;white-space:pre-wrap"><img \
src="https://lh3.googleusercontent.com/SfSwo77PKD8TgM6em8B7mlcqOV9_N-AoAASRBMbZ3PFYgdF \
IibiMdr3zR_AZbMScWJOeRN7me-R_nK6vn1rnHSbApJVGfEjendjUa7LiGewC_fPGilVYciUS7E9v4mpKpl--caud" \
style="border:none" alt="Logo_signature2.png" width="96" \
height="22"></span></a></p><p dir="ltr" \
style="line-height:1.2;margin-top:0pt;margin-bottom:0pt"><span \
style="font-size:8pt;font-family:Tahoma;color:rgb(153,153,153);background-color:transparent;vertical-align:baseline;white-space:pre-wrap">Vía \
de las dos Castillas, 33, Ática 4, 3 Planta</span></p><p dir="ltr" \
style="line-height:1.2;margin-top:0pt;margin-bottom:0pt"><span \
style="font-size:8pt;font-family:Tahoma;color:rgb(153,153,153);background-color:transparent;vertical-align:baseline;white-space:pre-wrap">28224 \
Pozuelo de Alarcón, Madrid, Spain </span></p><p dir="ltr" \
style="line-height:1.2;margin-top:0pt;margin-bottom:0pt"><span \
style="font-size:8pt;font-family:Tahoma;color:rgb(153,153,153);vertical-align:baseline;white-space:pre-wrap"><a \
href="tel:+34%20918%2028%2064%2073" value="+34918286473" target="_blank">+34 918 286 \
473</a> | </span><a href="http://www.stratio.com" target="_blank"><span \
style="font-size:8pt;font-family:Tahoma;color:rgb(17,85,204);vertical-align:baseline;white-space:pre-wrap">www.stratio.com</span></a></p><a \
href="https://twitter.com/stratiobd" target="_blank"><span \
style="font-size:10pt;font-family:Arial;color:rgb(17,85,204);background-color:transparent;vertical-align:baseline;white-space:pre-wrap"><img \
src="https://lh3.googleusercontent.com/hOlHqJK94rZ7nBo9gzYKhLiwgogX1sgXV81pPDpAHHyRVeC \
jHxw0THNCq19zhcZalZiYeVYt9r4T_7LhoeLMxN1eTMnG46IfttV83WkTGC3jL1z04craZ8mmUn9hNnxDTIgh4_cT" \
style="border:none" width="20" height="20"></span></a><a \
href="https://www.linkedin.com/company/stratiobd" target="_blank"><span \
style="font-size:10pt;font-family:Arial;color:rgb(17,85,204);background-color:transparent;vertical-align:baseline;white-space:pre-wrap"><img \
src="https://lh6.googleusercontent.com/xs2vZh2SrmNnOaJO9i07vQzFNoghAaZytG_Zh09D9-ESjgO \
v5LHzLrbVNOFa-e3g5FYdmeg-kj6Ur6hID_h1joaSCdsHETfxTNqNSXan5nxBdGtmxq6NMWoh6puraVK1JGPxzhzX" \
style="border:none" width="20" height="20"></span></a><a \
href="https://www.youtube.com/c/StratioBD" target="_blank"><span \
style="font-size:8pt;font-family:Arial;color:rgb(17,85,204);background-color:transparent;font-style:italic;vertical-align:baseline;white-space:pre-wrap"><img \
src="https://lh3.googleusercontent.com/GdwY5Qtr3qEaHfNZszPc0lGx52_bfO00F-ge1MzYQPNLQmX \
f2jO4z2kvWwIEKi2swQ-IfLjNvpS7o4wR0nwNhnhOFd7zZ1zJDtFVZLkp52XyaM0GpEXySD2iLbWq-dw0cmXUe7Jj" \
style="border:none" width="20" \
height="20"></span></a></span><br></div></div></div></div></div></div></div></div></div></div></div></div></div>
</div>
______________________________<wbr>______________________________<wbr>__________________<br>
Member info: <a href="https://lists.balabit.hu/mailman/listinfo/syslog-ng" \
rel="noreferrer" target="_blank">https://lists.balabit.hu/mailm<wbr>an/listinfo/syslog-ng</a><br>
Documentation: <a href="http://www.balabit.com/support/documentation/?product=syslog-ng" \
rel="noreferrer" target="_blank">http://www.balabit.com/support<wbr>/documentation/?product=syslog<wbr>-ng</a><br>
FAQ: <a href="http://www.balabit.com/wiki/syslog-ng-faq" rel="noreferrer" \
target="_blank">http://www.balabit.com/wiki/sy<wbr>slog-ng-faq</a><br> <br>
</blockquote></div></div></div></div></div>
<br>______________________________<wbr>______________________________<wbr>__________________<br>
Member info: <a href="https://lists.balabit.hu/mailman/listinfo/syslog-ng" \
rel="noreferrer" target="_blank">https://lists.balabit.hu/mailm<wbr>an/listinfo/syslog-ng</a><br>
Documentation: <a href="http://www.balabit.com/support/documentation/?product=syslog-ng" \
rel="noreferrer" target="_blank">http://www.balabit.com/support<wbr>/documentation/?product=syslog<wbr>-ng</a><br>
FAQ: <a href="http://www.balabit.com/wiki/syslog-ng-faq" rel="noreferrer" \
target="_blank">http://www.balabit.com/wiki/sy<wbr>slog-ng-faq</a><br> <br>
<br></blockquote></div><br><br clear="all"><br>-- <br><div \
class="m_3881913468889692784m_-1956735470216293634m_5907621731121508656m_-728596399201 \
4489531m_1591842896039309780m_-3601329726171256577m_5267951684435205629m_-4968185276606363308m_5899973325952193668m_-2103616535007880284gmail_signature" \
data-smartmail="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div \
dir="ltr"><div dir="ltr"><div dir="ltr"><br><div><div \
style="color:rgb(136,136,136);font-size:12.8000001907349px"><div dir="ltr"><div \
dir="ltr"><div style="color:rgb(0,0,0);font-family:'Times New \
Roman';font-size:12.7272720336914px"><span \
style="font-size:9pt;font-family:Tahoma;color:rgb(217,217,217);font-weight:700;vertical-align:baseline;white-space:pre-wrap">|</span><span \
style="font-size:9pt;font-family:Tahoma;color:rgb(102,102,102);vertical-align:baseline;white-space:pre-wrap"> \
Jose Angel Santiago</span><br></div><div \
style="color:rgb(0,0,0);font-family:'Times New \
Roman';font-size:12.7272720336914px"><div \
style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.7272720336914px"><span><br><p \
dir="ltr" style="line-height:1.656;margin-top:0pt;margin-bottom:0pt"><a \
href="http://www.stratio.com/" target="_blank"><span \
style="font-size:10pt;font-family:Arial;color:rgb(17,85,204);background-color:transparent;vertical-align:baseline;white-space:pre-wrap"><img \
src="https://lh3.googleusercontent.com/SfSwo77PKD8TgM6em8B7mlcqOV9_N-AoAASRBMbZ3PFYgdF \
IibiMdr3zR_AZbMScWJOeRN7me-R_nK6vn1rnHSbApJVGfEjendjUa7LiGewC_fPGilVYciUS7E9v4mpKpl--caud" \
style="border:none" alt="Logo_signature2.png" width="96" \
height="22"></span></a></p><p dir="ltr" \
style="line-height:1.2;margin-top:0pt;margin-bottom:0pt"><span \
style="font-size:8pt;font-family:Tahoma;color:rgb(153,153,153);background-color:transparent;vertical-align:baseline;white-space:pre-wrap">Vía \
de las dos Castillas, 33, Ática 4, 3 Planta</span></p><p dir="ltr" \
style="line-height:1.2;margin-top:0pt;margin-bottom:0pt"><span \
style="font-size:8pt;font-family:Tahoma;color:rgb(153,153,153);background-color:transparent;vertical-align:baseline;white-space:pre-wrap">28224 \
Pozuelo de Alarcón, Madrid, Spain </span></p><p dir="ltr" \
style="line-height:1.2;margin-top:0pt;margin-bottom:0pt"><span \
style="font-size:8pt;font-family:Tahoma;color:rgb(153,153,153);vertical-align:baseline;white-space:pre-wrap">+34 \
918 286 473 | </span><a href="http://www.stratio.com" target="_blank"><span \
style="font-size:8pt;font-family:Tahoma;color:rgb(17,85,204);vertical-align:baseline;white-space:pre-wrap">www.stratio.com</span></a></p><a \
href="https://twitter.com/stratiobd" target="_blank"><span \
style="font-size:10pt;font-family:Arial;color:rgb(17,85,204);background-color:transparent;vertical-align:baseline;white-space:pre-wrap"><img \
src="https://lh3.googleusercontent.com/hOlHqJK94rZ7nBo9gzYKhLiwgogX1sgXV81pPDpAHHyRVeC \
jHxw0THNCq19zhcZalZiYeVYt9r4T_7LhoeLMxN1eTMnG46IfttV83WkTGC3jL1z04craZ8mmUn9hNnxDTIgh4_cT" \
style="border:none" width="20" height="20"></span></a><a \
href="https://www.linkedin.com/company/stratiobd" target="_blank"><span \
style="font-size:10pt;font-family:Arial;color:rgb(17,85,204);background-color:transparent;vertical-align:baseline;white-space:pre-wrap"><img \
src="https://lh6.googleusercontent.com/xs2vZh2SrmNnOaJO9i07vQzFNoghAaZytG_Zh09D9-ESjgO \
v5LHzLrbVNOFa-e3g5FYdmeg-kj6Ur6hID_h1joaSCdsHETfxTNqNSXan5nxBdGtmxq6NMWoh6puraVK1JGPxzhzX" \
style="border:none" width="20" height="20"></span></a><a \
href="https://www.youtube.com/c/StratioBD" target="_blank"><span \
style="font-size:8pt;font-family:Arial;color:rgb(17,85,204);background-color:transparent;font-style:italic;vertical-align:baseline;white-space:pre-wrap"><img \
src="https://lh3.googleusercontent.com/GdwY5Qtr3qEaHfNZszPc0lGx52_bfO00F-ge1MzYQPNLQmX \
f2jO4z2kvWwIEKi2swQ-IfLjNvpS7o4wR0nwNhnhOFd7zZ1zJDtFVZLkp52XyaM0GpEXySD2iLbWq-dw0cmXUe7Jj" \
style="border:none" width="20" \
height="20"></span></a></span><br></div></div></div></div></div></div></div></div></div></div></div></div></div>
</div>
</div></div></blockquote></div><br><br clear="all"><br>-- <br><div \
class="m_3881913468889692784m_-1956735470216293634m_5907621731121508656m_-728596399201 \
4489531m_1591842896039309780m_-3601329726171256577m_5267951684435205629m_-4968185276606363308m_5899973325952193668gmail_signature" \
data-smartmail="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div \
dir="ltr"><div dir="ltr"><div dir="ltr"><br><div><div \
style="color:rgb(136,136,136);font-size:12.8000001907349px"><div dir="ltr"><div \
dir="ltr"><div style="color:rgb(0,0,0);font-family:'Times New \
Roman';font-size:12.7272720336914px"><span \
style="font-size:9pt;font-family:Tahoma;color:rgb(217,217,217);font-weight:700;vertical-align:baseline;white-space:pre-wrap">|</span><span \
style="font-size:9pt;font-family:Tahoma;color:rgb(102,102,102);vertical-align:baseline;white-space:pre-wrap"> \
Jose Angel Santiago</span><br></div><div \
style="color:rgb(0,0,0);font-family:'Times New \
Roman';font-size:12.7272720336914px"><div \
style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.7272720336914px"><span><br><p \
dir="ltr" style="line-height:1.656;margin-top:0pt;margin-bottom:0pt"><a \
href="http://www.stratio.com/" target="_blank"><span \
style="font-size:10pt;font-family:Arial;color:rgb(17,85,204);background-color:transparent;vertical-align:baseline;white-space:pre-wrap"><img \
src="https://lh3.googleusercontent.com/SfSwo77PKD8TgM6em8B7mlcqOV9_N-AoAASRBMbZ3PFYgdF \
IibiMdr3zR_AZbMScWJOeRN7me-R_nK6vn1rnHSbApJVGfEjendjUa7LiGewC_fPGilVYciUS7E9v4mpKpl--caud" \
style="border:none" alt="Logo_signature2.png" width="96" \
height="22"></span></a></p><p dir="ltr" \
style="line-height:1.2;margin-top:0pt;margin-bottom:0pt"><span \
style="font-size:8pt;font-family:Tahoma;color:rgb(153,153,153);background-color:transparent;vertical-align:baseline;white-space:pre-wrap">Vía \
de las dos Castillas, 33, Ática 4, 3 Planta</span></p><p dir="ltr" \
style="line-height:1.2;margin-top:0pt;margin-bottom:0pt"><span \
style="font-size:8pt;font-family:Tahoma;color:rgb(153,153,153);background-color:transparent;vertical-align:baseline;white-space:pre-wrap">28224 \
Pozuelo de Alarcón, Madrid, Spain </span></p><p dir="ltr" \
style="line-height:1.2;margin-top:0pt;margin-bottom:0pt"><span \
style="font-size:8pt;font-family:Tahoma;color:rgb(153,153,153);vertical-align:baseline;white-space:pre-wrap">+34 \
918 286 473 | </span><a href="http://www.stratio.com" target="_blank"><span \
style="font-size:8pt;font-family:Tahoma;color:rgb(17,85,204);vertical-align:baseline;white-space:pre-wrap">www.stratio.com</span></a></p><a \
href="https://twitter.com/stratiobd" target="_blank"><span \
style="font-size:10pt;font-family:Arial;color:rgb(17,85,204);background-color:transparent;vertical-align:baseline;white-space:pre-wrap"><img \
src="https://lh3.googleusercontent.com/hOlHqJK94rZ7nBo9gzYKhLiwgogX1sgXV81pPDpAHHyRVeC \
jHxw0THNCq19zhcZalZiYeVYt9r4T_7LhoeLMxN1eTMnG46IfttV83WkTGC3jL1z04craZ8mmUn9hNnxDTIgh4_cT" \
style="border:none" width="20" height="20"></span></a><a \
href="https://www.linkedin.com/company/stratiobd" target="_blank"><span \
style="font-size:10pt;font-family:Arial;color:rgb(17,85,204);background-color:transparent;vertical-align:baseline;white-space:pre-wrap"><img \
src="https://lh6.googleusercontent.com/xs2vZh2SrmNnOaJO9i07vQzFNoghAaZytG_Zh09D9-ESjgO \
v5LHzLrbVNOFa-e3g5FYdmeg-kj6Ur6hID_h1joaSCdsHETfxTNqNSXan5nxBdGtmxq6NMWoh6puraVK1JGPxzhzX" \
style="border:none" width="20" height="20"></span></a><a \
href="https://www.youtube.com/c/StratioBD" target="_blank"><span \
style="font-size:8pt;font-family:Arial;color:rgb(17,85,204);background-color:transparent;font-style:italic;vertical-align:baseline;white-space:pre-wrap"><img \
src="https://lh3.googleusercontent.com/GdwY5Qtr3qEaHfNZszPc0lGx52_bfO00F-ge1MzYQPNLQmX \
f2jO4z2kvWwIEKi2swQ-IfLjNvpS7o4wR0nwNhnhOFd7zZ1zJDtFVZLkp52XyaM0GpEXySD2iLbWq-dw0cmXUe7Jj" \
style="border:none" width="20" \
height="20"></span></a></span><br></div></div></div></div></div></div></div></div></div></div></div></div></div>
</div>
<br>______________________________<wbr>______________________________<wbr>__________________<br>
Member info: <a href="https://lists.balabit.hu/mailman/listinfo/syslog-ng" \
rel="noreferrer" target="_blank">https://lists.balabit.hu/mailm<wbr>an/listinfo/syslog-ng</a><br>
Documentation: <a href="http://www.balabit.com/support/documentation/?product=syslog-ng" \
rel="noreferrer" target="_blank">http://www.balabit.com/support<wbr>/documentation/?product=syslog<wbr>-ng</a><br>
FAQ: <a href="http://www.balabit.com/wiki/syslog-ng-faq" rel="noreferrer" \
target="_blank">http://www.balabit.com/wiki/sy<wbr>slog-ng-faq</a><br> <br>
<br></blockquote></div><br></div></div></div></div>
<br>______________________________<wbr>______________________________<wbr>__________________<br>
Member info: <a href="https://lists.balabit.hu/mailman/listinfo/syslog-ng" \
rel="noreferrer" target="_blank">https://lists.balabit.hu/mailm<wbr>an/listinfo/syslog-ng</a><br>
Documentation: <a href="http://www.balabit.com/support/documentation/?product=syslog-ng" \
rel="noreferrer" target="_blank">http://www.balabit.com/support<wbr>/documentation/?product=syslog<wbr>-ng</a><br>
FAQ: <a href="http://www.balabit.com/wiki/syslog-ng-faq" rel="noreferrer" \
target="_blank">http://www.balabit.com/wiki/sy<wbr>slog-ng-faq</a><br> <br>
<br></blockquote></div><br><br clear="all"><br>-- <br><div \
class="m_3881913468889692784m_-1956735470216293634m_5907621731121508656m_-728596399201 \
4489531m_1591842896039309780m_-3601329726171256577m_5267951684435205629gmail_signature" \
data-smartmail="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div \
dir="ltr"><div dir="ltr"><div dir="ltr"><br><div><div \
style="color:rgb(136,136,136);font-size:12.8000001907349px"><div dir="ltr"><div \
dir="ltr"><div style="color:rgb(0,0,0);font-family:'Times New \
Roman';font-size:12.7272720336914px"><span \
style="font-size:9pt;font-family:Tahoma;color:rgb(217,217,217);font-weight:700;vertical-align:baseline;white-space:pre-wrap">|</span><span \
style="font-size:9pt;font-family:Tahoma;color:rgb(102,102,102);vertical-align:baseline;white-space:pre-wrap"> \
Jose Angel Santiago</span><br></div><div \
style="color:rgb(0,0,0);font-family:'Times New \
Roman';font-size:12.7272720336914px"><div \
style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.7272720336914px"><span><br><p \
dir="ltr" style="line-height:1.656;margin-top:0pt;margin-bottom:0pt"><a \
href="http://www.stratio.com/" target="_blank"><span \
style="font-size:10pt;font-family:Arial;color:rgb(17,85,204);background-color:transparent;vertical-align:baseline;white-space:pre-wrap"><img \
src="https://lh3.googleusercontent.com/SfSwo77PKD8TgM6em8B7mlcqOV9_N-AoAASRBMbZ3PFYgdF \
IibiMdr3zR_AZbMScWJOeRN7me-R_nK6vn1rnHSbApJVGfEjendjUa7LiGewC_fPGilVYciUS7E9v4mpKpl--caud" \
style="border:none" alt="Logo_signature2.png" width="96" \
height="22"></span></a></p><p dir="ltr" \
style="line-height:1.2;margin-top:0pt;margin-bottom:0pt"><span \
style="font-size:8pt;font-family:Tahoma;color:rgb(153,153,153);background-color:transparent;vertical-align:baseline;white-space:pre-wrap">Vía \
de las dos Castillas, 33, Ática 4, 3 Planta</span></p><p dir="ltr" \
style="line-height:1.2;margin-top:0pt;margin-bottom:0pt"><span \
style="font-size:8pt;font-family:Tahoma;color:rgb(153,153,153);background-color:transparent;vertical-align:baseline;white-space:pre-wrap">28224 \
Pozuelo de Alarcón, Madrid, Spain </span></p><p dir="ltr" \
style="line-height:1.2;margin-top:0pt;margin-bottom:0pt"><span \
style="font-size:8pt;font-family:Tahoma;color:rgb(153,153,153);vertical-align:baseline;white-space:pre-wrap">+34 \
918 286 473 | </span><a href="http://www.stratio.com" target="_blank"><span \
style="font-size:8pt;font-family:Tahoma;color:rgb(17,85,204);vertical-align:baseline;white-space:pre-wrap">www.stratio.com</span></a></p><a \
href="https://twitter.com/stratiobd" target="_blank"><span \
style="font-size:10pt;font-family:Arial;color:rgb(17,85,204);background-color:transparent;vertical-align:baseline;white-space:pre-wrap"><img \
src="https://lh3.googleusercontent.com/hOlHqJK94rZ7nBo9gzYKhLiwgogX1sgXV81pPDpAHHyRVeC \
jHxw0THNCq19zhcZalZiYeVYt9r4T_7LhoeLMxN1eTMnG46IfttV83WkTGC3jL1z04craZ8mmUn9hNnxDTIgh4_cT" \
style="border:none" width="20" height="20"></span></a><a \
href="https://www.linkedin.com/company/stratiobd" target="_blank"><span \
style="font-size:10pt;font-family:Arial;color:rgb(17,85,204);background-color:transparent;vertical-align:baseline;white-space:pre-wrap"><img \
src="https://lh6.googleusercontent.com/xs2vZh2SrmNnOaJO9i07vQzFNoghAaZytG_Zh09D9-ESjgO \
v5LHzLrbVNOFa-e3g5FYdmeg-kj6Ur6hID_h1joaSCdsHETfxTNqNSXan5nxBdGtmxq6NMWoh6puraVK1JGPxzhzX" \
style="border:none" width="20" height="20"></span></a><a \
href="https://www.youtube.com/c/StratioBD" target="_blank"><span \
style="font-size:8pt;font-family:Arial;color:rgb(17,85,204);background-color:transparent;font-style:italic;vertical-align:baseline;white-space:pre-wrap"><img \
src="https://lh3.googleusercontent.com/GdwY5Qtr3qEaHfNZszPc0lGx52_bfO00F-ge1MzYQPNLQmX \
f2jO4z2kvWwIEKi2swQ-IfLjNvpS7o4wR0nwNhnhOFd7zZ1zJDtFVZLkp52XyaM0GpEXySD2iLbWq-dw0cmXUe7Jj" \
style="border:none" width="20" \
height="20"></span></a></span><br></div></div></div></div></div></div></div></div></div></div></div></div></div>
</div>
</div></div></blockquote></div><br><br clear="all"><br>-- <br><div \
class="m_3881913468889692784m_-1956735470216293634m_5907621731121508656m_-7285963992014489531m_1591842896039309780m_-3601329726171256577gmail_signature" \
data-smartmail="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div \
dir="ltr"><div dir="ltr"><div dir="ltr"><br><div><div \
style="color:rgb(136,136,136);font-size:12.8000001907349px"><div dir="ltr"><div \
dir="ltr"><div style="color:rgb(0,0,0);font-family:'Times New \
Roman';font-size:12.7272720336914px"><span \
style="font-size:9pt;font-family:Tahoma;color:rgb(217,217,217);font-weight:700;vertical-align:baseline;white-space:pre-wrap">|</span><span \
style="font-size:9pt;font-family:Tahoma;color:rgb(102,102,102);vertical-align:baseline;white-space:pre-wrap"> \
Jose Angel Santiago</span><br></div><div \
style="color:rgb(0,0,0);font-family:'Times New \
Roman';font-size:12.7272720336914px"><div \
style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.7272720336914px"><span><br><p \
dir="ltr" style="line-height:1.656;margin-top:0pt;margin-bottom:0pt"><a \
href="http://www.stratio.com/" target="_blank"><span \
style="font-size:10pt;font-family:Arial;color:rgb(17,85,204);background-color:transparent;vertical-align:baseline;white-space:pre-wrap"><img \
src="https://lh3.googleusercontent.com/SfSwo77PKD8TgM6em8B7mlcqOV9_N-AoAASRBMbZ3PFYgdF \
IibiMdr3zR_AZbMScWJOeRN7me-R_nK6vn1rnHSbApJVGfEjendjUa7LiGewC_fPGilVYciUS7E9v4mpKpl--caud" \
style="border:none" alt="Logo_signature2.png" width="96" \
height="22"></span></a></p><p dir="ltr" \
style="line-height:1.2;margin-top:0pt;margin-bottom:0pt"><span \
style="font-size:8pt;font-family:Tahoma;color:rgb(153,153,153);background-color:transparent;vertical-align:baseline;white-space:pre-wrap">Vía \
de las dos Castillas, 33, Ática 4, 3 Planta</span></p><p dir="ltr" \
style="line-height:1.2;margin-top:0pt;margin-bottom:0pt"><span \
style="font-size:8pt;font-family:Tahoma;color:rgb(153,153,153);background-color:transparent;vertical-align:baseline;white-space:pre-wrap">28224 \
Pozuelo de Alarcón, Madrid, Spain </span></p><p dir="ltr" \
style="line-height:1.2;margin-top:0pt;margin-bottom:0pt"><span \
style="font-size:8pt;font-family:Tahoma;color:rgb(153,153,153);vertical-align:baseline;white-space:pre-wrap">+34 \
918 286 473 | </span><a href="http://www.stratio.com" target="_blank"><span \
style="font-size:8pt;font-family:Tahoma;color:rgb(17,85,204);vertical-align:baseline;white-space:pre-wrap">www.stratio.com</span></a></p><a \
href="https://twitter.com/stratiobd" target="_blank"><span \
style="font-size:10pt;font-family:Arial;color:rgb(17,85,204);background-color:transparent;vertical-align:baseline;white-space:pre-wrap"><img \
src="https://lh3.googleusercontent.com/hOlHqJK94rZ7nBo9gzYKhLiwgogX1sgXV81pPDpAHHyRVeC \
jHxw0THNCq19zhcZalZiYeVYt9r4T_7LhoeLMxN1eTMnG46IfttV83WkTGC3jL1z04craZ8mmUn9hNnxDTIgh4_cT" \
style="border:none" width="20" height="20"></span></a><a \
href="https://www.linkedin.com/company/stratiobd" target="_blank"><span \
style="font-size:10pt;font-family:Arial;color:rgb(17,85,204);background-color:transparent;vertical-align:baseline;white-space:pre-wrap"><img \
src="https://lh6.googleusercontent.com/xs2vZh2SrmNnOaJO9i07vQzFNoghAaZytG_Zh09D9-ESjgO \
v5LHzLrbVNOFa-e3g5FYdmeg-kj6Ur6hID_h1joaSCdsHETfxTNqNSXan5nxBdGtmxq6NMWoh6puraVK1JGPxzhzX" \
style="border:none" width="20" height="20"></span></a><a \
href="https://www.youtube.com/c/StratioBD" target="_blank"><span \
style="font-size:8pt;font-family:Arial;color:rgb(17,85,204);background-color:transparent;font-style:italic;vertical-align:baseline;white-space:pre-wrap"><img \
src="https://lh3.googleusercontent.com/GdwY5Qtr3qEaHfNZszPc0lGx52_bfO00F-ge1MzYQPNLQmX \
f2jO4z2kvWwIEKi2swQ-IfLjNvpS7o4wR0nwNhnhOFd7zZ1zJDtFVZLkp52XyaM0GpEXySD2iLbWq-dw0cmXUe7Jj" \
style="border:none" width="20" \
height="20"></span></a></span><br></div></div></div></div></div></div></div></div></div></div></div></div></div>
</div>
<br>______________________________<wbr>______________________________<wbr>__________________<br>
Member info: <a href="https://lists.balabit.hu/mailman/listinfo/syslog-ng" \
rel="noreferrer" target="_blank">https://lists.balabit.hu/mailm<wbr>an/listinfo/syslog-ng</a><br>
Documentation: <a href="http://www.balabit.com/support/documentation/?product=syslog-ng" \
rel="noreferrer" target="_blank">http://www.balabit.com/support<wbr>/documentation/?product=syslog<wbr>-ng</a><br>
FAQ: <a href="http://www.balabit.com/wiki/syslog-ng-faq" rel="noreferrer" \
target="_blank">http://www.balabit.com/wiki/sy<wbr>slog-ng-faq</a><br> <br>
<br></blockquote></div><br></div></div></div></div></div>
<br>______________________________<wbr>______________________________<wbr>__________________<br>
Member info: <a href="https://lists.balabit.hu/mailman/listinfo/syslog-ng" \
rel="noreferrer" target="_blank">https://lists.balabit.hu/mailm<wbr>an/listinfo/syslog-ng</a><br>
Documentation: <a href="http://www.balabit.com/support/documentation/?product=syslog-ng" \
rel="noreferrer" target="_blank">http://www.balabit.com/support<wbr>/documentation/?product=syslog<wbr>-ng</a><br>
FAQ: <a href="http://www.balabit.com/wiki/syslog-ng-faq" rel="noreferrer" \
target="_blank">http://www.balabit.com/wiki/sy<wbr>slog-ng-faq</a><br> <br>
<br></blockquote></div><br><br clear="all"><br>-- <br><div \
class="m_3881913468889692784m_-1956735470216293634m_5907621731121508656gmail_signature" \
data-smartmail="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div \
dir="ltr"><div dir="ltr"><div dir="ltr"><br><div><div \
style="color:rgb(136,136,136);font-size:12.8000001907349px"><div dir="ltr"><div \
dir="ltr"><div style="color:rgb(0,0,0);font-family:'Times New \
Roman';font-size:12.7272720336914px"><span \
style="font-size:9pt;font-family:Tahoma;color:rgb(217,217,217);font-weight:700;vertical-align:baseline;white-space:pre-wrap">|</span><span \
style="font-size:9pt;font-family:Tahoma;color:rgb(102,102,102);vertical-align:baseline;white-space:pre-wrap"> \
Jose Angel Santiago</span><br></div><div \
style="color:rgb(0,0,0);font-family:'Times New \
Roman';font-size:12.7272720336914px"><div \
style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.7272720336914px"><span><br><p \
dir="ltr" style="line-height:1.656;margin-top:0pt;margin-bottom:0pt"><a \
href="http://www.stratio.com/" target="_blank"><span \
style="font-size:10pt;font-family:Arial;color:rgb(17,85,204);background-color:transparent;vertical-align:baseline;white-space:pre-wrap"><img \
src="https://lh3.googleusercontent.com/SfSwo77PKD8TgM6em8B7mlcqOV9_N-AoAASRBMbZ3PFYgdF \
IibiMdr3zR_AZbMScWJOeRN7me-R_nK6vn1rnHSbApJVGfEjendjUa7LiGewC_fPGilVYciUS7E9v4mpKpl--caud" \
style="border:none" alt="Logo_signature2.png" width="96" \
height="22"></span></a></p><p dir="ltr" \
style="line-height:1.2;margin-top:0pt;margin-bottom:0pt"><span \
style="font-size:8pt;font-family:Tahoma;color:rgb(153,153,153);background-color:transparent;vertical-align:baseline;white-space:pre-wrap">Vía \
de las dos Castillas, 33, Ática 4, 3 Planta</span></p><p dir="ltr" \
style="line-height:1.2;margin-top:0pt;margin-bottom:0pt"><span \
style="font-size:8pt;font-family:Tahoma;color:rgb(153,153,153);background-color:transparent;vertical-align:baseline;white-space:pre-wrap">28224 \
Pozuelo de Alarcón, Madrid, Spain </span></p><p dir="ltr" \
style="line-height:1.2;margin-top:0pt;margin-bottom:0pt"><span \
style="font-size:8pt;font-family:Tahoma;color:rgb(153,153,153);vertical-align:baseline;white-space:pre-wrap">+34 \
918 286 473 | </span><a href="http://www.stratio.com" target="_blank"><span \
style="font-size:8pt;font-family:Tahoma;color:rgb(17,85,204);vertical-align:baseline;white-space:pre-wrap">www.stratio.com</span></a></p><a \
href="https://twitter.com/stratiobd" target="_blank"><span \
style="font-size:10pt;font-family:Arial;color:rgb(17,85,204);background-color:transparent;vertical-align:baseline;white-space:pre-wrap"><img \
src="https://lh3.googleusercontent.com/hOlHqJK94rZ7nBo9gzYKhLiwgogX1sgXV81pPDpAHHyRVeC \
jHxw0THNCq19zhcZalZiYeVYt9r4T_7LhoeLMxN1eTMnG46IfttV83WkTGC3jL1z04craZ8mmUn9hNnxDTIgh4_cT" \
style="border:none" width="20" height="20"></span></a><a \
href="https://www.linkedin.com/company/stratiobd" target="_blank"><span \
style="font-size:10pt;font-family:Arial;color:rgb(17,85,204);background-color:transparent;vertical-align:baseline;white-space:pre-wrap"><img \
src="https://lh6.googleusercontent.com/xs2vZh2SrmNnOaJO9i07vQzFNoghAaZytG_Zh09D9-ESjgO \
v5LHzLrbVNOFa-e3g5FYdmeg-kj6Ur6hID_h1joaSCdsHETfxTNqNSXan5nxBdGtmxq6NMWoh6puraVK1JGPxzhzX" \
style="border:none" width="20" height="20"></span></a><a \
href="https://www.youtube.com/c/StratioBD" target="_blank"><span \
style="font-size:8pt;font-family:Arial;color:rgb(17,85,204);background-color:transparent;font-style:italic;vertical-align:baseline;white-space:pre-wrap"><img \
src="https://lh3.googleusercontent.com/GdwY5Qtr3qEaHfNZszPc0lGx52_bfO00F-ge1MzYQPNLQmX \
f2jO4z2kvWwIEKi2swQ-IfLjNvpS7o4wR0nwNhnhOFd7zZ1zJDtFVZLkp52XyaM0GpEXySD2iLbWq-dw0cmXUe7Jj" \
style="border:none" width="20" \
height="20"></span></a></span><br></div></div></div></div></div></div></div></div></div></div></div></div></div>
</div>
<br>______________________________<wbr>______________________________<wbr>__________________<br>
Member info: <a href="https://lists.balabit.hu/mailman/listinfo/syslog-ng" \
rel="noreferrer" target="_blank">https://lists.balabit.hu/mailm<wbr>an/listinfo/syslog-ng</a><br>
Documentation: <a href="http://www.balabit.com/support/documentation/?product=syslog-ng" \
rel="noreferrer" target="_blank">http://www.balabit.com/support<wbr>/documentation/?product=syslog<wbr>-ng</a><br>
FAQ: <a href="http://www.balabit.com/wiki/syslog-ng-faq" rel="noreferrer" \
target="_blank">http://www.balabit.com/wiki/sy<wbr>slog-ng-faq</a><br> <br>
<br></blockquote></div><br></div>
</div></div><br>______________________________<wbr>______________________________<wbr>__________________<br>
Member info: <a href="https://lists.balabit.hu/mailman/listinfo/syslog-ng" \
rel="noreferrer" target="_blank">https://lists.balabit.hu/mailm<wbr>an/listinfo/syslog-ng</a><br>
Documentation: <a href="http://www.balabit.com/support/documentation/?product=syslog-ng" \
rel="noreferrer" target="_blank">http://www.balabit.com/support<wbr>/documentation/?product=<wbr>syslog-ng</a><br>
FAQ: <a href="http://www.balabit.com/wiki/syslog-ng-faq" rel="noreferrer" \
target="_blank">http://www.balabit.com/wiki/sy<wbr>slog-ng-faq</a><br> <br>
<br></blockquote></div><br><br clear="all"><br>-- <br><div \
class="m_3881913468889692784gmail_signature" data-smartmail="gmail_signature"><div \
dir="ltr"><div><div dir="ltr"><div dir="ltr"><div dir="ltr"><div \
dir="ltr"><br><div><div \
style="color:rgb(136,136,136);font-size:12.8000001907349px"><div dir="ltr"><div \
dir="ltr"><div style="color:rgb(0,0,0);font-family:'Times New \
Roman';font-size:12.7272720336914px"><span \
style="font-size:9pt;font-family:Tahoma;color:rgb(217,217,217);font-weight:700;vertical-align:baseline;white-space:pre-wrap">|</span><span \
style="font-size:9pt;font-family:Tahoma;color:rgb(102,102,102);vertical-align:baseline;white-space:pre-wrap"> \
Jose Angel Santiago</span><br></div><div \
style="color:rgb(0,0,0);font-family:'Times New \
Roman';font-size:12.7272720336914px"><div \
style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.7272720336914px"><span><br><p \
dir="ltr" style="line-height:1.656;margin-top:0pt;margin-bottom:0pt"><a \
href="http://www.stratio.com/" target="_blank"><span \
style="font-size:10pt;font-family:Arial;color:rgb(17,85,204);background-color:transparent;vertical-align:baseline;white-space:pre-wrap"><img \
src="https://lh3.googleusercontent.com/SfSwo77PKD8TgM6em8B7mlcqOV9_N-AoAASRBMbZ3PFYgdF \
IibiMdr3zR_AZbMScWJOeRN7me-R_nK6vn1rnHSbApJVGfEjendjUa7LiGewC_fPGilVYciUS7E9v4mpKpl--caud" \
style="border:none" alt="Logo_signature2.png" width="96" \
height="22"></span></a></p><p dir="ltr" \
style="line-height:1.2;margin-top:0pt;margin-bottom:0pt"><span \
style="font-size:8pt;font-family:Tahoma;color:rgb(153,153,153);background-color:transparent;vertical-align:baseline;white-space:pre-wrap">Vía \
de las dos Castillas, 33, Ática 4, 3 Planta</span></p><p dir="ltr" \
style="line-height:1.2;margin-top:0pt;margin-bottom:0pt"><span \
style="font-size:8pt;font-family:Tahoma;color:rgb(153,153,153);background-color:transparent;vertical-align:baseline;white-space:pre-wrap">28224 \
Pozuelo de Alarcón, Madrid, Spain </span></p><p dir="ltr" \
style="line-height:1.2;margin-top:0pt;margin-bottom:0pt"><span \
style="font-size:8pt;font-family:Tahoma;color:rgb(153,153,153);vertical-align:baseline;white-space:pre-wrap">+34 \
918 286 473 | </span><a href="http://www.stratio.com" target="_blank"><span \
style="font-size:8pt;font-family:Tahoma;color:rgb(17,85,204);vertical-align:baseline;white-space:pre-wrap">www.stratio.com</span></a></p><a \
href="https://twitter.com/stratiobd" target="_blank"><span \
style="font-size:10pt;font-family:Arial;color:rgb(17,85,204);background-color:transparent;vertical-align:baseline;white-space:pre-wrap"><img \
src="https://lh3.googleusercontent.com/hOlHqJK94rZ7nBo9gzYKhLiwgogX1sgXV81pPDpAHHyRVeC \
jHxw0THNCq19zhcZalZiYeVYt9r4T_7LhoeLMxN1eTMnG46IfttV83WkTGC3jL1z04craZ8mmUn9hNnxDTIgh4_cT" \
style="border:none" width="20" height="20"></span></a><a \
href="https://www.linkedin.com/company/stratiobd" target="_blank"><span \
style="font-size:10pt;font-family:Arial;color:rgb(17,85,204);background-color:transparent;vertical-align:baseline;white-space:pre-wrap"><img \
src="https://lh6.googleusercontent.com/xs2vZh2SrmNnOaJO9i07vQzFNoghAaZytG_Zh09D9-ESjgO \
v5LHzLrbVNOFa-e3g5FYdmeg-kj6Ur6hID_h1joaSCdsHETfxTNqNSXan5nxBdGtmxq6NMWoh6puraVK1JGPxzhzX" \
style="border:none" width="20" height="20"></span></a><a \
href="https://www.youtube.com/c/StratioBD" target="_blank"><span \
style="font-size:8pt;font-family:Arial;color:rgb(17,85,204);background-color:transparent;font-style:italic;vertical-align:baseline;white-space:pre-wrap"><img \
src="https://lh3.googleusercontent.com/GdwY5Qtr3qEaHfNZszPc0lGx52_bfO00F-ge1MzYQPNLQmX \
f2jO4z2kvWwIEKi2swQ-IfLjNvpS7o4wR0nwNhnhOFd7zZ1zJDtFVZLkp52XyaM0GpEXySD2iLbWq-dw0cmXUe7Jj" \
style="border:none" width="20" \
height="20"></span></a></span><br></div></div></div></div></div></div></div></div></div></div></div></div></div>
</div>
</div></div></blockquote></div><br><br clear="all"><br>-- <br><div \
class="gmail_signature" data-smartmail="gmail_signature"><div dir="ltr"><div><div \
dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><br><div><div \
style="color:rgb(136,136,136);font-size:12.8000001907349px"><div dir="ltr"><div \
dir="ltr"><div style="color:rgb(0,0,0);font-family:'Times New \
Roman';font-size:12.7272720336914px"><span \
style="font-size:9pt;font-family:Tahoma;color:rgb(217,217,217);font-weight:700;vertical-align:baseline;white-space:pre-wrap">|</span><span \
style="font-size:9pt;font-family:Tahoma;color:rgb(102,102,102);vertical-align:baseline;white-space:pre-wrap"> \
Jose Angel Santiago</span><br></div><div \
style="color:rgb(0,0,0);font-family:'Times New \
Roman';font-size:12.7272720336914px"><div \
style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.7272720336914px"><span><br><p \
dir="ltr" style="line-height:1.656;margin-top:0pt;margin-bottom:0pt"><a \
href="http://www.stratio.com/" target="_blank"><span \
style="font-size:10pt;font-family:Arial;color:rgb(17,85,204);background-color:transparent;vertical-align:baseline;white-space:pre-wrap"><img \
src="https://lh3.googleusercontent.com/SfSwo77PKD8TgM6em8B7mlcqOV9_N-AoAASRBMbZ3PFYgdF \
IibiMdr3zR_AZbMScWJOeRN7me-R_nK6vn1rnHSbApJVGfEjendjUa7LiGewC_fPGilVYciUS7E9v4mpKpl--caud" \
width="96" height="22" style="border:none" \
alt="Logo_signature2.png"></span></a></p><p dir="ltr" \
style="line-height:1.2;margin-top:0pt;margin-bottom:0pt"><span \
style="font-size:8pt;font-family:Tahoma;color:rgb(153,153,153);background-color:transparent;vertical-align:baseline;white-space:pre-wrap">Vía \
de las dos Castillas, 33, Ática 4, 3 Planta</span></p><p dir="ltr" \
style="line-height:1.2;margin-top:0pt;margin-bottom:0pt"><span \
style="font-size:8pt;font-family:Tahoma;color:rgb(153,153,153);background-color:transparent;vertical-align:baseline;white-space:pre-wrap">28224 \
Pozuelo de Alarcón, Madrid, Spain </span></p><p dir="ltr" \
style="line-height:1.2;margin-top:0pt;margin-bottom:0pt"><span \
style="font-size:8pt;font-family:Tahoma;color:rgb(153,153,153);vertical-align:baseline;white-space:pre-wrap">+34 \
918 286 473 | </span><a href="http://www.stratio.com" target="_blank"><span \
style="font-size:8pt;font-family:Tahoma;color:rgb(17,85,204);vertical-align:baseline;white-space:pre-wrap">www.stratio.com</span></a></p><a \
href="https://twitter.com/stratiobd" target="_blank"><span \
style="font-size:10pt;font-family:Arial;color:rgb(17,85,204);background-color:transparent;vertical-align:baseline;white-space:pre-wrap"><img \
src="https://lh3.googleusercontent.com/hOlHqJK94rZ7nBo9gzYKhLiwgogX1sgXV81pPDpAHHyRVeC \
jHxw0THNCq19zhcZalZiYeVYt9r4T_7LhoeLMxN1eTMnG46IfttV83WkTGC3jL1z04craZ8mmUn9hNnxDTIgh4_cT" \
width="20" height="20" style="border:none"></span></a><a \
href="https://www.linkedin.com/company/stratiobd" target="_blank"><span \
style="font-size:10pt;font-family:Arial;color:rgb(17,85,204);background-color:transparent;vertical-align:baseline;white-space:pre-wrap"><img \
src="https://lh6.googleusercontent.com/xs2vZh2SrmNnOaJO9i07vQzFNoghAaZytG_Zh09D9-ESjgO \
v5LHzLrbVNOFa-e3g5FYdmeg-kj6Ur6hID_h1joaSCdsHETfxTNqNSXan5nxBdGtmxq6NMWoh6puraVK1JGPxzhzX" \
width="20" height="20" style="border:none"></span></a><a \
href="https://www.youtube.com/c/StratioBD" target="_blank"><span \
style="font-size:8pt;font-family:Arial;color:rgb(17,85,204);background-color:transparent;font-style:italic;vertical-align:baseline;white-space:pre-wrap"><img \
src="https://lh3.googleusercontent.com/GdwY5Qtr3qEaHfNZszPc0lGx52_bfO00F-ge1MzYQPNLQmX \
f2jO4z2kvWwIEKi2swQ-IfLjNvpS7o4wR0nwNhnhOFd7zZ1zJDtFVZLkp52XyaM0GpEXySD2iLbWq-dw0cmXUe7Jj" \
width="20" height="20" \
style="border:none"></span></a></span><br></div></div></div></div></div></div></div></div></div></div></div></div></div>
</div>
[Attachment #6 (text/plain)]
______________________________________________________________________________
Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
FAQ: http://www.balabit.com/wiki/syslog-ng-faq
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic