[prev in list] [next in list] [prev in thread] [next in thread] 

List:       ntop
Subject:    Re: [Ntop] [Ntop-dev] DNS name resolution stops working
From:       Filippo Fontanelli <fontanelli () ntop ! org>
Date:       2014-01-07 23:28:49
Message-ID: 1EE32784-8C34-465A-88F5-9AB22BA7B7E0 () ntop ! org
[Download RAW message or body]

Hi Andrew

Today i done some tests to recreate your issues but I can not do it. =


As anticipated by luca the behaviour you describe is normal because you can=
=92t increase the frequency with which the records are popped. =


In order to increase the numbers of dns.toresolve list,  you can increase t=
he MAX_NUM_QUEUED_ADDRS define how you want. Please test this change and le=
t me know if it works.

Best regards,

Filippo


On 06 Jan 2014, at 22:15, Andrew Rechenberg Lists <flux-ml@resurgent.com> w=
rote:

> Thanks for the reply Luca.
> =

> Here is the full command line I'm using:
> =

> ntopng -i tcp://127.0.0.1:5556 -i tcp://127.0.0.1:2001 -i tcp://127.0.0.1=
:2013 -i tcp://127.0.0.1:2018 -m [LOCALADDRESSES] -n 1 -X 200000 -p /etc/nt=
opng/protos.txt
> =

> We have multiple devices sending flow data to different ports and I have =
nprobe running as:
> =

> nprobe --zmq tcp://*:5556 -i none -n none --collector-port [INCOMINGPORT]
> =

> I have one nprobe running for each port on which I'd like to collect data=
.  The majority of the flow data is coming through the 5556 port and I've t=
ried just listening on one port with one nprobe collector and have DNS cach=
e disappear.
> =

> Even if I set the default -n 0 option the DNS resolution still halts afte=
r some time.
> =

> I've re-run my tests with the current SVN version and I'm still having th=
e resolution issues (here is the version reported via the web interface - G=
enerated by ntopng v.1.1.1 (r7147)).
> =

> How often are records in the dns.toresolve list popped to resolve?  Is th=
ere a way I can increase the number of records popped or increase the frequ=
ency with which they are popped?
> =

> Thanks again for the assistance.
> =

> Cheers,
> Andrew.
> =

> -----Original Message-----
> From: ntop-dev-bounces@listgateway.unipi.it [mailto:ntop-dev-bounces@list=
gateway.unipi.it] On Behalf Of Luca Deri
> Sent: Monday, January 6, 2014 11:56
> To: ntop-dev@unipi.it
> Cc: ntop@listgateway.unipi.it; ntop-dev@listgateway.unipi.it
> Subject: Re: [Ntop-dev] DNS name resolution stops working
> =

> Andrew
> before tuning the system, i would like to understand how you have configu=
red ntong. Can you please send me the complete command line you have used? =
By default we resolve only the local addresses as if you have a heavy loade=
d network the behaviour you describe is normal.
> =

> Please re-run your tests using the latest ntopng code in SVN
> =

> Regards Luca
> =

> On 03 Jan 2014, at 16:50, Andrew Rechenberg Lists <flux-ml@resurgent.com>=
 wrote:
> =

>> Good day,
>> =

>> I'm using ntopng 1.1 and redis 2.8.3 on CentOS 6.5 x86_64 on a system wi=
th 2 dual core Xeon 5130 2.00GHz with 8GB of RAM.  When ntopng is first sta=
rted name resolution works fine.  After a short period of time DNS name res=
olution stops working.
>> =

>> I understand that the name records are store in redis and looking at the=
 code they are cached for 300 seconds.  When I look at the redis keys after=
 about 5 minutes of ntopng uptime most of the dns cache records have been r=
eplaced IP.json records.  The dns.toresolve list in redis is also almost al=
ways at the 500 record limit.
>> =

>> If I trim the dns.toresolve list a few times by using the redis-cli comm=
and:
>> =

>> LTRIM dns.toresolve 0 0
>> =

>> and then refresh the Active Flows list a few times then the host IPs are=
 then resolved to names.
>> =

>> The web interface shows 2300-2600 hosts and 25000-140000 flows.
>> =

>> Currently the number of keys in the redis database range from about 512-=
2000.  It appears that the redis database isn't growing to accommodate all =
of the hosts that our instance needs to store.  The redis-server process is=
 running with the default options and is consuming about 50MB of system RAM=
.  I tried changing the maxmemorypolicy to noeviction but that didn't seem =
to make a difference.  =

>> =

>> It also appears that the name resolution queue isn't being processes fas=
t enough and the most frequent hosts with flows are having their dns.cache.=
* records expire in redis and never get placed back on the queue because it=
 is already 500+
>> =

>> What can be done to make sure that DNS name resolution always operates p=
roperly?  Do I need to change the MAX_NUM_QUEUED_ADDRS define?  Is there a =
way that I can increase the number of keys that the redis database will sto=
re in memory?  Is there a way to process the DNS host resolution queue fast=
er?
>> =

>> Thank you for your asssitance.
>> =

>> Cheers,
>> Andrew.
>> Confidentiality Notice: This e-mail message including attachments, if an=
y, is intended only for the person or entity to which it is addressed and m=
ay contain confidential and/or privileged material. Any unauthorized review=
, use, disclosure or distribution is prohibited. If you are not the intende=
d recipient, please contact the sender by reply e-mail and destroy all copi=
es of the original message. If you are the intended recipient, but do not w=
ish to receive communications through this medium, please so advise the sen=
der immediately.
>> _______________________________________________
>> Ntop-dev mailing list
>> Ntop-dev@listgateway.unipi.it
>> http://listgateway.unipi.it/mailman/listinfo/ntop-dev
> =

> _______________________________________________
> Ntop-dev mailing list
> Ntop-dev@listgateway.unipi.it
> http://listgateway.unipi.it/mailman/listinfo/ntop-dev
> Confidentiality Notice: This e-mail message including attachments, if any=
, is intended only for the person or entity to which it is addressed and ma=
y contain confidential and/or privileged material. Any unauthorized review,=
 use, disclosure or distribution is prohibited. If you are not the intended=
 recipient, please contact the sender by reply e-mail and destroy all copie=
s of the original message. If you are the intended recipient, but do not wi=
sh to receive communications through this medium, please so advise the send=
er immediately.
> _______________________________________________
> Ntop mailing list
> Ntop@listgateway.unipi.it
> http://listgateway.unipi.it/mailman/listinfo/ntop

_______________________________________________
Ntop mailing list
Ntop@listgateway.unipi.it
http://listgateway.unipi.it/mailman/listinfo/ntop
[prev in list] [next in list] [prev in thread] [next in thread]