[prev in list] [next in list] [prev in thread] [next in thread] 

List:       ntop
Subject:    RE: [Ntop] Amount of traffic displayed in data
From:       "Burton Strauss" <Burton () ntopSupport ! com>
Date:       2006-03-31 18:05:54
Message-ID: 094301c654ed$c3583cc0$648ea8c0 () burtonstrauss ! local
[Download RAW message or body]

Looks like I missed half the change - not only the mask has to change, but
the cast needs to be stripped - otherwise it's going to be reading 64 bits
where there are only 32 meaningful ones...

static void wrtLlongItm(FILE *fDescr, int lang, char* indent, char* name,
                        TrafficCounter value, char last, int numEntriesSent)
{
  char buf[80];

  safe_snprintf(__FILE__, __LINE__, buf, sizeof(buf),  "%lu", (long unsigned
int)value.value);
  wrtStrItm(fDescr, lang, indent, name, buf, last, numEntriesSent);
}

s/b

static void wrtLlongItm(FILE *fDescr, int lang, char* indent, char* name,
                        TrafficCounter value, char last, int numEntriesSent)
{
  char buf[80];

  safe_snprintf(__FILE__, __LINE__, buf, sizeof(buf),  "%llu", value.value);
  wrtStrItm(fDescr, lang, indent, name, buf, last, numEntriesSent);
} 

TrafficCounter is defined as 

typedef struct trafficCounter {
  Counter value;
  u_char modified;
} TrafficCounter;

Where Counter itself is:

#ifdef WIN32
typedef __int64 Counter;
#else
typedef unsigned long long Counter;
#endif

So you need both pieces of the change...

-----Burton



-----Original Message-----
From: ntop-bounces@unipi.it [mailto:ntop-bounces@unipi.it] On Behalf Of
Shawn Wall
Sent: Friday, March 31, 2006 10:44 AM
To: ntop@unipi.it
Subject: RE: [Ntop] Amount of traffic displayed in data
columncomparedtoexportfrom ntopdump

Port was not used for this test. I downloaded the source, change the
emitter.c as detailed by Burton and compiled. 

Shawn

-----Original Message-----
From: ntop-bounces@unipi.it [mailto:ntop-bounces@unipi.it] On Behalf Of Gary
Gatten
Sent: Friday, March 31, 2006 9:37 AM
To: ntop@unipi.it
Subject: RE: [Ntop] Amount of traffic displayed in data
columncomparedtoexport from ntopdump

I recall somone  - Burton maybe, saying don't use the ports - use CVS
instead.  Not sure if this will matter or not, just thought I'd through it
out.

Gary


>>> shawn@clearwave.ca 3/31/2006 10:32:31 AM >>>
I made the change, recompilied etc. Now the output is:

key|hostResolvedName|pktSent|pktRcvd|ipBytesSent|ipBytesRcvd|bytesMultic
key|hostResolvedName|pktSent|pktRcvd|ipBytesSent|ipBytesRcvd|astS
ent|pktMulticastSent|bytesMulticastRcvd|pktMulticastRcvd|bytesSent|bytes
ent|pktMulticastSent|bytesMulticastRcvd|pktMulticastRcvd|bytesSent|Rcvd
|ipBytesSent|ipBytesRcvd|ipv6Sent|ipv6Rcvd|tcpBytesSent|tcpBytesRcvd|udp
|ipBytesSent|ipBytesRcvd|ipv6Sent|ipv6Rcvd|tcpBytesSent|tcpBytesRcvd|Byt
|ipBytesSent|ipBytesRcvd|ipv6Sent|ipv6Rcvd|tcpBytesSent|tcpBytesRcvd|e
sSent|udpBytesRcvd|icmpSent|icmpRcvd|
x.x.x.x|x.x.x.x|2883923492468686851|2883923492468686849|2883923492468690928|
2883923492468687287|2883923492468686848|2883923492468686848|288392349246
2883923492468687287|2883923492468686848|2883923492468686848|8686
848|2883923492468686848|2883923492468690928|2883923492468687287|28839234
848|2883923492468686848|2883923492468690928|2883923492468687287|9246
8690928|2883923492468687287|2883923492468686848|2883923492468686848|2883
8690928|2883923492468687287|2883923492468686848|2883923492468686848|9234
92468690928|2883923492468687287|2883923492468686848|2883923492468686848|2883
923492468686848|2883923492468686848| 

I've run it several times, the results are the same for all hosts.

Any other ideas?

Thanks,

Shawn

-----Original Message-----
From: ntop-bounces@unipi.it [mailto:ntop-bounces@unipi.it] On Behalf Of
Burton Strauss
Sent: Wednesday, March 29, 2006 10:19 AM
To: ntop@unipi.it
Subject: RE: [Ntop] Amount of traffic displayed in data column
comparedtoexport from ntopdump

Probably truncation ... In emitter.c, change this:

static void wrtLlongItm(FILE *fDescr, int lang, char* indent, char* name,
                        TrafficCounter value, char last, int
numEntriesSent)
{
  char buf[80];

  safe_snprintf(__FILE__, __LINE__, buf, sizeof(buf),  "%lu", (long unsigned
int)value.value);
  wrtStrItm(fDescr, lang, indent, name, buf, last, numEntriesSent); }

To

static void wrtLlongItm(FILE *fDescr, int lang, char* indent, char* name,
                        TrafficCounter value, char last, int
numEntriesSent)
{
  char buf[80];

  safe_snprintf(__FILE__, __LINE__, buf, sizeof(buf),  "%llu", (long
unsigned int)value.value);
  wrtStrItm(fDescr, lang, indent, name, buf, last, numEntriesSent); } 

(%lu to %llu) and let us know.

-----Burton

-----Original Message-----
From: ntop-bounces@unipi.it [mailto:ntop-bounces@unipi.it] On Behalf Of
Shawn Wall
Sent: Wednesday, March 29, 2006 9:57 AM
To: ntop@listgateway.unipi.it
Subject: [Ntop] Amount of traffic displayed in data column compared toexport
from ntopdump

Hi list,

OK, I've searched the archives, read the FAQ and consulted the contrib docs
and I haven't been able to find anything specific to this topic. If it is
there, I have missed it and I apologize in advance.

Anyway, here's my setup:

Ntop v3.2 from FreeBSD ports
FreeBSD 6.0-Release
Libcap 0.9.4
Startup cmd: ntop -w 0 -W 3000 -doncgm x.x.0.0/16 --skip-version-check -u
ntop Using netflow plugin, receiving v5 from Cisco 2621

Network topology

|----Net 1 x.x.x.0/24----|
				  <--2924----2621----PIX---->www
|----Net 2 x.x.x.0/24----|  |
				    |
				   NTOP

Both net1 and net2 have the same 1st and 2nd octet. 

Everything works, ntop collects flows, reports usage, etc. Happy there.
Now
I'm starting to use cron jobs to dump data from ntop using wget. E.g.
/usr/local/bin/wget -O /usr/local/etc/ntopdump/ntopdumptbl
--no-check-certificate
"https://admin:xxxxx@127.0.0.1:3000/dumpData.html?language=txt&view=short".
Here's where I'm running into trouble. I'll use a specific host to
illustrate my problem.

The output from the wget cmd above produces a file containing all the
specific data ntop has collected in short form:

key|hostResolvedName|pktSent|pktRcvd|ipBytesSent|ipBytesRcvd|bytesMultic
key|hostResolvedName|pktSent|pktRcvd|ipBytesSent|ipBytesRcvd|astS
ent|pktMulticastSent|bytesMulticastRcvd|pktMulticastRcvd|bytesSent|bytes
ent|pktMulticastSent|bytesMulticastRcvd|pktMulticastRcvd|bytesSent|Rcvd
|ipBytesSent|ipBytesRcvd|ipv6Sent|ipv6Rcvd|tcpBytesSent|tcpBytesRcvd|udp
|ipBytesSent|ipBytesRcvd|ipv6Sent|ipv6Rcvd|tcpBytesSent|tcpBytesRcvd|Byt
|ipBytesSent|ipBytesRcvd|ipv6Sent|ipv6Rcvd|tcpBytesSent|tcpBytesRcvd|e
sSent|udpBytesRcvd|icmpSent|icmpRcvd|
x.x.x.x|x.x.x.x|32312562|33865785|3198778699|1752536103|0|0|0|0|3198778699|1
752536103|3198778699|1752536103|0|0|2471215495|966711354|727117640|77877
752536103|3198778699|1752536103|0|0|2471215495|966711354|727117640|9448
|445564|7045301|

As I understand it, if I were to add the ipBytesSent and ipBytesRcvd that
should give me the total amount of data this host has sent and/or received.
Let's try:

ipBytesSent+ipBytesRcvd = 3198778699 + 1752536103 = 4951314802
ipBytesSent+/1073741824
(convert to GB) = 4.611271249 GB

In the browser, ntop reports 26.7 GB in the data column for this host, which
is an accurate reflection of the amount of traffic accumulated in all the
columns listed. The two totals are not even close.

Now I'll add in all the traffic counters and see if that gives me the same
total as the ntop browser:

ipBytesSent+ipBytesRcvd+bytesMulticastSent+bytesMulticastRcvd+tcpBytesSent+t
cpBytesRcvd+udpBytesSent+updBytesRcvd+icmpSent+icmpRcvd (omitted 
cpBytesRcvd+udpBytesSent+updBytesRcvd+icmpSent+bytesSent,
bytesRvcd and the sent set of ipBytesSent, ipBytesRcvd since the values are
identical to the first ipBytesSent and ipBytesRcvd)

3198778699 + 1752536103 + 2471215495 + 966711354 + 727117640 +
778779448 +
445564 +	7045301 = 9902629604 /1073741824 = 9.222542498 GB

That's still not close at all. Now let's try using the long form of a dump.

key|index|hostNumIpAddress|hostResolvedName|firstSeen|lastSeen|minTTL|ma
key|index|hostNumIpAddress|hostResolvedName|firstSeen|lastSeen|minTTL|xT
key|index|hostNumIpAddress|hostResolvedName|firstSeen|lastSeen|minTTL|TL
|pktSent|pktRcvd|ipBytesSent|ipBytesRcvd|pktDuplicatedAckSent|pktDuplica
|pktSent|pktRcvd|ipBytesSent|ipBytesRcvd|pktDuplicatedAckSent|tedA
ckRcvd|pktBroadcastSent|bytesMulticastSent|pktMulticastSent|bytesMultica
ckRcvd|pktBroadcastSent|bytesMulticastSent|pktMulticastSent|stRc
vd|pktMulticastRcvd|bytesSent|bytesSentLoc|bytesSentRem|bytesRcvd|bytesR
vd|pktMulticastRcvd|bytesSent|bytesSentLoc|bytesSentRem|bytesRcvd|cvdL
oc|bytesRcvdFromRem|actualRcvdThpt|lastHourRcvdThpt|averageRcvdThpt|peak
oc|bytesRcvdFromRem|actualRcvdThpt|lastHourRcvdThpt|averageRcvdThpt|Rcvd
Thpt|actualSentThpt|lastHourSentThpt|averageSentThpt|peakSentThpt|actual
Thpt|actualSentThpt|lastHourSentThpt|averageSentThpt|peakSentThpt|TThp
t|averageTThpt|peakTThpt|actualRcvdPktThpt|averageRcvdPktThpt|peakRcvdPk
t|averageTThpt|peakTThpt|actualRcvdPktThpt|averageRcvdPktThpt|tThp
t|actualSentPktThpt|averageSentPktThpt|peakSentPktThpt|actualTPktThpt|av
t|actualSentPktThpt|averageSentPktThpt|peakSentPktThpt|actualTPktThpt|er
t|actualSentPktThpt|averageSentPktThpt|peakSentPktThpt|actualTPktThpt|ag
eTPktThpt|peakTPktThpt|ipBytesSent|ipBytesRcvd|ipv6Sent|ipv6Rcvd|tcpByte
eTPktThpt|peakTPktThpt|ipBytesSent|ipBytesRcvd|ipv6Sent|ipv6Rcvd|sSen
t|tcpBytesRcvd|udpBytesSent|udpBytesRcvd|icmpSent|icmpRcvd|tcpSentRem|ud
t|tcpBytesRcvd|udpBytesSent|udpBytesRcvd|icmpSent|icmpRcvd|tcpSentRem|pS
t|tcpBytesRcvd|udpBytesSent|udpBytesRcvd|icmpSent|icmpRcvd|tcpSentRem|en
tLoc|udpSentRem|tcpRcvdLoc|tcpRcvdFromRem|udpRcvdLoc|udpRcvdFromRem|tcpF
tLoc|udpSentRem|tcpRcvdLoc|tcpRcvdFromRem|udpRcvdLoc|udpRcvdFromRem|ragm
entsSent|tcpFragmentsRcvd|udpFragmentsSent|udpFragmentsRcvd|icmpFragment
entsSent|tcpFragmentsRcvd|udpFragmentsSent|udpFragmentsRcvd|sSen
t|icmpFragmentsRcvd|key|key|sentLoc|sentRem|rcvdLoc|rcvdFromRem|ethAddre
t|icmpFragmentsRcvd|key|key|sentLoc|sentRem|rcvdLoc|rcvdFromRem|ssSt
ring|

Same as before, this time I'm including every value that shows up in the
dump for the same host:

139.142.196.166|0|139.142.196.166|139.142.196.166|731748329|1143578455|0|0|3
2312373|33865636|3198770541|1752511588|68|5|32306679|0|0|0|0|3198770541|5700
53|3198200488|1752511588|788977|1751722611|9.07|4366.93|26843.82|2879347
53|3198200488|1752511588|788977|1751722611|.25|
9.05|2969.93|16714.95|2818646.00|18.12|43558.77|3465301.00|0.15|48.02|2993.7
5|0.18|45.82|3002.22|0.33|93.83|5995.97|3198770541|1752511588|0|0|247120
5|9960
|966691967|727115017|778774320|445564|7045301|2470950737|193398|726921619|59
806|966632161|729171|778045149|0|0|0|0|0|0|IP|FTP|0|591132|0|242518|HTTP|520
7|74858203|31804|1052500249|DNS|190942|230948|726668|896911|Telnet|0|13559|0
|17754|Mail|0|13464|0|19802|DHCP-BOOTP|0|588|0|819|SNMP|0|1223|0|232|NNTP|0|
15330|0|39091|NFS_AFS|0|1356|0|1803|VoIP|0|83680|0|83339|X11|0|390597|0|
15330|0|39091|3843
986|SSH|0|6114|0|5329|Gnutella|0|198342311|0|144756579|Kazaa|0|2614|0|2593|W
inMX|0|28130|0|47069|eDonkey|0|2591421389|0|2816039360|BitTorrent|180|26
inMX|0|28130|0|47069|eDonkey|0|2591421389|0|2816039360|BitTorrent|180|68
inMX|0|28130|0|47069|eDonkey|0|2591421389|0|2816039360|BitTorrent|180|93
4660|259|3418157189|Messenger|0|53914106|0|77345222|securityPkts|0|0|4|1|1|0
|0|0|0|0|20|12|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0
|0||

I won't spell it out, but the total in GB = 21.45643944 GB. 

So what am I missing? How come I don't get the same totals as ntop reports
in the browser? 

I've also attached this email as a doc, since it will get mangled.
								  
Shawn Wall

_______________________________________________
Ntop mailing list
Ntop@unipi.it
http://listgateway.unipi.it/mailman/listinfo/ntop 


--
No virus found in this incoming message.
Checked by AVG Free Edition.
Version: 7.1.385 / Virus Database: 268.3.3/296 - Release Date:
3/29/2006

_______________________________________________
Ntop mailing list
Ntop@unipi.it
http://listgateway.unipi.it/mailman/listinfo/ntop


_______________________________________________
Ntop mailing list
Ntop@unipi.it
http://listgateway.unipi.it/mailman/listinfo/ntop


-- 
No virus found in this incoming message.
Checked by AVG Free Edition.
Version: 7.1.385 / Virus Database: 268.3.4/299 - Release Date: 3/31/2006


_______________________________________________
Ntop mailing list
Ntop@unipi.it
http://listgateway.unipi.it/mailman/listinfo/ntop

_______________________________________________
Ntop mailing list
Ntop@unipi.it
http://listgateway.unipi.it/mailman/listinfo/ntop
[prev in list] [next in list] [prev in thread] [next in thread]