[prev in list] [next in list] [prev in thread] [next in thread]
List: ntop-dev
Subject: [Ntop-dev] What's new in ntop - one of a series of periodic reviews of the diffs...
From: "Burton M. Strauss III" <BStrauss3 () attbi ! com>
Date: 2002-02-28 23:35:38
Message-ID: HBEBKGPKAEJLGMLAANAOMEKIDLAA.BStrauss3 () attbi ! com
[Download RAW message or body]
(Comparing 05-Feb-2002 vs 28-Feb-2002) (In no particular order)
Common:
myGlobals - many of the global data items have been moved (in about 2 or 3
chunks) into a single structure, myGlobals). Allows id of global values, as
well as collecting them for initialization, etc. This has also entailed
some chasing of incorrect changes in comments (e.g. /* this version is */
became /* this myGlobals.version is */).
ntop now counts terminated tcp sessions along with attempted, established
and rejected.
Based on the hint in ntop_win32.c, I'm guessing it's official - 2.0.1 on
14Feb2002
The real biggie is the internal changes in hash.c - major cleanups.
Quoting Luca:
"I'm currently rewriting the hash code because too many people still
complain about it."
and then two weeks later:
"I have significantly reworked the way hash work. So [no] more rehash
and mapping errors anymore! As the code is quite new (and I have not
announced this) what do you think? Is ntop stable now?".
You will notice a number of new log messages:
Feb 28 16:20:54 tigger ntop[11623]: Extending hash size [newSize=256]
Feb 28 16:22:12 tigger ntop[11618]: Purging host (idx=0/192.168.0.254) (0
hosts purged)
Feb 28 16:22:12 tigger ntop[11618]: Purging completed in 0 sec [1 hosts
deleted]
Feb 28 16:27:13 tigger ntop[11618]: Purging host (idx=0/mail.attbi.com) (0
hosts purged)
Feb 28 16:27:13 tigger ntop[11618]: Purging host
(idx=1/dpmail09.doteasy.com) (1 hosts purged)
Feb 28 16:27:13 tigger ntop[11618]: Purging host (idx=2/wq2002.quicken.com)
(2 hosts purged)
Feb 28 16:27:13 tigger ntop[11618]: Purging host
(idx=3/q2002patch.quicken.com) (3 hosts purged)
Feb 28 16:27:13 tigger ntop[11618]: Purging host (idx=4/onlad.quicken.com)
(4 hosts purged)
Feb 28 16:27:13 tigger ntop[11618]: Purging host (idx=5/206.154.105.121) (5
hosts purged)
Feb 28 16:27:13 tigger ntop[11618]: Purging completed in 0 sec [6 hosts
deleted]
Feb 28 16:37:15 tigger ntop[11618]: Purging host (idx=0/128.121.26.136) (0
hosts purged)
Feb 28 16:37:15 tigger ntop[11618]: Purging host (idx=1/128.121.26.135) (1
hosts purged)
Feb 28 16:37:15 tigger ntop[11618]: Purging completed in 0 sec [2 hosts
deleted]
and now, two weeks later, after a few bug fixes, some minor tweaks, a couple
of segfaults, but also a VAST improvement, the unanounced & experimental
code has been proven and moved from #ifdef EXPERIMENTAL blocks to the real
thing, I have an answer:
Yes, Luca, ntop is a lot more stable now!
What do you think? Is it time for 2.1??
========================by
file================================================
acconfig.h - 64bit counters - tests for HAVE_U_INT64_T/HAVE_INT64_T
address.c - minor tweaks in resolveAddress
- ipaddr2str split to make part of it's function available as
fetchAddressFromCache
- handleDNSpacket won't return a too long answer
ChangeLog - updated through 12Feb2002 based on my notes - will push this
note to Luca as another update.
config.h.in - 64bit counters and zlib defines
configure.in - **MAJOR** Builds shared libraries by default
Specifically, AM_DISABLE_SHARED was commented out (dnl>
AM_DISABLE_SHARED) @ 195. This changes the generated configure file (help
text):
--enable-shared[=PKGS] build shared libraries [default=no]
becomes
--enable-shared[=PKGS] build shared libraries [default=yes]
- tests for 64bit counters added
dataFormat.c - Logic in formatPkts to scale calculations over 1,000,000,000
packets
emitter.c - Massive rewrite for XML
- free with no filter (discussed on this list) SEGV fixed.
globals-core.c - initNtopGlobals - lots of changes - mostly better defaults
and more things...
NOTE: access log filename disabled by default
by default ntop will merge network interfaces
globals-report.c - gutted - used to do some initialization for reporting,
that's now in the myGlobals inits.
globals-report.h - getProtocolDataSent and getProtocolDataReceived removed
from reportUtils.c to
- A few other function changes (parameters added)...
globals.h - more stuff related to myGlobals
graph.c - temp file names have changed from getNewRandomFile(fileName,
NAME_MAX); to fdopen(abs(myGlobals.newSock), "ab");
- Big diff, actualReportDeviceId is now
myGlobals.actualReportDeviceId
hash.c - LOTS OF UNDER THE COVERS CHANGES. Shouldn't affect the users
(other than to eliminate bad data, SEGVs, etc.), but hard on the developer.
- The serialCache is stored into @ 768 (see below)
http.c - minor clean up and what looks like a separate set of .html.j files
for border-sniffer-mode...
- parameter consolidation/cleanup in printHostsTraffic to prevent the
Send/Rcvd and sorting problems.
- Graph generation tests moved ?fix multithread problem??
initialize.c - experimental code to size the hash list now regular.
- serialCache.db now deleted if it exists. What is the
serialCache.db???
- The message "Possible solution: please use '-P <directory>'"
on gdbm open problems removed (it's as often a 2nd copy running as a bad -P
parameter)
- If no devices, a dummy is created to prevent crashes.
main.c - -s <max hash size (default 32768)> removed from usage() message and
the code, but NOT the getOpts string (OOPS)
-B <filter> added to getOpts string - it was already in the code.
Makefile.am - All .db files removed by make dist-clean
- ntop-rules.8 added as man file
- ntop.html build directly from ntop.8 not indirectly from
ntop.8 -> ntop.txt -> ntop.html
NEWS - added 2.0 release
ntop.8 - major change to add long named options and some other cleanups I
did
- Idle hosts scan loop delay changed from parameter
SESSION_SCAN_DELAY to 300 /* 5 minutes */
- lsof loop - not for win32
ntop.h - added terminated sessions function to plugins
- IDLE_HOST_PURGE_TIMEOUT changed from 30 -> 10 minutes (comment is
still wrong)
ntop_win32.c - internal version # is now "2.0.1", buildDate = "14/02/2002"
pbuf.c - getHostInfo moved to hash.c
instance in use (designed to that the other thread doing the
purging would skip over active entries, w/o locking the whole structure) is
gone.
plugin.c - sflowPlugin
- notify plugins of hash resize is gone!
- wap plugin is commented out
README - changed to this:
See files in docs/ for more info.
For instructions on how to build ntop from source, see
docs/BUILD-NTOP.txt (*nix) and docs/BUILD-MinGW.txt (Win32)
Online information is at http://www.ntop.org, including archives
of the ntop mailings lists.
Contributions of effort towards writing, rewriting and updating
ntop's documentation would be heartily welcomed!
report.c - the prev/next page buttons acquired ALT tags.
- printHostsTraffic looks like a total rewrite...
- Remote Traffic report in IP Protos | Distribution report.
*I really didn't mean for this to go it, it was to show that I was
having problems with all the traffic being classified as remote. But, with
a promisc. mode nic on a cable modem, it's occasionally interesting to see
what's out there. The real use would be for a sysadmin to see "remote"
traffic as a way of picking up pirate/unassigned networks*
reportUtils.c - whatever the "serialcache" is, it's referenced in
formatUsageCounter.
- heading generation cleanup in printHeader (the single report
code vs. two flags previously used)
- Added "Further Host Information" (a whois lookup) to the
"Info about host" report.
Note this is hard-coded to use http://www.radb.net/ I have to tell you,
some of the stuff in that database is a scream:
route: 63.160.0.0/12
descr: Proxy-registered route object for Sprint :-)
origin: AS1239
remarks: auto-generated route object
remarks: this next line gives the robot something to recognize
remarks: The quick brown fox jumped over the lazy dog.
remarks:
remarks: This route object is for a Sprint customer route
remarks: which is being exported under this origin AS.
remarks:
remarks: This route object was created because no existing
remarks: route object with the same origin was found, and
remarks: we really just wanted to help out those poor Sprint
remarks: folks who have an aversion to registering routes.
remarks:
remarks: We hope they have a sense of humor.
remarks:
remarks: Please contact WeLoveThoseCrazySprintFolks@Level3.net
remarks: if you have any questions regarding this object.
mnt-by: SPRINT-MNT
changed: WeLoveThoseCrazySprintFolks@Level3.net 20011018
source: LEVEL3
And completely different from ARIN
sessions.c - RESCAN_LIST: is gone!
- handlePluginSessionTermination is here
sql.c - mySQLnotifyTCPSession(session, actualDeviceId); added
util.c - Win32 "log" messages - which can't use syslog - will now have a
pretty prefix of the date/time.
webInterface.c - The configuration report pointer about UCD-SNMP is
commented out.
Top Hash size is gone too...
plugIns: traceEvent(TRACE_ERROR, "Buffer overflow!"); -> BufferOverflow()
change
sflowPlugin (Makefile.am, etc.)
============================================================================
=========
No real clue what the serialCache is... it's stored in hash.c in
getHostInfo, like this:
#ifdef HASH_DEBUG
traceEvent(TRACE_INFO, "Adding %s/%s
[idx=%d][device=%d][actualHashSize=%d]\n",
el->ethAddressString, el->hostNumIpAddress, list->idx,
actualDeviceId,
myGlobals.device[actualDeviceId].actualHashSize);
#endif
addSerialMapping(el);
addSerialMapping is (snipping for clarity):
if(el->hostNumIpAddress[0] != '\0')
data_data.dptr = el->hostNumIpAddress;
else
data_data.dptr = el->ethAddressString;
data_data.dsize = strlen(data_data.dptr)+1;
sprintf(tmpBuf, "%u", el->hostSerial);
key_data.dptr = tmpBuf;
key_data.dsize = strlen(tmpBuf)+1;
gdbm_store(myGlobals.serialCache, key_data, data_data, GDBM_REPLACE) != 0)
Which is:
char ethAddressString[18];
char hostNumIpAddress[17], *fullDomainName;
hostSerial is a global counter:
hash.c: 612 el->hostSerial =
myGlobals.serialCounter++;
It looks like this is just a database of addresses we've seen, perhaps to
eliminate some search in formatUsageCounter in reportUtils.c??? Doing a
strings on it give stuff like this:
192.168.42.3
204.127.198.7
00:A0:CC:79:37:3D
192.168.42.6
209.153.216.9
192.168.42.2
============================================================================
=========
As always, this is based on my own knowledge, reading and interpretation of
the diffs. It could be right, wrong or dangerous. YMMV...
============================================================================
=========
Prior messages:
Current build (02-02-12) is a MAJOR improvement
Tue, 12 Feb 2002 16:50:41 -0600
http://listmanager.unipi.it/pipermail/ntop/2002-February/001174.html
External Changes - New flags, options, code changes - and a warning on big
"diffs".
Mon, 4 Feb 2002 16:22:49 -0600
http://listmanager.unipi.it/pipermail/ntop/2002-February/001125.html
Summary of problems *I* know about...
Wed, 16 Jan 2002 19:49:41 -0600
http://listmanager.unipi.it/pipermail/ntop/2002-January/000963.html
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic