[prev in list] [next in list] [prev in thread] [next in thread]
List: timekeepers-announce
Subject: [time-announce] scripts for monitoring ntp client usage
From: wayne () schlitt ! net (wayne)
Date: 2005-04-16 12:55:54
Message-ID: x43btty02t.fsf () footbone ! schlitt ! net
[Download RAW message or body]
Hi NTP pool project members!
The use of the NTP pool has grown a lot over the last year or so,
which is great. This means that most pool servers now have more than
600 active clients, which is the most that the stock NTP servers can
track. Moreover, the NTP programs do not provide the detailed
statistics needed to identify abusive clients and to track long term
growth of the number of clients.
Over the last few months, I have developed a few scripts to solved
those short comings. Using these scripts I can see that I currently
have one NTP client that is sending over 10 requests per second for
the last several days and another that is sending one request every
16384 seconds (4.5 hours). I have also learned that most NTP
"clients" only send a few requests, probably as part of a boot up
script.
You can get a copy of these scripts from:
http://www.schlitt.net/scripts/ntp/
While it is great to see so many people (tens of millions?) using the
pool project, there have been some questions about the rate of growth
of the clients compared with the rate of growth of servers and also
whether the percentage of abusive clients has been growing.
One of the commands, ntp_pool_dns, will let you do long term tracking
of the pool usage of your server. I *really* would like to encourage
as many people to start collecting data as soon as possible. This is
the only way we will be able to predict problems before they happen
and to take the *correct* actions to prevent the problems.
The scripts produce about the same kind of information as the monlist
command, which is by default, available to everyone. So, I have made
these scripts produce output that, by default, is available to via
your web server. (If you have one.)
See the output generated by my system at:
http://www.schlitt.net/ntpstats/
If others could post to the timekeepers discussion list where their
data is being published, it would let any of us take the initiative to
do pool-wide analysis.
While the ntp_pool_dns script will let us do long-term analysis, the
ntp_clients_stats script will give you good short-term analysis. Here
is some example output of the ntp_clients_stats script:
--- cut here ---
Estimated active ntp pool clients: 1260
Estimated abusive ntp pool clients: 24
Estimated inactive ntp pool clients: 1218
Total ntp pool clients being tracked: 2478
Note: NTP is a stateless and connectionless (UDP based) protocol, so
exact numbers can't be determined.
5569802 ntp requests, in total, have been seen since 03/22/05 15:59:28
4498261 (80.8%) are from clients that are still active.
Long term request rate: 0.090 seconds between requests (11.10 req/sec)
Long term bandwidth in: 0.824 KBytes/s 6.593 Kbits/s
Long term bandwidth in: 2.037 GB/month 16.297 Gb/month
Current request rate: 0.070 seconds between requests (14.27 req/sec)
Current bandwidth in: 1.059 KBytes/s 8.475 Kbits/s
Current bandwidth in: 2.619 GB/month 20.949 Gb/month
(NTP packets are usually 76 bytes, UDP overhead included, in each direction.)
The dump file was written 16 seconds ago, at 03/28/05 11:19:45
Subnets with many clients:
# of Subnet Total Aggregate Abusive
IPs Count Rate Clients
8 146.88.3.x 3605 133.479 0
7 142.104.61.x 3269 154.065 0
6 208.34.86.x 8 6875.873 0
5 206.222.212.x 57501 17.624 0
4 146.88.1.x 1337 140.771 0
4 146.88.10.x 1401 360.059 0
4 64.32.210.x 46 178.182 0
Clients with rapid updates (min requests of 100):
Rank First Seen Client IP Requests Rate Usage Cumulative
1 03/23/05 04:29:38 194.66.78.15 2001526 0.08 44.50% 44.50% * !
2 03/22/05 15:59:29 134.231.11.189 427465 1.00 9.50% 54.00% * !
3 03/22/05 15:59:30 62.8.146.38 244809 1.75 5.44% 59.44% * !
4 03/22/05 15:59:31 207.173.206.5 107340 5.31 2.39% 61.83% * !
5 03/22/05 15:59:38 217.206.70.3 59829 7.79 1.33% 63.16% * !
6 03/22/05 16:00:07 64.19.16.106 40889 12.23 0.91% 64.07% * !
7 03/24/05 22:54:57 24.1.246.65 36112 7.79 0.80% 64.87% * !
8 03/22/05 16:06:36 69.17.112.3 32424 122.58 0.72% 65.59% * !
9 03/22/05 15:59:30 216.80.120.246 31893 15.01 0.71% 66.30% * !
10 03/22/05 15:59:36 212.76.33.65 31816 15.02 0.71% 67.01% * !
11 03/25/05 11:18:05 198.147.40.60 31573 7.80 0.70% 67.71% * !
12 03/22/05 15:59:41 68.6.159.149 28991 16.73 0.64% 68.35% * !
13 03/22/05 16:01:28 206.222.212.234 24419 26.99 ( 0.54%)
14 03/24/05 20:35:59 203.35.192.40 20058 15.01 0.45% 68.80% * !
15 03/22/05 15:59:33 66.178.130.209 19851 28.54 0.44% 69.24% * !
16 03/24/05 20:02:44 192.168.3.203 18577 16.00 0.41% 69.65% * !
17 03/22/05 15:59:53 83.149.199.68 17489 30.31 0.39% 70.04% * !
18 03/22/05 15:59:56 129.22.146.232 16644 30.15 0.37% 70.41% *
19 03/22/05 15:59:51 66.96.18.63 16357 26.43 0.36% 70.77% *
20 03/22/05 15:59:47 167.206.137.162 16074 30.17 0.36% 71.13% *
21 03/22/05 15:59:40 216.68.225.2 16068 30.15 0.36% 71.49% *
22 03/22/05 15:59:46 193.205.140.226 16050 30.16 0.36% 71.85% *
23 03/22/05 15:59:56 67.136.4.46 16042 30.06 0.36% 72.20% *
24 03/22/05 15:59:46 68.115.66.35 16026 30.16 0.36% 72.56% *
25 03/22/05 15:59:42 67.64.154.225 16023 30.06 0.36% 72.92% *
26 03/22/05 15:59:52 67.141.187.217 16022 30.11 0.36% 73.27% *
27 03/22/05 15:59:41 217.96.125.132 16009 30.29 0.36% 73.63% *
28 03/22/05 15:59:44 82.182.139.38 15989 30.14 0.36% 73.98% *
29 03/22/05 15:59:43 209.168.184.66 15883 30.12 ( 0.35%)
30 03/24/05 20:47:42 68.224.146.26 15345 16.03 0.34% 74.32% * !
31 03/22/05 15:59:47 80.86.183.219 14017 32.21 ( 0.31%)
32 03/22/05 16:02:32 69.163.11.52 13499 31.51 0.30% 74.62% *
33 03/23/05 14:18:04 69.22.130.50 13339 30.18 0.30% 74.92% *
34 03/24/05 06:03:27 132.248.139.150 11439 30.10 0.25% 75.18% *
35 03/26/05 08:31:04 12.217.244.144 11186 16.03 0.25% 75.42% * !
36 03/22/05 16:02:26 65.39.81.115 9251 30.07 0.21% 75.63% *
37 03/25/05 10:55:36 219.84.1.182 8634 30.41 0.19% 75.82% *
38 03/27/05 23:15:04 24.98.20.124 7815 5.03 0.17% 76.00% * !
39 03/23/05 04:22:06 200.182.145.118 6046 45.70 0.13% 76.13% *
40 03/27/05 22:36:37 67.234.71.140 5187 7.79 ( 0.12%)
41 03/26/05 14:59:53 81.226.0.44 4763 32.02 0.11% 76.24% *
42 03/27/05 02:06:27 81.179.201.97 4196 1.06 0.09% 76.33% * !
43 03/24/05 12:57:25 66.122.201.242 1833 36.42 ( 0.04%)
44 03/28/05 00:00:36 80.32.185.3 1187 21.15 ( 0.03%)
45 03/28/05 05:17:06 207.180.152.2 1105 15.11 0.02% 76.35% * !
46 03/28/05 06:25:12 68.62.121.1 947 16.80 0.02% 76.37% * !
47 03/28/05 07:49:09 67.10.5.19 601 16.25 0.01% 76.39% * !
48 03/28/05 07:42:27 24.201.161.172 587 15.94 ( 0.01%)
49 03/28/05 06:10:02 67.172.223.226 572 30.17 0.01% 76.40% *
50 03/28/05 06:46:52 24.42.54.14 518 29.12 0.01% 76.41% *
51 03/28/05 10:38:13 67.234.71.231 311 7.80 0.01% 76.42% * !
52 03/28/05 06:34:45 66.149.198.201 188 3.51 ( 0.00%)
53 03/27/05 20:01:49 67.143.40.110 183 16.47 0.00% 76.42% *
54 03/28/05 07:53:17 217.159.79.202 112 5.25 ( 0.00%)
55 03/28/05 08:02:42 217.159.94.38 108 5.31 ( 0.00%)
* = "active" = probably will send another request.
! = "abusive" = min requests of 100 and an average rate of less than 30s
between requests over the life of the entire connection.
Percentages are based on all packets from *active* clients. If the client
is not active, its percentage is what it would be if it were active.
Clients with very long updates (min requests of 10):
Rank First Seen Client IP Requests Rate Usage Cumulative
1 03/22/05 19:55:25 209.198.142.218 30 16384.29 0.00% 0.00% *
2 03/25/05 10:40:15 64.222.82.243 19 14399.87 0.00% 0.00% *
3 03/26/05 20:24:21 212.47.83.66 14 9525.36 0.00% 0.00% *
4 03/26/05 22:07:22 140.186.109.167 15 8724.21 0.00% 0.00% *
5 03/27/05 13:04:53 24.16.89.11 10 8390.50 0.00% 0.00% *
6 03/27/05 09:18:13 81.31.126.81 12 7815.75 0.00% 0.00% *
7 03/24/05 12:59:45 208.239.2.131 45 7758.68 0.00% 0.00% *
8 03/24/05 13:17:15 68.236.174.21 46 7677.66 0.00% 0.00% *
9 03/24/05 12:38:20 70.71.0.62 47 7328.33 0.00% 0.01% *
10 03/24/05 12:37:06 80.119.163.80 47 7328.15 0.00% 0.01% *
Clients distribution by count:
bin ( bin range ) count
1 ( 1 - 1) 278 ************************************
2 ( 2 - 2) 161 ********************
3 ( 3 - 3) 72 *********
4 ( 4 - 4) 346 *********************************************
5 ( 5 - 6) 65 ********
8 ( 7 - 9) 135 *****************
11 ( 10 - 13) 88 ***********
16 ( 14 - 19) 61 *******
22 ( 20 - 26) 140 ******************
32 ( 27 - 38) 53 ******
45 ( 39 - 53) 98 ************
64 ( 54 - 76) 50 ******
90 ( 77 - 107) 48 ******
128 ( 108 - 152) 33 ****
181 ( 153 - 215) 26 ***
256 ( 216 - 304) 29 ***
362 ( 305 - 430) 46 *****
512 ( 431 - 608) 320 *****************************************
724 ( 609 - 861) 50 ******
1024 ( 862 - 1217) 59 *******
1448 ( 1218 - 1722) 80 **********
2048 ( 1723 - 2435) 82 **********
2896 ( 2436 - 3444) 46 *****
4096 ( 3445 - 4870) 34 ****
5792 ( 4871 - 6888) 11 *
8192 ( 6889 - 9741) 31 ****
11585 ( 9742 - 13777) 4
16384 ( 13778 - 19483) 16 **
23170 ( 19484 - 27554) 3
32768 ( 27555 - 38967) 7
46340 ( 38968 - 55108) 1
65536 ( 55109 - 77935) 1
92681 ( 77936 - 110217) 1
131072 ( 110218 - 155871) 0
185363 ( 155872 - 220435) 0
262144 ( 220436 - 311743) 1
370727 ( 311744 - 440871) 1
524288 ( 440872 - 623487) 0
741455 ( 623488 - 881743) 0
1048576 ( 881744 - 1246974) 0
1482910 (1246975 - 1763487) 0
2097152 (1763488 - 2493948) 1
Clients distribution by rate (min requests of 10):
bin ( bin range ) count
1.0 ( 0.0 - 1.2) 10
1.4 ( 1.2 - 1.7) 1
2.0 ( 1.7 - 2.4) 1
2.8 ( 2.4 - 3.4) 0
4.0 ( 3.4 - 4.8) 1
5.7 ( 4.8 - 6.7) 10
8.0 ( 6.7 - 9.5) 11 *
11.3 ( 9.5 - 13.5) 9
16.0 ( 13.5 - 19.0) 76 *******
22.6 ( 19.0 - 26.9) 26 **
32.0 ( 26.9 - 38.1) 130 ************
45.3 ( 38.1 - 53.8) 9
64.0 ( 53.8 - 76.1) 49 ****
90.5 ( 76.1 - 107.6) 48 ****
128.0 ( 107.6 - 152.2) 71 ******
181.0 ( 152.2 - 215.3) 80 *******
256.0 ( 215.3 - 304.4) 124 ************
362.0 ( 304.4 - 430.5) 59 *****
512.0 ( 430.5 - 608.9) 58 *****
724.1 ( 608.9 - 861.1) 47 ****
1024.0 ( 861.1 - 1217.7) 464 *********************************************
1448.2 ( 1217.7 - 1722.2) 34 ***
2048.0 ( 1722.2 - 2435.5) 35 ***
2896.3 ( 2435.5 - 3444.3) 14 *
4096.0 ( 3444.3 - 4871.0) 32 ***
5792.6 ( 4871.0 - 6888.6) 9
8192.0 ( 6888.6 - 9742.0) 11 *
11585.2 ( 9742.0 - 13777.2) 0
16384.0 (13777.2 - 19484.0) 2
Clients distribution by amount of time client was active (in hours):
bin ( bin range ) count
1.0 ( 0.0 - 1.2) 1173 *********************************************
1.4 ( 1.2 - 1.7) 62 **
2.0 ( 1.7 - 2.4) 43 *
2.8 ( 2.4 - 3.4) 70 **
4.0 ( 3.4 - 4.8) 71 **
5.7 ( 4.8 - 6.7) 54 **
8.0 ( 6.7 - 9.5) 52 *
11.3 ( 9.5 - 13.5) 28 *
16.0 ( 13.5 - 19.0) 29 *
22.6 ( 19.0 - 26.9) 32 *
32.0 ( 26.9 - 38.1) 24
45.3 ( 38.1 - 53.8) 36 *
64.0 ( 53.8 - 76.1) 45 *
90.5 ( 76.1 - 107.6) 75 **
128.0 ( 107.6 - 152.2) 684 **************************
Clients distribution by time since first request (in hours):
bin ( bin range ) count
1.0 ( 0.0 - 1.2) 196 ************
1.4 ( 1.2 - 1.7) 89 *****
2.0 ( 1.7 - 2.4) 178 ***********
2.8 ( 2.4 - 3.4) 346 **********************
4.0 ( 3.4 - 4.8) 551 ************************************
5.7 ( 4.8 - 6.7) 87 *****
8.0 ( 6.7 - 9.5) 70 ****
11.3 ( 9.5 - 13.5) 31 **
16.0 ( 13.5 - 19.0) 30 *
22.6 ( 19.0 - 26.9) 32 **
32.0 ( 26.9 - 38.1) 27 *
45.3 ( 38.1 - 53.8) 36 **
64.0 ( 53.8 - 76.1) 46 ***
90.5 ( 76.1 - 107.6) 75 ****
128.0 ( 107.6 - 152.2) 684 *********************************************
Active clients distribution by time since last request (in secs):
bin ( bin range ) count
1.0 ( 0.0 - 1.2) 8 **
1.4 ( 1.2 - 1.7) 5 *
2.0 ( 1.7 - 2.4) 4 *
2.8 ( 2.4 - 3.4) 4 *
4.0 ( 3.4 - 4.8) 9 **
5.7 ( 4.8 - 6.7) 6 *
8.0 ( 6.7 - 9.5) 15 ****
11.3 ( 9.5 - 13.5) 20 ******
16.0 ( 13.5 - 19.0) 11 ***
22.6 ( 19.0 - 26.9) 37 ***********
32.0 ( 26.9 - 38.1) 34 **********
45.3 ( 38.1 - 53.8) 40 ************
64.0 ( 53.8 - 76.1) 58 ******************
90.5 ( 76.1 - 107.6) 68 *********************
128.0 ( 107.6 - 152.2) 73 ***********************
181.0 ( 152.2 - 215.3) 68 *********************
256.0 ( 215.3 - 304.4) 82 **************************
362.0 ( 304.4 - 430.5) 84 ***************************
512.0 ( 430.5 - 608.9) 112 ************************************
724.1 ( 608.9 - 861.1) 140 *********************************************
1024.0 ( 861.1 - 1217.7) 137 ********************************************
1448.2 ( 1217.7 - 1722.2) 38 ************
2048.0 ( 1722.2 - 2435.5) 36 ***********
2896.3 ( 2435.5 - 3444.3) 52 ****************
4096.0 ( 3444.3 - 4871.0) 33 **********
5792.6 ( 4871.0 - 6888.6) 33 **********
8192.0 ( 6888.6 - 9742.0) 29 *********
11585.2 ( 9742.0 - 13777.2) 17 *****
16384.0 (13777.2 - 19484.0) 7 **
Inactive clients distribution by time since last request (in secs):
bin ( bin range ) count
1.0 ( 0.0 - 1.2) 0
1.4 ( 1.2 - 1.7) 0
2.0 ( 1.7 - 2.4) 0
2.8 ( 2.4 - 3.4) 0
4.0 ( 3.4 - 4.8) 0
5.7 ( 4.8 - 6.7) 0
8.0 ( 6.7 - 9.5) 0
11.3 ( 9.5 - 13.5) 0
16.0 ( 13.5 - 19.0) 0
22.6 ( 19.0 - 26.9) 0
32.0 ( 26.9 - 38.1) 0
45.3 ( 38.1 - 53.8) 0
64.0 ( 53.8 - 76.1) 1
90.5 ( 76.1 - 107.6) 1
128.0 ( 107.6 - 152.2) 0
181.0 ( 152.2 - 215.3) 0
256.0 ( 215.3 - 304.4) 1
362.0 ( 304.4 - 430.5) 1
512.0 ( 430.5 - 608.9) 5
724.1 ( 608.9 - 861.1) 6
1024.0 ( 861.1 - 1217.7) 24 **
1448.2 ( 1217.7 - 1722.2) 28 ***
2048.0 ( 1722.2 - 2435.5) 35 ***
2896.3 ( 2435.5 - 3444.3) 59 ******
4096.0 ( 3444.3 - 4871.0) 55 *****
5792.6 ( 4871.0 - 6888.6) 138 **************
8192.0 ( 6888.6 - 9742.0) 127 *************
11585.2 ( 9742.0 - 13777.2) 418 *********************************************
16384.0 (13777.2 - 19484.0) 319 **********************************
--- cut here ---
-wayne
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic