[prev in list] [next in list] [prev in thread] [next in thread] 

List:       freeradius-users
Subject:    Re: Configurations for proxying radius requests to home-server using TCP
From:       Alan DeKok <aland () deployingradius ! com>
Date:       2021-08-26 21:40:42
Message-ID: D1A43726-B9CE-4A18-8DBD-F0D9EFCF5768 () deployingradius ! com
[Download RAW message or body]

On Aug 26, 2021, at 2:48 PM, Abhilash Y G <ygabhi@gmail.com> wrote:
> 
> Proxying to homeserver using a TCP connection is causing issue.
> I am seeing the response from the homeserver, it seems like freeradius
> running on proxy server is not honoring the response.
> I have attached the debug logs collected on the proxying server
> (debug_proxy_server.txt), homeservers (debug_homeserver.txt).
> 
> I had to trim the logs to focus on issue logs due to limitation in sending
> attachments more than 500KB.

  It's generally a good idea to trim the logs so that only the relevant information \
is sent to the list.  If you send megabytes of irrelevant logs, it's likely that \
people will just ignore the message.

  Plus, the debug log shows the server receiving many packets at the same time.  This \
makes it more difficult to see what's going on.

  One useful skill for debugging is knowing how to simplify the problem.  Don't try \
to debug issues in production, with tons of packets going by.  It will be impossible. \
Instead, set up one or more test systems.  You can then send ONE request, and follow \
it through the proxy chain.

> Please advise if any configuration changes are needed on the proxying
> server to handle the requests without timeouts.

  That is not a useful way to phrase things.  As you've been told, FreeRADIUS replies \
to all packets it gets, UNLESS you've done something to break the configuration.

  From the debug log on the proxy:

(28) Received Access-Request Id 27 from 172.29.93.13:2000 to 0.0.0.0:2083 length 687
(28)   User-Name = "03111800000899580"
(28)   NAS-Identifier = "wifi"
...

  This is proxied:

(28) Proxying request to home server 206.XX.XXX.70 port 1234 timeout 3.000000
(28) Sent Access-Request Id 120 from 172.19.0.2:37787 to 206.XX.XXX.70:1234 length \
691 (28)   User-Name = "03111800000899580"
(28)   NAS-Identifier = "wifi"


  Which then gets to the home server:

(13) Received Access-Request Id 120 from 206.XX.XXX.117:37787 to 0.0.0.0:1234 length \
691 (13)   User-Name = "03111800000899580"
(13)   NAS-Identifier = "wifi"

  The IDs match.  But the IP addresses are different.  So there's a NAT, or another \
proxy server involved.

  That request is proxied in turn to another server:

(13) Proxying request to home server 206.XX.XXX.71 port 1817 timeout 3.000000
(13) Sent Access-Request Id 54 from 0.0.0.0:44020 to 206.XX.XXX.71:1817 length 696
(13)   User-Name = "03111800000899580"

  Which replies:

(13) Received Access-Accept Id 54 from 206.XX.XXX.71:1817 to 172.18.0.2:44020 length \
640 (13)   Class = 0x4469616d657465722f6175732d653035342d726d722e636f72702e776179706f7 \
2742e6e65743b313632393739323030323b343531363b3033313131383030303030383939353840776c616 \
e2e6d6e633138302e6d63633331312e336770706e6574776f726b2e6f72673b6175732d653035342d726d722e636f72702e776179706f72742e6e6574


...

  And sends the Access-Challenge back to the proxy:

(13) Sent Access-Accept Id 120 from 0.0.0.0:1234 to 206.XX.XXX.117:37787 length 0
(13)   Class = 0x4469616d657465722f6175732d653035342d726d722e636f72702e776179706f72742 \
e6e65743b313632393739323030323b343531363b3033313131383030303030383939353840776c616e2e6 \
d6e633138302e6d63633331312e336770706e6574776f726b2e6f72673b6175732d653035342d726d722e636f72702e776179706f72742e6e6574


...

  And then nothing comes back to the proxy.

  But there are other weird things in the trace.  On the "home server", it says:

(5)   State = 0x4469616d657465722f68706e623032766f6c746568762e6570632e6d6e633138302e6d \
63633331312e336770706e6574776f726b2e6f72672f6570632e6d6e633138302e6d63633331312e336770 \
706e6574776f726b2e6f72672f6175732d653035342d726d722e636f72702e776179706f72742e6e65743b \
313632393739323030323b343531363b3033313131383030303030383939353840776c616e2e6d6e633138 \
302e6d63633331312e336770706e6574776f726b2e6f72673b6175732d653035342d726d722e636f72702e776179706f72742e6e6574


  What the heck is that?  Oh wait, it's just ASCII:

Diameter/hpnb02voltehv.epc.mnc180.mcc311.3gppnetwork.org/epc.mnc180.mcc311.3gppnetwork \
.org/aus-e054-rmr.corp.wayport.net;1629792002;4516;0311180000089958@wlan.mnc180.mcc311.3gppnetwork.org;aus-e054-rmr.corp.wayport.net


  So no, you are NOT using FreeRADIUS everywhere.  FreeRADIUS DOES NOT produce that \
kind of State attribute.  So you're using some kind of RADIUS to Diameter gateway.

  There's also this:

(5)   Proxy-State = 0x000102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f \
202122232425262728292a2b2c2d2e2f303132333435363738393a3b3c3d3e3f404142434445464748494a \
4b4c4d4e4f505152535455565758595a5b5c5d5e5f606162636465666768696a6b6c6d6e6f707172737475 \
767778797a7b7c7d7e7f808182838485868788898a8b8c8d8e8f909192939495969798999a9b9c9d9e9fa0 \
a1a2a3a4a5a6a7a8a9aaabacadaeafb0b1b2b3b4b5b6b7b8b9babbbcbdbebfc0c1c2c3c4c5c6c7c8c9cacb \
cccdcecfd0d1d2d3d4d5d6d7d8d9dadbdcdddedfe0e1e2e3e4e5e6e7e8e9eaebecedeeeff0f1f2f3f4f5f6f7f8f9fafbfc


  Again, that is ridiculous, and not created by FreeRADIUS.

  But 

  So to summarize:

* the debug output shows that many packets get replies, but some don't get replies.
* There's not one proxy. 
* There's two
* the home server isn't a home server.  It's a proxy to the "real" home server 
* Almost all of the packets are going back and forth just fine.  i.e. ALL RADIUS \
                servers reply
* the ACTUAL home server is some kind of horrific RADIUS to Diameter gateway.

  What's happening is that there's some kind of NAT gateway / firewall which is \
dropping the TCP traffic.  It's probably either blocking the TCP connections after a \
bit, or it's tearing them down.

  Don't put firewalls between your RADIUS servers.  Don't have firewalls drop RADIUS \
traffic.  It's bad.

  PLEASE set up a test system.  Trying to debug issues in production is just too \
difficult.  You get megabytes of logs, and no idea where to look for problems.

  There's nothing wrong with FreeRADIUS.  Your network is broken.  No amount of \
poking FreeRADIUS will fix the network.  Go fix the network.

  Alan DeKok.


-
List info/subscribe/unsubscribe? See http://www.freeradius.org/list/users.html


[prev in list] [next in list] [prev in thread] [next in thread]