[prev in list] [next in list] [prev in thread] [next in thread] 

List:       openbsd-bugs
Subject:    Proxy-arp behavior change between 5.9 and 6.0
From:       Andrew Hewus Fresh <andrew () afresh1 ! com>
Date:       2020-07-30 1:47:08
Message-ID: 20200730014708.GL12086 () us ! holligan ! net
[Download RAW message or body]

I am helping some folks with some OpenBSD stuff, including at some
point being more proactive about updating, but they currently have an
OpenBSD 5.9 machine that has been routing traffic happily for a while.
Unfortunately, they can't currently update because the proxy arp
configuration that they are using on 5.9 doesn't work on anything newer.


Here is the excellent description from someone there, and I was able to
reproduce the issue on my laptop, so I don't believe a dmesg will be
particularly helpful, but here's one for my laptop that's not terribly
old.

https://dmesgd.nycbug.org/index.cgi?do=view&id=5279

As I haven't ever actually used proxy arp on OpenBSD, so don't actually
know whether this was a glitch that was fixed or added.

I do see that 60.html says the routing table is now based on ART, so
that seems a possible culprit.

---


We've found that a proxy-arp configuration we've relied on (which works
perfectly on OpenBSD 5.9 and earlier) no longer works on OpenBSD 6.0 thru 6.7.

On OpenBSD 5.9, we can add an "arp -s" entry for a public IP on our
public-facing gateway router's external interface, followed by a host-route
for that same public IP (pointing to an internal subnet/interface). This
allows us to route public IPs to clients through our internal private subnets,
so a client can have a public IP assigned on their router, despite actually
being several hops deep inside our private network.

With this proxy-arp configuration, from both the client router's perspective
and from an external perspective, it looks as if the client router is in the
public IP subnet.

Unfortunately this method doesn't work in OpenBSD 6.0 and later.


Here's how it looks when you configure this on OpenBSD 5.9
(working as expected with no errors):

        root@openbsd-59-pub-gw-rtr~ # arp -s 1.2.3.4 [PUB-GW_em0_EXT-IF_MAC-ADDR] pub
        root@openbsd-59-pub-gw-rtr~ # route add 1.2.3.4/32 10.0.2.47
        add host 1.2.3.4/32: gateway 10.0.2.47
        root@openbsd-59-pub-gw-rtr~ # netstat -rn -finet|grep 1.2.3.4
        1.2.3.4        10.0.2.47        UGHS       0        0 -     8 em1
        1.2.3.4/32     [PUB-GW_em0_EXT-IF_MAC-ADDR]  ULS2 0        0     -     8 em0
     ##### ^ Success: Routing table shows both the host-route and the "arp -s" entry.


That same setup fails on OpenBSD 6.0 and later
(tested on OpenBSD 6.0, 6.2, 6.6, and 6.7).


If you add an arp -s entry before adding a host route for the same IP, the
"route add" fails with "File exists" error:

        root@openbsd-67-pub-gw-rtr~ # arp -s 1.2.3.4 [PUB-GW_em0_EXT-IF_MAC-ADDR] pub
        root@openbsd-67-pub-gw-rtr~ # route add 1.2.3.4/32 10.0.2.47
    !-> add host 1.2.3.4/32: gateway 10.0.2.47: File exists
        root@openbsd-67-pub-gw-rtr~ # netstat -rn -finet|grep 1.2.3.4
        1.2.3.4/32     [PUB-GW_em0_EXT-IF_MAC-ADDR]  ULS2 0        0     -     8 em0
     ##### ^ Failure: Routing table only shows the "arp -s" entry (failed to add host-route)


If you add the route before the arp -s entry, "arp -s" fails with error
"set: proxy entry exists for non 802 device":

        root@openbsd-67-pub-gw-rtr~ # route add 1.2.3.4/32 10.0.2.47
        add host 1.2.3.4/32: gateway 10.0.2.47
        root@openbsd-67-pub-gw-rtr~ # arp -s 1.2.3.4 [PUB-GW_em0_EXT-IF_MAC-ADDR] pub
    !-> set: proxy entry exists for non 802 device
        root@openbsd-67-pub-gw-rtr~ # netstat -rn -finet|grep 1.2.3.4
        1.2.3.4        10.0.2.47         UGHS       0        0 -     8 em1
     ##### ^ Failure: Routing table only shows the host-route (failed to add arp -s entry)


For most of our clients who request a dedicated public IP, we're able to
assign their router a private IP, which we then bi-NAT to a dedicated public
IP on our gateway router. Unfortunately the bi-NAT method doesn't work for
certain routers' built-in VPNs, so we're looking for a way to re-create the
functionality we had with proxy-arp on OpenBSD v5.9. I imagine there's some
workaround or alternate method to achieve the same thing on OpenBSD 6.x, but I
haven't found it yet.

[prev in list] [next in list] [prev in thread] [next in thread]