[prev in list] [next in list] [prev in thread] [next in thread]
List: strongswan-users
Subject: Re: [strongSwan] IPSec route based VPN - VTI interface TX Errors NoRoute
From: Tiago Stoco <tmsblink () msn ! com>
Date: 2021-08-31 23:46:16
Message-ID: CWLP123MB5133DF728DD626F29CBDD5CACDCC9 () CWLP123MB5133 ! GBRP123 ! PROD ! OUTLOOK ! COM
[Download RAW message or body]
[Attachment #2 (text/plain)]
Hi Noel,
Thanks for the help.
I have moved the route for IPSec back into the main routing table.
[root@arch-linux ~]# ip route
default via 192.168.45.1 dev ens18
10.10.10.0/30 dev ip_vti1 scope link src 10.10.10.2
192.168.45.0/24 dev ens18 proto kernel scope link src 192.168.45.30
[root@arch-linux ~]# ip route show table 220
[root@arch-linux ~]#
And the connmark plugin has been disabled.
[root@arch-linux ~]# ipsec statusall | grep -i connmark
[root@arch-linux ~]#
Also, I have noticed that even after the connmark plugin is disabled some mark rules \
are added to iptables.
[root@arch-linux ~]# ipsec stop
Stopping strongSwan IPsec...
[root@arch-linux ~]# iptables-save | grep -i mark
[root@arch-linux ~]#
[root@arch-linux ~]# ipsec start
Starting strongSwan 5.9.3 IPsec [starter]...
[root@arch-linux ~]# swanctl --load-all
plugin 'mysql' failed to load: libmariadb.so.3: cannot open shared object file: No \
such file or directory loaded ike secret 'ike-0'
no authorities found, 0 unloaded
no pools found, 0 unloaded
loaded connection 'ipseclab'
successfully loaded 1 connections, 0 unloaded
[root@arch-linux ~]# iptables-save | grep -i mark
-A PREROUTING -d 10.10.10.0/30 -j MARK --set-xmark 0x2a/0xffffffff
-A PREROUTING -s 192.168.45.10/32 -d 192.168.45.30/32 -p esp -m esp --espspi \
3419086685 -j MARK --set-xmark 0x2a/0xfffffff f
-A OUTPUT -d 10.10.10.0/30 -j MARK --set-xmark 0x2a/0xffffffff
The scenario is the same at the moment :
pings from 10.10.10.1 ( pfSense ) to 10.10.10.2 ( Linux ) ✅ seem as received by \
Linux pings from 10.10.10.2 ( Linux ) to 10.10.10.1 ( pfSense ) ❌ seem as Errors \
NoRoute
ip_vti1: ip/ip remote 192.168.45.10 local 192.168.45.30 ttl inherit nopmtudisc key 42
RX: Packets Bytes Errors CsumErrs OutOfSeq Mcasts
66095 5551980 0 0 0 0
TX: Packets Bytes Errors DeadLoop NoRoute NoBufs
0 0 133788 0 133788 0
I am at work and not able to capture packets to further investigate but will do as \
soon possible.
[root@arch-linux ~]# ipsec statusall
Status of IKE charon daemon (strongSwan 5.9.3, Linux 5.13.12-arch1-1, x86_64):
uptime: 9 minutes, since Sep 01 00:24:27 2021
malloc: sbrk 2936832, mmap 0, used 1328288, free 1608544
worker threads: 11 of 16 idle, 5/0/0/0 working, job queue: 0/0/0/0, scheduled: 115
loaded plugins: charon ldap pkcs11 aes des rc2 sha2 sha3 sha1 md5 mgf1 random nonce \
x509 revocation constraints pubkey p kcs1 pkcs7 pkcs8 pkcs12 pgp dnskey sshkey pem \
openssl fips-prf gmp curve25519 agent chapoly xcbc cmac hmac ntru drbg newho pe bliss \
curl sqlite attr kernel-netlink resolve socket-default forecast farp stroke vici \
updown eap-identity eap-sim eap- aka eap-aka-3gpp2 eap-simaka-pseudonym \
eap-simaka-reauth eap-md5 eap-gtc eap-mschapv2 eap-dynamic eap-radius eap-tls eap-t \
tls eap-peap xauth-generic xauth-eap xauth-pam xauth-noauth dhcp radattr unity \
counters Listening IP addresses:
192.168.45.30
10.10.10.2
Connections:
ipseclab: 192.168.45.30...192.168.45.10 IKEv2, dpddelay=10s
ipseclab: local: [ipsec-lab-openwrt] uses pre-shared key authentication
ipseclab: remote: [ipsec-lab-pfsense] uses pre-shared key authentication
con1: child: 10.10.10.0/30 === 10.10.10.0/30 TUNNEL, dpdaction=restart
Security Associations (2 up, 0 connecting):
ipseclab[54]: ESTABLISHED 9 seconds ago, \
192.168.45.30[ipsec-lab-openwrt]...192.168.45.10[ipsec-lab-pfsense] ipseclab[54]: \
IKEv2 SPIs: 840c3fd77eb0efee_i b3f6cec233130e6a_r*, rekeying in 6 hours \
ipseclab[54]: IKE proposal: \
AES_CBC_256/HMAC_SHA2_256_128/PRF_HMAC_SHA2_256/MODP_2048
con1{54}: INSTALLED, TUNNEL, reqid 1, ESP SPIs: c206ad8c_i c631ebba_o
con1{54}: AES_GCM_16_256, 0 bytes_i, 0 bytes_o, rekeying in 48 minutes
con1{54}: 10.10.10.0/30 === 10.10.10.0/30
ipseclab[53]: ESTABLISHED 19 seconds ago, \
192.168.45.30[ipsec-lab-openwrt]...192.168.45.10[ipsec-lab-pfsense] ipseclab[53]: \
IKEv2 SPIs: 133d2066ebf6a358_i d8da41bca97601ca_r*, rekeying in 6 hours \
ipseclab[53]: IKE proposal: \
AES_CBC_256/HMAC_SHA2_256_128/PRF_HMAC_SHA2_256/MODP_2048
con1{53}: INSTALLED, TUNNEL, reqid 1, ESP SPIs: cb169570_i c35823e7_o
con1{53}: AES_GCM_16_256, 0 bytes_i, 0 bytes_o, rekeying in 51 minutes
con1{53}: 10.10.10.0/30 === 10.10.10.0/30
I will add to the thread later but NFLOG is able to capture MARKS in the packets. The \
packets captured by NFLOG has a special section with quite useful information.
...
Linux Netfilter NFLOG
Family: IPv4 (2)
Version: 0
Resource id: 7
TLV Type: NFULA_PACKET_HDR (1), Length: 8
...
Once again, thanks for the help.
Tiago Stoco.
________________________________
From: Noel Kuntze
Sent: Tuesday, August 31, 2021 2:20 PM
To: Tiago Stoco; Tobias Brunner; users@lists.strongswan.org
Subject: Re: [strongSwan] IPSec route based VPN - VTI interface TX Errors NoRoute
Hello Tiago,
> And, I have moved the route for the VTI to table 220 because it seems to be the \
> right way to config routed based IPSec VPN.
> [root@arch-linux ~]# ip rule
> 0: from all lookup local
> 220: from all lookup 220
> 32766: from all lookup main
> 32767: from all lookup default
Don't do that.
220 is managed by strongSwan. Keep them in the main table.
> -A PREROUTING -d 10.10.10.0/30 -c 2352 230776 -j MARK --set-xmark 0x2a/0xffffffff
> -A OUTPUT -d 10.10.10.0/30 -c 3605 336028 -j MARK --set-xmark 0x2a/0xffffffff
Disable the connmark plugin.
> I have added a few more NFLOG captures into my iptables and I am a bit confused \
> with the results.
> A tcpdump capture in the VTI interface with a ping from the remote ( pfSense - \
> 10.10.10.1 ) shows :
> No Time Source Destination
>
> 1 0.000000 10.10.10.1 10.10.10.2 ICMP 84 Echo (ping) request id=0x9877, \
> seq=471/55041, ttl=64 (reply in 2) 2 0.000038 10.10.10.2 > 10.10.10.1 ICMP 84 Echo \
> (ping) reply id=0x9877, seq=471/55041, ttl=64 (request in 1)
> I do not see the IPSec MARK in these packets.
They are only visible in the output of the TRACE target. I suspect they are note \
copied into the buffer passed to the applications.
>
> Also, the NAT chain is not having packets passing through it.
>
> [root@arch-linux ~]# snat
> -P PREROUTING ACCEPT -c 0 0
> -P INPUT ACCEPT -c 0 0
> -P OUTPUT ACCEPT -c 0 0
> -P POSTROUTING ACCEPT -c 0 0
> -A PREROUTING -c 0 0 -j NFLOG --nflog-group 9
>
> That is odd cause I am not able to manipulate the packets.
>
> I will run a ping from the local Linux (10.10.10.2) and see how the packets are \
> flowing through the iptables chains and will update in another email.
The *nat table is only consulted for the first packet of a connection.
Kind regards
Noel
Am 31.08.21 um 17:22 schrieb Tiago Stoco:
> Hi Tobias,
>
> First of all, THANKS for replying and clarifying some settings.
>
> I have completely disabled the bypass-lan plugin since I do not have a use for it \
> right now.
> [root@arch-linux ~]# cat /etc/strongswan.conf
> ...
> plugins {
> include strongswan.d/charon/*.conf
> bypass-lan {
> load = no
> }
> ...
>
> And, I have moved the route for the VTI to table 220 because it seems to be the \
> right way to config routed based IPSec VPN.
> [root@arch-linux ~]# ip rule
> 0: from all lookup local
> 220: from all lookup 220
> 32766: from all lookup main
> 32767: from all lookup default
>
> [root@arch-linux ~]# ip r s t 220
> 10.10.10.0/30 via 10.10.10.2 dev ip_vti1 src 10.10.10.2
>
> [root@arch-linux ~]# ip route
> default via 192.168.45.1 dev ens18
> 192.168.45.0/24 dev ens18 proto kernel scope link src 192.168.45.30
>
> I am going to add some more details of my configs because the TX Errors NoRoute are \
> still present.
> 7: ip_vti1@NONE: <POINTOPOINT,NOARP,UP,LOWER_UP> mtu 1436 qdisc noqueue state \
> UNKNOWN group default qlen 1000 link/ipip 192.168.45.30peer \
> 192.168.45.10promiscuity 0 minmtu 0 maxmtu 0 vti remote 192.168.45.10 local \
> 192.168.45.30 ikey 0.0.0.42 okey 0.0.0.42 numtxqueues 1 numrxqueues 1 gso_max_size \
> 65536 gso_max_segs 65535 inet 10.10.10.2peer 10.10.10.1/32 scope global ip_vti1
> valid_lft forever preferred_lft forever
> inet6 fe80::5efe:c0a8:2d1e/64 scope link
> valid_lft forever preferred_lft forever
>
> I can also see that the IPSec added some rules to MARK packets in my iptables.
>
> -A PREROUTING -d 10.10.10.0/30 -c 2352 230776 -j MARK --set-xmark 0x2a/0xffffffff
> -A OUTPUT -d 10.10.10.0/30 -c 3605 336028 -j MARK --set-xmark 0x2a/0xffffffff
>
> The counters confirms that the packets are being marked. I am not sure if I should \
> keep the MARK in iptables or remove it allowing routing decisions to send the \
> packets to the VTI device that will MARK the packets but according to my \
> understanding it should not matter.
> [root@arch-linux ~]# ip xfrm policy
> src 0.0.0.0/0 dst 0.0.0.0/0
> socket in priority 0 ptype main
> src 0.0.0.0/0 dst 0.0.0.0/0
> socket out priority 0 ptype main
> src 0.0.0.0/0 dst 0.0.0.0/0
> socket in priority 0 ptype main
> src 0.0.0.0/0 dst 0.0.0.0/0
> socket out priority 0 ptype main
> src ::/0 dst ::/0
> socket in priority 0 ptype main
> src ::/0 dst ::/0
> socket out priority 0 ptype main
> src ::/0 dst ::/0
> socket in priority 0 ptype main
> src ::/0 dst ::/0
> socket out priority 0 ptype main
>
> Above are the policies installed. Again, because it is a routed base VPN seems \
> correct.
> [root@arch-linux ~]# ip xfrm state
> src 192.168.45.30 dst 192.168.45.10
> proto esp spi 0xc2239b57 reqid 1 mode tunnel
> replay-window 0 flag af-unspec
> mark 0x2a/0xffffffff
> aead rfc4106(gcm(aes)) \
> 0x264acee3119a4e523af2fbf5905b50c5acc1f7be9079ff23ffa2c6473a9c507fe1ae936b 128 \
> anti-replay context: seq 0x0, oseq 0x0, bitmap 0x00000000 src 192.168.45.10 dst \
> 192.168.45.30 proto esp spi 0xc661b9e5 reqid 1 mode tunnel
> replay-window 32 flag af-unspec
> aead rfc4106(gcm(aes)) \
> 0x69a86fa6ca9448bece6ffdff77893f0e9ce5ebef604040f681b5cdd2d5976438ed005df1 128 \
> anti-replay context: seq 0x656, oseq 0x0, bitmap 0xffffffff
> I have added a few more NFLOG captures into my iptables and I am a bit confused \
> with the results.
> A tcpdump capture in the VTI interface with a ping from the remote ( pfSense - \
> 10.10.10.1 ) shows :
> No Time Source Destination
>
> 1 0.000000 10.10.10.1 10.10.10.2 ICMP 84 Echo (ping) request id=0x9877, \
> seq=471/55041, ttl=64 (reply in 2) 2 0.000038 10.10.10.2 > 10.10.10.1 ICMP 84 Echo \
> (ping) reply id=0x9877, seq=471/55041, ttl=64 (request in 1)
> I do not see the IPSec MARK in these packets.
> The reply packets end up in the OUTPUT chain marked but not encrypted as an ESP \
> packet. By the way I do not see the replies even being encapsulated at all by \
> IPSec.
> Also, the NAT chain is not having packets passing through it.
>
> [root@arch-linux ~]# snat
> -P PREROUTING ACCEPT -c 0 0
> -P INPUT ACCEPT -c 0 0
> -P OUTPUT ACCEPT -c 0 0
> -P POSTROUTING ACCEPT -c 0 0
> -A PREROUTING -c 0 0 -j NFLOG --nflog-group 9
>
> That is odd cause I am not able to manipulate the packets.
>
> I will run a ping from the local Linux (10.10.10.2) and see how the packets are \
> flowing through the iptables chains and will update in another email.
> In the meantime, if someone sees something that I am missing. Please let me know.
>
> Many Thanks.
> ------------------------------------------------------------------------------------ \
> ------------------------------------------------------------------------------------ \
> ------------------------------------------------------------------------------------ \
> ------------------------------------------------------------------------------------ \
> ------------------------------------------------------------------------------------ \
> ------------------------------------------------------------------------------------ \
> ------------------------------------------------------------------------------------ \
> ------------------------------------------------------------------------------------ \
> ------------------------------------------------------------------------------------ \
> ------------------------------------------------------------------------------------ \
> ------------------------------------------------------------------------------------------------------------------------------------------------------
>
> *From:* Tobias Brunner <tobias@strongswan.org>
> *Sent:* Tuesday, August 31, 2021 5:51 AM
> *To:* Tiago Stoco <tmsblink@msn.com>; users@lists.strongswan.org \
> <users@lists.strongswan.org>
> *Subject:* Re: [strongSwan] IPSec route based VPN - VTI interface TX Errors NoRoute
> Hi Tiago,
>
> > Pings from the Linux system are being seem as errors NoRoute by the tunnel. > ...
> > Shunted Connections:
> > Bypass LAN 10.10.10.0/30: 10.10.10.0/30 === 10.10.10.0/30 PASS
>
> The reason is most likely this passthrough IPsec policy installed by the
> bypass-lan plugin for the subnet that is reachable (according to the
> main routing table) via ip_vti1. For a ping from 10.10.10.2 to
> 10.10.10.1, the VTI interface won't find an IPsec policy to protect the
> packet (the passthrough policy has a higher priority), so it gets dropped.
>
> To avoid that, either install the routes via VTI in table 220 (which is
> ignored by the bypass-lan plugin automatically), exclude the VTI
> interface explicitly via charon.plugins.bypass-lan.interfaces_ignore, or
> just disable the bypass-lan plugin completely if you don't need it.
>
> Regards,
> Tobias
[Attachment #3 (text/html)]
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<style type="text/css" style="display:none;"> P {margin-top:0;margin-bottom:0;} \
</style> </head>
<body dir="ltr">
<div style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; color: \
rgb(0, 0, 0);"> Hi Noel,</div>
<div style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; color: \
rgb(0, 0, 0);"> <br>
</div>
<div style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; color: \
rgb(0, 0, 0);"> Thanks for the help.</div>
<div style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; color: \
rgb(0, 0, 0);"> <br>
</div>
<div style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; color: \
rgb(0, 0, 0);"> <span style="font-size:12pt">I have moved the route for IPSec back \
into the main routing table.</span> <br>
</div>
<div style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; color: \
rgb(0, 0, 0);"> <br>
</div>
<div style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; color: \
rgb(0, 0, 0);"> <span style="font-family:monospace"><span \
style="color:#000000;background-color:#ffffff">[root@arch-linux ~]# ip route \
</span><br> default via 192.168.45.1 dev ens18 <br>
10.10.10.0/30 dev ip_vti1 scope link src 10.10.10.2 <br>
192.168.45.0/24 dev ens18 proto kernel scope link src 192.168.45.30</span><br>
</div>
<div style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; color: \
rgb(0, 0, 0);"> <br>
</div>
<div style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; color: \
rgb(0, 0, 0);"> <span style="font-family:monospace"><span \
style="color:#000000;background-color:#ffffff">[root@arch-linux ~]# ip route show \
table 220 </span><br>
[root@arch-linux ~]#</span><br>
</div>
<div style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; color: \
rgb(0, 0, 0);"> <br>
</div>
<div style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; color: \
rgb(0, 0, 0);"> And the connmark plugin has been disabled.</div>
<div style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; color: \
rgb(0, 0, 0);"> <br>
</div>
<div style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; color: \
rgb(0, 0, 0);"> <span style="font-family:monospace"><span \
style="color:#000000;background-color:#ffffff">[root@arch-linux ~]# ipsec statusall | \
grep -i connmark </span><br>
[root@arch-linux ~]# </span><br>
</div>
<div style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; color: \
rgb(0, 0, 0);"> </div>
<div style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; color: \
rgb(0, 0, 0);"> <br>
</div>
<div style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; color: \
rgb(0, 0, 0);"> Also, I have noticed that even after the connmark plugin is disabled \
some mark rules are added to iptables.</div> <div style="font-family: Calibri, \
Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);"> <br>
</div>
<div style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; color: \
rgb(0, 0, 0);"> <span style="font-family:monospace"><span \
style="color:#000000;background-color:#ffffff">[root@arch-linux ~]# ipsec stop \
</span><br> Stopping strongSwan IPsec... <br>
[root@arch-linux ~]# iptables-save | grep -i mark <br>
[root@arch-linux ~]#</span><br>
</div>
<div style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; color: \
rgb(0, 0, 0);"> <br>
</div>
<div style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; color: \
rgb(0, 0, 0);"> <span style="font-family:monospace"><span \
style="color:#000000;background-color:#ffffff">[root@arch-linux ~]# ipsec start \
</span><br> Starting strongSwan 5.9.3 IPsec [starter]... <br>
[root@arch-linux ~]# swanctl --load-all <br>
plugin 'mysql' failed to load: libmariadb.so.3: cannot open shared object file: No \
such file or directory <br>
loaded ike secret 'ike-0' <br>
no authorities found, 0 unloaded <br>
no pools found, 0 unloaded <br>
loaded connection 'ipseclab' <br>
successfully loaded 1 connections, 0 unloaded <br>
[root@arch-linux ~]# iptables-save | grep -i mark <br>
-A PREROUTING -d 10.10.10.0/30 -j MARK --set-xmark 0x2a/0xffffffff <br>
-A PREROUTING -s 192.168.45.10/32 -d 192.168.45.30/32 -p esp -m esp --espspi \
3419086685 -j MARK --set-xmark 0x2a/0xfffffff<br> f <br>
-A OUTPUT -d 10.10.10.0/30 -j MARK --set-xmark 0x2a/0xffffffff</span><br>
</div>
<div style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; color: \
rgb(0, 0, 0);"> <br>
</div>
<div style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; color: \
rgb(0, 0, 0);"> <br>
</div>
<div style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; color: \
rgb(0, 0, 0);"> The scenario is the same at the moment :</div>
<div style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; color: \
rgb(0, 0, 0);"> <br>
</div>
<div style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; color: \
rgb(0, 0, 0);"> pings from 10.10.10.1 ( pfSense ) to 10.10.10.2 ( Linux ) <span \
id="✅">✅ seem as received by Linux<br> </span></div>
<div style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; color: \
rgb(0, 0, 0);"> <span id="✅">pings from 10.10.10.2 ( Linux ) to 10.10.10.1 ( \
pfSense ) <span id="❌"> ❌</span></span> seem as Errors NoRoute<br>
</div>
<div style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; color: \
rgb(0, 0, 0);"> <span style="font-family:monospace"><span \
style="color:#000000;background-color:#ffffff">ip_vti1: ip/ip remote 192.168.45.10 \
local 192.168.45.30 ttl inherit nopmtudisc key 42 </span><br>
RX: Packets Bytes Errors \
CsumErrs OutOfSeq Mcasts <br> 66095 \
5551980 0 \
0 0 \
0 \
<br>
TX: Packets Bytes Errors \
DeadLoop NoRoute NoBufs <br> 0 \
0 \
133788 0 \
133788 0</span><br> </div>
<div style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; color: \
rgb(0, 0, 0);"> <br>
</div>
<div style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; color: \
rgb(0, 0, 0);"> I am at work and not able to capture packets to further investigate \
but will do as soon possible.</div> <div style="font-family: Calibri, Helvetica, \
sans-serif; font-size: 12pt; color: rgb(0, 0, 0);"> <br>
</div>
<div style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; color: \
rgb(0, 0, 0);"> <span style="font-family:monospace"><span \
style="color:#000000;background-color:#ffffff">[root@arch-linux ~]# ipsec statusall \
</span><br> Status of IKE charon daemon (strongSwan 5.9.3, Linux 5.13.12-arch1-1, \
x86_64): <br> uptime: 9 minutes, since Sep 01 00:24:27 2021 <br>
malloc: sbrk 2936832, mmap 0, used 1328288, free 1608544 <br>
worker threads: 11 of 16 idle, 5/0/0/0 working, job queue: 0/0/0/0, scheduled: \
115 <br>
loaded plugins: charon ldap pkcs11 aes des rc2 sha2 sha3 sha1 md5 mgf1 random \
nonce x509 revocation constraints pubkey p<br> kcs1 pkcs7 pkcs8 pkcs12 pgp dnskey \
sshkey pem openssl fips-prf gmp curve25519 agent chapoly xcbc cmac hmac ntru drbg \
newho<br> pe bliss curl sqlite attr kernel-netlink resolve socket-default forecast \
farp stroke vici updown eap-identity eap-sim eap-<br> aka eap-aka-3gpp2 \
eap-simaka-pseudonym eap-simaka-reauth eap-md5 eap-gtc eap-mschapv2 eap-dynamic \
eap-radius eap-tls eap-t<br> tls eap-peap xauth-generic xauth-eap xauth-pam \
xauth-noauth dhcp radattr unity counters <br>
Listening IP addresses: <br>
192.168.45.30 <br>
10.10.10.2 <br>
Connections: <br>
ipseclab: 192.168.45.30...192.168.45.10 IKEv2, \
dpddelay=10s <br> ipseclab: local: \
[ipsec-lab-openwrt] uses pre-shared key authentication <br> \
ipseclab: remote: [ipsec-lab-pfsense] uses pre-shared \
key authentication <br> con1: \
child: 10.10.10.0/30 === 10.10.10.0/30 TUNNEL, dpdaction=restart \
<br> Security Associations (2 up, 0 connecting): <br>
ipseclab[54]: ESTABLISHED 9 seconds ago, \
192.168.45.30[ipsec-lab-openwrt]...192.168.45.10[ipsec-lab-pfsense] <br>
ipseclab[54]: IKEv2 SPIs: 840c3fd77eb0efee_i b3f6cec233130e6a_r*, \
rekeying in 6 hours <br>
ipseclab[54]: IKE proposal: \
AES_CBC_256/HMAC_SHA2_256_128/PRF_HMAC_SHA2_256/MODP_2048 <br>
con1{54}: INSTALLED, TUNNEL, reqid 1, \
ESP SPIs: c206ad8c_i c631ebba_o <br> \
con1{54}: AES_GCM_16_256, 0 bytes_i, \
0 bytes_o, rekeying in 48 minutes <br> \
con1{54}: 10.10.10.0/30 === \
10.10.10.0/30 <br> ipseclab[53]: ESTABLISHED 19 seconds ago, \
192.168.45.30[ipsec-lab-openwrt]...192.168.45.10[ipsec-lab-pfsense] <br>
ipseclab[53]: IKEv2 SPIs: 133d2066ebf6a358_i d8da41bca97601ca_r*, \
rekeying in 6 hours <br>
ipseclab[53]: IKE proposal: \
AES_CBC_256/HMAC_SHA2_256_128/PRF_HMAC_SHA2_256/MODP_2048 <br>
con1{53}: INSTALLED, TUNNEL, reqid 1, \
ESP SPIs: cb169570_i c35823e7_o <br> \
con1{53}: AES_GCM_16_256, 0 bytes_i, \
0 bytes_o, rekeying in 51 minutes <br> \
con1{53}: 10.10.10.0/30 === \
10.10.10.0/30</span></div> <div style="font-family: Calibri, Helvetica, sans-serif; \
font-size: 12pt; color: rgb(0, 0, 0);"> <br>
</div>
<div style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; color: \
rgb(0, 0, 0);"> I will add to the thread later but NFLOG is able to capture MARKS in \
the packets. The packets captured by NFLOG has a special section with quite useful \
information.</div> <div style="font-family: Calibri, Helvetica, sans-serif; \
font-size: 12pt; color: rgb(0, 0, 0);"> <pre>...<br>Linux Netfilter NFLOG
Family: IPv4 (2)
Version: 0
Resource id: 7
TLV Type: NFULA_PACKET_HDR (1), Length: 8<br>...<br></pre>
</div>
<div style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; color: \
rgb(0, 0, 0);"> <br>
</div>
<div style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; color: \
rgb(0, 0, 0);"> Once again, thanks for the help.</div>
<div style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; color: \
rgb(0, 0, 0);"> <br>
</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; \
color:rgb(0,0,0);"> Tiago Stoco.</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; \
color:rgb(0,0,0);"> <br>
<hr tabindex="-1" style="display:inline-block; width:98%;">
<b>From:</b> Noel Kuntze<br>
<b>Sent:</b> Tuesday, August 31, 2021 2:20 PM<br>
<b>To:</b> Tiago Stoco; Tobias Brunner; users@lists.strongswan.org<br>
<b>Subject:</b> Re: [strongSwan] IPSec route based VPN - VTI interface TX Errors \
NoRoute <div><br>
</div>
</div>
<div>
<div class="BodyFragment"><font size="2"><span style="font-size:11pt;">
<div class="PlainText">Hello Tiago,<br>
<br>
> And, I have moved the route for the VTI to table 220 because it seems to be the \
right way to config routed based IPSec VPN.<br> > <br>
> [root@arch-linux ~]# ip rule<br>
> 0: from all lookup local<br>
> 220: from all lookup 220<br>
> 32766: from all lookup main<br>
> 32767: from all lookup default<br>
<br>
Don't do that.<br>
220 is managed by strongSwan. Keep them in the main table.<br>
<br>
> -A PREROUTING -d 10.10.10.0/30 -c 2352 230776 -j MARK --set-xmark \
0x2a/0xffffffff<br> > -A OUTPUT -d 10.10.10.0/30 -c 3605 336028 -j MARK \
--set-xmark 0x2a/0xffffffff<br> <br>
Disable the connmark plugin.<br>
<br>
<br>
> I have added a few more NFLOG captures into my iptables and I am a bit confused \
with the results.<br> > <br>
> A tcpdump capture in the VTI interface with a ping from the remote ( pfSense - \
10.10.10.1 ) shows :<br> > <br>
> No Time \
Source Destination<br> > <br>
> 1 0.000000 10.10.10.1 10.10.10.2 ICMP 84 Echo (ping) request id=0x9877, \
seq=471/55041, ttl=64 (reply in 2)<br> > 2 0.000038 10.10.10.2 > 10.10.10.1 \
ICMP 84 Echo (ping) reply id=0x9877, seq=471/55041, ttl=64 (request \
in 1)<br> > <br>
> I do not see the IPSec MARK in these packets.<br>
<br>
They are only visible in the output of the TRACE target. I suspect they are note \
copied into the buffer passed to the applications.<br> <br>
> <br>
> Also, the NAT chain is not having packets passing through it.<br>
> <br>
> [root@arch-linux ~]# snat<br>
> -P PREROUTING ACCEPT -c 0 0<br>
> -P INPUT ACCEPT -c 0 0<br>
> -P OUTPUT ACCEPT -c 0 0<br>
> -P POSTROUTING ACCEPT -c 0 0<br>
> -A PREROUTING -c 0 0 -j NFLOG --nflog-group 9<br>
> <br>
> That is odd cause I am not able to manipulate the packets.<br>
> <br>
> I will run a ping from the local Linux (10.10.10.2) and see how the packets are \
flowing through the iptables chains and will update in another email.<br> > <br>
<br>
The *nat table is only consulted for the first packet of a connection.<br>
<br>
Kind regards<br>
Noel<br>
<br>
<br>
Am 31.08.21 um 17:22 schrieb Tiago Stoco:<br>
> Hi Tobias,<br>
> <br>
> First of all, THANKS for replying and clarifying some settings.<br>
> <br>
> I have completely disabled the bypass-lan plugin since I do not have a use for \
it right now.<br> > <br>
> [root@arch-linux ~]# cat /etc/strongswan.conf<br>
> ...<br>
> plugins {<br>
> \
include strongswan.d/charon/*.conf<br> > \
bypass-lan \
{<br> > load \
= no<br> > }<br>
> ...<br>
> <br>
> And, I have moved the route for the VTI to table 220 because it seems to be the \
right way to config routed based IPSec VPN.<br> > <br>
> [root@arch-linux ~]# ip rule<br>
> 0: from all lookup local<br>
> 220: from all lookup 220<br>
> 32766: from all lookup main<br>
> 32767: from all lookup default<br>
> <br>
> [root@arch-linux ~]# ip r s t 220<br>
> 10.10.10.0/30 via 10.10.10.2 dev ip_vti1 src 10.10.10.2<br>
> <br>
> [root@arch-linux ~]# ip route<br>
> default via 192.168.45.1 dev ens18<br>
> 192.168.45.0/24 dev ens18 proto kernel scope link src 192.168.45.30<br>
> <br>
> I am going to add some more details of my configs because the TX Errors NoRoute \
are still present.<br> > <br>
> 7: ip_vti1@NONE: <POINTOPOINT,NOARP,UP,LOWER_UP> mtu 1436 qdisc noqueue \
state UNKNOWN group default qlen 1000<br> > link/ipip \
192.168.45.30peer 192.168.45.10promiscuity 0 minmtu 0 maxmtu 0<br> > \
vti remote 192.168.45.10 local 192.168.45.30 ikey 0.0.0.42 okey \
0.0.0.42 numtxqueues 1 numrxqueues 1 gso_max_size 65536 gso_max_segs 65535<br> \
> inet 10.10.10.2peer 10.10.10.1/32 scope global \
ip_vti1<br> > valid_lft forever \
preferred_lft forever<br> > inet6 fe80::5efe:c0a8:2d1e/64 \
scope link<br> > valid_lft forever \
preferred_lft forever<br> > <br>
> I can also see that the IPSec added some rules to MARK packets in my \
iptables.<br> > <br>
> -A PREROUTING -d 10.10.10.0/30 -c 2352 230776 -j MARK --set-xmark \
0x2a/0xffffffff<br> > -A OUTPUT -d 10.10.10.0/30 -c 3605 336028 -j MARK \
--set-xmark 0x2a/0xffffffff<br> > <br>
> The counters confirms that the packets are being marked. I am not sure if I \
should keep the MARK in iptables or remove it allowing routing decisions to send the \
packets to the VTI device that will MARK the packets but according to my \
understanding it should not matter.<br>
> <br>
> [root@arch-linux ~]# ip xfrm policy<br>
> src 0.0.0.0/0 dst 0.0.0.0/0<br>
> socket in priority 0 ptype \
main<br> > src 0.0.0.0/0 dst 0.0.0.0/0<br>
> socket out priority 0 ptype \
main<br> > src 0.0.0.0/0 dst 0.0.0.0/0<br>
> socket in priority 0 ptype \
main<br> > src 0.0.0.0/0 dst 0.0.0.0/0<br>
> socket out priority 0 ptype \
main<br> > src ::/0 dst ::/0<br>
> socket in priority 0 ptype \
main<br> > src ::/0 dst ::/0<br>
> socket out priority 0 ptype \
main<br> > src ::/0 dst ::/0<br>
> socket in priority 0 ptype \
main<br> > src ::/0 dst ::/0<br>
> socket out priority 0 ptype \
main<br> > <br>
> Above are the policies installed. Again, because it is a routed base VPN seems \
correct.<br> > <br>
> [root@arch-linux ~]# ip xfrm state<br>
> src 192.168.45.30 dst 192.168.45.10<br>
> proto esp spi 0xc2239b57 reqid 1 \
mode tunnel<br> > replay-window 0 \
flag af-unspec<br> > mark \
0x2a/0xffffffff<br> > aead \
rfc4106(gcm(aes)) 0x264acee3119a4e523af2fbf5905b50c5acc1f7be9079ff23ffa2c6473a9c507fe1ae936b \
128<br> > anti-replay context: seq \
0x0, oseq 0x0, bitmap 0x00000000<br> > src 192.168.45.10 dst 192.168.45.30<br>
> proto esp spi 0xc661b9e5 reqid 1 \
mode tunnel<br> > replay-window 32 \
flag af-unspec<br> > aead \
rfc4106(gcm(aes)) 0x69a86fa6ca9448bece6ffdff77893f0e9ce5ebef604040f681b5cdd2d5976438ed005df1 \
128<br> > anti-replay context: seq \
0x656, oseq 0x0, bitmap 0xffffffff<br> > <br>
> I have added a few more NFLOG captures into my iptables and I am a bit confused \
with the results.<br> > <br>
> A tcpdump capture in the VTI interface with a ping from the remote ( pfSense - \
10.10.10.1 ) shows :<br> > <br>
> No Time \
Source Destination<br> > <br>
> 1 0.000000 10.10.10.1 10.10.10.2 ICMP 84 Echo (ping) request id=0x9877, \
seq=471/55041, ttl=64 (reply in 2)<br> > 2 0.000038 10.10.10.2 > 10.10.10.1 \
ICMP 84 Echo (ping) reply id=0x9877, seq=471/55041, ttl=64 (request in \
1)<br> > <br>
> I do not see the IPSec MARK in these packets.<br>
> The reply packets end up in the OUTPUT chain marked but not encrypted as an ESP \
packet. By the way I do not see the replies even being encapsulated at all by \
IPSec.<br> > <br>
> Also, the NAT chain is not having packets passing through it.<br>
> <br>
> [root@arch-linux ~]# snat<br>
> -P PREROUTING ACCEPT -c 0 0<br>
> -P INPUT ACCEPT -c 0 0<br>
> -P OUTPUT ACCEPT -c 0 0<br>
> -P POSTROUTING ACCEPT -c 0 0<br>
> -A PREROUTING -c 0 0 -j NFLOG --nflog-group 9<br>
> <br>
> That is odd cause I am not able to manipulate the packets.<br>
> <br>
> I will run a ping from the local Linux (10.10.10.2) and see how the packets are \
flowing through the iptables chains and will update in another email.<br> > <br>
> In the meantime, if someone sees something that I am missing. Please let me \
know.<br> > <br>
> Many Thanks.<br>
> --------------------------------------------------------------------------------- \
-------------------------------------------------------------------------------------- \
-------------------------------------------------------------------------------------- \
-------------------------------------------------------------------------------------- \
-------------------------------------------------------------------------------------- \
-------------------------------------------------------------------------------------- \
-------------------------------------------------------------------------------------- \
-------------------------------------------------------------------------------------- \
-------------------------------------------------------------------------------------- \
-------------------------------------------------------------------------------------- \
---------------------------------------------------------------------------------------------------------------------------------------<br>
> *From:* Tobias Brunner <tobias@strongswan.org><br>
> *Sent:* Tuesday, August 31, 2021 5:51 AM<br>
> *To:* Tiago Stoco <tmsblink@msn.com>; users@lists.strongswan.org \
<users@lists.strongswan.org><br> > *Subject:* Re: [strongSwan] IPSec route \
based VPN - VTI interface TX Errors NoRoute<br> > Hi Tiago,<br>
> <br>
>> Pings from the Linux system are being seem as errors NoRoute by the tunnel. \
> ...<br> >> Shunted Connections:<br>
>> Bypass LAN 10.10.10.0/30: 10.10.10.0/30 === 10.10.10.0/30 PASS<br>
> <br>
> The reason is most likely this passthrough IPsec policy installed by the<br>
> bypass-lan plugin for the subnet that is reachable (according to the<br>
> main routing table) via ip_vti1. For a ping from 10.10.10.2 to<br>
> 10.10.10.1, the VTI interface won't find an IPsec policy to protect the<br>
> packet (the passthrough policy has a higher priority), so it gets dropped.<br>
> <br>
> To avoid that, either install the routes via VTI in table 220 (which is<br>
> ignored by the bypass-lan plugin automatically), exclude the VTI<br>
> interface explicitly via charon.plugins.bypass-lan.interfaces_ignore, or<br>
> just disable the bypass-lan plugin completely if you don't need it.<br>
> <br>
> Regards,<br>
> Tobias<br>
<br>
</div>
</span></font></div>
</div>
</body>
</html>
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic