Issue creating subflow 2/ backup subflow to a remote endpoint

[DanielP2D]

Pre-requisites

• A similar question has not been reported before.
• mptcp.dev website does not cover my case.
• The wiki doesn't cover my case.
• This is not a question related to the current behavior, an issue or a feature requst: if it is, please use another template even if it is a question: we will need details about your system: kernel version, config, etc.

My question

Hi All,
My question is somewhat similar to what was asked in: #506 (comment), but however I still could not find a solution even after trying different combinations. It would be a great help if someone can guide me on this, thanks!

The entire setup is as follows:
Network configuration:
(privateNetwork_ns & publicNetwork_ns are dummy network namespaces at the moment, they will simple forward the packets)
OS: Ubuntu 24.04 LTS
Kernel version: 6.6.59

daniel@daniel-hp:~$ sudo ip netns exec gatewayProxyClient_ns sysctl -a | grep mptcp
net.mptcp.add_addr_timeout = 120
net.mptcp.allow_join_initial_addr_port = 1
net.mptcp.checksum_enabled = 0
net.mptcp.enabled = 1
net.mptcp.pm_type = 0
net.mptcp.scheduler = default
net.mptcp.stale_loss_cnt = 4
Go script:

daniel@daniel-hp:~$ sudo ip netns exec gatewayProxyServer_ns sysctl -a | grep mptcp
net.mptcp.add_addr_timeout = 120
net.mptcp.allow_join_initial_addr_port = 1
net.mptcp.checksum_enabled = 0
net.mptcp.enabled = 1
net.mptcp.pm_type = 0
net.mptcp.scheduler = default
net.mptcp.stale_loss_cnt = 4
Go script:

GOAL: Path 1 must be used primarily and switch to path 2 only in case of a break down of path 1 (OR use both paths in parallel). Could you please tell me how to configure the end points for this scenaraio?

daniel@daniel-hp:~/go/src/6mptcp$ sudo ip netns exec gatewayProxyClient_ns ip mptcp endpoint show
10.0.1.1 id 1 subflow dev veth_gpc_prn
10.0.3.1 id 2 subflow backup dev veth_gpc_pun

daniel@daniel-hp:~/go/src/6mptcp$ sudo ip netns exec gatewayProxyServer_ns ip mptcp endpoint show
10.0.2.2 id 1 signal dev veth_gps_prn
10.0.4.2 id 2 signal dev veth_gps_pun

With the above configuration, the path 2 (subflow 2) was not created, here are the outputs,

daniel@daniel-hp:~$ sudo ip netns exec gatewayProxyClient_ns ip mptcp monitor
[ CREATED] token=f13fa7ad remid=0 locid=0 saddr4=10.0.1.1 daddr4=10.0.2.2 sport=51318 dport=8080
[ ESTABLISHED] token=f13fa7ad remid=0 locid=0 saddr4=10.0.1.1 daddr4=10.0.2.2 sport=51318 dport=8080
[ ANNOUNCED] token=f13fa7ad remid=2 daddr4=10.0.4.2 dport=8080
[ SF_CLOSED] token=f13fa7ad remid=0 locid=2 saddr4=10.0.3.1 daddr4=10.0.2.2 sport=33039 dport=8080 backup=0 ifindex=96

daniel@daniel-hp:~$ sudo ip netns exec gatewayProxyServer_ns ip mptcp monitor
[ CREATED] token=57435ab2 remid=0 locid=0 saddr4=10.0.2.2 daddr4=10.0.1.1 sport=8080 dport=51318
[ ESTABLISHED] token=57435ab2 remid=0 locid=0 saddr4=10.0.2.2 daddr4=10.0.1.1 sport=8080 dport=51318

Traces from Wireshark:
10:45:39.110992535 10.0.1.1 10.0.2.2 51318 8080 MPTCP 51318 → 8080 [SYN] Seq=0 Win=64240 Len=0 MSS=1460 SACK_PERM TSval=1709536355 TSecr=0 WS=128

10:45:39.111018798 10.0.2.2 10.0.1.1 8080 51318 MPTCP 8080 → 51318 [SYN, ACK] Seq=0 Ack=1 Win=65160 Len=0 MSS=1460 SACK_PERM TSval=3263432456 TSecr=1709536355 WS=128

10:45:39.111032146 10.0.1.1 10.0.2.2 51318 8080 MPTCP 51318 → 8080 [ACK] Seq=1 Ack=1 Win=64256 Len=0 TSval=1709536355 TSecr=3263432456

10:45:39.111074112 10.0.2.2 10.0.1.1 8080 51318 MPTCP [TCP Window Update] 8080 → 51318 [ACK] Seq=1 Ack=1 Win=65280 Len=0 TSval=3263432456 TSecr=1709536355
Options: (28 bytes), No-Operation (NOP), No-Operation (NOP), Timestamps, MPTCP

TCP Option - No-Operation (NOP) 
TCP Option - No-Operation (NOP) 
TCP Option - Timestamps 
Multipath Transmission Control Protocol: **Add Address** 
    Kind: Multipath TCP (30) 
    Length: 16 
    0011 .... = Multipath TCP subtype: Add Address (3) 
    .... ...0 = Echo: 0 
    Address ID: 2 
    Advertised IPv4 Address: **10.0.4.2** 
    Truncated HMAC: 185465443590933752 

###############################################################
ISSUE
The [SF_CLOSED] event on the client-side, combined with the Wireshark output,

1. The server correctly announces Path 2 (10.0.4.2) using the ADD_ADDR option.
2. The client acknowledges this announcement but attempts to establish the second subflow to the server's private IP address (10.0.2.2) instead of the announced public IP address (10.0.4.2).
3. This incorrect connection attempt is likely causing the subflow to be closed.

I also tried to check if there are any links from path 2 network component to the endpoint of the path 1 network,
daniel@daniel-hp:~/go/src/6mptcp$ sudo ip netns exec publicNetwork_ns ping 10.0.2.2
ping: connect: Network is unreachable

I do not know where the issue lies, either in the gatewayProxyClient_ns.go & gatewayProxyServer.go scrtips or in network_configuration.sh or in configuring the endpoints.

can you please tell me if setting metric values for the paths is essential or MPTCP handles it irrespective of the priority when the other path fails?
network_configuration: network_configuration.txt

Sorry for the long query, just tried to add as much details as I possibly can.
Thank you!

[DanielP2D]

Hi, I have an update!
I had missed the add_addr_accepted command on the client side even though it was clearly mentioned on the MPTCP website. After setting the limit to 1 and setting the server endpoint as signal, I was able to use both the subflows in parallel. Even if one subflow fails, the other kept running.

And now for the other case, where one is the main subflow and other is the backup subflow, I did the following:
On the client side: ip mptcp endpoint add 10.x.x.x subflow backup dev y
ip mptcp limits set add_addr_accepted 1
on the server side: ip mptcp endpoint add 10.x.x.x signal dev x

The issue is fixed and hence closing the ticket, thanks!