Routing in overlapping ethernet segments

I have come across a few cases where you have (parts of) the same Ethernet segment (IP subnet) on two different interfaces. Overlapping segments only works when it is known in which segment the IP addresses are located, because explicit routes are used. If clients dynamically move between subnets, you are better off using bridging. With these constraints, let's move on.

Note: Please substitute addresses and interfaces names to match those of your actual situation when entering commands.

Router on the "left", network on the "right" side:

This situation can come up if the router is configured to use a certain subnet, and you want all traffic to go through your own server without having to resort to bridging or NAT. Bridging (in Linux) makes firewall scripts more complex and is a bit harder to debug because kernel/tcpdump seems to get packet flow tracing wrong. NAT is undesired because all clients in your local network will appear with the IP address of our server.

It is further assumed that the router does not provide any segment-dependent services (like DHCP), and/or our server handles the DHCP.

In this scenario, 192.168.60.1 is the router, which is set up to use 192.168.60.0/22. Our own server is 192.168.60.2, from which we run `ip addr` and `ip route`:

2: iet0: mtu 1500 qdisc pfifo_fast qlen 1000
    link/ether 00:20:18:8c:ce:79 brd ff:ff:ff:ff:ff:ff
    inet 192.168.60.2 peer 192.168.60.1/32 scope global iet0
    inet6 fe80::220:18ff:fe8c:ce79/64 scope link
      valid_lft forever preferred_lft forever
3: bond0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast qlen 1000
    link/ether 00:0a:e6:98:ed:d7 brd ff:ff:ff:ff:ff:ff
    inet 192.168.60.2/22 brd 192.168.63.255 scope global bond0
    inet6 fe80::fe80::20a:e6ff:fe98:edd7/64 scope link
       valid_lft forever preferred_lft forever

192.168.60.1 dev iet0 proto kernel scope link src 192.168.60.2
192.168.60.0/22 dev bond0 proto kernel scope link src 192.168.60.2
default via 192.168.60.1 dev iet0

[192.168.60.1]-----[192.168.60.2]-----[192.168.60.0/22]

The router is the only device on the iet0 segment. If there is another (static!) device in the iet0 segment, add another route for it. The router operates with 192.168.60.0/22, as do the clients in the actual 192.168.60.0/22%bond0 network. This creates an interesting situation:

1. The clients will not be able to reach 192.168.60.1, because it is not on the bond0 segment. They usually do not need to; define 192.168.60.2 as a router, which is just the right thing. If they do need 192.168.60.1 for whatever reason, you need to add an extra routing rule on each client, because 192.168.60.1 is only reachable through 192.168.60.2.

2. (Assuming the clients already have a route to 192.168.60.1.) The router will not be able to reach the clients, because there is only 192.168.60.2 on the iet0 segment. It may be problematic to add rules to the router, either because it is locked up by your ISP, or it does not provide a way to add routes of the required complexity (it would need 192.168.60.2/32 and 192.168.60.0/22 via 192.168.60.2).

So what is required here is that we fake the "existence" of the clients on the iet0 segment by making 192.168.60.2 send ARP replies for any ARP queries from the router. Because all clients are only reachable through our server anyway.

Network on the "left", single node at the "right" side

Another view (of the same case actually) is when you have a Bluetooth (or other sort of device) attached to your desktop machine. The router is 192.168.1.1, the desktop machine 192.168.1.127 and the BT device is at 192.168.1.128.

2: eth0: <BROADCAST,MULTICAST,NOTRAILERS,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast qlen 1000
    link/ether 00:15:f2:16:aa:46 brd ff:ff:ff:ff:ff:ff
    inet 192.168.1.127/24 brd 192.168.1.255 scope global eth0
    inet6 fe80::215:f2ff:fe16:aa46/64 scope link
       valid_lft forever preferred_lft forever
3: ppp0: <POINTOPOINT,MULTICAST,NOARP,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast qlen 3
    link/ppp
    inet 192.168.1.127 peer 192.168.1.128/32 scope global ppp0

192.168.1.128 dev ppp0 proto kernel scope link src 192.168.1.127
192.168.1.0/24 dev eth0 proto kernel scope link src 192.168.1.127
default via 192.168.1.1 dev eth0

The same as above happens: 192.168.1.128 does not know how to contact any hosts in 192.168.1.0%eth0, and hosts on eth0 do not know about 192.168.1.128%ppp0.

Solution

As previously mentioned, the solution is faking up ARP responses on the intermedia machine (example 1: 192.168.60.2; example 2: 192.168.1.127). To do this, we need brctl (package: bridge-utils) and ebtables. That is because the ebt_arpreply kernel module is only usable with ebtables, and that needs a bridge interface.

Well, did not we want to avoid bridging? Yes. And we do. The fact that we create a bridge does not imply bridging. (Just "close" the bridge and force everything to go through the valley.) This is what we will be doing. In fact, we will create what you could call a "half-bridge". Just remember that you have one (or more) "single" hosts and the usual network. In the first step, we will attach each network interface which needs arpreply to its own logical bridge:

brctl addbr br0
brctl setfd br0 1
brctl addif br0 iet0
ip link set br0 up

(Create new bridge interfaces br1, br2, etc. if there are more interfaces to fake ARP on.)

To make arpreply work, ARP packets must hit the ebtables kernel code, which happens only on bridging. So ARP packets continue to be 'bridged' (but are dropped then on arpreply, see --arpreply-target) while the rest is routed:

ebtables -t broute -P BROUTING DROP
ebtables -t broute -A BROUTING -p arp --arp-opcode request -j ACCEPT

In case ebtables complains about a missing table, make sure that the ebtables kernel module is already loaded; ebtables does not do that automatically, unfortunately.

Then add the ARP reply target:

ebtables -t nat -A PREROUTING -p arp --arp-opcode request -j arpreply --arpreply-mac aa:bb:cc:dd:ee:ff --arpreply-target DROP

Done. You may want to add this to a start script for the bridge to be set up at boot automatically.

External links

Linux Ethernet Bridging