Background and Motivation
Currently we keep the EVE device networking and the different network instances for application networking separated using a combination of ip rules (PBR = policy based routing) and ACLs (iptables).
Our desired semantics is to have network instances separated from each other as much as possible. For example, applications deployed on different network instances should not be able to communicate with each other directly. Only through so-called "port maps", which are effectively port-forwarding ACL rules, network communication can be established between applications on different network. But even in this case traffic does not get forwarded between networks directly, instead it is hairpined via an uplink interface where the port mapping is configured. For networks using different uplink interfaces it is even required to hairpin the traffic outside box, even if the communicating applications are deployed on the same edge device.
The IPsec-based VPN network instances should be even more isolated from each other and other networks. Application inside a VPN network should only be able to talk to apps in the same network or with endpoints on the other side of the tunnel.
The following drawing visually depict the current EVE implementation with an example of 4 apps, 2 local NIs, one switch NI and some (non-overlapping) IP addressing:
While the current implementation is successful in prohibiting a direct communication between network instances and facilitating hairpinning inside or outside the device based on the aforementioned criteria, it fails to isolate networks when it comes to IP address allocation.
For example, it is not possible to deploy multiple network instances with the same or overlapping IP subnets. A significant risk of IP address collisions exists also between external (uplink) and internal (downlink) networks. A local network instance should be completely isolated and independent from the outside networks with a NAT placed in-between. Similarly, traffic selectors of multiple VPN networks could overlap, thus preventing from opening the tunnels at the same time, which is also against our semantics of network instance isolation.
We can get better isolation, including IP address isolation if we split network instances using either VRFs or with network namespaces. Furthemore, if we use a containerd task to run network instance networking (especially the external processes like dnsmasq, radvd, etc.), we can even isolate resource usage and apply limiting. We will now describe VRFs and network instances separately, with a bit more focus on VRFs, which, after some internal discussion, is now the preferred choice.
VRFs
TODO
TODO
Network Namespaces
TODO
Proof of Concept
TODO
Development Steps (VRF Proposal)
TODO