In part 2 of this series, we will demystify how multiple containers running on the same host communicates with the host and vice versa.
Topics to be covered
Setting up multiple containers on the host
We will understand about Linux bridge
Set up container networking from scratch using Linux commands
Setting up the multiple containers on the host
In part 1 , we had single container running on the host, and it was able to communicate with the host. The implementation of it is shown below.
Now consider that we want to have one more container named container 2 as shown below. We created veth pair named veth2 and ceth2, attached one end (ceth2) inside the container and another end (veth2) in the host. This way the container 2 will be able to talk with the host.
But with this, how will container1 and container2 will communicate with each other? One possible way which I could think of now is to create another veth pair and connect two containers with each other as shown below.
You might now have a question is what was so difficult with this. Think of having one more container named container3, and further more containers. Now if all the containers wants to communicate with each other, we will end up creating many veth pairs which is not the ideal way.
How can we achieve that 🤔?. Think for few minutes ….
Okay, it’s the right time to introduce one more container networking constituent element.
Linux Bridge
Linux bridge is a virtual device which allow forwarding packets to the interfaces that are connected to it.
In the above diagram, we have two networks LAN1 and LAN2 connected by a bridge allowing host1 in LAN1 to send packets to host4 in LAN2. Any devices connected with bridge can communicate with each other by forwarding packets.
In our multi container scenario, we can create a bridge and attach our containers to it that will facilitate all containers to communicate with each other.
Continuing with setting up the multiple containers on the host
We will create a bridge and to attach containers to it, we can put one end of veth pair in container and attach other end with the bridge as shown.
Now inside the containers we can set bridge as the default gateway to send packets to each other and the host. If any packets are sent to the bridge, the bridge can forward to the destination connected to it.
Now let’s switch back to the terminal and write some scripts 😎
Prerequisites
In this article also, we are using the vagrant ubuntu/jammy64 virtual machine. Make sure you have the vagrant installed. Here is the link to the vagrant file to set up the VM.
To start the VM, go to the directory where you downloaded the vagrant file and then execute the below commands:
#To start the VM vagrant up #To SSH into the VM vagrant ssh
Note: All the commands in the following articles are run with root privileges. Either you can add sudo before each command or switch to different a user with root privileges by running sudo su
Set up container networking from scratch using Linux commands
In part 1 of this series, we already saw how to create network namespaces, add routes, run command inside network namespace etc. In this I will quickly run the commands. We will focus more on the bridge part to help you understand how containers are attached to it and how packets are sent via it.
To create two network namespace container1 and container2 ip netns add container1 ip netns add containe2
To create two veth pairs ip link add veth1 type veth peer name ceth1 ip link add veth2 type veth peer name ceth2
To attach one end of each veth pair with containers ip link set ceth1 netns container1 ip link set ceth2 netns container2
To enables the interfaces inside the containers ip netns exec container1 ip link set lo up ip netns exec container2 ip link set lo up ip netns exec container1 ip link set ceth1 up ip netns exec container2 ip link set ceth2 up
To assign ip address to interface inside containers ip netns exec container1 ip addr add 172.16.0.2/24 dev ceth1 ip netns exec container2 ip addr add 172.16.0.3/24 dev ceth2
To create a bridge, run ip link add br0 type bridge This will create a bridge named br0. Run ip link to list the bridge.
To attach the veth1 and veth2 to the bridge ip link set veth1 master br0 ip link set veth2 master br0
To enable veth1, veth2 and bridge br0 ip link set veth1 up ip link set veth2 up ip link set br0 up
To assign an ip address to the bridge, Run ip addr add 172.16.0.1/24 dev br0 A route is added in the host, it means any traffic which is destined for 172.16.0.0/24 send it via bridge and the source ip of the packet will the bridge ip address 172.16.0.1
To ping container2 form container1, Run ip netns exec container1 ping 172.16.0.3 You can now ping container 1 and container 2 from the host, ping containers from each other. But if you try to ping host from any containers ( ip netns exec container1 ping 10.0.0.10 ), it will fail saying the host is unreachable because there is no route in containers saying how to forward packets destined for host.
To add a default route in containers For container1, Run ip netns exec container1 ip route add default via 172.16.0.1 dev ceth1 For container2, Run ip netns exec container2 ip route add default via 172.16.0.1 dev ceth2 This adds a default route in the containers saying if no route matches for the destination, use bridge(172.16.0.1) as the default gateway to forward all packets and the out interface is ceth1 and ceth2.
To ping to internet from container 1, Run ip netns exec container1 ping 1.1.1.1 The ping to internet did not succeed!! Give yourself some time to think why cannot the containers reach to the internet…. In part 3, we will see how the containers can connect to the internet.
Conclusion
In this, you learned how multiple containers running on a single host can communicate with each other. You also learned about linux bridges and created a linux bridge to attach containers to it.
In part 3, we will try to resolve the connection between the containers and the internet. You will learn about iptables, network address translation etc.
Other related articles
Container Networking Deep Dive - Part 1
Container Networking Deep Dive - Part 3