I have multiple network interfaces configured on my Linux host system to route between multiple VLANs and use different internet providers. I organize most of my stuff with Docker and Docker Compose, and for some of these services and containers, I want to use a different network route to access the internet. One example would be to balance outgoing traffic between multiple providers or have a download-heavy container use a network with a faster or less used connection.

To achieve this, it’s necessary to create a new Docker bridge network and connect containers to it. Unfortunately, it’s also required to adjust the routing table on the host system to make everything work together, but it’s relatively simple.

Docker Setup: compose.yaml

Here’s an example compose.yaml file. I have defined two alpine test containers that both run wget to icanhazip.com to print out the IP address they are using for outgoing traffic to make it more clear.

The networks part at the end of the file is important for creating the new Docker network. This network and the chosen IP address space are just for the Docker containers. For the first Alpine service, you can see that I configured vlan3_network as the network to use for this container.

services:
  alpine:
    image: alpine
    command: "wget -qO- https://icanhazip.com"
    networks:
      - vlan3_network

  alpine2:
    image: alpine
    command: "wget -qO- https://icanhazip.com"

networks:
  vlan3_network:
    driver: bridge
    driver_opts:
      com.docker.network.bridge.name: docker_vlan3
    ipam:
      config:
        - subnet: 192.168.100.0/24

Linux Routing Configuration

echo "101 vlan3_table" | tee -a /etc/iproute2/rt_tables
ip rule add from 192.168.100.0/24 lookup vlan3_table
ip route add default via 192.168.3.1 dev enp2s0.vlan3 table vlan3_table

Essentially, we’re creating a new routing table with the name vlan3_table and adding a new default route so that all traffic from the subnet 192.168.100.0/24 goes through the host network interfaceenp2s0.vlan3.

Test

Now, when you run docker compose up, you should see the output from both containers, ideally displaying a different public IP address.

Persist Configuration

The ip commands from above are not saved when the system reboots. This is how I configure them using systemd when the system boots up.

  1. Create a Script to Execute on Boot:
vim /usr/local/bin/docker_routing.sh

With these contents:

#!/bin/sh

ip rule add from 192.168.200.0/24 lookup eth1_table
ip route add default via 192.168.2.1 dev eth1 table eth1_table

Make it executable:

chmod +x /usr/local/bin/docker_routing.sh
  1. Create the systemd Service:
vim /etc/systemd/system/routing-vlan3.service

With this:

[Unit]
Description=vlan3 routing
Requires=network-online.target
After=sys-subsystem-net-devices-enp2s0.vlan3.device
After=network-online.target

[Service]
Type=oneshot
ExecStart=/usr/local/bin/docker_routing.sh
RemainAfterExit=true

[Install]
WantedBy=multi-user.target

Hint: You can find the device name with systemctl list-units | grep vlan3

  1. Reload systemd and Enable the Service:
systemctl daemon-reload
systemctl enable routing-vlan3.service