How to fix Nomad using wrong network interface for jobs

This guide will help you to fix the issue when Nomad / Consul are binding jobs on the wrong interface.

Get your private interface

First, run ifconfig to check if the interface is available and get it's name.

enp7s0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1450
        inet 10.4.0.3  netmask 255.255.255.255  broadcast 10.4.0.3
        inet6 fe80::8400:ff:fe74:ac5c  prefixlen 64  scopeid 0x20<link>
        ether 86:00:00:74:af:6a  txqueuelen 1000  (Ethernet)
        RX packets 8370677  bytes 1497631969 (1.4 GB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 7785709  bytes 1158079549 (1.1 GB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

If your networking is configured correctly you will find your private interface named like ens..., enp....

Fix

You can either hardcode the interface name or use the go-sockaddr/template templating language to get the private interface name.

The following example will get the name of the first network interface in the 10.0.0.0/8 address space (this will match all ip addresses from 10.0.0.0 to 10.255.255.255).

client {
  # use go-sockaddr/template
  network_interface = "{{ GetPrivateInterfaces | include \"network\" \"10.0.0.0/8\" | attr \"name\" }}"
  
  # hardcode the interface name
  network_interface = "enp7s0"
}

It's important to restart Nomad and all jobs afterwards. Only restarting Nomad will not be enough since the job will keep running in the background.

References

See Nomad network_interface Docs

Troubleshooting

To see all configuration values and debug info, run nomad operator debug and quit using Ctrl+C. A .tar.gz file will be created in the current directory. Download, extract it and check the cluster/agent-self.json file for the NetworkInterface value to see if the value has been set correctly.