Nomad reserved memory & CPU "OOM killed" issue
Fix the issue where Nomad reserves too much memory or CPU which results in tasks being killed by OOMKill.
Roman Zipp, April 24th, 2023
I recently ran into an issue where Nomad has reserved too much CPU & memory althoough configured otherwise. This can result in tasks being killed by the OOMKill.
OOM Killed
Workaround
There is a workaround by overriding the available memory and/or CPU power in your Nomad client config file.
Nomad client config
You can compute your "real" CPU compute power by multiplying the number of cores with the MHz. Example: 4 core CPU with 2 GHz each = 4 * 2000 = 8000.
To get your CPUs clock speed, use the lscpu util:
lscpu | grep MHz
The Nomad client config is located at /etc/nomad/nomad.hcl by default.
client {
memory_total_mb = 16000
cpu_total_compute = 8000
}