I’m running k3s, a lightweight Kubernetes / k8s environment, on the oracle cloud free tier. However, Oracle does deploy some weird iptables rules onto all of their distributions which one should not delete, otherwise the system will not boot since it can’t mount remote file systems. I’ve done this by accident with ufw rules to deny certain traffic. With this change, I deleted all existing rules and couldn't start the instance anymore. 🤷‍♂️ There's more about this topic here.

Fix for Failed to connect to example.com port 443 after 9 ms: Host is unreachable

Unfortunately, for some reason the deployed iptables rules by Oracle also prevent some pods under some circumstances (when run as a cron, but not interactively for instance) to not be able to resolve DNS and connect out to the internet. However, I’ve got a few pods running which need to connect to the internet to retrieve data.

If you see some errors in your pod logs, such as:

Failed to connect to example.com port 443 after 9 ms: Host is unreachable

You might want to update your iptables rules like so to allow DNS from the Kubernetes Node:

Run this command on your Oracle Cloud instance:

sudo iptables -t nat -A PREROUTING -p udp --dport 53 -j DNAT --to 10.0.0.163:53

replace 10.0.0.163 with the IP address of your Kubernetes node, you can get the IP address either with kubectl get nodes -o wide or directly with kubectl get nodes -o jsonpath="{.items[*].status.addresses[].address}"

There’s a related issue on GitHub which suggested this change. There’s also still another open related issue.