I updated my kubernetes cluster this weekend to 1.7 and noticed my Datadog metrics stopped reporting. Took a few hours of digging but I eventually found the culprit as a change kubernetes did that disables cAdvisor from port 4194 which the Datadog agent relies on to get information.

I found the relevant PR here which shows the change and reasoning (and the reasoning makes sense to me). I brought this to the attention of datadog and they said they are aware of it and are working on either updating the documentation with the changes or using the stats endpoint instead.

That said it is possible to re-enable the cAdvisor port should you have taken precautions to prevent public access to it which you can find here.

The gist is to run the following:

1
2
3
sed -e "/cadvisor-port=0/d" -i /etc/systemd/system/kubelet.service.d/10-kubeadm.conf
systemctl daemon-reload
systemctl restart kubelet

Hopefully this saves you the hours it took me to find this!