Easily Install Kubernetes Metrics Server in minikube Today!

How To Solve The "cannot validate certificate for" x509 IP SANs Error

The following practice question for the Certified Kubernetes Application Developer (CKAD) exam came up this weekend while reviewing the observability exercises available on Dimitris-Ilias Gkanatsios’ CKAD Exercises project page:

Get CPU/memory utilization for nodes (metrics-server must be running)

Installing the Kubernetes (K8s) Metrics Server appears to be a straightforward one-liner from the command line (CLI) however I ran into problems while using minikube on both Ubuntu and OSX. Below I have the full details of the issue along with a full example of a working solution (the issue on GitHub includes the correct command however it doesn’t indicate where exactly it needs to be placed — we cover this below).

In this example, we rely on Kubectl version v1.19.4, Kubernetes version v1.17.3, and Docker version v19.03.6.

The Problem: Unexpected x509 certificate validation errors

According to the kubernetes-sigs / metrics-server page we should be able to install the Metrics Server with a single line (step one):

kubectl apply -f https://github.com/kubernetes-sigs/metrics-server/releases/latest/download/components.yaml

This results in the following, which looks promising:

creationTimestamp: “2020-11-29T22:07:21Z”
serviceaccount/metrics-server created
clusterrole.rbac.authorization.k8s.io/system:aggregated-metrics-reader created
clusterrole.rbac.authorization.k8s.io/system:metrics-server created
rolebinding.rbac.authorization.k8s.io/metrics-server-auth-reader created
clusterrolebinding.rbac.authorization.k8s.io/metrics-server:system:auth-delegator created
clusterrolebinding.rbac.authorization.k8s.io/system:metrics-server created
service/metrics-server created
deployment.apps/metrics-server created
apiservice.apiregistration.k8s.io/v1beta1.metrics.k8s.io created

— however on closer inspection, we can see that the Metrics Server isn’t working. For example, executing:

kubectl top nodes

returns:

Error from server (ServiceUnavailable): the server is currently unable to handle the request (get nodes.metrics.k8s.io)

and executing:

kubectl get deployment metrics-server -n kube-system

returns:

NAME READY UP-TO-DATE AVAILABLE AGE
metrics-server 0/1 1 0 105s

— which is also wrong. I’ve included an image below with blue arrows that point to the command and several orange arrows that direct your attention to the output that shows that the Metrics Server is unavailable.

Terminal output for kubectl commands that show that the Kubernetes Metrics Server running in k8s minikube has a problem "x509: cannot validate certificate for 192.168.99.100 because it doesn't contain any IP SANs."
Kubernetes Metrics Server Incomplete Installation

The block in orange is directly related to the issue on Github entitled “metrics-server error because it doesn’t contain any IP SANs” — buried in the conversation is one suggested solution which works and we’ll cover that next.

The Solution: Add commands to the Kubernetes Metrics Server

In order to fix this we need to add the following to the components.yaml file (step two):

command:
- /metrics-server
- --kubelet-insecure-tls
- --kubelet-preferred-address-types=InternalIP

Below we have the complete file with notes, see lines 141 to 215, specifically as this is where the change needs to be applied.

Once we’ve edited the configuration directly, we can restart minikube and we should see something that looks like what we have below. Note that the green arrows point to output that indicates that the Metrics Server is now running correctly — you should see something similar to this.

Note that you can edit your component.yaml file however you’ll need to apply it once the change has been added — so either (step three):

kubectl edit deployment -n kube-system metrics-server

and add this text to the appropriate place or modify the components.yaml file and then run:

kubectl apply -f ./components.yaml

Finally you’ll need to stop and start minikube using (step four & five):

minikube [stop | start]

The image below details what this should look like along with the expected output.

Shell output with pointers to minikube restart commands, a view of the top nodes, and evidence that the Kubernetes Metrics Server is running.
Kubernetes Metrics Server Working Installation

As the Metrics Server is now running correctly, this issue has been addressed.

Easy Kubernetes Metrics Server Install Conclusion

If you found this article to be helpful you may also like the article I wrote recently entitled “Answers to Five Kubernetes CKAD Practice Questions (2020)” where we review five questions as they pertain to the Certified Kubernetes Application Developer (CKAD) exam and include answers along with commentary and verification that the given solutions work.

Questions and comments are welcomed.

Tags: , , ,

thospfuller

I am a Web Design, Technical SEO, and WordPress Specialist based in Northern Virginia. I am interested in software development, content engineering, and business. I'm originally from Chicago, IL, and currently reside in Reston, VA.