As you may already know, certificates plays an important role in Kubernetes clusters. This is because the communications between various Kubernetes components are secured using TLS. For this encryption of communication and for authentication purposes, we need certificates. Here, the authentication method is 2 way authentication. So this means we need certificates for both the client and the server.
Server certificates in Kubernetes include certificates of, the Kube-API server, the Etcd server and the Kubelet servers. Client components having client certificates includes, Kube-Scheduler, Kube-Controller manager and the Kube-Proxy. Aside from the above, administrators trying to authenticate through kubectl or REST API also need to have client certificates. Apart from the client and server certificates, we need to have the root level certificates. Also known as CA (Certificate Authority) certificates.
This is a lot.. It is important to keep these certificates current and renew them soon as they near their expiration. If not there will be consequences. There are both automatic and manual ways of managing the certificates. I intend to write a 4 part blog series that covers most aspects of this.
In this blog, I intend to write about how we can create certificates and how we can view the certificate details manually.
Generating a certificate.
In this section we will see how we can generate certificates for different components in a Kubernetes cluster. Firstly, let’s look at the general workflow on creating certificates and then let’s look at some specific situations.
1 – Create a key.
Let’s say we are creating a key called admin_user.key. The syntax for this is openssl genrsa -out <keyname> <number of bits for the key>. So our command will look something like this.
openssl genrsa -out admin_user.key 2048
2 – Create a certificate signing request (CSR).
Next, we must create a certificate signing request. For this, the command syntax is, openssl req -new -key <keyfile> -subj "/CN=<who or what component you intend to authenticate as>/O=<Name of a group(optional)>" -out <name for csr file>. Therefore our command for this situation looks like this.
openssl req -new -key admin_user.key -subj "/CN=kube-admin/O=system:masters" out admin.csr
The O=system:masters specifies the group and we use this when we are creating a CSR for a user. When it comes to other Kubernetes system components such as the Kube-Scheduler, Kube-Controller-Manager and Kube Proxy, we need to prefix the CN part with a “system:”. i.e. subj "/CN=system:kube-proxy"
3 – Generating the certificate.
Lastly, we need to create the certificate itself. The syntax for this is openssl x509 -req -in <CSR file> -CA <Certification Authority’s certificate> -CAkey <Root A.K.A Certificate Authority’s key> -out <Certificate file name>. An example command is as follows.
openssl x509 -req -in admin.csr -CA ca.crt -CAkey ca.key -out admin_user.crt
You may be wondering about the CA key above. Do we generate it ourselves? Yes. We can generate this ourselves and this would be the first certificate that needs to be generated before any other certificates (for any component or user) are generated. The steps to do that are the same 3 steps above with one exception. For the final command, we don’t specify a -CA as there are no CAs at this point. We call this a self signed certificate because we are using the same key generated for the certificate to sign the certificate itself.
Another exception to the above steps is generating certificates for the Kube-API server. The Kube-API server can have many names. To fulfil this requirement, you can create config file and specify subject alternative names. More on that here.
Once we create all the certificates required for various Kubernetes components, we can put the them and the keys in /var/lib/kubernetes or /etc/kubernetes/pki (or anywhere we wish). We need to then, specify where we put the certificates and the keys in either in the service configuration file or in the YAML files if the Kubernetes components are run as static pods.
Viewing the certificates
You may already know that we can deploy Kubernetes components as either services or static pods. Depending on which method we use, how to locate the existing certificate files would differ. If they are running as services, we can look at the service configuration file for the certificate location. Else, we can read the YAML file of the static pods to know the location of the certificates.
Next, we can CD to the location where we have the certificate files.
Let’s assume that we have a certificate named apiserver.crt and we need to view its details. For this we can use a command like this. openssl x509 -in <location of the certificate> -text -noout. Therefore in our case this will be something like this.
openssl x509 -in /etc/kubernetes/pki/apiserver.crt -text -noout
Using the above command, we can get information like who is the issuer of the certificate, when is the expiry date and other alternative names given.
While viewing information such as the expiry date is possible in the above method, it is not the most ideal way to do it. This could be as there could be multiple clusters having multiple components. Which implies multiple certificates. This way there is a chance that a certificate may expire without being renewed as it was not brought in to attention at the right time.
To address the aforementioned challenge, we can use a monitoring system to detect when certificates are about to expire and notify us. After which we can automatically renew these certificates. And that is a topic for another blog 🙂
To learn how to get a free certificate for your website/blog check this out.