X509 credentials for kubernetes cluster access

Introduction

This document describes a secure method for creating X509 credentials to grant API access to a Kubernetes cluster. It has the advantage that no confidential information such as a private key needs to be transmitted. This is a general procedure that works for providing access to any user, but in this context a Kubernetes cluster admin is providing access to a Harvester admin.

References

• How to configure RBAC in your kubernetes cluster
• Kubernetes authorization overview
• Kubernetes authentication overview
• Managing TLS certificates in a cluster
• Client certificate authentication
• Important note on approving certificate signing requests
• Certificate signers

Secure the cluster

The Kubernetes admin should ensure that RBAC and PSP (along with the relevant admission controller) are applied and enabled before granting external access to the cluster. That is outside the scope of this document.

Configure certificate expiry time

When creating the cluster, you can use the cluster-signing-duration kubeadm option to increase the lifetime of certificates issued by the cluster, so that the credentials don't need to be renewed as frequently.

Creation of user credentials

On Harvester node

• The Harvester admin creates a new private key, or could re-use an existing key. It must be stored securely. openssl genrsa -out k8s-private.pem
• The Harvester admin creates a CSR from the key. The username and group must be coordinated with the Kubernetes admin, and should follow a naming scheme. For example, use the group harvester and a username like aiatlasXYZ identifying a Harvester node or which private key was used. openssl req -new -key k8s-private.pem -out <username>.csr -subj "/CN=<username>/O=<group>"
• The Harvester admin sends the .csr file to the Kubernetes admin. It is advisable to verify the authenticity and integrity of the file e.g. by confirming the sha256sum via a separate trusted channel.

On Kubernetes cluster

• The Kubernetes admin should verify that the username and group(s) in the CSR were specified as expected. Otherwise, the credential could grant different privileges than intended.
  • openssl req -in <username>.csr -noout -text
• Verify that the checksum of the .csr file matches what the Harvester admin attested.
• Create a CertificateSigningRequest like this, where the request field is the output of base64 -w 0 <username>.csr

---
apiVersion: certificates.k8s.io/v1
kind: CertificateSigningRequest
metadata:
  name: csr-name
spec:
  request: LS0......0tCg==
  signerName: kubernetes.io/kube-apiserver-client
  #expirationSeconds: 94608000
  usages:
  - digital signature
  - client auth

• Note:
  • Although the username and group can be specified in the CSR object definition, apparently it has no effect and does not override the user-provided values imported from the CSR file (as of v1beta1), hence the need to explicitly confirm them in a prior step.
  • The expirationSeconds field can optionally be specified to request a certificate lifetime longer than the default, if allowed by the configuration of the maximum ClusterSigningDuration of the cluster.
• Approve the CSR object: kubectl certificate approve csr-name.
• Extract the public certificate for the user: kubectl get csr csr-name -o jsonpath='{.status.certificate}' | base64 -d
  • This needs to be done soon after the approval, or it will expire and disappear, requiring re-approval.

Prepare kubeconfig for Harvester admin

To get the CA of the cluster: kubectl config view --raw --minify --flatten -o jsonpath='{.clusters[].cluster.certificate-authority-data}' | base64 -d > cluster-ca.crt

Provide the user certificate (as extracted above) and the cluster CA certificate to the Harvester admin. Along with the private key retained by the Harvester admin, now all 3 pieces of information needed for a kubeconfig file are available. Each one (key, cert, CA) can be defined in-line in a kubeconfig file (base64 encoded using base64 -d) or referenced as a separate file relative to the location of the kubeconfig file. See the links provided.

You can check the expiry date of the user certificate by doing openssl x509 -text -noout on it.

To produce an example kubeconfig file you can do something like this:

• kubectl config set-cluster MyCluster --server=https://<address>:6443 --certificate-authority=dummyca
• kubectl config set-credentials k8s-user --client-certificate=dummycert --client-key=dummykey
• kubectl config set-context MyContext --cluster=MyCluster --namespace=MyNamespace --user=k8s-user
• kubectl config use-context MyContext

Allow access to the cluster

The Kubernetes admin can now configure RBAC and PSP as needed, based on the username and group configured. For example, the subjects portion of a RoleBinding would look like this to allow the harvester group certain privileges.

subjects:
  - kind: Group
    name: harvester
    apiGroup: rbac.authorization.k8s.io