EKS Anywhere, validating KeyCloak OIDC SSO access to clusters for kubectl

This article is part of the EKS Anywhere series EKS Anywhere., extending the Hybrid cloud momentum

In the previous two related articles, we have already setup the KeyCloak server and also configured our EKS Anywhere cluster for OIDC access.

In this article, we will observe how to access the OIDC enabled cluster via kubectl. While there are many plugins/methods available to do so., it largely depends on whether you prefer a browser based or a browserless authentication method to access the cluster. We will document the latter (browserless) method as it greatly simplifies the challenges one would face in case of logins via jump hosts or where browser-based authentications are not feasible.

We can test the below procedure right from the EKS Anywhere Administrative machine as it already has kubectl installed. Alternatively, in this specific example, I will assume a developer machine with WSL2 Ubuntu installed with kubectl.

The main goal is to create the kubectl contexts for various users and validate access levels. Recall our RBAC arrangement (oidc user/groups, k8s rbac role mappings) via the previous two articles

• user-admin | kube-admin |cluster-admin role
• user-dev | kube-dev | edit role
• user-view-only | kube-view-only | view role

Let’s begin by performing the below procedure on any Linux machine which has kubectl installed.

• Create a new file named create-oidc-contexts.sh under $HOME and make it executable
• Copy and paste the contents of the below given raw gist to the above created file 

https://gist.github.com/thecloudgarage/fdf0cdc89a49ed0fcf75c19b69e5eb8e

Repeat the below script for each of the OIDC user & cluster pairs. In this case, since we have only one cluster and 3 users, a total of 3 contexts needs to be created, i.e., the below script needs to be run thrice with each of the OIDC usernames. Repeat the script for additional clusters & user pairs.

 

source $HOME/create-oidc-contexts.shThe script will prompt you for the below parameters on a per context basis. I have provided my setup's parameters. Yours could be different depending on what you have set the values in the previous two exercises (KeyCloak server setup and OIDC enablement on EKS Anywhere cluster)* oidcClusterName: oidctestcluster01* fqdnOfKeyCloakServer: keycloak.thecloudgarage.com* oidcClientId: kube (yours will be the same)* oidcClientSecret: kube-client-secret (yours will be the same)* oidcUsername: user-admin or user-dev or user-view-only* oidcPassword: user-admin or user-dev or user-view-only* apiServerEndpoint: 172.24.165.11 (yours will be different)

 

Let’s observe the actual execution of this script in my machine as I intend to access the EKS Anywhere cluster named testwk01 which is configured for KeyCloak SSO OIDC

NOTE:

• The client secret for all contexts will be kube-client-secret
• The password for OIDC users is same as the username

Let’s create the context for user-admin and oidctestcluster01 cluster

 

source create-oidc-contexts.shInput OIDC enabled cluster name for kubectl contextoidClusterName: oidctestcluster01Input your OIDC servers FQDNfqdnOfKeyCloakServer: keycloak.thecloudgarage.comInput your OIDC client idoidcClientId: kubeInput your OIDC SecretoidcClientSecret:Input your OIDC UsernameoidcUsername: user-adminInput your OIDC PasswordoidcPassword:Provide the API server endpoint in the format https://172.24.165.11:6443Ensure that https and port number 6443 is mentioned as specififed in the above formatOnly in cases e.g. eks public clusters, you can omit the port number as it provides a load-balancer URLapiServerEndpoint: https://172.24.165.11:6443Cluster "oidctestcluster01" set.depth=0 C = IN, ST = MH, L = Mumbai, O = stack, OU = devops, CN = keycloak.thecloudgarage.comverify error:num=20:unable to get local issuer certificateverify return:1depth=0 C = IN, ST = MH, L = Mumbai, O = stack, OU = devops, CN = keycloak.thecloudgarage.comverify error:num=21:unable to verify the first certificateverify return:1DONE% Total    % Received % Xferd  Average Speed   Time    Time     Time  CurrentDload  Upload   Total   Spent    Left  Speed100  3816  100  3673  100   143  29861   1162 --:--:-- --:--:-- --:--:-- 31024% Total    % Received % Xferd  Average Speed   Time    Time     Time  CurrentDload  Upload   Total   Spent    Left  Speed100  3816  100  3673  100   143  31127   1211 --:--:-- --:--:-- --:--:-- 32338deleted user user-admin from /home/ubuntu/.kube/configwarning: this removed your active context, use "kubectl config use-context" to select a different onedeleted context user-admin-oidctestcluster01 from /home/ubuntu/.kube/configProperty "current-context" unset.User "user-admin" set.Context "user-admin-oidctestcluster01" created.Switched to context "user-admin-oidctestcluster01".

 

Let’s create the context for user-dev and oidctestcluster01 cluster

 

source create-oidc-contexts.shInput OIDC enabled cluster name for kubectl contextoidClusterName: oidctestcluster01Input your OIDC servers FQDNfqdnOfKeyCloakServer: keycloak.thecloudgarage.comInput your OIDC client idoidcClientId: kubeInput your OIDC SecretoidcClientSecret:Input your OIDC UsernameoidcUsername: user-devInput your OIDC PasswordoidcPassword:Provide the API server endpoint in the format https://172.24.165.11:6443Ensure that https and port number 6443 is mentioned as specififed in the above formatOnly in cases e.g. eks public clusters, you can omit the port number as it provides a load-balancer URLapiServerEndpoint: https://172.24.165.12:6443Cluster "oidctestcluster01" set.depth=0 C = IN, ST = MH, L = Mumbai, O = stack, OU = devops, CN = keycloak.thecloudgarage.comverify error:num=20:unable to get local issuer certificateverify return:1depth=0 C = IN, ST = MH, L = Mumbai, O = stack, OU = devops, CN = keycloak.thecloudgarage.comverify error:num=21:unable to verify the first certificateverify return:1DONE% Total    % Received % Xferd  Average Speed   Time    Time     Time  CurrentDload  Upload   Total   Spent    Left  Speed100  3774  100  3635  100   139  28849   1103 --:--:-- --:--:-- --:--:-- 29952% Total    % Received % Xferd  Average Speed   Time    Time     Time  CurrentDload  Upload   Total   Spent    Left  Speed100  3774  100  3635  100   139  29795   1139 --:--:-- --:--:-- --:--:-- 30934deleted user user-dev from /home/ubuntu/.kube/configdeleted context user-dev-oidctestcluster01 from /home/ubuntu/.kube/configProperty "current-context" unset.User "user-dev" set.Context "user-dev-oidctestcluster01" created.Switched to context "user-dev-oidctestcluster01".

 

And lastly for user-view-only and cluster oidctestcluster01

 

source create-oidc-contexts.shInput OIDC enabled cluster name for kubectl contextoidClusterName: oidctestcluster01Input your OIDC servers FQDNfqdnOfKeyCloakServer: keycloak.thecloudgarage.comInput your OIDC client idoidcClientId: kubeInput your OIDC SecretoidcClientSecret:Input your OIDC UsernameoidcUsername: user-view-onlyInput your OIDC PasswordoidcPassword:Provide the API server endpoint in the format https://172.24.165.11:6443Ensure that https and port number 6443 is mentioned as specififed in the above formatOnly in cases e.g. eks public clusters, you can omit the port number as it provides a load-balancer URLapiServerEndpoint: https://172.24.165.12:6443Cluster "oidctestcluster01" set.depth=0 C = IN, ST = MH, L = Mumbai, O = stack, OU = devops, CN = keycloak.thecloudgarage.comverify error:num=20:unable to get local issuer certificateverify return:1depth=0 C = IN, ST = MH, L = Mumbai, O = stack, OU = devops, CN = keycloak.thecloudgarage.comverify error:num=21:unable to verify the first certificateverify return:1DONE% Total    % Received % Xferd  Average Speed   Time    Time     Time  CurrentDload  Upload   Total   Spent    Left  Speed100  3898  100  3747  100   151  31487   1268 --:--:-- --:--:-- --:--:-- 32756% Total    % Received % Xferd  Average Speed   Time    Time     Time  CurrentDload  Upload   Total   Spent    Left  Speed100  3898  100  3747  100   151  31225   1258 --:--:-- --:--:-- --:--:-- 32483deleted user user-view-only from /home/ubuntu/.kube/configdeleted context user-view-only-oidctestcluster01 from /home/ubuntu/.kube/configProperty "current-context" unset.User "user-view-only" set.Context "user-view-only-oidctestcluster01" created.Switched to context "user-view-only-oidctestcluster01".

 

Now that all the three contexts are set., we can verify them in our kube config file

 

KUBECONFIG=$HOME/.kube/configkubectl config get-contextsKUBECONFIG=$HOME/.kube/configkubectl config get-contextsCURRENT   NAME                      CLUSTER    AUTHINFO         NAMESPACEuser-admin-oidctestcluster01       oidctestcluster01   user-adminuser-dev-oidctestcluster01         oidctestcluster01   user-dev*         user-view-only-oidctestcluster01   oidctestcluster01   user-view-only

 

Let’s start validating the access levels by switching the contexts

Let’s begin by switching context to user-admin that has a cluster role of cluster-admin. We can see that this context can create namespaces and retrieve node information

 

kubectl config use-context user-admin-oidctestcluster01Switched to context "user-admin-oidctestcluster01".kubectl create namespace testonenamespace/testone createdkubectl get nodesNAME                             STATUS   ROLES                  AGE   VERSIONoidctestcluster01-ftqg7                   Ready    control-plane,master   18h   v1.21.13-eks-b88cc51oidctestcluster01-kv7q6                   Ready    control-plane,master   18h   v1.21.13-eks-b88cc51oidctestcluster01-md-0-7595c49d8d-5vc59   Ready18h v1.21.13-eks-b88cc51 oidctestcluster01-md-0-7595c49d8d-6khmx Ready18h v1.21.13-eks-b88cc51

 

Next let’s switch the context to user-dev which has a cluster role of edit. As you can see that this role restricts the permissions model to certain actions as implicitly defined in the default “edit” role

 

kubectl config use-context user-dev-oidctestcluster01Switched to context "user-dev-oidctestcluster01".kubectl get namespaceNAME              STATUS   AGEdefault           Active   18hkube-node-lease   Active   18hkube-public       Active   18hkube-system       Active   18htestone           Active   3m27skubectl create namespace testtwoError from server (Forbidden): namespaces is forbidden: User "user-dev@emaildomainname" cannot create resource "namespaces" in API group "" at the cluster scopekubectl get nodesError from server (Forbidden): nodes is forbidden: User "user-dev@emaildomainname" cannot list resource "nodes" in API group "" at the cluster scopekubectl get podsNo resources found in default namespace.

 

Lastly, let’s switch the context to user-dev. Likewise, as per the permissions implicit to the default “view” cluster role, the user can only perform certain restricted actions

 

kubectl config use-context user-view-only-oidctestcluster01Switched to context "user-view-only-oidctestcluster01".kubectl get namespaceNAME              STATUS   AGEdefault           Active   18hkube-node-lease   Active   18hkube-public       Active   18hkube-system       Active   18htestone           Active   6m25skubectl get podsNo resources found in default namespace.kubectl get nodesError from server (Forbidden): nodes is forbidden: User "user-view-only@emaildomainname" cannot list resource "nodes" in API group "" at the cluster scope

 

That’s it! Hopefully, you would have understood the interaction model between EKS Anywhere clusters & KeyCloak SSO OIDC., and how you can effectively create organization wide policy derivatives to secure Kubernetes access

cheers

Ambar Hassani

#iwork4dell