Sukanta Maikap

How to connect to a container running in k8s as 'root' user

Kubernetes is built around the philosophy of immutable infrastructure. You may still need to inspect the pods by connecting to them, especially during cluster development. Connection to a pod running in Kubernetes is easy:

kubectl exec -it order-6b78846998-5zt49 --container order -- /bin/bash

But, it won’t give you root access unless the image is built with root as the current user. And, many times, you won’t have access to the underlying Dockerfile to make the necessary changes.

In the world of docker, connecting to a docker container as root is very easy and does not require a Dockerfile change :

docker -u root exec -it {{container name or id}} bash

But when you are running the same container on a Kubernetes cluster, it is not straightforward. Unfortunately, the below command won’t work:

kubectl -u root exec -it {{pod name}} bash

The solution is a bit convoluted but doable. You need to connect to the node and then connect to the container from there using docker. Let’s assume you have two replicas of a container named order running on a Kubernetes cluster. Here are the steps :

Find the node for that corresponding pod running the container you would like to connect as root.

>> kubectl get pods -o wide
NAME                    READY  STATUS   RESTARTS  AGE    IP            NODE                                       NOMINATED NODE
order-7595956475-9t6w9  1/1    Running  0         3h55m  192.168.1.3   gke-ms-cluster-default-pool-1bc2a6cd-kz0l  <none>
order-7595956475-rxv52  1/1    Running  0         3h55m  192.168.2.4   gke-ms-cluster-default-pool-6098e4ac-ld2l  <none>

Let’s say, I want to connect to order-7595956475-9t6w9 as root user. The corresponding node is gke-ms-cluster-default-pool-1bc2a6cd-kz0l.

First, inspect the pod in question to get the docker container you want to connect to.

>> kubectl describe pod order-7595956475-9t6w9
Name: order-7595956475-9t6w9
Namespace: default
Priority: 0
PriorityClassName: <none>
Node: gke-toms-test-cluster-default-pool-1bc2a6cd-kz0l/10.40.30.16
Start Time: Thu, 22 Aug 2019 10:35:19 +1000
Labels: app=order
pod-template-hash=7595956475
Annotations: kubernetes.io/limit-ranger: LimitRanger plugin set: cpu request for container order
Status: Running
IP: 192.168.1.3
Controlled By: ReplicaSet/order-7595956475
Containers:
order:
Container ID: docker://404bbb83e469f04925f9dd7a8ffe387ca3c3baa84e6ed428d865ce13aa6ddf71
Image: asia.gcr.io/petcircle-main-cloud/order:8.5.39
Image ID: docker-pullable://asia.gcr.io/petcircle-main-cloud/order@sha256:2ee9fcd11eab2bba260c8e6c1049227097dabff6dfadcd1a0130b0987e69fec1
Port: 80/TCP
Host Port: 0/TCP
State: Running
Started: Thu, 22 Aug 2019 10:35:46 +1000
Ready: True
Restart Count: 0
Requests:
cpu: 100m
Environment: <none>
Mounts:
/var/run/secrets/kubernetes.io/serviceaccount from default-token-4rx9g (ro)
Conditions:
Type Status
Initialized True
Ready True
ContainersReady True
PodScheduled True
Volumes:
default-token-4rx9g:
Type: Secret (a volume populated by a Secret)
SecretName: default-token-4rx9g
Optional: false
QoS Class: Burstable
Node-Selectors: <none>
Tolerations: node.kubernetes.io/not-ready:NoExecute for 300s
node.kubernetes.io/unreachable:NoExecute for 300s
Events: <none>

From the above output note down the below details:

Container ID: 404bbb83e469f04925f9dd7a8ffe387ca3c3baa84e6ed428d865ce13aa6ddf71

ssh this node and run:

>> docker exec -u root -it 404bbb83e469f04925f9dd7a8ffe387ca3c3baa84e6ed428d865ce13aa6ddf71 /bin/bash

And, voila, you are inside the container, as root.

Published Nov 21, 2020

I'm a father, husband, life long learner, maker / hacker, avid reader, traveller, photographer and foodie in this exact order of priority.