Kubernetes Interview Questions and Answers for 2025 (Beginner to Advanced)

Introduction: Why Kubernetes Skills Actually Matter in 2025

Look, I’ve been running production Kubernetes clusters for about six years now, and I can tell you the interview landscape has changed dramatically. Back in 2019, you could get away with knowing what a pod was and maybe spinning up a deployment. These days? Interviewers expect you to debug networking issues, understand security contexts, and explain why your cluster just ate through the cloud budget.

The thing is, Kubernetes isn’t just a buzzword anymore. It’s the backbone of modern infrastructure at companies ranging from startups to Fortune 500s. During interviews, I’ve watched candidates freeze up when asked to troubleshoot a CrashLoopBackOff, even though it’s something you’ll deal with weekly in production.

This guide covers what you’ll actually face in Kubernetes interviews in 2025. I’ve pulled from real questions I’ve asked candidates, stuff I’ve been asked myself, and the kinds of problems that pop up at 2 AM when something breaks. No fluff, no copied documentation, just practical knowledge you can use.

How to Use This Guide

I’ve organized questions by difficulty, but here’s the thing: even “beginner” questions can have deeper answers depending on who’s interviewing you. A senior engineer might ask “What’s a pod?” expecting you to discuss pod lifecycle, init containers, and ephemeral storage. Read the full answer even if the question seems basic.

For each question, I’ve tried to give the answer I’d want to hear if I were conducting the interview, including the gotchas and edge cases that separate okay candidates from great ones.

Beginner Level Questions: Kubernetes Interview Questions and Answers

These cover fundamental concepts. If you’re interviewing for a junior DevOps or platform engineering role, expect these to come up early.

1. What is Kubernetes and why would you use it instead of just Docker?

Kubernetes is a container orchestration platform that manages the lifecycle of containerized applications across a cluster of machines. Docker runs containers, but it doesn’t solve the production problems: what happens when a container dies? How do you scale to 100 instances? How do you route traffic between them?

I learned this the hard way at my first job. We had a Docker Compose setup that worked great on staging, then completely fell apart when we tried to scale it in production. No health checks, no automatic restarts, no load balancing. Kubernetes handles all that out of the box. It’s self-healing, meaning if a container crashes, K8s automatically spins up a replacement. It also handles networking, storage, secrets management, and a bunch of other operational concerns.

You use Kubernetes when you need reliability, scalability, and automation. You don’t use it for a simple blog that runs on a single server, that’s overkill.

2. Explain what a Pod is and why Kubernetes uses Pods instead of containers directly.

A pod is the smallest deployable unit in Kubernetes. It’s a wrapper around one or more containers that share the same network namespace, storage volumes, and lifecycle.

Here’s why pods exist: containers inside a pod can communicate over localhost and share storage volumes. This is huge for patterns like sidecar containers. I’ve used this dozens of times, like when you need a logging agent alongside your main application container, or when you’re running Istio service mesh where it injects an Envoy proxy as a sidecar.

The other reason is atomic deployment. Kubernetes guarantees that all containers in a pod start together on the same node. Either they all run or none do. This prevents the nightmare scenario where half your application stack is running on one server and the other half is somewhere else.

Most of the time, you’ll have one container per pod. But understanding the multi-container use case shows you actually know why pods exist.

3. What’s the difference between a Deployment and a Pod?

A pod is just an instance of your application. A Deployment is the control mechanism that manages those pods.

Think of it this way: you don’t create pods directly in production. You create a Deployment that specifies “I want 3 replicas of this pod running at all times.” The Deployment creates a ReplicaSet (more on that later), which actually creates the pods. If one pod crashes, the Deployment ensures a new one gets created.

Deployments also handle rolling updates. When you change the container image, the Deployment gradually replaces old pods with new ones, ensuring zero downtime. I’ve seen production deployments running 50+ replicas get updated with zero dropped requests because Deployments handle the orchestration properly.

Here’s a basic Deployment YAML:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-app
spec:
  replicas: 3
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx:1.21
        ports:
        - containerPort: 80

4. What are Namespaces and when should you use them?

Namespaces are virtual clusters within your physical cluster. They provide scope for names and allow you to partition resources.

In practice, I use namespaces to separate environments and teams. At my current company, we have namespaces like dev, staging, production, monitoring, and team-specific ones like payments-team and data-pipeline. This prevents conflicts where two teams accidentally name their service the same thing, and it lets you apply different resource quotas and network policies to each namespace.

You can also use namespaces for multi-tenancy, though be careful with that. Namespaces aren’t a security boundary by themselves. You need to combine them with RBAC and Network Policies.

Quick commands:

kubectl create namespace staging
kubectl get pods -n staging
kubectl config set-context --current --namespace=staging

5. How do you check the logs of a running pod?

kubectl logs <pod-name>

If the pod has multiple containers, you need to specify which one:

kubectl logs <pod-name> -c <container-name>

For streaming logs in real-time, add the -f flag (like tail -f):

kubectl logs -f <pod-name>

And here’s a tip that’s saved me countless times: if a pod is crash-looping, you can view logs from the previous instance:

kubectl logs <pod-name> --previous

This is critical for debugging because once a pod crashes, its logs are gone unless you use that --previous flag.

6. What is a Service in Kubernetes?

A Service is a stable network endpoint that routes traffic to a set of pods. Pods are ephemeral and their IP addresses change, but a Service provides a consistent DNS name and IP that doesn’t change.

There are four types of Services:

ClusterIP (default): Only accessible within the cluster. This is what you use for internal microservices communication. Your frontend pod talks to your backend service via something like http://backend-service:8080.

NodePort: Exposes the service on a static port on each node’s IP. Honestly, I rarely use this in production. It’s useful for quick testing but not great for real workloads.

LoadBalancer: Creates an external load balancer (if you’re in a cloud provider that supports it). This gives you a public IP address. Be careful with these, they cost money and each LoadBalancer Service creates a separate cloud load balancer.

ExternalName: Maps the service to an external DNS name. I’ve used this when migrating from legacy infrastructure to Kubernetes, pointing to an external database that wasn’t in the cluster yet.

Example ClusterIP service:

apiVersion: v1
kind: Service
metadata:
  name: backend-service
spec:
  selector:
    app: backend
  ports:
  - protocol: TCP
    port: 80
    targetPort: 8080
  type: ClusterIP

7. What’s the difference between a ReplicaSet and a Deployment?

A ReplicaSet ensures that a specified number of pod replicas are running at any time. A Deployment is a higher-level concept that manages ReplicaSets and provides declarative updates.

In practice, you almost never create a ReplicaSet directly. You create a Deployment, which automatically creates and manages ReplicaSets for you. When you update a Deployment (like changing the container image), it creates a new ReplicaSet with the new configuration and gradually scales it up while scaling down the old ReplicaSet.

This enables rolling updates and rollbacks. If you push a bad deployment, you can roll back:

kubectl rollout undo deployment/my-app

Kubernetes keeps the old ReplicaSets around (by default, the last 10) so you can roll back instantly without rebuilding anything.

8. How do you scale a Deployment?

There are two ways. Imperatively (on the command line):

kubectl scale deployment my-app --replicas=5

Or declaratively by updating the YAML and applying it:

spec:
  replicas: 5

Then run kubectl apply -f deployment.yaml.

The declarative approach is better for production because it’s version-controlled and auditable. But the imperative approach is faster when you’re fighting a production fire and need to scale NOW.

I’ve also set up Horizontal Pod Autoscalers (HPA) that automatically scale based on CPU or memory usage. More on that in the intermediate section.

9. What happens when you run kubectl apply -f deployment.yaml?

This is a deceptively simple question that tests whether you understand the Kubernetes control loop.

When you run that command, kubectl sends the YAML to the API server. The API server validates it, stores it in etcd (the cluster’s database), and returns a response. But nothing has actually happened to your cluster yet.

Now the control loop kicks in. The Deployment controller (running as part of the controller manager) notices there’s a new Deployment object or an update to an existing one. It creates or updates a ReplicaSet. The ReplicaSet controller sees it needs to create some pods. It tells the API server “I need 3 pods with this spec.”

The scheduler watches for unscheduled pods. It picks nodes for each pod based on resource requirements, node selectors, taints, tolerations, and a bunch of other factors. Once scheduled, the kubelet on each node sees it has a new pod assignment. The kubelet tells the container runtime (like containerd or CRI-O) to pull the image and start the container.

This whole process usually takes a few seconds, but understanding that it’s asynchronous and controller-based is important. Kubernetes isn’t executing commands, it’s reconciling desired state with actual state.

10. What is a ConfigMap and when would you use it?

ConfigMaps store non-sensitive configuration data as key-value pairs. You use them to decouple configuration from container images, which means you can use the same image across different environments.

I use ConfigMaps constantly for things like application config files, environment variables, and feature flags. For example, your app might need different database connection strings in dev vs. production. Instead of baking that into the image, you store it in a ConfigMap.

Creating one from a file:

kubectl create configmap app-config --from-file=config.json

Using it in a pod as environment variables:

spec:
  containers:
  - name: app
    image: myapp:latest
    envFrom:
    - configMapRef:
        name: app-config

Or mount it as a volume if you need it as a file inside the container:

volumes:
- name: config-volume
  configMap:
    name: app-config
volumeMounts:
- name: config-volume
  mountPath: /etc/config

One gotcha: ConfigMaps aren’t automatically reloaded when you update them. The pod needs to restart to pick up changes, unless you’ve built your application to watch for file changes.

11. What’s the difference between a ConfigMap and a Secret?

Secrets are like ConfigMaps but for sensitive data. They’re base64 encoded (not encrypted, just encoded) and have additional access controls.

Use Secrets for passwords, API keys, TLS certificates, anything you don’t want in plain text in your Git repo. Although honestly, base64 encoding isn’t real security. Most engineers I know use external secret management tools like Vault, AWS Secrets Manager, or Sealed Secrets for production.

Creating a secret:

kubectl create secret generic db-password --from-literal=password=mysecretpassword

Kubernetes stores secrets in etcd, and they’re transmitted to nodes only when needed. But etcd encryption needs to be explicitly enabled, which a lot of people forget.

12. What is a kubelet and what does it do?

The kubelet is the primary node agent. It runs on every node in the cluster and is responsible for making sure containers are running in pods.

Here’s what it actually does: it watches the API server for pods assigned to its node. When it sees a new pod assignment, it tells the container runtime to pull images and start containers. It continuously monitors the health of containers and reports status back to the API server. If a container fails a health check, the kubelet restarts it.

The kubelet doesn’t manage containers that weren’t created by Kubernetes. If you SSH onto a node and run a container manually with Docker, the kubelet ignores it.

I’ve debugged kubelet issues by checking its logs:

journalctl -u kubelet -f

Usually when I see kubelet problems, it’s because the node ran out of disk space or memory, and the kubelet can’t pull images or start new pods.

13. What are Labels and Selectors?

Labels are key-value pairs attached to objects like pods. Selectors use labels to identify groups of objects.

This is fundamental to how Kubernetes works. Services use selectors to find pods. Deployments use selectors to manage pods. You use labels for organization, filtering, and grouping.

Example labels:

metadata:
  labels:
    app: web
    environment: production
    version: v2.1

Then you can query pods with specific labels:

kubectl get pods -l app=web
kubectl get pods -l environment=production,version=v2.1

Most engineers don’t put enough thought into their labeling strategy, then regret it later when they can’t find anything. I recommend establishing conventions early: what labels are required, what the allowed values are, stuff like that.

Intermediate Level Questions: Kubernetes Interview Questions and Answers

These questions test whether you’ve actually run Kubernetes in production and dealt with real operational challenges.

14. Explain how Kubernetes networking works at a high level.

Kubernetes has a flat network model. Every pod gets its own IP address, and pods can communicate with each other directly without NAT, regardless of which node they’re on. This is different from Docker’s default bridge networking.

Here’s how it works in practice: when a pod tries to reach another pod, the network plugin (like Calico, Cilium, or Flannel) handles the routing. If the destination pod is on the same node, traffic goes through the local network bridge. If it’s on a different node, traffic goes over the network overlay.

Services add another layer. When a pod connects to a service by its ClusterIP, kube-proxy (or eBPF if you’re using something modern like Cilium) rewrites the destination to one of the backend pod IPs using load balancing.

There’s also the concept of Network Policies, which are essentially firewalls for your pods. By default, all pods can talk to all other pods, which is convenient but terrible for security.

Diagram Description: Picture showing three nodes, each with pods. Arrows showing pod-to-pod communication within a node and across nodes via the overlay network. Another diagram showing a Service with a ClusterIP load balancing requests to multiple backend pods.

15. What is an Ingress and how does it differ from a Service?

An Ingress is an API object that manages external HTTP/HTTPS access to services. While a LoadBalancer Service creates a separate load balancer for each service, an Ingress lets you define rules for routing different URLs to different services using a single entry point.

Here’s the practical difference: say you have three services (web, api, admin). With LoadBalancer Services, you’d get three different IP addresses and pay for three load balancers. With an Ingress, you have one load balancer that routes traffic based on paths or hostnames.

Example Ingress:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: my-ingress
spec:
  rules:
  - host: myapp.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: web-service
            port:
              number: 80
      - path: /api
        pathType: Prefix
        backend:
          service:
            name: api-service
            port:
              number: 8080

Important: Ingress is just the API definition. You need an Ingress Controller (like nginx-ingress, Traefik, or HAProxy) to actually implement the routing. I’ve seen candidates get confused about this, thinking Ingress alone does something. It doesn’t. The controller does the work.

16. What is a StatefulSet and when would you use it?

StatefulSets are for stateful applications that need stable network identities and persistent storage. Unlike Deployments where pods are interchangeable, StatefulSet pods have sticky identities.

Use cases: databases (PostgreSQL, MySQL), distributed systems (Kafka, Zookeeper, Elasticsearch), anything that needs to maintain state and identity across restarts.

Key differences from Deployments:

• Pods are created and terminated in order (pod-0, then pod-1, then pod-2)
• Each pod gets a stable hostname like myapp-0.myapp-service.default.svc.cluster.local
• Pods maintain the same PersistentVolume across restarts
• Scaling down doesn’t immediately delete PersistentVolumeClaims

I’ve run PostgreSQL clusters on StatefulSets. It works, but honestly, for databases, I prefer using managed services when possible. Running stateful workloads on Kubernetes is doable but adds complexity.

17. What are DaemonSets and give me a real example of when you’d use one?

A DaemonSet ensures that a copy of a pod runs on every node (or on a subset of nodes matching a selector). When you add a node to the cluster, the DaemonSet automatically schedules its pod there.

Real-world examples:

• Log collectors (Fluentd, Filebeat) that need to collect logs from every node
• Monitoring agents (Node Exporter, Datadog agent) that gather metrics from each node
• Network plugins (kube-proxy, Cilium agents) that need to run on every node for networking to work
• Security tools that scan each node for vulnerabilities

I recently set up a DaemonSet for a security scanning tool that needed to inspect running containers on every node. Without DaemonSets, I’d have to manually ensure the agent runs on every node, and it wouldn’t automatically deploy to new nodes when we scaled.

18. Explain Kubernetes resource requests and limits.

Requests are the guaranteed resources a container will get. Limits are the maximum resources a container can use.

This is critical for cluster stability and I’ve seen many production issues from misconfigured resource settings.

When you specify a request, the scheduler only places the pod on a node that has enough available resources. The kubelet guarantees the container gets at least that much CPU and memory. Limits prevent a single container from consuming all resources on a node.

Example:

resources:
  requests:
    memory: "256Mi"
    cpu: "250m"
  limits:
    memory: "512Mi"
    cpu: "500m"

Here’s what happens when you hit limits:

• CPU: The container gets throttled. It doesn’t crash, just runs slower.
• Memory: The container gets OOMKilled (Out Of Memory Killed) and restarted.

Most engineers set limits too low or forget to set requests at all. Then they wonder why their pods keep getting OOMKilled or why scheduling fails. My rule of thumb: set requests based on actual usage (use metrics), set limits 50-100% higher than requests to allow for spikes.

19. What is a PersistentVolume and how does it relate to PersistentVolumeClaim?

PersistentVolumes (PV) are storage resources in the cluster. PersistentVolumeClaims (PVC) are requests for storage by users.

The abstraction works like this: cluster admins create PVs that represent actual storage (EBS volumes in AWS, NFS shares, local disks, whatever). Developers create PVCs that say “I need 10GB of storage with ReadWriteOnce access.” Kubernetes matches PVCs to PVs that satisfy the requirements.

This separation means developers don’t need to know the underlying storage implementation. They just request storage and use it.

Example PVC:

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: app-storage
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 10Gi
  storageClassName: fast-ssd

Then mount it in a pod:

volumes:
- name: storage
  persistentVolumeClaim:
    claimName: app-storage
volumeMounts:
- name: storage
  mountPath: /data

Access modes matter: ReadWriteOnce (RWO) means only one node can mount it read-write. ReadWriteMany (RWX) means multiple nodes can mount it simultaneously, but not all storage backends support this.

20. What is a StorageClass and why is it useful?

StorageClasses enable dynamic provisioning of storage. Instead of pre-creating PersistentVolumes, you define a StorageClass that describes how to dynamically create storage when needed.

When a PVC references a StorageClass, Kubernetes automatically provisions the underlying storage. In AWS, this might create an EBS volume. In GCP, a persistent disk. On-premises, it might use Ceph or another storage system.

Example StorageClass:

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: fast-ssd
provisioner: kubernetes.io/aws-ebs
parameters:
  type: gp3
  iopsPerGB: "50"

This is way better than manually creating volumes. I’ve worked at places that had hundreds of statically provisioned volumes, and it was a nightmare to manage. Dynamic provisioning with StorageClasses scales much better.

21. How does the Kubernetes scheduler decide which node to place a pod on?

The scheduler goes through a two-phase process: filtering and scoring.

Filtering eliminates nodes that can’t run the pod. It checks:

• Does the node have enough CPU and memory for the pod’s requests?
• Do node selectors or affinity rules match?
• Are there taints on the node that the pod doesn’t tolerate?
• Is the required port available?
• Is the required volume available if it’s node-local?

Scoring ranks the remaining nodes. It considers:

• Resource utilization (tries to spread pods evenly)
• Affinity preferences (place related pods together or apart)
• Image locality (prefer nodes that already have the image)
• Various scoring plugins you can configure

The pod goes to the highest-scoring node. If multiple nodes tie, it picks one randomly.

I’ve tuned scheduler behavior for specific workloads using node affinity and pod anti-affinity. For example, spreading database replicas across availability zones to handle zone failures.

22. What are taints and tolerations?

Taints allow a node to repel pods. Tolerations allow pods to be scheduled on tainted nodes.

Think of taints as “this node is special” and tolerations as “I can handle that special node.” This is how you dedicate nodes to specific workloads.

Real example: you might taint GPU nodes so only pods that explicitly need GPUs land there:

kubectl taint nodes gpu-node-1 gpu=true:NoSchedule

Then your GPU workload needs a toleration:

tolerations:
- key: "gpu"
  operator: "Equal"
  value: "true"
  effect: "NoSchedule"

There are three taint effects:

• NoSchedule: Pods won’t be scheduled unless they tolerate the taint
• PreferNoSchedule: Scheduler tries to avoid the node but isn’t required to
• NoExecute: Evicts existing pods that don’t tolerate the taint

I’ve used taints to drain nodes for maintenance. Taint them with NoExecute, watch the pods migrate, then upgrade the node.

23. Explain Horizontal Pod Autoscaling (HPA).

HPA automatically scales the number of pod replicas based on observed metrics. Most commonly, it scales based on CPU utilization, but you can use custom metrics too.

You define target metrics, and HPA adjusts replicas to try to maintain those targets. If average CPU across all pods exceeds 70%, HPA scales up. If it drops to 30%, HPA scales down.

Example HPA:

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: web-app-hpa
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: web-app
  minReplicas: 2
  maxReplicas: 10
  metrics:
  - type: Resource
    resource:
      name: cpu
      target:
        type: Utilization
        averageUtilization: 70

Or with kubectl:

kubectl autoscale deployment web-app --cpu-percent=70 --min=2 --max=10

Important: HPA requires the metrics server to be installed. I’ve debugged HPAs that weren’t working because metrics-server wasn’t running or was misconfigured.

Also, scaling decisions happen every 15 seconds by default, but actual scaling respects cooldown periods to prevent flapping.

24. What are liveness probes, readiness probes, and startup probes?

These are health checks that tell Kubernetes about your container’s status.

Liveness Probe: Checks if the container is still alive. If it fails, Kubernetes kills and restarts the container. Use this to catch deadlocks or situations where the process is running but unresponsive.

Readiness Probe: Checks if the container is ready to serve traffic. If it fails, the pod gets removed from Service endpoints, so it stops receiving traffic, but the container isn’t restarted. Use this when your app needs time to warm up or when it’s temporarily overloaded.

Startup Probe: Checks if the application has started. It disables liveness and readiness probes until it succeeds. Use this for slow-starting containers to prevent them from getting killed during initialization.

Example probes:

livenessProbe:
  httpGet:
    path: /healthz
    port: 8080
  initialDelaySeconds: 30
  periodSeconds: 10
  failureThreshold: 3

readinessProbe:
  httpGet:
    path: /ready
    port: 8080
  initialDelaySeconds: 5
  periodSeconds: 5
  failureThreshold: 3

I can’t stress enough how important proper probe configuration is. I’ve seen so many CrashLoopBackOff issues caused by liveness probes that were too aggressive or readiness probes that checked the wrong endpoint.

25. What is RBAC in Kubernetes?

Role-Based Access Control (RBAC) determines what users and service accounts can do in the cluster. It’s how you implement the principle of least privilege.

RBAC has four main objects:

Role/ClusterRole: Defines permissions (what actions on what resources). A Role is namespaced, a ClusterRole is cluster-wide.

RoleBinding/ClusterRoleBinding: Grants permissions to users or service accounts.

Example Role that allows reading pods:

apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  namespace: default
  name: pod-reader
rules:
- apiGroups: [""]
  resources: ["pods"]
  verbs: ["get", "list", "watch"]

Then bind it to a user:

apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: read-pods
  namespace: default
subjects:
- kind: User
  name: jane
  apiGroup: rbac.authorization.k8s.io
roleRef:
  kind: Role
  name: pod-reader
  apiGroup: rbac.authorization.k8s.io

In production, I give developers read-only access to production namespaces and full access to dev namespaces. Admins get cluster-admin ClusterRole, but we’re careful about who gets that.

You can check permissions with:

kubectl auth can-i delete pods --namespace=production

26. What are Init Containers and give a use case?

Init containers run before app containers in a pod. They run to completion sequentially. If an init container fails, Kubernetes restarts the pod until it succeeds.

Use cases:

• Waiting for dependencies (database, cache) to be ready before starting your app
• Fetching configuration from external systems
• Running setup scripts or database migrations
• Setting up volume permissions

Example where init container waits for a service:

initContainers:
- name: wait-for-db
  image: busybox:1.28
  command: ['sh', '-c', 'until nslookup db-service; do echo waiting for db; sleep 2; done']

I’ve used init containers to run database migrations before the application starts. This ensures the schema is up to date before any app pods start serving traffic, which prevents race conditions during deployments.

27. How do you debug a pod that’s in CrashLoopBackOff status?

CrashLoopBackOff means the pod keeps starting and crashing, and Kubernetes is backing off on restart attempts.

My debugging process:

1. Check pod status and events:

kubectl describe pod <pod-name>

Look at the Events section at the bottom. This often tells you immediately what’s wrong (missing ConfigMap, image pull failure, etc.).

2. Check the logs:

kubectl logs <pod-name> --previous

The --previous flag shows logs from the crashed container. This is critical.

3. Check resource limits. Is the pod getting OOMKilled?
4. Check liveness probes. Are they too aggressive?
5. If you need interactive debugging, you can run a similar container and exec into it:

kubectl run -it debug --image=myapp:latest --restart=Never -- /bin/sh

Common causes I’ve seen:

• Application crashes due to missing environment variables or ConfigMaps
• OOMKills because memory limits are too low
• Liveness probe fails immediately because initialDelaySeconds is too short
• The entrypoint command in the Dockerfile is wrong

28. What’s the difference between kubectl apply and kubectl create?

kubectl create is imperative. It creates a new resource. If the resource already exists, it fails with an error.

kubectl apply is declarative. It creates the resource if it doesn’t exist, or updates it if it does. It tracks changes by storing the last-applied configuration in an annotation.

In production, always use apply. It’s idempotent and works with GitOps workflows. You can run the same manifest multiple times without errors.

Behind the scenes, apply uses a three-way merge: it compares your new manifest with the last-applied configuration and the current cluster state to determine what needs to change. This is why you can use apply on resources created by other means.

29. Explain what Jobs and CronJobs are.

Jobs run pods to completion. Unlike Deployments that keep pods running indefinitely, a Job runs a task, waits for it to finish successfully, and then stops.

Use Jobs for:

• Batch processing
• Database backups
• One-time data migrations
• Running tests

Example Job:

apiVersion: batch/v1
kind: Job
metadata:
  name: data-migration
spec:
  template:
    spec:
      containers:
      - name: migrate
        image: migration-tool:latest
        command: ["python", "migrate.py"]
      restartPolicy: Never
  backoffLimit: 3

CronJobs are Jobs that run on a schedule, like cron. I use these all the time for recurring tasks:

apiVersion: batch/v1
kind: CronJob
metadata:
  name: backup
spec:
  schedule: "0 2 * * *"  # 2 AM daily
  jobTemplate:
    spec:
      template:
        spec:
          containers:
          - name: backup
            image: backup-tool:latest
            command: ["./backup.sh"]
          restartPolicy: OnFailure

Important: CronJobs use UTC timezone by default. I’ve had to explain this to confused developers multiple times when their jobs ran at unexpected times.

Advanced Level Questions: Kubernetes Interview Questions and Answers

These separate senior engineers from intermediate ones. You should have production battle scars to answer these well.

30. Explain how the Kubernetes API server works internally.

The API server is the front door to Kubernetes. Every operation goes through it. It’s a REST API that handles authentication, authorization, admission control, and writes to etcd.

Here’s the request flow:

1. Authentication: Who are you? (TLS certs, service account tokens, OIDC, etc.)
2. Authorization: What are you allowed to do? (RBAC, ABAC, Node authorization, Webhook authorization)
3. Admission Control: Is this request valid and should it be modified? (MutatingWebhooks run first, then ValidatingWebhooks)
4. Validation: Does the object match the API schema?
5. Persistence: Write to etcd
6. Response: Return the result to the client

The API server is stateless. All state lives in etcd. This is why you can run multiple API server replicas for high availability.

An important detail: the API server doesn’t actually do anything to your cluster. It just validates and stores objects. Controllers watch for changes and do the actual work. This decoupling is fundamental to Kubernetes’ design.

31. What are Admission Controllers and why are they important?

Admission controllers intercept requests to the API server before objects are persisted to etcd. They can mutate or validate objects, or reject requests entirely.

There are two types:

Mutating Admission Controllers modify objects. Examples:

• Setting default values for fields not specified
• Injecting sidecar containers (like Istio does)
• Adding labels or annotations

Validating Admission Controllers reject invalid requests. Examples:

• Enforcing naming conventions
• Requiring resource limits
• Preventing privileged containers in certain namespaces

Built-in admission controllers include NamespaceLifecycle, LimitRanger, ResourceQuota, PodSecurityPolicy (deprecated), and more. You enable them with flags on the API server.

You can also write custom admission controllers using MutatingWebhookConfiguration and ValidatingWebhookConfiguration. I’ve written webhook admission controllers to enforce company policies, like requiring all images to come from our private registry or preventing pods from running as root.

32. What is a Custom Resource Definition (CRD) and when would you create one?

CRDs extend the Kubernetes API with your own resource types. You define a new kind of object, and Kubernetes treats it just like built-in resources.

When to use CRDs:

• When you’re building an operator to manage complex applications
• When you need domain-specific abstractions on top of Kubernetes primitives
• When you want to use kubectl and RBAC with your custom resources

Example CRD for a custom database type:

apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
  name: databases.example.com
spec:
  group: example.com
  versions:
  - name: v1
    served: true
    storage: true
    schema:
      openAPIV3Schema:
        type: object
        properties:
          spec:
            type: object
            properties:
              version:
                type: string
              replicas:
                type: integer
  scope: Namespaced
  names:
    plural: databases
    singular: database
    kind: Database

After creating this CRD, you can use kubectl get databases and create database objects. But the CRD alone does nothing, you need a controller (an operator) to watch for these objects and take action.

I’ve used CRDs at scale when building internal platforms. They let developers think in terms of “I need a database” rather than manually configuring StatefulSets, Services, PVCs, and backups.

33. What is an Operator pattern?

An Operator is a Kubernetes controller that uses CRDs to manage applications and their components. It encodes operational knowledge into software.

The pattern: you define the desired state in a custom resource, and the operator continuously reconciles actual state to match. It’s like having a DevOps engineer in a loop watching your application and fixing problems automatically.

Popular examples:

• Prometheus Operator: Manages Prometheus instances
• etcd Operator: Manages etcd clusters
• PostgreSQL Operator: Handles backups, failovers, upgrades

I’ve written operators using Kubebuilder and the Operator SDK. The typical operator:

1. Watches for custom resources
2. Creates/updates underlying Kubernetes resources (Deployments, Services, etc.)
3. Monitors application health
4. Performs automatic remediation (backup, failover, scaling)
5. Handles upgrades

The reconciliation loop is key. If you manually delete a pod that the operator manages, it recreates it. This is way more powerful than Helm charts or static manifests.

34. How does pod scheduling work with node affinity and pod affinity/anti-affinity?

These are constraints that influence or require where pods get scheduled.

Node Affinity is like a more flexible nodeSelector. It lets you say “prefer these nodes” or “require these nodes” based on labels.

affinity:
  nodeAffinity:
    requiredDuringSchedulingIgnoredDuringExecution:
      nodeSelectorTerms:
      - matchExpressions:
        - key: instance-type
          operator: In
          values:
          - c5.xlarge
          - c5.2xlarge

Pod Affinity schedules pods near other pods. Use it to co-locate tightly coupled services on the same node for lower latency.

Pod Anti-Affinity schedules pods away from other pods. Use it to spread replicas across nodes or availability zones for high availability.

Example anti-affinity to spread database replicas:

affinity:
  podAntiAffinity:
    requiredDuringSchedulingIgnoredDuringExecution:
    - labelSelector:
        matchExpressions:
        - key: app
          operator: In
          values:
          - database
      topologyKey: kubernetes.io/hostname

This says “don’t schedule this pod on a node that already has a pod with label app=database.” The topologyKey defines what “different” means (different hostname = different node).

I use anti-affinity extensively for production databases and critical services. You don’t want all three Postgres replicas on the same node.

35. What are Network Policies and how do you implement zero-trust networking?

Network Policies are firewall rules for pods. By default, all pods can communicate with all other pods. Network Policies restrict this.

A Network Policy is a whitelist. Once you apply any Network Policy to a pod, all traffic not explicitly allowed is denied.

Example: only allow traffic to a database from the application tier:

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: database-policy
  namespace: production
spec:
  podSelector:
    matchLabels:
      app: database
  policyTypes:
  - Ingress
  ingress:
  - from:
    - podSelector:
        matchLabels:
          app: web-app
    ports:
    - protocol: TCP
      port: 5432

This says “for pods labeled app=database, only allow ingress from pods labeled app=web-app on port 5432.”

For zero-trust networking:

1. Apply a default deny-all policy to every namespace
2. Explicitly whitelist required communication paths
3. Use namespaceSelectors to allow cross-namespace traffic only where needed
4. Consider using a service mesh (Istio, Linkerd) for mTLS and finer-grained control

Important: Network Policies require a network plugin that supports them (Calico, Cilium, Weave). Standard kubenet doesn’t enforce them.

36. Explain Pod Security Standards and how they replace PodSecurityPolicies.

Pod Security Standards are a newer way to enforce pod security configurations. They replaced PodSecurityPolicies, which were deprecated and removed in Kubernetes 1.25.

There are three levels:

Privileged: Unrestricted. No restrictions at all.

Baseline: Minimally restrictive. Prevents known privilege escalations but allows most workloads.

Restricted: Heavily restricted. Follows current pod hardening best practices.

You enforce these at the namespace level using labels:

apiVersion: v1
kind: Namespace
metadata:
  name: production
  labels:
    pod-security.kubernetes.io/enforce: restricted
    pod-security.kubernetes.io/audit: restricted
    pod-security.kubernetes.io/warn: restricted

This namespace now rejects pods that don’t meet the restricted standard. Common restrictions include:

• No privileged containers
• No hostPath volumes
• No host networking or host PID
• Must run as non-root user
• Must drop all capabilities and can’t add dangerous ones
• Read-only root filesystem encouraged

I’ve had to update a lot of deployments to comply with restricted standards. Usually it means adding:

securityContext:
  runAsNonRoot: true
  runAsUser: 1000
  allowPrivilegeEscalation: false
  capabilities:
    drop:
      - ALL
  seccompProfile:
    type: RuntimeDefault

37. How do you secure secrets in Kubernetes?

Out of the box, Kubernetes secrets are not very secure. They’re base64 encoded (not encrypted) and stored in etcd. If someone gets etcd access, they have all your secrets.

Security measures I implement:

1. Enable etcd encryption at rest: Configure the API server with an EncryptionConfiguration. This encrypts secret data before writing to etcd.

2. Use external secret management: Tools like HashiCorp Vault, AWS Secrets Manager, Azure Key Vault, or Google Secret Manager. Use an operator like External Secrets Operator or Sealed Secrets to sync external secrets into Kubernetes.

3. RBAC: Restrict who can read secrets. Most users shouldn’t be able to kubectl get secrets.

4. Audit logging: Enable audit logs to track who accessed secrets.

5. Use service accounts carefully: Every pod has a service account token automatically mounted. If you don’t need it, disable it:

automountServiceAccountToken: false

At my current job, we use Vault with the Vault Agent Injector. Secrets get injected into pods at runtime and never stored in Kubernetes. It’s more complex to set up but way more secure.

38. What is a ServiceMesh and what problems does it solve?

A service mesh is an infrastructure layer that handles service-to-service communication. Popular options are Istio, Linkerd, and Consul Connect.

Without a service mesh, you implement cross-cutting concerns (mTLS, retries, circuit breaking, observability) in every microservice. With a service mesh, you get these features transparently.

Key features:

mTLS: Automatic encryption of all service-to-service traffic with certificate rotation

Traffic management: Advanced routing, canary deployments, traffic splitting, fault injection

Observability: Automatic metrics, distributed tracing, and logging for all service calls

Resilience: Retries, timeouts, circuit breakers without changing application code

How it works: The mesh injects a sidecar proxy (usually Envoy) into every pod. All traffic goes through the proxy, which handles the service mesh features.

I’ve used Istio in production. It’s powerful but complex. You get amazing observability and security, but you also get another layer to debug when things break. Not every team needs a service mesh. If you have a dozen microservices, maybe not. If you have hundreds, definitely consider it.

39. How do you handle zero-downtime deployments?

Several strategies depending on requirements:

Rolling Updates (default with Deployments): Gradually replace old pods with new ones. Configure maxUnavailable and maxSurge:

strategy:
  type: RollingUpdate
  rollingUpdate:
    maxUnavailable: 0
    maxSurge: 1

This creates one new pod before terminating an old pod, ensuring capacity never drops.

Blue/Green Deployment: Run both versions simultaneously, then switch traffic instantly by changing the Service selector. This requires double the resources but allows instant rollback.

Canary Deployment: Route a small percentage of traffic to the new version. If metrics look good, gradually increase traffic. Requires advanced traffic management (Ingress controller or service mesh).

Critical considerations:

1. Readiness probes: New pods must pass readiness before receiving traffic
2. PreStop hooks: Give containers time to finish in-flight requests:

lifecycle:
  preStop:
    exec:
      command: ["/bin/sh", "-c", "sleep 15"]

3. PodDisruptionBudgets: Ensure minimum availability during updates:

apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
  name: web-app-pdb
spec:
  minAvailable: 2
  selector:
    matchLabels:
      app: web-app

I’ve handled deployments for high-traffic services. The key is testing deployments in staging first and having a clear rollback plan. Monitoring during the rollout is critical.

40. Explain the concept of GitOps and how it works with Kubernetes.

GitOps uses Git as the single source of truth for declarative infrastructure and applications. You commit changes to Git, and automation tools sync those changes to your cluster.

Core principles:

1. Everything is declared in Git
2. Git is the source of truth
3. Changes happen via pull requests with review
4. Automated agents ensure cluster state matches Git state

Popular GitOps tools:

Flux CD: Watches Git repos and applies changes. It can also watch container registries and auto-update image tags.

Argo CD: Provides a UI and more opinionated workflows. It continuously monitors cluster state and auto-syncs or alerts on drift.

How I use it:

All our Kubernetes manifests live in Git. When a developer wants to deploy a change, they update the manifest and open a PR. After code review, the PR merges, and Argo CD automatically applies the change to the cluster within minutes.

Benefits:

• Audit trail: Git history shows who changed what and when
• Rollbacks: Just revert the Git commit
• Disaster recovery: Spin up a new cluster and point Argo CD at your Git repo
• Declarative: No kubectl commands, no drift

The tricky part is secrets. You can’t store them plain in Git. Solutions include Sealed Secrets, Mozilla SOPS, or external secret management.

GitOps has transformed how we operate. Before, engineers would kubectl apply stuff manually, and nobody knew what was actually running. Now, if it’s not in Git, it doesn’t exist.

41. What happens during a node failure?

When a node becomes unhealthy or unreachable, here’s the sequence:

1. Node Controller detects failure: By default, it marks the node as NotReady after 40 seconds of not receiving heartbeats
2. Pod eviction: After 5 minutes (default pod-eviction-timeout), the Node Controller marks pods on that node for deletion
3. Scheduler reschedules pods: Pods from Deployments, StatefulSets, etc. get scheduled to healthy nodes
4. PersistentVolumes: This is where it gets interesting. If you’re using cloud PVs with ReadWriteOnce access, the volume is attached to the failed node. Kubernetes can’t detach it immediately because it doesn’t know if the node is truly dead or just network partitioned. It waits (6 minutes by default) before force-detaching and attaching to a new node.

I’ve dealt with this in production. We had an EC2 instance crash, and it took 11 minutes total before the pod came back up on a new node with its volume. This is why high-availability architectures use multiple replicas and avoid single points of failure.

For stateless apps with multiple replicas, node failures are transparent. For stateful apps, you need to understand these timings and potentially tune them for your requirements.

42. How does the etcd cluster work and why is it critical?

etcd is a distributed key-value store that holds all Kubernetes cluster data. Everything, every pod, service, secret, ConfigMap, lives in etcd. No etcd means no cluster state.

It uses the Raft consensus algorithm to maintain consistency across multiple etcd instances (you should run at least 3, ideally 5 for production). Raft ensures all instances agree on the current state even in the face of network partitions or node failures.

Why it’s critical:

• The API server reads from and writes to etcd for every operation
• If etcd is unavailable, the API server can’t function
• If you lose etcd data without backups, you lose your entire cluster configuration

Operations I care about:

Backups: I snapshot etcd regularly. Without backups, a catastrophic etcd failure means rebuilding the entire cluster from scratch.

ETCDCTL_API=3 etcdctl snapshot save /backup/snapshot.db

Performance: etcd is sensitive to disk latency. Use SSDs and monitor disk latency. Slow disks cause cascading performance problems in the entire cluster.

Sizing: etcd has a default 2GB storage limit. For large clusters, you need to increase it and monitor usage. If etcd fills up, the cluster stops accepting new objects.

I treat etcd like a database because that’s what it is. Regular backups, monitoring, proper hardware, and never mess with it directly unless you know exactly what you’re doing.

43. What are the different types of volume plugins in Kubernetes?

Kubernetes supports many volume types, but they fall into categories:

Cloud Provider Volumes:

• awsElasticBlockStore (EBS)
• gcePersistentDisk
• azureDisk
• These directly integrate with cloud provider storage APIs

Network Storage:

• nfs: Network File System, useful for ReadWriteMany
• cephfs, glusterfs: Distributed file systems
• iscsi: Block-level storage over networks

Local Storage:

• hostPath: Mounts a file or directory from the node (dangerous, avoid in production)
• local: Similar to hostPath but with proper scheduling (pod must stay on the same node)

Special Types:

• configMap, secret: Inject configuration
• emptyDir: Temporary storage that lives with the pod
• projected: Combine multiple volume sources

CSI (Container Storage Interface): Modern storage plugins use CSI, which is a standard interface. This replaced the old in-tree volume plugins. CSI drivers are external and can be updated independently of Kubernetes.

In practice, I use cloud provider volumes via StorageClasses for most persistent storage, emptyDir for temporary scratch space, and ConfigMaps/Secrets for configuration. I avoid hostPath because it couples pods to specific nodes and creates security risks.

44. How would you implement multi-tenancy in Kubernetes?

True multi-tenancy, where you don’t trust tenants, is hard in Kubernetes. The typical approaches:

Namespace-based soft multi-tenancy:

• Each tenant gets namespaces
• Use ResourceQuotas to limit resource usage per namespace
• Use NetworkPolicies to isolate network traffic
• Use RBAC so tenants can’t see each other’s resources
• Use PodSecurityStandards to enforce security baselines

This works when tenants trust each other somewhat, like different teams in a company.

Cluster-per-tenant: Each tenant gets their own cluster. More expensive but stronger isolation. This is what I recommend for true multi-tenancy where tenants don’t trust each other.

Virtual clusters: Tools like vcluster or Kamaji create virtual Kubernetes clusters inside a physical cluster. Each virtual cluster has its own API server but shares the underlying nodes. This provides better isolation than namespaces but is cheaper than full clusters.

What doesn’t work: Just using namespaces without NetworkPolicies and RBAC. I’ve seen setups where “multi-tenancy” meant different namespaces, but tenants could still see and access each other’s stuff.

Key point: Namespaces are not a security boundary. You need layered security: NetworkPolicies, RBAC, PodSecurityStandards, and potentially separate node pools.

45. What is the difference between ClusterIP, NodePort, and LoadBalancer services in detail?

Let me get into the actual implementation:

ClusterIP: Creates a virtual IP address that’s only reachable from inside the cluster. kube-proxy (or eBPF with Cilium) sets up iptables or IPVS rules on every node to intercept traffic to the ClusterIP and load-balance it to backend pod IPs. When a pod sends traffic to the ClusterIP, the node’s networking intercepts it and rewrites the destination to a pod IP.

NodePort: Builds on ClusterIP. In addition to the ClusterIP, it opens a static port (30000-32767 range by default) on every node. Traffic to <NodeIP>:<NodePort> gets proxied to the ClusterIP, which then load-balances to pods. You can hit any node on that port and reach the service. This is useful for development or when you have your own external load balancer that needs a static endpoint.

LoadBalancer: Builds on NodePort. In addition to the ClusterIP and NodePort, it calls out to the cloud provider’s API to provision an external load balancer (ELB in AWS, Cloud Load Balancer in GCP, etc.). The cloud load balancer points to all nodes on the NodePort. Traffic flows: external IP → cloud LB → NodePort on nodes → ClusterIP → pods.

Each type is a superset of the previous one. LoadBalancer includes NodePort functionality, which includes ClusterIP.

Cost consideration: Every LoadBalancer Service creates a separate cloud load balancer, which costs money. If you have 20 services exposed externally, that’s 20 load balancers. This is why Ingress exists: one load balancer can handle many services.

Senior-Level Scenario Questions

These are open-ended challenges that test your ability to design solutions and troubleshoot complex production issues. There’s rarely one right answer, interviewers want to see your thought process.

Scenario 1: The Mysterious Memory Leak

Situation: Your production cluster has been running fine for months. Suddenly, pods are getting OOMKilled randomly across different applications. Memory usage keeps climbing over days until nodes run out of memory. The monitoring shows normal application memory usage. What do you investigate?

My approach:

First, I’d check if it’s actually an application memory leak or something else. I’ve seen this exact scenario before, and it wasn’t the applications at all.

Things to investigate in order:

Check the kernel memory usage on nodes. Sometimes kernel memory leaks happen, especially with older kernels or buggy network drivers. I’d SSH to a node and check slabtop and /proc/meminfo. Look for unusually high kernel slab usage.

Look at the CNI plugin. I’ve seen Calico and Cilium have memory leaks in certain versions. Check the CNI pod memory usage over time. If those pods are consuming gigabytes, that’s your culprit.

Examine zombie processes and orphaned containers. Sometimes the container runtime doesn’t properly clean up stopped containers. Run docker ps -a | wc -l on nodes and see if you have thousands of stopped containers consuming memory.

Check if you’re actually setting memory limits on all pods. If some pods don’t have limits, they can consume all available memory. Run a quick audit: kubectl get pods --all-namespaces -o json | jq '.items[].spec.containers[].resources.limits.memory' to find pods without limits.

Look at sidecars and DaemonSets. Logging or monitoring agents running on every node can slowly leak memory. If you have 50 nodes and each logging agent leaks 10MB per day, that adds up.

The last time I dealt with this, it was Calico’s Felix component leaking memory due to a bug in the version we were running. We upgraded and the problem disappeared. The key was checking the infrastructure components, not just application pods.

Scenario 2: Cross-Region Disaster Recovery

Situation: You’re running Kubernetes in AWS us-east-1. Management wants a disaster recovery plan that allows you to fail over to us-west-2 within 30 minutes if the entire region becomes unavailable. How do you architect this?

My solution:

True cross-region DR for Kubernetes is complex, and 30 minutes is aggressive. Here’s what I’d build:

Infrastructure as Code for the entire cluster. Use Terraform or similar to define the cluster in both regions. This ensures you can spin up an identical cluster quickly. Keep this code updated as you make changes.

GitOps for all applications. Everything deployed via Argo CD or Flux, pulling from Git repos. In a disaster, point the new cluster’s GitOps tool at the same repos, and it reconstructs all applications.

Database strategy is the hard part. For stateful data, you need cross-region replication. Options include AWS RDS with read replicas in the DR region (can be promoted during failover), or running your own database with replication. Document the promotion process thoroughly because you’ll be stressed when executing it.

Persistent volumes are tricky. EBS volumes are regional. For critical data, I’d use solutions that replicate across regions like EFS (if acceptable for your performance needs) or third-party storage solutions with cross-region replication. Alternatively, continuous backups to S3 with versioning enabled.

Secrets and configuration. Use AWS Secrets Manager or Parameter Store and replicate to both regions. Don’t rely solely on Kubernetes secrets because those don’t replicate.

DNS and traffic management. Use Route53 with health checks. In normal operation, traffic goes to us-east-1. During failover, update Route53 to point to us-west-2. Use short TTLs on these records.

Regular DR drills. I schedule quarterly failover tests where we actually switch to the DR region for a day. This reveals problems before the real emergency happens. The first test always finds issues you didn’t think about.

The 30-minute requirement is tight. In reality, the technical failover might take 10-15 minutes, but validating everything works correctly and making the traffic switch takes longer. I’d set stakeholder expectations that 30 minutes is the goal but 45 minutes is more realistic, especially if databases need manual intervention.

Scenario 3: The Noisy Neighbor Problem

Situation: You’re running a shared Kubernetes cluster with multiple teams. One team’s application suddenly causes CPU throttling for other teams’ applications. The cluster has CPU available, but pods are getting throttled. What’s happening and how do you fix it?

Analysis:

This is a classic resource contention issue, and I’ve debugged this several times. Here’s what’s likely happening:

The problem is probably with CPU limits. When a container hits its CPU limit, it gets throttled even if the node has spare CPU capacity. This is by design in the Linux kernel’s CFS (Completely Fair Scheduler).

To investigate, I’d check the metrics for CPU throttling:

kubectl top pods -A --sort-by=cpu

Then look at the actual throttling metrics if you have a metrics system:

container_cpu_cfs_throttled_seconds_total

What probably happened: One team deployed an application without properly tuning resource requests and limits. They either set the limit too low (causing throttling) or set requests very high (preventing other pods from being scheduled).

Solutions depend on the root cause:

If it’s low limits causing throttling, increase the CPU limit or remove it entirely. I generally don’t set CPU limits anymore except for known resource-intensive batch jobs. CPU is compressible, so it’s safe to overcommit.

If it’s high requests monopolizing node capacity, reduce the requests to match actual usage. Use VPA (Vertical Pod Autoscaler) to recommend appropriate values based on historical usage.

Implement LimitRanges to prevent teams from setting unreasonable values:

apiVersion: v1
kind: LimitRange
metadata:
  name: cpu-limit-range
  namespace: team-a
spec:
  limits:
  - max:
      cpu: "4"
    min:
      cpu: "100m"
    default:
      cpu: "500m"
    defaultRequest:
      cpu: "200m"
    type: Container

Long-term solution: Better cluster organization. Consider namespace-per-team with ResourceQuotas to limit total resource consumption per team. This prevents one team from impacting others.

I’d also implement better monitoring and alerting on CPU throttling so you catch these issues before they affect users.

Scenario 4: The $50,000 Cloud Bill

Situation: Your AWS bill jumped from $15,000 to $50,000 this month. Finance is asking questions. The culprit is Kubernetes-related. How do you investigate and prevent this?

Investigation process:

I’ve lived through this, except our bill went from $30k to $80k. Fun times.

First, identify what specifically increased. In AWS, check Cost Explorer and break down by service. Common Kubernetes-related cost spikes:

LoadBalancer Services creating dozens of ELBs. Each costs money hourly plus data transfer. I’d audit all Services with type LoadBalancer and migrate appropriate ones to Ingress with a shared load balancer.

kubectl get svc --all-namespaces -o json | jq -r '.items[] | select(.spec.type=="LoadBalancer") | "\(.metadata.namespace)/\(.metadata.name)"'

Persistent volumes. EBS volumes cost money even if pods aren’t using them. Abandoned PVCs from deleted deployments can accumulate. I’d list all PVCs and cross-reference with actual usage:

kubectl get pvc --all-namespaces

Check for unbound PVCs or PVCs from pods that no longer exist.

Data transfer costs. Traffic between availability zones costs money. If your pods are scattered across AZs and making lots of cross-AZ calls, the charges add up. Review topology spread constraints and consider concentrating related services in the same AZ.

Overprovisioning. If you’re running nodes that are mostly empty, you’re wasting money. Check node utilization:

kubectl top nodes

If nodes are consistently under 30% utilized, you have too much capacity.

NAT Gateway data transfer. If your pods make lots of external API calls, the NAT Gateway charges can be significant. Consider VPC endpoints for AWS services or proxy solutions.

Prevention measures I implement:

Set up cost monitoring and budget alerts in AWS. Get notified before the bill gets crazy.

Implement ResourceQuotas to limit how much compute resources each namespace can consume. This prevents runaway scaling.

Use cluster autoscaler and right-size node types. Don’t run m5.4xlarge instances if m5.large would suffice.

Regular audits of resources. I have a monthly review where we identify orphaned resources (unattached EBS volumes, unused load balancers, old snapshots).

Educate teams on costs. Many developers don’t realize that every LoadBalancer Service costs money or that leaving test deployments running all month adds up.

Implement pod disruption budgets and use Spot instances for non-critical workloads. Spot can save 70% on compute costs.

The key lesson: Cloud costs in Kubernetes are death by a thousand cuts. It’s rarely one giant thing, but many small inefficiencies that accumulate.

Scenario 5: The Upgrade That Went Wrong

Situation: You’re upgrading your production Kubernetes cluster from 1.26 to 1.27. Halfway through, pods start failing with API errors. Some deployments are broken. How do you handle this and what went wrong?

Response:

First rule: Don’t panic. Second rule: Don’t try to upgrade production during business hours. But assuming we’re here…

Immediate triage:

What’s the current state? Check API server health, etcd health, and which nodes are on which version. In a managed Kubernetes service (EKS, GKE, AKS), check the control plane version versus worker node versions.

Are control plane components healthy? Check API server, controller manager, scheduler logs for errors. Version skew between components can cause issues.

What exactly is failing? Are pod creations failing? Are existing pods having issues? Check specific error messages from kubectl and pod events.

What likely went wrong:

API version deprecations. Kubernetes regularly deprecates old API versions. If you’re using manifests with deprecated APIs, they stop working after upgrade. For example, the 1.25 upgrade removed several long-deprecated APIs like extensions/v1beta1 for Ingress.

To check for deprecated APIs before upgrading, I use tools like Pluto or kubectl-convert:

pluto detect-files -d .

The fix during the incident: If deprecated APIs are the issue, update the manifests to use current API versions. This is why you test upgrades in staging first.

CRD compatibility issues. Custom Resource Definitions might not be compatible with the new Kubernetes version. Check if any operators or controllers are having issues.

Webhook timeouts. Admission webhooks that aren’t compatible with the new version can cause API requests to hang or fail.

My upgrade process to avoid this:

Test the upgrade thoroughly in staging first with an exact copy of production workloads.

Use upgrade compatibility tools:

kubectl-convert -f old-manifest.yaml --output-version apps/v1

Upgrade one minor version at a time. Never skip versions. Going from 1.24 to 1.27 is three separate upgrades: 1.24 → 1.25 → 1.26 → 1.27.

Read the release notes carefully. Each release has a “Changes by Kind” section that lists deprecations and breaking changes.

Backup etcd before starting. If everything goes wrong, you can restore.

Upgrade control plane first, then worker nodes gradually. Modern Kubernetes supports the control plane being one or two minor versions ahead of nodes.

Have a rollback plan. For managed Kubernetes, understand whether you can roll back (usually you can’t) or if you need to restore from backup.

The scenario you described sounds like someone tried to upgrade too quickly, skipped testing, or didn’t check for API deprecations. I’ve seen this happen, and recovery involves updating manifests while under pressure, which is not fun.

Scenario 6: Global Traffic Routing

Situation: Your company has users worldwide. You’re running Kubernetes clusters in AWS regions: us-east-1, eu-west-1, and ap-southeast-1. You need to route users to the nearest cluster with automatic failover if a region goes down. How do you architect this?

My architecture:

This requires components outside Kubernetes itself. Here’s what I’d build:

DNS-based geographic routing using Route53 with geolocation routing policies. Create records that route European users to eu-west-1, Asian users to ap-southeast-1, and American users to us-east-1.

Health checks at the cluster level, not just individual services. Create a health check endpoint that validates the cluster and critical services are healthy. Route53 health checks poll this endpoint and automatically fail over to the next closest region if it fails.

Each cluster has identical deployments managed via GitOps. The same Git repos deploy to all regions, ensuring consistency.

The database layer needs careful design. Options:

Use a global database service like AWS Aurora Global Database or DynamoDB Global Tables that replicate across regions with low latency.

Or accept eventual consistency and have regional databases with asynchronous replication.

Or use a hybrid approach: master database in one region for writes, read replicas in other regions for reads, with application logic that handles the replication lag.

Session management: Sticky sessions break geographic routing because users get pinned to one region. Instead, use a shared session store like Redis with global replication, or JWT tokens that are stateless and can be validated in any region.

Observability across regions: Centralized logging and metrics (CloudWatch, Datadog, whatever) that aggregate from all clusters so you can see global health at a glance.

The tricky parts:

Cost. Running full clusters in three regions is expensive. Make sure the business justification is there.

Data consistency. CAP theorem says you can’t have everything. With geographic distribution, you typically choose availability and partition tolerance over strong consistency. Make sure the application can handle this.

Testing failover. Regularly test that failover works by intentionally failing health checks in one region and verifying traffic shifts to others.

Deployment coordination. When deploying updates, do you deploy all regions simultaneously or roll out region by region? I prefer the latter with monitoring between each region to catch issues early.

I implemented something similar at a previous job. The biggest lesson: Start with two regions and get that working perfectly before adding a third. The complexity scales non-linearly.

Quick Reference: Essential kubectl Commands

These are the commands I use daily. Bookmark this section.

Pod Operations:

kubectl get pods -n <namespace>
kubectl describe pod <pod-name>
kubectl logs <pod-name> -f
kubectl logs <pod-name> --previous
kubectl exec -it <pod-name> -- /bin/bash
kubectl delete pod <pod-name>
kubectl top pods

Deployment Operations:

kubectl get deployments
kubectl scale deployment <name> --replicas=5
kubectl rollout status deployment/<name>
kubectl rollout history deployment/<name>
kubectl rollout undo deployment/<name>
kubectl set image deployment/<name> container=image:tag

Service and Network:

kubectl get svc
kubectl get ingress
kubectl port-forward pod/<name> 8080:80
kubectl proxy

Cluster Information:

kubectl get nodes
kubectl top nodes
kubectl get events --sort-by='.lastTimestamp'
kubectl cluster-info
kubectl api-resources

Debugging and Troubleshooting:

kubectl describe <resource> <name>
kubectl get events -n <namespace>
kubectl logs -l app=<label> --tail=100
kubectl get pods --field-selector status.phase=Failed
kubectl auth can-i <verb> <resource>

Resource Management:

kubectl apply -f manifest.yaml
kubectl delete -f manifest.yaml
kubectl diff -f manifest.yaml
kubectl get <resource> -o yaml
kubectl get <resource> -o json | jq '.items[].metadata.name'

Namespace Operations:

kubectl create namespace <name>
kubectl get all -n <namespace>
kubectl config set-context --current --namespace=<name>

Kubernetes Glossary: Key Terms Explained

API Server: The front door to Kubernetes. All operations go through the API server, which validates requests, handles authentication and authorization, and persists state to etcd.

Container Runtime: The software that actually runs containers. Common runtimes include containerd, CRI-O, and Docker (though Docker is deprecated as a runtime). Kubernetes talks to runtimes through the Container Runtime Interface (CRI).

Control Plane: The collection of components that manage the cluster, including the API server, scheduler, controller manager, and etcd. These can run on dedicated master nodes or be managed by your cloud provider.

CRI (Container Runtime Interface): The standard API between Kubernetes and container runtimes. This abstraction lets you swap runtimes without changing Kubernetes.

CSI (Container Storage Interface): The standard API for storage plugins. Modern storage solutions use CSI drivers instead of being built into Kubernetes.

Declarative Configuration: You specify the desired state (three replicas of this pod), and Kubernetes figures out how to achieve it. Contrast with imperative configuration where you specify explicit actions (create pod, then create another, then…).

DaemonSet: A controller that ensures a pod runs on every node in the cluster or on a subset of nodes matching specific criteria.

etcd: A distributed key-value store that holds all cluster state. It’s the cluster’s source of truth. If you lose etcd without backups, you lose everything.

Helm: A package manager for Kubernetes. It lets you define, install, and upgrade applications using charts, which are packaged collections of Kubernetes resources.

Ingress: An API object that manages external HTTP/HTTPS access to services, providing load balancing, SSL termination, and name-based virtual hosting.

Ingress Controller: The actual software that implements Ingress rules. Popular controllers include nginx-ingress, Traefik, and HAProxy. The Ingress object is just configuration, the controller does the work.

Kubelet: The agent running on every node that manages pods assigned to that node. It ensures containers are running and healthy.

Kube-proxy: A network proxy that runs on each node and maintains network rules for Service routing. In modern setups, this functionality might be replaced by eBPF-based solutions like Cilium.

Label: A key-value pair attached to objects for organization and selection. Labels are how Services find pods, how deployments manage replicas, and how you filter resources.

Namespace: A virtual cluster within your cluster. Namespaces provide scope for names and enable resource isolation through quotas and policies.

Node: A worker machine (physical or virtual) in the cluster where pods actually run. Each node runs a kubelet, container runtime, and kube-proxy.

Operator: A Kubernetes controller that uses Custom Resource Definitions to manage applications and their components. Operators encode operational knowledge into software.

Pod: The smallest deployable unit in Kubernetes. A pod wraps one or more containers that share network and storage resources.

PersistentVolume (PV): A piece of storage in the cluster that has been provisioned by an administrator or dynamically provisioned using a StorageClass.

PersistentVolumeClaim (PVC): A request for storage by a user. It’s like a pod requesting compute resources but for storage.

ReplicaSet: Ensures a specified number of pod replicas are running. You typically don’t create these directly, Deployments manage them.

Scheduler: The control plane component that watches for newly created pods and assigns them to nodes based on resource requirements, constraints, and policies.

Service: An abstraction that defines a logical set of pods and a policy for accessing them. Services provide stable networking for ephemeral pods.

StatefulSet: A controller for stateful applications that need stable network identities and persistent storage. Unlike Deployments, pods in a StatefulSet have sticky identities.

Taint: A property of a node that repels pods unless those pods have a matching toleration. Used to dedicate nodes to specific workloads.

Toleration: Allows a pod to schedule onto nodes with matching taints. This is how you opt-in to running on tainted nodes.

Volume: A directory accessible to containers in a pod. Unlike container storage (which is ephemeral), volumes persist beyond container restarts.

Conclusion: Preparing for Your Kubernetes Interview

After conducting and taking dozens of Kubernetes interviews, here’s what I’ve learned about preparation:

Don’t just memorize definitions. Understand the why behind concepts. When someone asks “What’s a pod?” you should be able to explain not just what it is but why Kubernetes uses pods instead of containers directly.

Get hands-on experience. Spin up a cluster using Minikube or kind on your laptop. Break things on purpose. Create a pod without resource limits and watch what happens. Misconfigure a Service selector and figure out why traffic isn’t routing. The best learning comes from fixing your own mistakes in a safe environment.

Run something in production or as close as you can get. Many concepts don’t click until you’ve been paged at 3 AM because a pod is CrashLooping. If you don’t have production access, contribute to open source projects that use Kubernetes, or run a side project that you actually care about keeping up.

Read the official documentation, but don’t just read it, use it. When you’re stuck, the Kubernetes docs are usually excellent. Learning to navigate them efficiently is itself a valuable skill.

Stay current with new versions. Kubernetes releases a new minor version every four months. You don’t need to know every feature of every version, but understanding the major themes (like the shift from PSPs to Pod Security Standards) shows you’re engaged with the ecosystem.

For the interview itself, think out loud. Even if you don’t know the answer, talking through your reasoning shows how you approach problems. I’ve hired candidates who didn’t know specific answers but demonstrated strong troubleshooting methodology.

Finally, remember that Kubernetes is just a tool. The best engineers understand when to use it and when not to. If an interviewer asks “Why would you use Kubernetes?” and you can articulate both the benefits and the costs, that shows maturity that many candidates lack.

Good luck with your interviews. Feel free to revisit this guide as you prepare. The questions and scenarios here cover what you’ll actually face in 2025, not theoretical concepts that sound good but never come up.

And remember, everyone started somewhere. I still have a note from my first Kubernetes interview where I completely blanked on how Ingress works under pressure. You learn, you improve, and eventually you’re the one interviewing others.