https://koti.dev Real Kubernetes lessons from 7+ years in production. en-us https://koti.dev/posts/firewall-restriction https://koti.dev/posts/firewall-restriction Thu, 16 Apr 2026 00:00:00 GMT Cluster-to-cluster is fine. Cluster-to-Stripe is dead. The rule that hides in plain sight. https://koti.dev/posts/network-connectivity-issues https://koti.dev/posts/network-connectivity-issues Wed, 15 Apr 2026 00:00:00 GMT One default-deny egress policy, one black-hole namespace, one very long pager night. https://koti.dev/posts/dns-resolution-failure https://koti.dev/posts/dns-resolution-failure Tue, 14 Apr 2026 00:00:00 GMT The pod is healthy, the Service is up, nslookup hangs. The five-hop debug. https://koti.dev/posts/ingress-configuration-issue https://koti.dev/posts/ingress-configuration-issue Mon, 13 Apr 2026 00:00:00 GMT IngressClass, TLS secret, host, path. One of these four is why your Ingress is silent. https://koti.dev/posts/loadbalancer-service-misconfig https://koti.dev/posts/loadbalancer-service-misconfig Sun, 12 Apr 2026 00:00:00 GMT The cloud provider is fine. Your Service selector matches nothing. Here is the five-minute fix. https://koti.dev/posts/port-binding-issues https://koti.dev/posts/port-binding-issues Sat, 11 Apr 2026 00:00:00 GMT Two containers, one host port, zero useful error messages. Here is what is happening. https://koti.dev/posts/service-port-mismatch https://koti.dev/posts/service-port-mismatch Fri, 10 Apr 2026 00:00:00 GMT The pod runs, the service exists, curl returns refused. One number was wrong. https://koti.dev/posts/crash-due-to-insufficient-disk-space https://koti.dev/posts/crash-due-to-insufficient-disk-space Thu, 09 Apr 2026 00:00:00 GMT Your pod is healthy. The node is out of disk. Guess who gets evicted. https://koti.dev/posts/disk-io-errors https://koti.dev/posts/disk-io-errors Wed, 08 Apr 2026 00:00:00 GMT The container is crashing. The node is the reason. Here is how to prove it in 90 seconds. https://koti.dev/posts/file-permissions-on-mounted-volumes https://koti.dev/posts/file-permissions-on-mounted-volumes Tue, 07 Apr 2026 00:00:00 GMT readOnlyRootFilesystem is a security win and a 2AM footgun. Here is how to get both. https://koti.dev/posts/persistent-volume-claim-issues https://koti.dev/posts/persistent-volume-claim-issues Mon, 06 Apr 2026 00:00:00 GMT Pending forever. No events. No provisioner. Here is the five-minute diagnosis. https://koti.dev/posts/volume-mount-issue https://koti.dev/posts/volume-mount-issue Sun, 05 Apr 2026 00:00:00 GMT API rejects the pod, the error scrolls past, and you lose an hour to one missing block. https://koti.dev/posts/cgroup-issues https://koti.dev/posts/cgroup-issues Sat, 04 Apr 2026 00:00:00 GMT Your pod metrics look fine. The kernel disagrees. Here is what lives beneath kubectl. https://koti.dev/posts/failed-resource-limits https://koti.dev/posts/failed-resource-limits Fri, 03 Apr 2026 00:00:00 GMT Pod created. Pod killed. Pod created. Pod killed. Welcome to the forever loop. https://koti.dev/posts/oom-killed https://koti.dev/posts/oom-killed Thu, 02 Apr 2026 00:00:00 GMT Memory limits, cgroups, and the OOM score nobody reads. Why your container is dead and the node is perfectly fine. https://koti.dev/posts/resource-requests-limits-mismatch https://koti.dev/posts/resource-requests-limits-mismatch Wed, 01 Apr 2026 00:00:00 GMT One of the few Kubernetes errors that fails loudly at apply time, and the one most people still misread. https://koti.dev/posts/cluster-autoscaler-issues https://koti.dev/posts/cluster-autoscaler-issues Tue, 31 Mar 2026 00:00:00 GMT Twenty replicas asking for CPU the cluster does not have, and an autoscaler that stays silent. Here is the debug path. https://koti.dev/posts/taints-and-tolerations-mismatch https://koti.dev/posts/taints-and-tolerations-mismatch Mon, 30 Mar 2026 00:00:00 GMT Half the Pending pods I have debugged in my career were a missing toleration. Here is the mental model that ends the confusion. https://koti.dev/posts/node-affinity-issue https://koti.dev/posts/node-affinity-issue Sun, 29 Mar 2026 00:00:00 GMT One wrong letter in a hostname inside a nodeAffinity block, and the scheduler goes silent for an hour. https://koti.dev/posts/affinity-rules-violation https://koti.dev/posts/affinity-rules-violation Sat, 28 Mar 2026 00:00:00 GMT The scheduler will wait forever for an affinity rule that can never be satisfied, and it will never tell you out loud. https://koti.dev/posts/pod-stuck-pending https://koti.dev/posts/pod-stuck-pending Fri, 27 Mar 2026 00:00:00 GMT 45 minutes staring at a Pending pod, a Slack channel on fire, and one line of kubectl output that finally made sense. https://koti.dev/posts/liveness-readiness-failure https://koti.dev/posts/liveness-readiness-failure Thu, 26 Mar 2026 00:00:00 GMT Two probes, one bug, two outages. Read the event ages and you will know the order of operations. https://koti.dev/posts/liveness-probe-failure https://koti.dev/posts/liveness-probe-failure Wed, 25 Mar 2026 00:00:00 GMT Exit code 137 with clean app logs means the probe is the murderer. Here is how to catch it in the act. https://koti.dev/posts/readiness-probe-failure https://koti.dev/posts/readiness-probe-failure Tue, 24 Mar 2026 00:00:00 GMT STATUS says Running. READY says 0/1. Users see 503s. Here is where the pod is hiding. https://koti.dev/posts/image-pull-backoff https://koti.dev/posts/image-pull-backoff Mon, 23 Mar 2026 00:00:00 GMT Same cause as ErrImagePull, plus a backoff clock that makes the cluster look broken when it is just being polite. https://koti.dev/posts/image-pull-error https://koti.dev/posts/image-pull-error Sun, 22 Mar 2026 00:00:00 GMT One status, three completely different fixes. The describe event tells you which one you are actually looking at. https://koti.dev/posts/crashloopbackoff https://koti.dev/posts/crashloopbackoff Sat, 21 Mar 2026 00:00:00 GMT A status, a timer, and an exit code. Learn to read all three and you debug in sixty seconds instead of forty minutes. https://koti.dev/posts/wrong-container-command https://koti.dev/posts/wrong-container-command Fri, 20 Mar 2026 00:00:00 GMT The kubelet pulled the image, runc tried to exec, and nothing happened. Here is the one line that tells you why. https://koti.dev/posts/day-0-cluster-setup https://koti.dev/posts/day-0-cluster-setup Thu, 19 Mar 2026 00:00:00 GMT Before you debug a single Pending pod, you need a real cluster. Here is how to get one on AWS, GCP, or Azure with a single command — and the Terraform that makes it boring.