authenticate and authorize every pod connection by default
Zero Trust networking in Kubernetes: network policies and mTLS with Cilium
14 min read
Default Kubernetes networking lets any pod reach any other pod. Combine default-deny NetworkPolicy, SPIRE-backed mutual authentication, and Cilium eBPF enforcement to segment east-west traffic and prove service identity—without a sidecar on every pod.
Why flat Kubernetes networking fails in production
Most clusters still run implicit-allow networking: any pod can reach any other pod across namespaces until something blocks it. In a small staging cluster with trusted services, that is convenient. In production with dozens of teams, compliance boundaries, and multi-tenant workloads, it is a liability. Three gaps drive incidents. Lateral movement: a compromised pod in payments can probe databases, internal APIs, and caches in other namespaces because the default trust boundary is flat. Missing service identity: TLS at the ingress protects north-south traffic, but east-west calls often cross the pod network in plaintext with no cryptographic proof the caller is authorized—any workload on the network can impersonate a client. Operational friction: segmentation built from manual iptables rules or vendor-locked APIs cannot keep pace with daily deploys. You need policy beside application code, version-controlled in Git, enforced consistently on every cluster. Without a zero-trust networking layer, the perimeter is strong but the courtyard inside is open.
Authorization, identity, and eBPF enforcement as one model
Zero trust here means three complementary layers. NetworkPolicy is authorization: which pods may talk to which peers, on which ports, from which namespaces—firewall rules scoped to Kubernetes objects instead of IP spreadsheets. Namespace-level default-deny plus label-based allows gives strong multi-tenant segmentation without unmaintainable CIDR lists. mTLS is identity: policies say who may connect, but only mutual TLS proves who is actually sending bytes. Both sides present certificates; spoofing and confused-deputy attacks at the transport layer fail before application logic runs. Cilium is the enforcement engine: eBPF programs apply NetworkPolicy, mutual authentication, and Hubble observability at the socket level before packets walk deep iptables chains. Identity-aware rules use pod labels, not fragile IP ranges. Hubble exposes L3/L4/L7 flows and policy verdicts without extra agents. Transparent mutual authentication through SPIRE avoids per-pod sidecars for baseline east-west trust, while embedded Envoy still enables L7 HTTP rules where you need method and path control.
Stage 0: map real traffic with Hubble before you deny anything
The top cause of zero-trust migration failure is default-deny before you understand legitimate flows. Install Cilium with Hubble relay, UI, and drop metrics enabled, then observe at least one full business cycle—including batch jobs, nightly cron, and month-end processing. Export unique source and destination pairs into a policy blueprint. Only then start enforcement namespace by namespace.
helm repo add cilium https://helm.cilium.io
helm repo update
helm upgrade --install cilium cilium/cilium \
--version 1.16.0 \
--namespace kube-system \
--set hubble.enabled=true \
--set hubble.relay.enabled=true \
--set hubble.ui.enabled=true \
--set hubble.metrics.enabled="{dns,drop,tcp,flow,port-distribution}"hubble observe --since 168h --output json | \
jq -r '[.source.namespace, .source.pod.name, .destination.namespace, .destination.pod.name] | @tsv' | \
sort -u > observed-flows.tsvStage 1–2: default-deny per namespace, then explicit allows
Apply deny-all ingress and egress in one application namespace at a time. Always add an explicit DNS egress rule to kube-dns—without it, every pod loses name resolution silently. Validate with hubble observe that legitimate flows still pass before moving to the next namespace. From observed-flows.tsv, add targeted ingress allows: frontend gateway to API on 8080, API to Postgres on 5432. Prefer namespace labels for cross-namespace rules and pod labels for intra-namespace service boundaries—namespace labels change rarely; pod labels represent service edges.
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: default-deny-all
namespace: payments
spec:
podSelector: {}
policyTypes:
- Ingress
- Egress
---
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: allow-dns
namespace: payments
spec:
podSelector: {}
policyTypes:
- Egress
egress:
- to:
- namespaceSelector:
matchLabels:
kubernetes.io/metadata.name: kube-system
podSelector:
matchLabels:
k8s-app: kube-dns
ports:
- protocol: UDP
port: 53
- protocol: TCP
port: 53apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: allow-frontend-to-api
namespace: payments
spec:
podSelector:
matchLabels:
app: payments-api
policyTypes:
- Ingress
ingress:
- from:
- namespaceSelector:
matchLabels:
kubernetes.io/metadata.name: frontend
podSelector:
matchLabels:
app: web-gateway
ports:
- protocol: TCP
port: 8080apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: allow-api-to-postgres
namespace: payments
spec:
podSelector:
matchLabels:
app: postgres
policyTypes:
- Ingress
ingress:
- from:
- podSelector:
matchLabels:
app: payments-api
ports:
- protocol: TCP
port: 5432Stage 3: mutual authentication with SPIRE and Cilium
Cilium mutual authentication (Beta) validates workload identity through SPIFFE/SPIRE before connections proceed. Enable authentication and the bundled SPIRE install in Helm, then add authentication.mode required on ingress rules in CiliumNetworkPolicy. The first packet on a new flow may drop briefly while agents complete the handshake—expect a small retry on cold connections. Certificate issuance and rotation are handled by the Cilium agent and SPIRE; you do not manage per-pod cert files in application images. Pair mutual authentication with WireGuard or IPsec encryption in Helm when compliance requires encrypted transport in addition to identity proof.
helm upgrade cilium cilium/cilium \
--namespace kube-system \
--reuse-values \
--set authentication.enabled=true \
--set authentication.mutual.spire.enabled=true \
--set authentication.mutual.spire.install.enabled=trueapiVersion: cilium.io/v2
kind: CiliumNetworkPolicy
metadata:
name: mtls-payments-api
namespace: payments
spec:
endpointSelector:
matchLabels:
app: payments-api
ingress:
- fromEndpoints:
- matchLabels:
app: web-gateway
io.kubernetes.pod.namespace: frontend
authentication:
mode: "required"
toPorts:
- ports:
- port: "8080"
protocol: TCPStage 4: L7 HTTP rules for API surface control
Network allow rules stop unauthorized peers; they do not stop an authorized pod from calling admin endpoints. For HTTP services, CiliumNetworkPolicy can enforce method and path patterns through the embedded Envoy proxy on matched ports. Even when mTLS proves identity, L7 rules limit which URLs that identity may invoke—defense in depth for internal APIs.
apiVersion: cilium.io/v2
kind: CiliumNetworkPolicy
metadata:
name: l7-payments-api
namespace: payments
spec:
endpointSelector:
matchLabels:
app: payments-api
ingress:
- fromEndpoints:
- matchLabels:
app: web-gateway
toPorts:
- ports:
- port: "8080"
protocol: TCP
rules:
http:
- method: "GET"
path: "/api/v1/(balance|transactions).*"
- method: "POST"
path: "/api/v1/transfer"Operational practices: Git, tiers, monitoring, and runtime security
Store NetworkPolicy and CiliumNetworkPolicy manifests in the same Git repo as Helm charts or Kustomize overlays—every change through pull request review with rollback history. Enable Hubble drop and TCP flow metrics from day one; a policy that blocks readiness probes cascades into outage—alert on new DROPPED verdicts in critical namespaces. In shared clusters, place platform guardrails in CiliumClusterwideNetworkPolicy—block cloud metadata endpoints, deny kube-system egress surprises—so tenant policies cannot weaken global segmentation. Automate certificate rotation: SPIRE short-lived identities beat year-long certs that make revocation meaningless. Test new policies in staging against captured flow logs before production; regressions at 3 AM are expensive. Layer network segmentation with runtime tools such as Falco or Tetragon for unexpected process execution, filesystem writes, and credential access inside pods. Admission Policy as Code from Kyverno or Gatekeeper can require NetworkPolicy labels on every namespace before workloads deploy—closing the gap between documented standards and enforced segmentation. Zero trust is verified at every layer, not only at the ingress edge.
Default-deny networking extends the baseline from our Kubernetes security hardening guide for production clusters.
Hubble flow visibility and eBPF datapath details are covered in our eBPF kernel observability guide with Cilium Hubble.