Sidecar-less Service Mesh: Securing Kubernetes with Cilium eBPF

For years, the standard blueprint for a Kubernetes service mesh has been synonymous with the sidecar pattern. If you wanted mutual TLS (mTLS), observability, and fine-grained traffic control, you accepted the "sidecar tax": an Envoy proxy injected into every single pod. While Istio and Linkerd revolutionized microservices communication, they also introduced significant operational complexity, increased memory footprints, and added measurable latency to every network hop.

Enter Cilium and the eBPF-powered service mesh. By moving logic from user-space proxies into the Linux kernel, we can now achieve the same security and observability goals without the overhead of sidecars. This article explores how to implement a sidecar-less service mesh using Cilium, focusing on mutual authentication and Hubble-based observability.

The Problem with the Sidecar Tax

In a traditional service mesh, every application pod consists of the app container and a proxy container (the sidecar). This architecture has three primary drawbacks:

Resource Consumption: Even a lightweight proxy consumes 30MB–70MB of RAM. In a cluster with 1,000 pods, you are losing 50GB+ of memory just to move packets.
Latency: Traffic must traverse the TCP/IP stack multiple times—from the app to the sidecar, out to the network, into the destination sidecar, and finally to the destination app. Each jump adds milliseconds.
Operational Complexity: Upgrading the mesh often requires restarting every pod in the cluster to inject a new proxy version. Managing certificate rotation and debugging failed injections becomes a full-time job.

Cilium leverages eBPF (Extended Berkeley Packet Filter) to solve this. eBPF allows us to run sandboxed programs in the Linux kernel without changing kernel source code or loading modules. Because the kernel sees every packet, it can handle routing, security, and telemetry at the source, bypassing the need for a middleman proxy in most scenarios.

Rethinking Mutual Authentication (mTLS)

In the sidecar model, mTLS is handled by the Envoy proxies. They manage the handshakes, certificates, and encryption. In a sidecar-less model with Cilium, we decouple the authentication of the identity from the encryption of the data.

Identity-Based Security

Cilium does not rely on IP addresses for security, as IPs are ephemeral in Kubernetes. Instead, Cilium assigns a numeric Security Identity to every pod based on its labels. When Pod A talks to Pod B, Cilium attaches this identity to the packet (using a custom header or via the overlay network). The receiving node checks if Identity A is allowed to talk to Identity B based on the loaded eBPF programs.

Transparent Encryption

For the "encryption" part of mTLS, Cilium uses WireGuard or IPsec at the node level. This is a significant shift: instead of encrypting at the pod-to-pod level via a proxy, Cilium encrypts at the host-to-host level.

When you enable WireGuard in Cilium, the kernel automatically encrypts all traffic between nodes. Because this happens in the kernel, it is significantly faster than user-space encryption. From the application’s perspective, the traffic is plain text; from the wire's perspective, it is fully encrypted.

To enable this, the configuration is remarkably simple compared to managing Istio's PeerAuthentication resources:

cilium config set encryption-enabled true
cilium config set encryption-type wireguard

Implementing Layer 7 Policy without Sidecars

One common argument for sidecars is the need for L7 visibility (HTTP methods, paths, headers). Cilium handles this using a per-node Envoy proxy rather than a per-pod proxy.

When a CiliumNetworkPolicy requires L7 inspection, Cilium’s eBPF program redirects only the necessary traffic to a single Envoy instance running on the node. This reduces the number of proxies from $N$ (number of pods) to $M$ (number of nodes), drastically reducing memory overhead.

Example: Restricting HTTP Methods

Suppose you have a maintenance service that should only allow GET requests to the /health endpoint. In a sidecar-less mesh, you define a CiliumNetworkPolicy like this:

apiVersion: "cilium.io/v2"
kind: CiliumNetworkPolicy
metadata:
  name: "restrict-maintenance-access"
spec:
  endpointSelector:
    matchLabels:
      app: maintenance
  ingress:
  - fromEndpoints:
    - matchLabels:
        app: monitor
    toPorts:
    - ports:
      - port: "8080"
        protocol: TCP
      rules:
        http:
        - method: "GET"
          path: "/health"

Cilium’s eBPF programs will intercept the traffic at the socket level and, if it matches the L7 criteria, ensure it is validated before reaching the application. This provides the granularity of a service mesh with the efficiency of a CNI.

Deep Observability with Hubble

Observability is often the "killer feature" that drives teams to adopt a service mesh. Cilium provides this through Hubble, a distributed networking and security observability platform built on top of Cilium.

Because Hubble pulls data directly from eBPF hooks, it provides a level of visibility that proxies cannot match, including drops at the kernel level, DNS resolution failures, and TCP retransmissions.

The Hubble Stack

Hubble Relay: Aggregates data from all nodes in the cluster.
Hubble UI: Provides a graphical service map showing how services communicate.
Hubble CLI: A powerful tool for real-time traffic debugging.

To see real-time traffic between services without having to look at sidecar logs, you can run:

hubble observe --pod my-app-pod --follow

This output shows you exactly which flows are being allowed or denied, the identity of the source and destination, and the L7 metadata—all without the application ever knowing it’s being watched.

Performance Comparison: The eBPF Advantage

When evaluating a sidecar-less approach, performance is usually the deciding factor. In standard benchmarks, Cilium's eBPF-based data path consistently outperforms sidecar-based meshes in two key metrics:

Throughput: By avoiding the context switches between kernel-space and user-space (which happen every time a packet hits an Envoy sidecar), Cilium can saturate 10Gbps or 40Gbps links with significantly less CPU usage.
Tail Latency (P99): Sidecars introduce "jitter" due to garbage collection or buffer management within the proxy. eBPF programs are executed in the kernel context with deterministic execution times, leading to much flatter P99 latency curves.

In high-frequency trading, gaming, or large-scale API gateways, these millisecond gains translate into significant cost savings and better user experiences.

When Do You Still Need a Proxy?

It is important to be realistic: eBPF is not a magic wand that replaces every feature of Envoy. While Cilium is moving more logic into the kernel, certain complex L7 functions are still better handled by a proxy (even if it's the per-node Envoy managed by Cilium):

Complex Retries: Implementing sophisticated exponential backoff and retry logic entirely in eBPF is difficult due to the kernel's verification constraints.
Payload Transformation: If you need to modify the body of an HTTP request on the fly (e.g., stripping headers or transforming JSON), a proxy is still the right tool.
WAF Functionality: Deep packet inspection for SQL injection or XSS is better suited for a user-space process.

The beauty of Cilium is that it uses a hybrid approach. It uses eBPF for everything it can (L3/L4, identity, encryption, simple L7) and transparently offloads to a node-local Envoy only when complex L7 processing is required.

Real-World Implementation Strategy

If you are considering migrating from a sidecar-based mesh to Cilium, follow this phased approach:

Phase 1: CNI Replacement

Start by replacing your existing CNI (like Flannel or standard Calico) with Cilium. This gives you basic identity-based networking and the performance benefits of eBPF without changing how your apps work.

Phase 2: Transparent Encryption

Enable WireGuard. This provides immediate security for all traffic in the cluster. Unlike Istio mTLS, this works for all protocols (TCP, UDP, ICMP) and requires zero changes to your application manifests.

Phase 3: Identity-Based Policies

Replace your standard NetworkPolicies with CiliumNetworkPolicies. Start with L4 (IP/Port) and move to L7 (HTTP/Path) only where necessary. Use Hubble to audit traffic before switching to a "default-deny" posture.

Phase 4: Hubble Integration

Integrate Hubble metrics into Prometheus and Grafana. Use the Hubble service map to identify bottlenecked services or unexpected traffic patterns that were previously invisible.

Conclusion

The shift toward sidecar-less service meshes represents the next evolution of Kubernetes networking. By leveraging eBPF, Cilium allows us to treat the network as a first-class citizen of the operating system rather than an afterthought managed by a fleet of sidecar containers.

Actionable Takeaways:

Audit your current overhead: Check how much CPU and RAM your sidecars are consuming. If it's more than 10% of your cluster capacity, Cilium will provide an immediate ROI.
Benchmark your latency: Use tools like fortio to compare your current mesh latency against a Cilium-enabled cluster.
Start small: You don't need to enable every feature on day one. Cilium works perfectly well as a high-performance CNI first, and a service mesh second.

By moving the mesh into the kernel, we simplify our architecture, reduce our attack surface, and reclaim the performance that microservices were always meant to deliver.