Sidecar-less Service Mesh: Securing Kubernetes with Cilium eBPF

For years, the industry standard for securing microservices in Kubernetes has been the sidecar-based service mesh. Whether you are using Istio or Linkerd, the recipe was the same: inject a proxy (usually Envoy) into every pod to handle traffic, encryption, and observability. While this solved the 'spaghetti networking' problem, it introduced a significant 'sidecar tax' in the form of latency, memory consumption, and operational complexity.

As a senior engineer, you've likely felt the pain of managing thousands of sidecars, debugging why a proxy didn't inject correctly, or seeing your infrastructure bill climb because every tiny microservice suddenly required an extra 100MB of RAM for its proxy.

Enter Cilium and eBPF. By moving the service mesh logic from the user-space (sidecars) into the Linux kernel (eBPF), we can achieve the same security and observability goals with significantly less overhead. This article explores how to implement a sidecar-less service mesh using Cilium, focusing on mutual authentication (mTLS) and Hubble for deep visibility.

Why Move Away from Sidecars?

Before diving into the implementation, we need to understand the fundamental limitations of the sidecar model. In a traditional mesh, every packet sent from Service A to Service B follows a tortuous path:

1. Service A sends a packet.
2. The packet is intercepted by the local Envoy proxy (Context switch 1).
3. Envoy processes the packet, applies mTLS, and sends it out.
4. The packet travels across the network.
5. The packet is intercepted by Service B's Envoy proxy (Context switch 2).
6. Envoy decrypts and validates the packet.
7. Envoy forwards the packet to Service B (Context switch 3).

This 'triple-hop' architecture adds milliseconds of latency. Furthermore, managing the lifecycle of these proxies—upgrading them, tuning their resources, and troubleshooting their configurations—becomes a full-time job at scale.

The eBPF Advantage

eBPF (Extended Berkeley Packet Filter) allows us to run sandboxed programs inside the Linux kernel without changing kernel source code or loading kernel modules. In the context of Cilium, eBPF allows the networking layer to become 'identity-aware.'

Instead of intercepting traffic and passing it to a user-space proxy, Cilium uses eBPF to handle networking logic directly at the socket level. When a packet leaves a container, the kernel already knows the identity of the source and destination. It can enforce security policies, encrypt traffic, and collect metrics without the packet ever leaving the kernel's data path for a proxy.

Implementing Mutual Authentication (mTLS) without Sidecars

One of the biggest arguments for sidecars was mTLS. How do you ensure encrypted, authenticated communication without a proxy to handle the handshakes?

Cilium 1.14 and later introduced a revolutionary approach to mTLS. It separates the authentication handshake from the data path encryption.

How it works:

1. Identity-based Control: Cilium assigns a unique security identity to every pod based on its labels.
2. Authentication Handshake: When two pods need to communicate, the Cilium agents on their respective nodes perform a handshake to verify identities using certificates (often integrated with SPIFFE/SPIRE or Kubernetes secrets).
3. Data Path Encryption: Once authenticated, the actual data is encrypted using WireGuard or IPsec at the kernel level.

This approach is significantly faster because the kernel handles the encryption of the actual payload, while the 'heavy lifting' of the TLS handshake happens out-of-band and only when necessary.

Practical Example: Enabling mTLS

To enable mTLS in a Cilium-managed cluster, you don't need to annotate pods for injection. Instead, you apply a CiliumNetworkPolicy. Here is a real-world example of requiring mTLS for a specific service:

apiVersion: "cilium.io/v2"
kind: CiliumNetworkPolicy
metadata:
  name: "secure-app-access"
  namespace: production
spec:
  endpointSelector:
    matchLabels:
      app: backend-api
  ingress:
  - fromEndpoints:
    - matchLabels:
        app: frontend-web
    authentication:
      mode: "required"

In this configuration, Cilium ensures that any traffic coming from frontend-web to backend-api is authenticated via the mTLS handshake. If the handshake fails or the identity doesn't match, the kernel drops the packets before they even reach the application layer.

Observability with Hubble

In a sidecar mesh, observability is achieved by the proxy logging every request. In a sidecar-less mesh, we use Hubble.

Hubble is built on top of Cilium and eBPF to provide deep visibility into the communication between services. Because it operates in the kernel, it sees everything—even traffic that bypasses standard networking tools.

The Hubble Stack

• Hubble Relay: Aggregates data from multiple nodes.
• Hubble UI: Provides a graphical service map showing how traffic flows.
• Hubble CLI: A powerful tool for real-time flow debugging.

Debugging with Hubble CLI

Imagine you have a developer complaining that their service is getting 403 Forbidden errors. In a traditional environment, you'd be looking at proxy logs and tcpdumps. With Hubble, you can see the exact flow and why it was dropped:

hubble observe --pod backend-api-6f8c --follow

Output example:

Jan 24 10:15:02.123: frontend-web-9a2b:45232 -> backend-api-6f8c:8080 http-request FORWARDED (HTTP/1.1 GET /api/v1/data)
Jan 24 10:15:02.125: frontend-web-9a2b:45232 <- backend-api-6f8c:8080 http-response FORWARDED (HTTP/1.1 200 OK)

If a policy were blocking the traffic, Hubble would explicitly state which CiliumNetworkPolicy caused the drop. This level of transparency, without the overhead of a proxy, is a game-changer for platform teams.

Performance Comparison: Sidecar vs. eBPF

When evaluating this for your stack, consider these three metrics:

1. Memory Footprint: A sidecar (Envoy) typically consumes 50MB–150MB per pod. In a cluster with 500 pods, that's up to 75GB of RAM just for the mesh. Cilium runs as a daemonset (one pod per node), meaning memory usage scales with the number of nodes, not the number of pods.
2. Latency: Sidecars add 2–5ms of latency per hop due to the user-space context switching. eBPF processing happens in nanoseconds/microseconds within the kernel path.
3. CPU Overhead: Handling TLS in user-space is CPU-intensive. WireGuard (used by Cilium) is implemented in the kernel and is exceptionally efficient, often outperforming IPsec and user-space TLS implementations.

Operational Considerations

Transitioning to a sidecar-less model isn't just about performance; it's about simplifying the 'Day 2' operations.

• No More Injection: You don't have to worry about sidecar.istio.io/inject: "true" annotations or ensuring that the sidecar container starts before the application container.
• Simplified Upgrades: To upgrade the mesh, you upgrade the Cilium agent on the node. You don't need to restart every single application pod in the cluster to roll out a security patch to the proxy.
• App-Agnostic: Because the logic is in the kernel, it doesn't matter if your application is written in Go, Java, or a legacy COBOL wrapper; the security and observability remain consistent.

When Should You Still Use a Sidecar?

As a senior engineer, you know there is no silver bullet. While eBPF handles Layer 3/4 (IP/TCP) and basic Layer 7 (HTTP) exceptionally well, there are cases where a user-space proxy like Envoy is still beneficial:

• Complex L7 Retries/Circuit Breaking: If you need highly specific, complex application-level retry logic that requires deep buffering of requests, a sidecar might still be appropriate.
• Legacy Environments: If you are running on very old Linux kernels (pre-5.10) that don't fully support the latest eBPF features.

Cilium actually allows for a hybrid approach. It can spin up a 'node-local' Envoy proxy only when complex L7 processing is required, rather than forcing a sidecar into every pod. This 'Envoy on-demand' model provides the best of both worlds.

Conclusion: Actionable Next Steps

Implementing a sidecar-less service mesh with Cilium is no longer an experimental endeavor; it is a production-ready strategy used by organizations like Google, Adobe, and S&P Global. To get started:

1. Audit your current 'Sidecar Tax': Calculate the total CPU/Memory consumed by proxies in your cluster. This is your potential ROI.
2. Verify Kernel Compatibility: Ensure your worker nodes are running a modern Linux kernel (5.10+ is recommended for optimal eBPF features).
3. Install Cilium with Hubble: Use the Cilium CLI or Helm chart to install Cilium. Enable Hubble during the installation to immediately gain visibility.
4. Start with Visibility, then Security: Deploy Hubble first to map your service dependencies. Once you understand the traffic patterns, start enforcing CiliumNetworkPolicies with mTLS enabled.

By moving the mesh into the kernel, you reclaim your cluster's resources, reduce your attack surface, and simplify the lives of your developers.