Back to updates
API Gateway Redesign
API Gateway: Envoy WASM filter for custom rate limiting
Moved rate limiting logic from a sidecar service into an Envoy WASM filter, cutting tail latency by 60% and simplifying the deployment topology.
envoywasmrate-limitingperformance
Why Move to WASM?
Our previous rate limiting architecture had a sidecar Redis proxy next to each Envoy instance. Every request required a round-trip to the sidecar, adding ~2ms of latency at p50 and ~15ms at p99.
By implementing the rate limiting logic directly in an Envoy WASM filter, we eliminated the network hop entirely.
Implementation
- Written in Rust compiled to WASM using
wasm-bindgen - Sliding window algorithm with Redis as the state backend
- Shared Redis connection pool within the WASM sandbox via Envoy’s hostcall API
Performance Impact
| Metric | Before (sidecar) | After (WASM) | Change |
|---|---|---|---|
| P50 latency | 2.1ms | 0.3ms | -86% |
| P99 latency | 15.8ms | 6.2ms | -60% |
| Memory per instance | 180MB | 45MB | -75% |
Lessons Learned
WASM debugging is still rough. We spent a couple of days fighting opaque error messages before realizing the issue was a missing hostcall capability in the Envoy config. Better tooling is needed here.