FlashScale.

The problem: concurrent cache misses

FlashScale is Sumit Kushwaha’s local Spring Boot demonstration of distributed caching and shared rate limiting. When several application replicas miss the same cache key at once, all of them can query the database for identical work. The project makes the coordination behind cache-stampede protection visible and measurable.

Three Spring Boot application replicas run behind Nginx in Docker Compose. They share Redis for cached product data, coordination, and rate-limit state, with PostgreSQL as the source of truth. Prometheus scrapes the replicas and Grafana presents the resulting metrics.

Architecture and the cache-aside path

1. Look up: a product request checks Redis first. A cached value can be returned without reading PostgreSQL.
2. Coordinate: a miss attempts a per-product Redis SET NX lock with an ownership token and expiry. The winner checks the cache again before querying the database.
3. Populate: the lock holder reads PostgreSQL, caches the product with a TTL, and releases its lock through a Lua script that checks ownership.
4. Wait: callers that lose the lock poll the cache for a bounded period. They do not use JVM-local synchronization, so the coordination spans all application replicas.

Lock cleanup runs in a finally block on database failure. Missing products return 404, and Redis or PostgreSQL data-access failures return 503 instead of leaving callers waiting indefinitely.

Shared rate limiting and observability

A Redis-backed fixed-window limiter applies to product reads across replicas. Rejected requests receive HTTP 429 and Retry-After. The limiter deliberately fails open when Redis is unavailable; product cache operations can still report a dependency failure. This separates the limiter’s availability decision from the data path’s ability to serve a product.

Metrics distinguish initial cache hits and misses from lock acquisition, lock failures, waits, and database reads. Grafana correlates these counters with HTTP latency and JVM memory. Replica-local counters must be aggregated across instances when examining system behavior.

The repository includes k6 scenarios for hot-cache reads, stampedes, bursts, and rate limiting. It records no reproducible throughput or latency benchmark yet, so this portfolio makes no capacity claim.

Engineering trade-offs

Fixed windows can allow bursts at a window boundary. Cache waiters briefly occupy servlet threads, and a lock TTL can expire during an unusually slow database query. The project documents these limitations rather than treating a distributed lock as an unlimited correctness guarantee.

Unit tests cover cache hits, population after misses, the lock winner’s double-check, bounded waits, and lock release after failures. Docker Compose provides the integration environment. Future work documented in the repository includes authenticated administration, jittered waits, lock renewal or fencing where needed, and reproducible benchmark results.