CORTEX
All projects

TypeAheadX

PROTOTYPE

Distributed autocomplete over three Redis nodes, hardened by breaking it: node loss, a frozen database and concurrent writes, each measured before and after.

Course project (the original app), audited and hardened afterwards · Jun 2026 – Oct 2026 · 15 original commits over two days, then an audit

THE PROBLEM

Autocomplete is read-heavy but latency-sensitive: one search generates a query per keystroke, hot prefixes create Zipfian skew, and a single Redis instance becomes a bottleneck.

HOW IT WORKS

  • FastAPI serves suggestions from PostgreSQL behind a Redis cache-aside layer; a 128-bit MD5 consistent-hash ring (500 virtual nodes) spreads prefixes across three Redis nodes. A Next.js frontend calls the API directly.
  • Writes are batched in memory and flushed as one atomic SQL statement; a decaying trending score is applied when results are read, not when they are stored.
  • Failure handling was written after trying to break each claim at runtime: a per-node circuit breaker with no client retries for Redis, a deadline, bulkhead and breaker for database reads, a bounded write buffer, and single-flight for concurrent identical misses.

ENGINEERING EVIDENCE

  • The consistent-hash ring balances keys evenly (about 33.9/34.6/31.6% at 500 virtual nodes; 1,000 is tighter, and 150 gives 39.7/28.6/31.7%). Adding a fourth node moves about 26.4% of keys against about 74.8% under modulo hashing. claims.json ↗
  • Stopping or freezing one of three Redis nodes at 100 requests per second caused 36-38% errors and about 17 s without a successful request. The cause included redis-py's default retries (a 250 ms timeout took 7.65 s). With a per-node circuit breaker and retries off: 0 errors, p99 about 9 ms, in repeated runs. 004-cache-failure-handling.md ↗
  • Six concurrent flushers lost 67.5% and 70.0% of the counter updates (234 and 216 of 720 stored) until the merge moved into one SQL statement: 720 of 720, and a flush of 100 queries from about 500 ms to about 14 ms. lost_update.jsonl ↗
  • A 20 s database freeze left 14 s with no successful request. A read deadline, bulkhead and breaker cut that to 0 s, with the slowest request at 2.0 s. Writes are deliberately not guarded, because a timed-out write may still execute. 006-write-path-durability-and-db-deadlines.md ↗
  • The audit also found that cache invalidation never invalidated (two different key strings), that the trending rank was ignored by the SQL, that LIKE wildcards in the search box changed the meaning of a query, and that one oversized query dropped a whole batch. Each has a regression test. postmortems.md ↗
  • 136 backend tests (71 need no services, 65 run against real PostgreSQL and Redis), 11 frontend tests, a 17-defect mutation check that the suite catches, and CI on every pull request. mutation_smoke.py ↗
  • An open-loop, interleaved benchmark replaced a closed-loop one. On a free GitHub-hosted Linux runner the original code collapsed at 500 requests per second with one worker, while the current code served 900 and, with two workers, 1,500 with no errors. benchmarks.md ↗

DECISIONS & INVESTIGATIONS

WHAT ISN'T DONE

  • A prototype, not a production service: no real users or traffic. The main results come from one Windows laptop with one uvicorn worker, a 150,000-row sample of a public autocomplete dataset, and synthetic keystroke traffic; the Linux CI run is on a shared, noisy runner.
  • There is no authentication or rate limiting on POST /search, so anyone can inject suggestions; that needs a gateway or a product decision. Debug endpoints are on by default, and an unknown cache provider silently falls back to an unbounded in-memory cache.
  • Writes still block while the database is frozen, and the write buffer lives in process memory, so a crash loses up to about 10 s of popularity counters. The breaker and single-flight are per process.
  • Balanced keys did not mean balanced traffic: under a Zipfian workload one node still took about 60% of requests, because the most popular queries hash to it. That is a property of the workload, and the proposed fix (an L1 cache or hot-key replication) is not built.
  • A warm cache hit has a p50 of about 1.1-1.5 ms in the open-loop laptop runs, not the 'sub-millisecond' the original README claimed, and the 97-98% hit rate in the original report came from 10,000 synthetic keys, which cap even a cold cache's hit rate at about 95.6%.
  • Not measured: more than two workers, a separate load-generator host, a million-row corpus, a soak longer than two minutes, real network latency or TLS. Fault injection runs locally, not in CI, there are no browser end-to-end tests, and the full 240-point matrix ran an earlier build (two later changes were validated in isolation).
  • The trending parameters are the documented defaults and have not been tuned on real traffic.

NEXT STEPS

Each one comes from a gap listed above. It says what fixing the gap would take; it is not a promise.

  • Add authentication or rate limiting to POST /search.
  • Build the proposed hot-key fix (an L1 cache or hot-key replication) and re-measure the 60% hot-node share.
  • Run fault injection in CI and measure beyond two workers on a separate load-generator host.
  • Give writes an idempotency key so they can have a deadline too.

STACK

  • Python
  • FastAPI
  • PostgreSQL
  • Redis
  • Next.js
  • TypeScript
  • GitHub Actions