Operate the Web Crawler
Hard About 1 h 30 min Katabench Pro
Synthesize durable discovery, partitioned ownership, polite fetching, content identity, replayable indexing, quarantine, secrets, and telemetry.
The full brief is part of Katabench Pro
Pro opens this kata's brief and its grader. To see how the track works first, try Handle Celebrity Fan-out, which is free with an account.
How it is graded
You build the design on a canvas, and Katabench checks the topology against the challenge's design rules.
What it practices
- Sharding and partitioning
- Split data and ownership by key when one node cannot hold or serve it.
- Idempotency and deduplication
- Make retries and redeliveries safe with idempotency keys and canonical identity.
- Secrets and network isolation
- Keep data stores private and secrets in a managed vault.
- Observability
- Carry traces, metrics and logs across synchronous and asynchronous hops.
Do these first
Related katas
- Operate the Reliable Checkout Hard Pro Synthesize redundant checkout, idempotent payment, reliable order events, managed secrets, and correlated telemetry.
- Observe the Shortener Hard Pro Redirects feel slow in some regions and click counts lag by hours, and nobody can prove where the time goes. Instrument the split system you built before guessing.
- Observe the Video Pipeline Hard Pro Carry useful telemetry across an asynchronous video-processing pipeline and into an operations stack.
- Operate the Notification Platform Hard Pro Synthesize a production notification plane with redundant APIs and consumers, shared preferences, managed secrets, and end-to-end telemetry.