Start the URL Frontier
Easy About 25 min Katabench Pro
Turn seed URLs into a durable crawl loop instead of keeping the entire crawl in one fragile process.
The full brief is part of Katabench Pro
Pro opens this kata's brief and its grader. To see how the track works first, try Publish the First Post, which is free with an account.
How it is graded
You build the design on a canvas, and Katabench checks the topology against the challenge's design rules.
What it practices
- Message queues
- Accept quickly, buffer durably and process asynchronously with background workers.
Related katas
- Checkpoint Before Parsing Medium Pro Split unreliable network fetches from CPU-heavy parsing with an immutable page checkpoint and a second durable handoff.
- Count the Clicks Medium Pro Marketing wants click counts, but writing a row during every redirect doubles latency. Stream clicks through a durable queue into a dedicated Click Store, and keep them off the links table.
- Persist Chat Messages Medium Pro Slow message storage blocks socket loops. Publish accepted messages to a durable broker and persist them with redundant writers.
- Process Video Uploads Medium Pro Move large video uploads off the request path and process them through a durable asynchronous pipeline.