Deduplicate Page Content
Hard About 45 min Katabench Pro
Hash fetched bytes before expensive parsing so mirrors and alias URLs share one canonical content record.
The full brief is part of Katabench Pro
Pro opens this kata's brief and its grader. To see how the track works first, try Handle Celebrity Fan-out, which is free with an account.
How it is graded
You build the design on a canvas, and Katabench checks the topology against the challenge's design rules.
What it practices
- Idempotency and deduplication
- Make retries and redeliveries safe with idempotency keys and canonical identity.
Do these first
Related katas
- Operate the Reliable Checkout Hard Pro Synthesize redundant checkout, idempotent payment, reliable order events, managed secrets, and correlated telemetry.
- Operate the Web Crawler Hard Pro Synthesize durable discovery, partitioned ownership, polite fetching, content identity, replayable indexing, quarantine, secrets, and telemetry.
- Refund the Late Payment Hard Pro A payment result arrives 31 seconds after the hold expired. The seat has gone, the money has moved, and support hears about it two days later. Give every result one owner.
- Deduplicate Discovered URLs Medium Pro Canonicalize every discovered link and claim it in one shared seen-URL ledger before it can re-enter the frontier.