Section 11: Open Doors Attract Everyone - Trust and Safety: Spam, Vote Fraud and Moderation
Executive Summary: Open writes and valuable reputation attract spammers, sockpuppets and script injectors. Human pre-moderation of 1.4M posts a day is impossible, so the defense is layered and every action is reversible: rate limits by reputation tier and IP, a 50 ms synchronous rules check, an async ML classifier that hides posts after the fact, community flags, and human review only for what machines can't decide. Vote fraud is caught nightly as a pattern and reversed exactly through Section 8's vote transaction and the reputation ledger. Markdown is sanitized once, at write time, and the cache rules keep private data out of the shared page.
Section 10 tested load and failure. The interviewer turns to adversaries.
Interviewer: "You said spam is a real problem. Who attacks this, and how do you stop them without a human reading every post?"
You: "Not with humans first. We get 1.4M posts a day; at 30 seconds a review that's 42M seconds, about 11,700 human-hours a day, and Daniel's answer would wait in a queue instead of appearing within 10 seconds. So: cheap layers, each catching what the last missed, with humans seeing only what machines can't decide."
Know the Adversary
| Threat | What the attacker wants | Where it lands | First defense |
|---|---|---|---|
| Link spam | Clicks and search ranking for their site | New posts with links | Rate limits, fast path, rel="nofollow ugc" |
| Low-quality floods | Volume: bots, copy-paste campaigns | Post endpoints | Buckets by reputation tier |
| Sockpuppets and vote rings | Reputation, privileges, ranking | The vote path, 86% of writes | Nightly fraud job |
| XSS via markdown | Script running in readers' browsers | body_html, served 300M times a day |
Write-time sanitizer, CSP |
| Cache poisoning, private data in cache | One bad response served to everyone | The CDN, 85% of traffic | No cookies on cacheable responses |
| Scraping | Bulk copies, load on expensive endpoints | Search, viewer-state | Per-IP buckets on origin-only endpoints |
The XSS row deserves the most fear. One stored script on Priya's question would have run in a million browsers during its viral hour.
Sign in to continue reading
The rest of this lesson is available with a free account. Signing in with Google or Microsoft is free.
Sign in to read the full lesson