How to find the spam among 10,000 posts a day

“The best approach is to fight magic with magic: the bad actors flood us with spam, so we use language models to read, at scale, who the spammers are.”

Two lines of defense against spam

The first is the behavioral layer: during sign-up, login, and when liking or tipping, a reCAPTCHA／Turnstile A little quiz or background check used at sign-up or login to tell whether “you're a real person or a bot.” blocks bots (since 2020). The second is the content layer: since 2024, a language model analyzes what each post is actually saying, one by one.

What is the content model, and how does the machine learn?

Because the goal is a model that “understands meaning,” the team trains one built for comprehension — an Encoder An AI model built specifically to “read and understand” the meaning of text — a different job from the chat-style models that “generate” text; BERT is a well-known example of this kind. — on training material of roughly 40,000 hand-labeled positive and negative examples marked up by the operations team (technically, a granite embedding model fine-tuned with LoRA). The current approach is that a post judged to be in violation is not deleted; it simply no longer appears on public pages. Through continued effort in 2026, the block success rate rose from 96% to 99.7%, and the false-positive rate fell from 3.1% to 0.13%. For comments, a smaller e5-small model is used instead, with a false-positive rate of just 0.33% — the small model turns out to be more precise than the unwieldy large one.

The model can easily “poison itself”

The old model retrained every day, and over time it amplified its own mistakes — the problem was that the wrong content was never pulled out in time. Once an account got flagged as spam one day, the normal posts it wrote afterward kept being treated as spam too — like pouring cola into dough, the more you knead it the less you can tell things apart. Fortunately this score was only used in the backend and didn't affect ranking. Our fix was to use a stronger LLM Large AI models like ChatGPT and Claude that can read, write, and understand language. (Claude Sonnet) to re-label an initial 5,000 items, at a cost of about 1 US dollar, which cut the false-kill rate to one-tenth of what it was.

Ring detection: catching the behavior of a whole cluster of accounts

There's a kind of spam the model scores very low (one “Kaohsiung translation agency” scored just 0.09) that is actually a whole cluster of accounts flooding the site with the same template. Ring A group of accounts run by one person or team that flood the site with the same template at scale. The source code calls this a Ring. detection targets exactly this kind of behavioral pattern: the same fingerprint, lots of new accounts, concentrated in a short window. A key step is merged Traditional–Simplified matching: searching “披著…” in Traditional returns 0 hits, but fold it into Simplified “披着…” and search again, and it instantly matches 28 accounts. On top of the spam model, we further designed batch-freezing of ring accounts, easing the load on moderators and improving the experience for normal users.

There's also a cheap and effective fix

For short-form spam, the team didn't rely on the model alone: the “Chit-chat” channel switched to a community design using an opt-in Closed by default — anyone who wants to join has to actively apply and pass review. approach, which cleared the spam to zero within half a month — “cheaper and more effective than the model” — though the downside is that normal users have to actively apply before they can join.

Further reading: how the model learns, how it poisons itself, and how it's detoxified