Theme · False positives in spam-detection models and the ethics of the threshold

How do innocent users get hurt?

Any system that “scores and sets a threshold” makes its decisions the same way: it blocks spam but also wrongly removes legitimate posts. This method can never be perfect, so the fault does not lie with sloppy engineering.

False positive	Legitimate content is misjudged as spam → it wrongly harms normal users; this is the price paid for freedom of expression
False negative	Spam is not caught and slips through → this is the price paid for the quality of the public space

Now you try

Drag toward the stricter end and spam is blocked more cleanly, but legitimate content starts getting wrongly removed; drag toward the looser end and wrongful removals fall, but spam slips back in. The ideal spot lands in the middle — and real-world content is far more complex than these 22 kinds of posts.

Why the rarer the good content, the more easily it is wrongly removed

There is a common statistical base-rate The base-rate trap: the share a thing already occupies within the whole. When spam makes up an overwhelming proportion and the good content worth protecting is tiny, even the most accurate filtering easily harms that small handful. phenomenon: if spam is inherently rare, then even a very accurate filter will — because legitimate content is so plentiful — wrongly flag quite a lot of content that is actually fine. Matters is in exactly the opposite, and trickier, situation: 70–80% of the platform is spam, so the model's absolute wrongful-removal rate is in fact very low (falling from 3.1% all the way to 0.13%); but the civic, political, and investigative content that truly needs protecting makes up only about 1%. The trouble is that when the model “amplifies its errors,” it tends to misjudge precisely that 1% as spam. Even with a wrongful-removal rate as low as 0.13%, the small handful that gets wrongly removed happens to be the very 1% that is the platform's whole reason for existing.

When in doubt, better to let one piece of spam slip by than to wrongly remove a precious article.

The engineering response: don't pass judgment when you're not sure

In the new generation of the model we added one more layer, a conformal Lets the model hold back from forcing a call when it isn't sure, answering “uncertain — hand it to a human” instead of classifying reluctantly. It can also use statistics to guarantee that the “probability of wrongly blocking legitimate content” stays below a set ceiling. mechanism, which returns one of three outcomes for each post — pass / block / send for human review — so whatever it isn't sure about is not forced through but handed back to a real person to judge. Thanks to this step, the article model's actual wrongful-removal rate was pushed from 24% down to 1.6%. And human judgment, in turn, should also follow clear standards.

What to do when wrongly removed: appeals and remedies

However you tune the threshold, wrongful judgments never drop to zero. So the real crux, beyond “removing a little less,” is making sure the wrongly removed can be brought back — the very thing many large social platforms are criticized for, because they offer no such transparent, trustworthy service. A trustworthy remedy mechanism has to meet at least three conditions: a transparent process (users can see how they were handled, and why), appeals that are simple to use (the channel is findable and easy to file), and remedies that actually work (appeals are genuinely reviewed, and wrongful judgments are genuinely reversed).

Platform governance needs two design choices for the harm of wrongful removal to shrink. First, a measure at the article level only lowers visibility, it does not delete: a suspect article no longer appears in recommendations and channels, but its content, page, and original links are all kept, so a wrongful judgment is recoverable — you just put the visibility back, nothing has to be conjured out of thin air. Second, the new generation of detection adds the “pass / block / send for human review” layer, so whatever it isn't sure about is not judged but handed to a human, driving down at the source the volume of cases that would end up as appeals.

The “due process” flow will not be the same for different measures. The most complete is the neighborhood-watch team's “clean-up”: every removal has a public record, an appeal status, and admin review (the verdict split into uphold, overturn, or adjust the reason), and anyone can look up “who, at what time, on what grounds, removed what.” This is because “users” are granted part of an admin's powers, and we use transparent records to practice public oversight of one another. By contrast, the article-level automatic noise-reduction is mostly silent, and users often do not know they have been acted on. The reason the site keeps stressing “transparency” is precisely to make the algorithms, the training data, and the wrongly-removed cases public, so that appeals and remedies have something to stand on.

When in doubt, better to let one piece of spam slip by than to wrongly remove a precious article.

This standard applies not only inward but outward too — it can even let the platform protect itself. During the 2025 Lunar New Year, a flood of spam into Matters triggered Taiwanese ISP The telecom / network company that gives you your internet connection, such as Chunghwa Telecom. ' fraud-blocking process in the Domain Name System (DNS) (carried out by the Taiwan Network Information Center (TWNIC) via rewritten domain queries (DNS-RPZ)), and for three days users across Taiwan could not reach the platform. In this case, the platform itself became the victim that was “misjudged and cut off.” The platform demands transparent process, usable appeals, and effective remedies of itself toward its users; when it is the one being acted on by others, it deserves the same treatment. Whether a remedy mechanism is trustworthy is tested exactly in this situation where “the governor is at the same time the governed.”

The threshold should not be an engineer's call alone

Where that line should be drawn is, in the end, a value trade-off; technology can only tell you the consequences, it cannot decide the answer. That is why this project holds a symposium, taking “whether the threshold should lean toward wrongly removing legitimate content or toward tolerating spam” as an important question to discuss together.

Further reading: how the model learns, how it gets poisoned, and how it's rescued