Rules
Part of Content moderation policy: the parts worth your attention
Content moderation policy checklist: the items people skip
A content moderation policy checklist for a small community: what to decide before opening, the run list per decision, and the stop conditions worth honoring.
This is for a small operation. A forum, a comments section, a community server, a marketplace with reviews: somewhere a handful of people make decisions about other people's material, without a policy team and without tooling.
The failures at this scale are different from the ones at platform scale, and so is the fix.
What to take away
- Decide the outcomes before the rules. A team that can only remove or ignore will remove too much and ignore too much.
- Every decision needs a record with the rule cited and the date. Without it there is no consistency, only memory.
- The two rules you cannot skip are an appeal to someone who was not involved, and a written escalation route for anything involving risk of harm.
Before you open: eight decisions
- What is this space for.One sentence. Every hard call afterwards gets easier if this exists and harder if it does not.
- Which outcomes exist.Not just remove or leave. A workable set is: leave, edit or ask the author to edit, add a note, limit visibility, hold for review, remove, warn the account, restrict the account, remove the account.
- Who decides.Named people, not a role that nobody holds.
- Who decides an appeal.A different named person. This is the single most valuable structural choice available to a small team and it costs nothing.
- What gets escalated out of the ordinary process.Anything involving risk of harm to a person, anything involving a minor, anything that looks like a legal demand. These do not go through a normal queue and the route must exist before it is needed.
- What you keep.The item, the decision, the rule, the decider, the date. Decide the retention period now.
- What you tell the author.A category name, the rule, or the reasoning. Small operations can afford the third, which is their main advantage over large ones.
- Who reviews the rules, and when.
The run list for one decision
The run list
- Read the item in full, and read what it is replying to. Context collapse is the main cause of wrong calls, and at small scale you can afford the context.
- Name the rule you think it breaks. If you cannot name one, that is a finding: either the rule is missing or the item is fine.
- Ask what the author was doing. Quoting abuse to condemn it, documenting something, or joking looks identical to the thing itself at the level of the words.
- Pick the lightest outcome that solves the problem. Removal is the loudest option and often not the smallest sufficient one.
- Write the recorditem, rule, outcome, decider, date, one line of reasoning.
- Tell the author, including how to appeal.
Six steps, and at small volume it takes a couple of minutes. The step people drop is the record, and dropping it is what makes the operation inconsistent within a month.
Run list for one decision
- Read item and its context
- Name the rule it breaks
- Ask what the author was doing
- Pick the lightest outcome
- Write the record
- Tell the author and appeal route
Stop conditions
Stop and escalate where there is risk of harm to a person. This is not a moderation decision. Where someone appears to be in immediate danger, emergency services come first and the queue is irrelevant.
When to stop and escalate
Risk of harm to a person?
emergency services first, queue irrelevant
continue normal process
Stop where a decision is about you. Anyone with a personal stake in an item hands it to someone else. This is the rule that gets broken in small communities because the moderator is also a participant, and it is the one that destroys trust fastest.
Stop where the demand is legal. A formal complaint, a takedown demand, or anything asserting a legal right is not something to answer from a policy. Route it to whoever handles that for you, and do not reply substantively in the meantime.
Stop where you are deciding a fact you cannot check. Whether a claim is true is frequently not something a moderator can establish. Deciding what is verifiable is a different job from deciding what is permitted, and confusing them is how a moderation team ends up adjudicating a dispute it has no way to resolve.
The record fields
| Field | Why it is there |
|---|---|
| Item reference and a copy | The original may be edited or deleted |
| Rule cited, by number | Consistency is measurable only if rules are cited |
| Outcome | Which of your defined outcomes was applied |
| Decider | Accountability, and pattern detection |
| Date and time | The rule in force at the time is the operative one |
| One line of reasoning | The only field that helps the next decider |
| Appeal, if any, and who heard it | Shows the appeal is real |
Six months of this is worth more than any policy document, because it is the only way to find out whether two people applying your rules reach the same answer.
Record fields to keep
- Item reference and a copy
- Rule cited, by number
- Outcome applied
- Decider
- Date and time
- One line of reasoning
- Appeal and who heard it
What to do about tools
Very little, at first: small operations mostly need a queue and a record, both of which a spreadsheet does adequately.
Automated detection at small volume creates more work than it saves. Every flag still needs a human decision, and false positives are yours to sort. The categories of tooling and when they start earning their cost are covered separately in the discussion of what moderation tooling actually does.
The rules themselves
Keep them short, numbered, and each one testable: two people applying it to the same item should reach the same answer. A rule that fails that test is a value statement, which is useful in a preamble and useless in a decision.
Testable rule vs value statement
Testable rule
- Two deciders agree
- Yes
- Numbered
- Yes
- Use in a decision
- Directly
- Best home
- Rule list
Value statement
- Two deciders agree
- No
- Numbered
- Rarely
- Use in a decision
- Useless
- Best home
- Preamble
Drafting vague prohibitions is not a neutral choice. One legal system has a whole doctrine about rules too unclear to be applied fairly, described at an explanation of the vagueness doctrine; the reasoning behind it holds for private rules too, even where nothing legal is engaged.
And the reason to tell someone what rule they broke, in terms specific enough to answer, is the same reason set out in a general explanation of notice: a person cannot respond to a decision they cannot understand.
Where this connects to everything else
Rules that apply to people in an organization, rather than to strangers in a community, are a different problem with different constraints: that is the subject of guidelines people actually use.
Rules in a school setting are different again, because a separate safeguarding structure overrides the ordinary sequence, which is covered in which document governs which incident. And where your community sits on somebody else's service, their rules apply to you as well as yours, which is the subject of what the agreement lets a service do.
Common questions
How many rules is the right number?
Few enough that a moderator can hold them in mind. If you cannot list them from memory, neither can the people you are asking to follow them.
Should we publish the reasoning for decisions?
Publish the rule and the outcome. Reasoning to the author privately is usually better than in public, since a public explanation turns every decision into a debate and discourages moderators from deciding at all.
What do we do when the moderators disagree?
Record the disagreement and use it to fix the rule. Persistent disagreement about the same rule means the rule is not testable, and no amount of discussion about individual items will fix that.
Is one moderator enough?
For the decisions, sometimes. For appeals, no: a system where the same person decides and reviews is not an appeal, and everyone involved knows it.
