Process steps
| Step | Meaning |
|---|
| intake | A review case can come from self-application, member nomination, target-requested repeat review, or admin-triggered review. |
| eligibility | Self-owned Apply cases and nominations that open review cases pass eligibility before review opens. |
| reviewer sample | Each reviewer receives random subsamples in two streams: Reviews and Reports. Reports are separate report cases, not review-case intake. In Reviews, self-applications, member nominations, target-requested repeat review, and admin-triggered review are sampled together. Samples stay random. Rating is not review evidence. |
| vote | Reviewers and admins can submit one of three review votes or skip their assigned case. Reviewer/admin nomination automatically submits a clear vote on the nominated case. |
| result | A review decision closes as clear, uncertain, or cheater. |
Eligibility:
- Clearance review: at least 30 rated Codeforces contests, no skipped contests, and no external cheater database match. External matches create external-match cheater status and are not reviewed as report cases.
- Reviewer applications: clear member, historical max rating at least 1600, and either at least 50 rated Codeforces contests or regular Codeforces round problemsetting. Admins can appoint clear users directly.
Decision rules are summarized below. Admins can override any review decision. Any decision source and status are public. More on it in Statuses and roles.
Review votes
| Vote | Meaning |
|---|
| clear | Reviewer supports clearance for ranking inclusion. |
| not enough evidence | Reviewer does not support clearance or a cheater result under current evidence. |
| cheater | Reviewer supports a project cheating finding. |
Review details
- A not enough evidence vote means the case lacks enough reliable evidence to support clearance or a cheating finding.
- Reviewer assignments use random visible subsets of open cases to decorrelate votes.
- Reviewers and admins can skip assigned reviews before voting, or convert a recent completed vote to a skip while the case remains open. A skip leaves no verdict on the case. Reviewer-pool exhaustion does not close a case by itself.
- Reviewers can add shared evidence for facts, links, observations, caveats, and counter-evidence. Private vote notes remain visible only to their author and admins.
- Reports are reviewed as report cases. Users whose last decision was uncertain or cheater can reappeal once through Apply.
Nominations
| Rule | Meaning |
|---|
| who can nominate | Members, reviewers, and admins can nominate unprocessed eligible Codeforces handles they believe are clear enough for the ranking. |
| review pool | Nominated cases enter the same Reviews stream as self-applications, repeat reviews, and admin-triggered reviews. |
| who can vote | Reviewer or admin nominations automatically cast clear votes. Member nominations from non-reviewers do not vote. |
| nominator identity | Reviewers can see that a case came from member nomination, but not who nominated it. Admins can see the nominator. |
Review decision thresholds
| Outcome | Public threshold |
|---|
| clear | At least 4 submitted clear votes. A reviewer/admin nomination clear vote counts toward this number and does not use one of the 5 ordinary review slots. |
| uncertain | 5 ordinary submitted reviews without 4 clear votes, 4 cheater votes, or the 3-cheater admin-escalation trigger. It can also close earlier when no final outcome remains possible. |
| cheater | At least 4 submitted cheater votes. If 5 ordinary submitted reviews include exactly 3 cheater votes, admins review the case before any public cheater status. |
Review visibility
| Role | Can see | Cannot see |
|---|
| admins | Final status, decision source, votes, voter identities, notes, shared reviewer evidence, applicant notes, report details, admin notes, and admin reasons. | none |
| reviewers | Final status, decision source, assigned-case handles and context, assigned report evidence, assigned shared reviewer evidence, assigned admin notes, and their own private vote notes. | Vote counts, other reviewers' identities or notes, reporter details, report counts, and admin reasons. |
| other users | Final status and decision source. | Votes, notes, private case context, report details, and admin reasons. |