Sell-readiness
We publish the thresholds the tactic corpus must meet, and where it stands against them, so you see the corpus as it is rather than a marketing claim. A script checks these numbers; nobody rounds them up.
| Criterion | Threshold at first sale | Six months later | Current |
|---|---|---|---|
| Published tactics | ≥ 30, each with ≥ 2 evidence sources | ≥ 60 | not yet measured |
| Approved examples per published tactic | ≥ 5 with crops | ≥ 15 | not yet measured |
| Corpus sites captured and annotated | ≥ 100 | ≥ 300 | not yet measured |
| Golden-set precision per published tactic | ≥ 0.85 | ≥ 0.9 | not yet measured |
| Two-pass agreement (rolling 200) | ≥ 0.8 | ≥ 0.85 | not yet measured |
| Audit p95 wall time | ≤ 3 min | ≤ 2 min | not yet measured |
| Tactics graded tested or better | ≥ 10 | ≥ 25 | not yet measured |
| Tactics graded measured | 0 allowed, disclosed | ≥ 5 | not yet measured |
Current values: not yet measured. They are filled in by the readiness check once it runs against the corpus.
What the criteria mean
- Published tactics counts tactics that passed the publish rule. Drafts do not count.
- Approved examples are cropped screenshots of the tactic on real sites that passed annotation review.
- Golden-set precision is how often the system is right when it says a tactic is present, checked against hand-labelled crops.
- Two-pass agreement is Cohen's kappa between two independent annotation passes (different model and prompt) over the last 200 observations, with positive agreement (agreement on the cases where either pass saw the tactic) reported alongside. It is never raw agreement: most crops show neither pass finding a given tactic, and those both-absent pairs push raw agreement up whatever the passes actually do (in our pilot, raw 0.974 against kappa 0.838).
- Audit p95 wall time is the time within which 95 in 100 page audits finish.
- Tested or better and measured count tactics by evidence grade.
What happens when a tactic slips
Any tactic whose golden-set precision falls below 0.85, or whose two-pass kappa falls below 0.75 over its last 200 observations, is demoted to draft automatically and stops being returned to agents. A figure counts only once there is enough data behind it: at least 20 labelled presences for precision, and at least 50 two-pass pairs including 5 where a pass saw the tactic for kappa. Below that, the figure is reported as not measured.
Every tool response carries tactic_count and corpus_version, so your agent always sees the current size of the corpus.