DraftBench.

§ 1The Problem

Drafting is the highest-leverage step in the patent lifecycle. A weak specification can’t be fixed by a strong prosecution. AI tools that draft specifications and claim sets are entering the market without a reproducible measure of whether their output survives prosecution.

Vendors today benchmark drafting on internal datasets with internal rubrics. There is no neutral way for an in-house IP team or a law-firm partner to compare how DeepIP, Solve Intelligence, IP Author, Rowan, or a frontier LLM perform on the drafting task as it actually plays out in front of a USPTO examiner.

DraftBench fills that gap. It scores AI-drafted specifications and claim sets against (a) the prosecution outcome of the same or analog application and (b) a panel of registered practitioners.

§ 2Domains

DraftBench evaluates four drafting domains. Domain weights apply to the merged composite; per-track scores publish separately per § 4.

§ 3Difficulty Tiers

Five tiers calibrated to the drafting complexity human practitioners encounter. Tier counts in v2026.1 are provisional pending CBlindspot’s test-set release.

Human-baseline figures are calibration estimates from CBlindspot’s practitioner panel; final figures publish with v1.0.

§ 4Four-Layer Scoring

DraftBench combines four scoring layers. Composite score equals the weighted sum, except where a hard floor applies (§ 7).

§ 5Evaluation Rubrics

Two rubrics govern Track B scoring. Both publish with v1.0; weights below are provisional.

Specification Quality Rubric

• Enablement — can a person of ordinary skill make and use the invention from the spec? (1.5× weight)
• Written description — does the spec demonstrate possession of the claimed subject matter? (1.5×)
• Best mode — is the inventor’s preferred embodiment disclosed? (1.0×)
• Definiteness foundation — does the spec define terms used in the claims with sufficient precision? (1.0×)

Claim Quality Rubric

• Scope ladder — does the dependent-claim structure provide a defensible fall-back position? (2.0×)
• Breadth vs. novelty — are the claims as broad as the disclosure permits without reading on the prior art? (1.5×)
• Antecedent basis — is every term properly introduced? (1.0×)
• Restriction resilience — do the claim sets anticipate likely restriction practice? (1.5×)
• Professional quality — readability, consistency, style. (0.5×)

§ 6Reproducibility (Glass Box)

Per Charter § 8.3, every published DraftBench run is reproducible by third parties within a 5% scoring delta. Five pillars apply:

1. Test set publication. The full test set publishes one release cycle after initial scoring (to prevent training contamination). Held-out sets are documented.
2. Rubric transparency. Both rubrics are public, versioned, and annotated with calibration examples.
3. Output availability. Vendor outputs (with consent) and panel scores publish under Apache-2.0. Successes and failures both published.
4. Failure-mode analysis. Each scored run includes a failure-mode taxonomy (fabricated citation, missing antecedent basis, overbroad claim, missing best mode, etc.).
5. Continuous reporting. Public leaderboard updates monthly. Methodology revisions go through Methodology Council review per Charter § 7.2.

§ 7Hallucination Floor (Therasense Standard)

Any cited US patent or publication number in an AI-drafted output (IDS, spec background, claim-construction-support argument) must verify against an authoritative source — USPTO PatentsView API, USPTO Patent Public Search, or the relevant jurisdictional equivalent (EPO Open Patent Services, J-PlatPat, CNIPA Patent Search). Citations that fail verification trigger a hard floor: composite equals 0.0.

This is not a deduction. A draft with one fabricated citation does not earn 95%; it earns 0%. Catastrophic conduct is not averaged into a passing grade.

The Therasense Standard is non-negotiable for OIPC accreditation. Maintainers may not relax it.

§ 8Data Repository

The DraftBench repository will publish at github.com/cblindspot/draftbench at OIPC accreditation. Planned contents:

• METHODOLOGY.md — this document, versioned.
• test_set.jsonl — invention disclosures (input) and target outcomes (Track A ground truth).
• rubrics/spec_quality.json, rubrics/claim_quality.json — full rubric specifications.
• panel_scores/ — per-release panel ratings (anonymized practitioner ID, rubric breakdown, free-text rationale).
• vendor_outputs/ — vendor-submitted outputs with their consent. Apache-2.0.
• harness/ — the evaluation harness (Track A scoring against USPTO outcomes, citation-verification client, panel-scoring intake).
• leaderboard.md — auto-generated rolling results.

§ 9Governance

DraftBench is owned by CBlindspot. Per Charter § 2 principle 2, the Open IP Council does not own this benchmark; it accredits it. Methodology decisions sit with CBlindspot and any future co-maintainers; methodology disputes between maintainers go to the OIPC Methodology Council per Charter § 7.2.

No single AI vendor, law firm, or platform holds majority influence over DraftBench’s test set, rubrics, or panel composition. CBlindspot publishes its conflict-of-interest disclosures per Charter § 8.2.

Annual reaccreditation per Charter § 6: CBlindspot publishes a conformance report each calendar year. Failure to publish triggers a 90-day cure period before de-accreditation.

§ 10Get Involved

• Vendors — submit your tool for evaluation. Outputs publish under Apache-2.0 with your written consent.
• Practitioners — apply to the Track B panel. USPTO registration and 5+ years drafting experience required.
• Buyers — cite DraftBench accreditation as a procurement requirement in your AI-tool RFPs.
• Academics — the Methodology Council welcomes peer-review papers analyzing DraftBench results, calibration data, and rubric construct validity.
• Public comment — methodology objections via the OIPC public-comment process. Subject line should include DRAFTBENCH-METHODOLOGY-v0.1.