Anti-Autoresearch 🛡️
·
🔬 The field has tolerated unreliable autoresearch long enough — Anti-Autoresearch is the read that finally catches it.
天下苦 autoresearch 久矣 —— Anti-Autoresearch 替研究者们一眼看穿不靠谱的工作。
🏆 Built on a battle-tested foundation: ARIS (~12.5k★ · HuggingFace Daily Papers #1 · 78+ skills across 7+ platforms). Anti-Autoresearch points ARIS's production audit DNA (experiment-audit · paper-claim-audit · citation-audit · kill-argument) outward — auditing a third party's submission instead of your own.
Autoresearch has gone mainstream, and a fast-growing share of what reaches the review pile is machine-generated — and a lot of it doesn't hold up: tables that don't match the text, baselines that aren't there, open-sourced code that won't reproduce its own paper. Reviewers, area chairs, and honest authors increasingly need to verify that, not just suspect it.
Regardless of who or what wrote a paper, does the science hold together and reflect its own evidence? Anti-Autoresearch audits a submission for self-consistency and fabrication, and produces a span-anchored, reviewer-ready report. It is not an opaque AI-text classifier (no authorship probabilities, no "AI-written" verdict) and does not judge misconduct — it surfaces discrepancies a human reviewer should investigate. Separately, it lists transparent, itemized AI writing-style impressions in a quarantined, zero-verdict-weight section (a paper can be integrity-
CLEANwhile listing many), because reviewers react to them.
📰 News
-
v0.5 (2026-06) — Added the AIS track (AI Writing-Style Impressions): 13 transparent, itemized writing-style signals (defensive hedging, LLM phrasing tics, clause-then-formula walls, bullet/bold spam, invented codenames, single-style figures, …) reported in a separate, zero-verdict-weight section — a paper can be integrity-
CLEAN_GIVEN_EVIDENCEwhile listing many. The 5 pure-style patterns moved out of family F into AIS. Taxonomy restructured to 46 integrity patterns (A–H) + 13 AIS + 2 advisory; new/ai-style-impressionsskill; the adjudicator now provably excludes zero-weight findings from the verdict (regression-tested). These are transparent impressions, never an authorship verdict — we are not an opaque AI-text classifier. -
v0.4 (2026-06) — Taxonomy v0.4: 51 hack-patterns across 8 families — A. Numeric self-consistency (数值自洽:表内·表文·增量算术对得上) · B. Method & scope (方法与范围:说的方法/范围≠实际做的) · C. Baseline integrity (baseline 诚信:对比基线缺失·偏弱·不公平) · D. Experiment integrity (实验诚信:假 GT·幽灵结果·代码≠数字,需代码) · E. Citation integrity (引用诚信:伪造·张冠李戴·撤稿) · F. Presentation & surface signals (表面信号:排版·文风·配图) · G. Proof & derivation integrity (证明诚信:漏证·循环论证·无效推导) · H. Evaluation design & validity (评测设计有效性:数据泄漏·LLM 裁判可信度·选择性报告, new). The deterministic eval gate grew 3→8 patterns (GRIM / GRIMMER / statcheck, plus a conservative defensive-hedge density screen); added CI, the
eval-design-forensicsskill, theHP-INVENTED-CODENAMEsurface pattern, and a prior-art acknowledgments section. Two more checkable self-consistency patterns —HP-ACRONYM-DRIFT(family B) andHP-UNDEFINED-NOTATION(family G) — were distilled from a "vibe-paper tells" thread while refusing its pure-stylometry items (we are not a vibe classifier). -
v0.1 (2026-06) — Initial release: reviewer-side integrity forensics for autoresearch / AI-Scientist papers. Ships the evidence ledger, deterministic adjudicator, and observability tiers. Not an AI-text detector.
🚀 Quickstart
Agent workflow (normal use)
Anti-Autoresearch runs as a Claude Code skill workflow — the Python tools are the deterministic spine inside that workflow, not the usual interface.
# 1) Install the skills + workflow (global, or pass a project's .claude/skills dir)
git clone https://github.com/wanshuiyin/Anti-Autoresearch.git
./Anti-Autoresearch/tools/install_anti_autoresearch.sh # → ~/.claude/skills
# project-local instead: ./Anti-Autoresearch/tools/install_anti_autoresearch.sh ./.claude/skills
# 2) Wire the cross-model reviewer (end state: Claude Code exposes mcp__codex__codex)
claude mcp add codex -- codex mcp-server
claude mcp list
# 3) Audit a paper
claude
> /anti-autoresearch ~/papers/submission
The run writes REPORT.md + report.json + claims.json + per-skill
*.findings.json into the paper directory. Put the code/result artifacts alongside
the paper to unlock L2 checks; PDF/source-only runs are observability-limited by
design.
Zero verdict weight — the AIS + advisory tracks (reported, never moves the verdict)
Three skills produce outputs that are reported but carry zero weight on the integrity
verdict — the non-integrity categories that round out a report: the AIS writing-style
track and the advisory memos. They matter to a human reviewer (a style impression, the
worst-case rejection paragraph, prior-art overlap), so the report shows them in their own
section — but the deterministic verdict stays driven only by the 46 integrity patterns. A
paper can be CLEAN_GIVEN_EVIDENCE while listing many. /anti-autoresearch runs them
automatically; to run one standalone, build the ledger first (next section) and invoke it like
any auditor.
| Skill | What it writes |
|---|---|
/ai-style-impressions |
(AIS · separate report section) AI writing-style impressions: defensive hedging, LLM phrasing tics, clause-then-formula walls, bullet/bold spam, invented codenames, single-style figures |
/adversarial-case-builder |
(memo, no verdict) the single strongest evidence-bound rejection paragraph a hostile reviewer would write |
/novelty-duplication-advisory |
(memo, no verdict) prior-work overlap: trivial-combination ("缝合 / stapling") and duplicate-publication candidates, laid out for a human to weigh |
Single-skill use
Every auditor is also a standalone skill — the installer drops all of them plus the workflow, so you can run just the axis you care about. They share one contract, so run it in order:
claude
# 1) Build the evidence ledger ONCE — the spine every auditor anchors to. Skip it and
# any auditor stops with: NO_LEDGER: claims.json not found. Run /evidence-ledger FIRST
> /evidence-ledger ~/papers/submission # → claims.json + observability level (L0/L1/L2)
# 2) Then run any auditor below against that ledger → <skill>.findings.json
The verdict-bearing auditors — each takes the paper dir, reads the ledger, and proposes span-anchored findings the deterministic adjudicator turns into the verdict (the zero-weight AIS + advisory skills are in the section above):
| Skill | What it catches |
|---|---|
/consistency-audit |
the paper against itself: inflated / mismatched numbers, method & scope drift, appendix-vs-body contradictions |
/citation-forensics |
citations: hallucinated references, and real papers cited for a claim they don't make |
/baseline-comparison-audit |
the missing / weak / mistuned baselines hiding behind a "SOTA" or "outperforms" claim |
/experiment-forensics |
(L2 — needs code+results) fake / derived ground truth, score self-normalization, phantom results, placeholder data, code output ≠ reported numbers |
/proof-derivation-forensics |
(L1 — needs LaTeX source) the written proof: skipped obligations, circularity, invalid steps, symbol drift, smuggled assumptions |
/eval-design-forensics |
the evaluation's validity: train/test leakage, a conflicted or unvalidated LLM-judge metric, selective reporting (dropped conditions / switched metrics) |
/presentation-signals |
(capped at minor → at most SOFT) checkable surface tells: duplicate tables, leftover pipeline/template strings, LLM-generated figures, page-padding — context, never a verdict |
A single skill only proposes span-anchored findings — it never returns a verdict.
To get one, feed the findings to the deterministic adjudicator (the
python3 tools/adjudicate_findings.py … --ledger … command in the next section); the
model never grades. Two more notes: consistency-audit, presentation-signals, and
ai-style-impressions also write a *.deterministic.findings.json (works with no
cross-model reviewer wired); and /anti-autoresearch runs every auditor above in one
shot, adding ingest
(arxiv-id / pdf → workdir + pdftotext), automatic observability, auto-selection of
which auditors apply, and the final cross-dimension verdict + REPORT.md.