OfficeBuddy

Most document agents write bytes into a file and hope. OfficeBuddy renders the document through Microsoft Word and Excel themselves after every single edit, diffs the pages, and hands the screenshot to a separate multimodal verifier that has to sign off before the next step begins. When the verifier objects, it says which page, which element, what is wrong — so the repair is targeted, not a blind retry.

The renderer is not a lookalike engine. It is Word.

Proof, not promises

Nothing below is a mockup. This is a real run on a real .docx, on a real Mac, rendered by real Word — instruction: "Make the title bold, 24pt and centered; then add a 2×3 table at the end with headers 任务/负责人/状态 and one data row, with black solid borders."

That red box is not decoration. It is drawn by a pixel diff against the last verified render, so the verifier is told exactly where to look instead of re-reading the whole page.

The complete evidence trail — the model's plan, every tool call, the rendered PDFs, and the verifier's structured verdicts — is checked into examples/harness-walkthrough/. Read it and grade the claim yourself.

Measured against another agent

Byte-level assertions and schema validation share a blind spot: a file can be structurally valid, pass every assertion, and still be visibly broken once a real layout engine renders it. Render-Truth Bench is 9 tasks built on exactly that gap — each pairs an ordinary edit instruction with a literal-execution trap, where doing what was asked, verbatim, yields a file that validates cleanly and renders wrong.

A 2×2 factorial over two systems × visual-feedback availability, 3 repeats per cell, 108 runs, every artefact rendered through real Microsoft Word and Excel:

DDR — defect delivery rate; share of runs whose delivered file carries the defect, decided by deterministic detectors with no model in the loop. SFR — silent failure rate; the file is broken and the user was never told. DAR — defect awareness rate. DAR* — DAR excluding the two tasks this project has already hardened against, so the number cannot be explained by a baked-in default.

Read the caveats before quoting any of this. The task set is deliberately aimed at the one dimension this architecture targets — it does not measure capability breadth, speed, or token cost, and on those the comparison runs the other way (see bench/REPORT.md). At n=3 per cell, differences of a few points are not distinguishable. The harness, the three-state calibration gate that every task had to pass, and the full report with rendered evidence are all in bench/ — including RESULTS.md and visual_report.html.

Why this is more than a retry wrapper

• The verifier is a separate, stateless call. It never sees the edit history or the model's own reasoning — only the screenshot and the step description. It cannot talk itself into believing an edit worked.
• The baseline ratchets. Each new render is diffed against the last render that passed, not merely the last render produced. A failed step cannot quietly become the new normal.
• Failure escalates instead of repeating. Errors are normalized into signatures; the same signature twice forces a different strategy, a third time asks you. Every step and every task has a hard budget ceiling.

Quick start

git clone https://github.com/richardChenzhihui/OfficeBuddy.git
cd OfficeBuddy
pip install -e .
export MINIMAX_API_KEY=...      # Anthropic-compatible endpoint, model MiniMax-M3
officebuddy doctor              # one-time automation-permission setup + self-check

Then just talk to it:

# one-shot, then drop into a REPL to keep going
officebuddy "把第一段改成 Times New Roman 12 号，并加粗标题" report.docx

# pure one-shot
officebuddy "add a totals row and bold it" sales.xlsx --one-shot

# interactive session
officebuddy

Useful flags:

Requirements: macOS, Microsoft Word / Excel (they are the renderer), Python 3.10+, and a MiniMax API key. office-agent still works as an alias of the officebuddy command.

What it can edit

Word — text editing and find/replace (paragraph- and run-level), character styling (font, size, bold/italic/underline, color, per-script CJK font slots), paragraph styling (alignment, indent, spacing), inserting and deleting elements (paragraphs, tables, page breaks), table and cell borders (tblBorders / tcBorders), and structure inspection.

Excel — cell read/write with type preservation, formulas, cell and range styling (font, fill, alignment, number format, borders), conditional selection (row[Salary>5000]-style predicates), row/column insertion and deletion, sheet management, freeze panes, charts, and a fidelity guard that inventories the workbook's parts before and after a save and reports exactly what the underlying reader would have dropped.

Safety, because it edits your actual files

• Your original is never written until you say so. All work happens on an isolated copy; the default output is <name>.edited.<ext>. Overwriting the original requires an interactive confirmation or --yes.
• Every change is snapshotted byte-for-byte with a persistent index — undo and restore work at any point.
• Document content is data, not instructions. Text read out of your files is never allowed to steer the agent (prompt-injection defense).
• No permission dialogs during normal use. Working copies live inside each Office app's own sandbox container, exports pre-delete their target, alerts are suppressed, and focus is never stolen. macOS asks for automation permission exactly once, and doctor walks you through it.
• The Excel fidelity guard warns you up front when a workbook contains parts the reader cannot round-trip (see Limitations).

Architecture

cli.py                  REPL / one-shot / doctor
agent/
  loop.py               main harness: plan → clarify → execute → render → verify → repair
  verifier.py           independent stateless visual verification (forced structured verdict)
  budget.py             error signatures + escalation ladder (retry → new strategy → ask)
  history.py            message history (images are kept out of the main loop's context)
tools/
  registry.py           pydantic models → tool schemas; uniform error envelope; auto-snapshot
  word_tools.py         word_edit_text / edit_style / insert_element / delete_element /
                        find_replace / read_content
  excel_tools.py        excel_write_cells / edit_formula / edit_style / conditional_select /
                        create_chart / manage_sheet / freeze_panes / fidelity_report / …
  interaction_tools.py  propose_plan / update_plan / ask_user / render_preview
render/
  applescript.py        Word/Excel → PDF (in-container, dialog-free, timeout + error classes)
  pdf_to_images.py      PDF → PNG (PyMuPDF, 144 dpi)
  page_diff.py          changed-page detection + red bounding-box annotation
  renderer.py           content-addressed render cache + verified-baseline ratchet
core/
  session.py            working-copy isolation (the original is touched only on explicit save)
  snapshot_manager.py   per-step byte snapshots + persistent index (undo / restore)
adapters/               stateless python-docx / o