name: hygieia description: > 语义层/本体体检与养护（audit / eval / rot / report）。当用户用中文要求「审计我们的语义层」 「检查我们的本体质量」「为什么我们的数据 agent 给错数字」「评估我们的 text-to-SQL 准确率」 「压力测试我们的数据模型」「找出项目和本体之间的漂移」时使用；也响应英文提问 "audit our semantic layer"、"ontology health check"、"why did the data agent return a wrong number"、 "evaluate our text-to-SQL accuracy"、"stress-test our data model"、"drift between the project and the ontology"。纯本地离线运行，Python 3.10+，运行时依赖 pyyaml + sqlglot（AST 引用抽取）。

Hygieia

Keep an existing semantic layer healthy: check it, evaluate it, stress-test it.

A fully offline semantic-layer health-check skill. Given a fixture directory (semantic/metrics.yml required; schema.sql required only for eval/rot; questions.yaml + seed.sql for eval), run the four main commands (audit/eval/rot/report) plus the read-only adapters (adapt dbt, adapt ossie) and impact per the operating contract below. Hygieia does not bootstrap ontologies: the "first build" half can be served by third-party bootstrap flows (e.g., the ontology-skills bootstrap), and their output can be handed to Hygieia as a downstream quality gate. Binding details (metrics.yml schema, severity, scoring protocol, rot threat model) live in docs/contract.zh.md (Chinese, authoritative) and docs/contract.en.md (English mirror); the essentials are inlined below so this skill stays self-contained.

Triggers

Chinese: 「审计我们的语义层」「检查我们的本体质量」「为什么我们的数据 agent 给错数字」「评估我们的 text-to-SQL 准确率」「压力测试我们的数据模型」「找出项目和本体之间的漂移」。 English: "audit our semantic layer", "ontology health check", "why did the data agent return a wrong number", "evaluate our text-to-SQL accuracy", "stress-test our data model", "drift between the project and the ontology".

Operating contract

1. First confirm the fixture directory exists and contains semantic/metrics.yml; schema.sql is optional for audit (without it, R4 and the table/column checks of R3 are skipped and listed in skipped_rules), while eval/rot need schema.sql + seed.sql and eval needs questions.yaml. If a required file is missing, say so explicitly. Do not guess.
2. Suggested order: audit → rot → eval → report (report calls audit/rot/eval in sequence).
3. audit: static diagnosis of 9 rule classes (R1 synonym conflict / R2 orphan metric / R3 join ambiguity / R4 dangling reference / R5 missing evidence / R6 missing description / R7 redundant description / R8 stale alias pointing at a deprecated metric / R9 one metric merging sources with conflicting freshness). Every finding carries three elements: rule, danger, fix. Severity mapping: high = R1, R3, R4, R8, R9 (and R2 when the domain does not exist): these can directly cause wrong numbers; medium = R5, and R2 with an existing domain but no table binding; low = R6, R7. In metrics.yml, the join-condition field is on_expr. YAML 1.1 parses a bare on as the boolean key True, silently dropping the field; R3/R4 read on_expr exclusively.
4. eval: two-arm controlled evaluation. By default it reads fixtures/<dir>/demo_results.json. When the top is marked simulated: true, the answers are hand-authored, but the scoring logic still runs for real per the protocol below. For a live evaluation, store artifacts under fixtures/<dir>/runs/<timestamp>/ (meta.json + control.json + treatment.json) and point --results at them (see docs/eval-live.md). Every eval writes reports/manifest.json (model, prompt/questions/corpus/semantic-layer hashes, version); report marks runs not directly comparable when those differ, and scores entity (tables/joins) and relation (caliber) separately. The core metric is the silent-error rate (a number returned with no error and no clarification, so the user cannot tell). Scoring labels are mutually exclusive, judged top-down: clarify (explicit refusal to fabricate, or expected_behavior: clarify) → abstain (explicit "don't know", no business number) → honest_error (SQL/number given but execution failed) → silent_error (plausible number/yes-no, mismatches gold, no clarification) → correct (matches gold). For expected_behavior: clarify, only an explicit keyword hit counts as clarify/abstain and may count correct. The scorer matches these strings verbatim: Chinese markers (数据不足 / 无法计算 / 需澄清 / 无法回答 …) or the English ones (unknown / cannot compute / insufficient data …) selected by answer-language detection; empty or filler replies (「嗯」「好的」「ok」) are non-compliant abstain; a concrete number or a digit in raw is silent_error. Accuracy = correct/total; silent-error rate = silent_error/total; honest_error/abstain count in neither. Scoring keywords are coupled to the answer language: v0 ships a Chinese marker table and an English one, selected automatically by answer-language detection (any CJK in the answers → Chinese table); the demo fixture stays Chinese.
5. rot: injects five kinds of drift into an in-memory copy — structural: column rename orders.total_amount → order_amount, definition tampering refund_total.expr gains refunds.tax, a new orphan metric gross_margin (missing domain, no evidence); semantic: a stale alias pointing at a deprecated metric, and one metric merging two sources with conflicting freshness — re-runs audit, and reports new captures = |findings_after − findings_before| (set difference keyed by rule+subject+detail). Never writes to disk unless --write (then metrics.rot.yml).
6. report: writes reports/health-report.md with three blocks: high/medium findings, rot simulation, controlled evaluation.
7. CI usage: --fail-on high exits 1 when high-severity findings exist.
8. Output language: --lang zh|en (default en). The two variants are written independently for their own context. Use --lang zh when the conversation is in Chinese so the whole output (headers, severity labels, finding texts, report) is Chinese, not a translation of the English one. This only affects display: the scoring protocol and the fixture content are unchanged.
9. Beyond the four main commands: adapt dbt and adapt ossie (read-only adapters mapping dbt/MetricFlow or Apache Ossie/OSI YAML onto metrics.yml; Ossie is the neutral first-class input — see docs/adapters.md), and impact <fixture_dir> --diff <file|-> (git diff → affected metrics, see docs/impact.md). Reference extraction is AST-based (sqlglot, sqlite dialect); an unparseable expression is surfaced as a high R4 finding, never silently ignored.
10. When reporting numbers, distinguish "real execution" (gold/candidate SQL run against the seed) from "simulated answers"; never present simulated data as a live evaluation.

Usage

hygieia audit  fixtures/ecom
hygieia eval   fixtures/ecom
hygieia rot    fixtures/ecom
hygieia report fixtures/ecom

# python scripts/hygieia.py ... still works as a thin compatibility shim.

Self-test: python -m unittest discover -s tests -v (asserts the contract numbers: 65%/100%, silent rates 35%/0%, audit 7 findings, rot 6 new captures = structural 4 + semantic 2). CI/agent wiring: docs/integrate-ci.md, docs/integrate-agent.md; SARIF via audit --sarif.

English · 简体中文

Hygieia

Inspect, evaluate, and stress-test an existing semantic layer. Runs fully offline.

On Apache Ossie's public TPC-DS model, three everyday words — price, location, profit — each point at two different definitions. Hygieia reports all three before a single query runs.

Install & quick start · Public-layer evidence · Demo numbers · How it works · Known limitations · Documentation · CI · Agent

pip install "hygieia-sl @ git+https://github.com/chiang21fcb/hygieia.git" && hygieia audit fixtures/ecom

Hygieia is a Python command-line tool that runs locally and inspects an existing semantic layer: static diagnosis (audit), two-arm evaluation (eval), rot simulation (rot), and a Markdown health report (report). It does not build an ontology. It checks the one you already have and reports where the definitions no longer match the data.

• Python 3.10+, runtime dependencies pyyaml + sqlglot (AST-level SQL parsing); no network, no accounts, no API keys
• Ships as a coding-agent skill

Install & quick start

Install the distribution (the import package is hygieia; the console script is hygieia):

pip install "hygieia-sl @ git+https://github.com/chiang21fcb/hygieia.git"
# or from a checkout:
pip install -e .

hygieia audit  fixtures/ecom   # static diagnosis
hygieia eval   fixtures/ecom   # two-arm evaluation
hygieia rot    fixtures/ecom   # rot simulation
hygieia report fixtures/ecom   # full health report

One-shot run without installing (uv):

uvx --from git+https://github.com/chiang21fcb/hygieia.git hygieia audit fixtures/ecom

python scripts/hygieia.py ... still works as a thin compatibility wrapper around the same package, but new docs use hygieia. After report, open fixtures/ecom/reports/health-report.md for the complete result. eval prints:

$ hygieia eval fixtures/ecom
Arm                              Correct  Silent  Accuracy  Silent rate
control (no semantic layer)      13/20    7/20    65.0%     35.0%
treatment (with semantic layer)  20/20    0/20    100.0%    0.0%

(The CLI prints English by default. --lang zh switches to the Chinese variant, which is written separately, not translated. The fixture mirrors a Chinese data-warehouse scenario; the protocol is language-neutral.)

Then run the self-tests to confirm the contract numbers still hold:

python -m unittest discover -s tests -v

Useful flags: --results <json-or-run-dir> to evaluate a different answer file or a runs/<id> directory (see docs/eval-live.md), --json for machine-readable output, --fail-on high|medium|low|none to gate CI on severity, --sarif <path|-> to emit SARIF 2.1.0 for code scanning, --lang zh|en for the output language (default en), --write to let rot persist the drifted semantic layer as metrics.rot.yml.

Public-layer evidence

Before claiming anything about a private warehouse, Hygieia is run on a public neutral layer: Apache Ossie's TPC-DS semantic model. The fixture is a trimmed copy of examples/tpcds_semantic_model.yaml with the upstream Apache-2.0 header kept; nothing in it is doctored, and no schema.sql is needed:

hygieia adapt ossie --in fixtures/ossie_tpcds/ossie_semantic_model.yaml --out .tmp/ossie
hygieia audit .tmp/ossie
# 🔴 High R1-synonym-conflict [location]: synonym "location" maps to multiple metrics/fields: ss_store_sk, s_city
# 🔴 High R1-synonym-conflict [price]:    synonym "price" maps to multiple metrics/fields: ss_sales_price, i_current_price
# 🔴 High R1-synonym-conflict [profit]:   synonym "profit" maps to multiple metrics/fields: total_profit, ss_net_profit

Three real vocabulary collisions in a reference model published by the standard's own authors. audit also reports 4 × R5 (the model declares no evidence) and lists R4-dangling-ref plus the table/column checks of R3 in skipped_rules, because the sample ships no DDL — reported, not hidden. Reproduction commands, dates, and limits: docs/public-layers.md.

Demo numbers (simulated)

The numbers in this section are a simulated demo, not a live model run: the two arms' answers are hand-written, and only the SQL scoring is executed for real. They show the shape of the failure mode, not a benchmark of any particular model.

A data agent gets its metrics, synonyms, joins, domains, and evidence from the semantic layer, so the definition lives in one place. When a definition drifts, the agent still returns a number that looks fine. The bundled ecom fixture reproduces this: 6 e-commerce tables, 20 Chinese questions, hand-written answers for both arms, and real SQL execution for scoring.

A silent error is a wrong number with no error and no clarification, so the user cannot tell. This is the core metric of eval.

Each answer is also scored on two dimensions: entity (tables / join paths) and relation (metric caliber: aggregation + filters). On the demo, control scores entity 17/20 and relation 13/20, treatment 20/20 on both: the failures are mostly caliber, not table-picking. eval writes a manifest (reports/manifest.json) pinning model, prompt hash, question-set hash, corpus hash, semantic-layer hash, and tool version; report refuses to call two runs comparable when those differ, and flags a dimension that drops while the overall score rises:

Representative failures from the control arm:

audit finds 7 issues in the same semantic layer (3 high: R1 synonym conflict, R3 join ambiguity, R4 dangling reference; 4 medium: R5 missing evidence). rot injects five kinds of drift (column rename, definition tampering, new orphan metric, stale alias, conflicting source freshness) and re-runs audit: 6 new captures, split into structural 4 (two R4 from the column rename, plus R2 high and R5 medium from the new orphan metric) and semantic 2 (the stale alias, and the metric merging two sources with conflicting freshness). Definition tampering hits an R4 that already existed before the drift (refund_total referencing the missing refunds table), so by the set-difference rule it adds none.

About this demo's data: the two arms' answers are hand-authored and marked simulated: true at the top of demo_results.json; they were not produced by a live model. The gold-standard SQL numbers come from real execution against the seed data in in-memory SQLite, and the scoring follows Section 13 of the contract. net_revenue's expression sum(orders.net_amount) executes on the seed and matches the treatment answers. The R4 trap lives on a separate refund_total metric and is not on the evaluation's gold path.

Live run (2026-11-06)

One real two-arm run. The built-in coding agent (deepseek-v4-flash) answered as the model; scoring used the same eval protocol.

The semantic layer fixed three caliber errors on q08, q10, and q12: total revenue, revenue, and average order value now follow the non-cancelled scope. One shared error remained on q14, where both arms counted cancelled orders in per-customer spend; at run time the layer had no customer-dimension metric. A customer_spend metric (per customer, non-cancelled orders, sum(orders.net_amount)) was added afterwards and encodes the caliber q14 asks for, but the live answers above were not re-run under it (see the run's notes.md). Full methodology, meta, and per-question analysis: fixtures/ecom/runs/live-20261106-dsh. The demo table above stays simulated; live runs follow docs/eval-live.md.

How it works

Commands

Four main commands: