digline
dev.digline/digline-mcp · 10 stars · Python · Apache-2.0
MCP server Read, measure and explain LLM eval runs of a digline suite. Cannot promote a baseline.
Install
The repo has no one-line install. Follow its README.
Published as
- PyPI
digline-mcpstdio
From the server's entry in the official MCP Registry.
Files
digline-mcp
The MCP server for digline: it lets a coding agent read a digline result and measure a new one, and it does not let the agent promote a baseline.
Not because promotion is refused. Because there is no such tool.
The thirteen tools
| tool | what it does |
|---|---|
list_runs |
every stored run of a suite, newest first, with the baseline marked |
get_run |
one stored run — the verdicts, never the payload |
get_baseline |
the approved reference, same shape |
log |
which model answered, read down the stored runs; never a gate |
compare |
a run against the baseline: did it get worse? |
diff |
two runs, neither of them a reference: should I switch? |
explain |
one run read back as typed facts, with the exit code |
run |
execute the suite, with the call count acknowledged first |
where |
where the project is, and the command for the next step |
record_status |
whether each tenant records, and how much |
record_listing |
a suite's recorded answers, counted by the shape of the output; never a yes or a no |
record_compare |
a suite's recorded answers by case and by setup, each by its number in the answer |
suggest |
a first suite, proposed as facts, with the command that writes it; writes nothing, asks no model |
promote is absent by construction, and so are migrate, view,
report, a record import that writes a yes or a no, and record on and
off. A baseline is an approved reference, and promote writes into
.digline/<tenant>/baselines/, which is committed — anything landing there
arrives in somebody's diff and must arrive because they put it there. A refusal
would be a conversation an agent could argue with; an absence is not.
Every tool, the reading ones included, loads the suite it is named. For a
suite.py that executes the project's code, in the server's environment.
The reasoning is ADR 0011, and the
judgement layer it turns into surface is AGENTS.md.
Install and configure
uv pip install digline-mcp
One server serves one repository: the perimeter is the repo, and multi-project is N named servers rather than a registry.
{
"mcpServers": {
"digline-northwind": {
"command": "digline-mcp",
"args": ["--root", "/Users/you/src/northwind", "--names", "clear"]
}
}
}
--tenant and --env are accepted and verify; they never override. The
suite decides, as it does everywhere else in digline.
--names is required, with no default: clear lets the store's names leave
as they are, and project projects every answer, so the names leave as tokens
from the name table that --name-table module:attribute names; a refusal is
not projected yet, and is a sentence (ADR 0054 §8). A configuration
with --root alone no longer starts; --names clear keeps what it did.
ADR 0054
has the reasons.
What run costs, before it costs it
run takes a mandatory acknowledge_calls that must equal the suite's planned
calls to the target. Called without it, the tool refuses and hands back the
number:
run(suite="suite.py")
→ refused: this suite plans 100 calls to the target.
20 cases × 5 samples = 100 calls to the target; each answer is judged
3 times by llm_rubric.
Call again with acknowledge_calls=100.
run(suite="suite.py", acknowledge_calls=100) → executes
The first call is the probe. An agent cannot spend a hundred model calls without
having stated the number — AGENTS.md §7 as a contract rather than as advice.
What never crosses
An MCP response goes into a model's context, and from there into transcripts and caches nobody controls. So the run tools return a verdict-only projection: the name, the identity, the status, the score, the threshold, the tolerance, the measured interval, and the metadata a suite disclosed.
Never the judge's reason — the judge quotes the output, so the reason is the
output. Never the sentence explaining a suspension, never undisclosed metadata,
never a case's inputs, and never a prompt or its digest unless the suite
declares Disclosure(artifact_body=True).
A declared file outside the perimeter is refused when the run is made; one under
.digline or .git is refused by the command that would let it cross. Neither is
dropped in silence.
Facts
- Kind
- MCP server
- Repo
- digline/digline
- Group
- Uncategorized
- Stars
- 10
- License
- Apache-2.0
- Language
- Python
- Last push
- 2026-11-14
- Forks
- 1
- MCP Registry
- dev.digline/digline-mcp
- Homepage
- digline.dev
- Topics
- baseline, ci, evals, github-actions, llm, llm-evaluation, llmops, mcp, prompt-engineering, python, regression-testing, testing, mcp-server, pytest, pytest-plugin
Next on this shelf
- 6443Lovabledev.lovable/mcpOfficial MCP server for Lovable, the AI-powered full-stack app builder.10
- 6444NYC Charter, Laws and Rulesio.github.BetaNYC/nyc-charter-laws-rulesSearch and read the NYC Charter, Administrative Code and Rules of the City of New York10
- 6445io.github.ClawCall-Dev/clawcallio.github.ClawCall-Dev/clawcallReal US phone calls for AI agents: navigates phone menus, waits on hold, returns transcripts.10
- 6446io.github.Digilac/simap-mcpio.github.Digilac/simap-mcpMCP server for simap.ch - Swiss public procurement platform10
- 6447Funplay Unreal MCPio.github.FunplayAI/funplay-unreal-mcpstdio bridge for the local Unreal Editor MCP server from FunplayAI/funplay-unreal-mcp.10
- 6448Awesome Mineflayer MCPio.github.G0Osey99/awesome-mineflayer-mcpControl a Mineflayer Minecraft bot with 120+ MCP tools: move, mine, craft, fight, build, and see.10