ClaudeCodeMod

All shelves / MCP servers

digline

dev.digline/digline-mcp · 10 stars · Python · Apache-2.0

MCP server Read, measure and explain LLM eval runs of a digline suite. Cannot promote a baseline.

Install

The repo has no one-line install. Follow its README.

Published as

  • PyPIdigline-mcpstdio

From the server's entry in the official MCP Registry.

Open the repo

Files

packages/digline-mcp/README.md

digline-mcp

The MCP server for digline: it lets a coding agent read a digline result and measure a new one, and it does not let the agent promote a baseline.

Not because promotion is refused. Because there is no such tool.

The thirteen tools

tool what it does
list_runs every stored run of a suite, newest first, with the baseline marked
get_run one stored run — the verdicts, never the payload
get_baseline the approved reference, same shape
log which model answered, read down the stored runs; never a gate
compare a run against the baseline: did it get worse?
diff two runs, neither of them a reference: should I switch?
explain one run read back as typed facts, with the exit code
run execute the suite, with the call count acknowledged first
where where the project is, and the command for the next step
record_status whether each tenant records, and how much
record_listing a suite's recorded answers, counted by the shape of the output; never a yes or a no
record_compare a suite's recorded answers by case and by setup, each by its number in the answer
suggest a first suite, proposed as facts, with the command that writes it; writes nothing, asks no model

promote is absent by construction, and so are migrate, view, report, a record import that writes a yes or a no, and record on and off. A baseline is an approved reference, and promote writes into .digline/<tenant>/baselines/, which is committed — anything landing there arrives in somebody's diff and must arrive because they put it there. A refusal would be a conversation an agent could argue with; an absence is not.

Every tool, the reading ones included, loads the suite it is named. For a suite.py that executes the project's code, in the server's environment.

The reasoning is ADR 0011, and the judgement layer it turns into surface is AGENTS.md.

Install and configure

uv pip install digline-mcp

One server serves one repository: the perimeter is the repo, and multi-project is N named servers rather than a registry.

{
  "mcpServers": {
    "digline-northwind": {
      "command": "digline-mcp",
      "args": ["--root", "/Users/you/src/northwind", "--names", "clear"]
    }
  }
}

--tenant and --env are accepted and verify; they never override. The suite decides, as it does everywhere else in digline.

--names is required, with no default: clear lets the store's names leave as they are, and project projects every answer, so the names leave as tokens from the name table that --name-table module:attribute names; a refusal is not projected yet, and is a sentence (ADR 0054 §8). A configuration with --root alone no longer starts; --names clear keeps what it did. ADR 0054 has the reasons.

What run costs, before it costs it

run takes a mandatory acknowledge_calls that must equal the suite's planned calls to the target. Called without it, the tool refuses and hands back the number:

run(suite="suite.py")
  → refused: this suite plans 100 calls to the target.
    20 cases × 5 samples = 100 calls to the target; each answer is judged
    3 times by llm_rubric.
    Call again with acknowledge_calls=100.

run(suite="suite.py", acknowledge_calls=100)   → executes

The first call is the probe. An agent cannot spend a hundred model calls without having stated the number — AGENTS.md §7 as a contract rather than as advice.

What never crosses

An MCP response goes into a model's context, and from there into transcripts and caches nobody controls. So the run tools return a verdict-only projection: the name, the identity, the status, the score, the threshold, the tolerance, the measured interval, and the metadata a suite disclosed.

Never the judge's reason — the judge quotes the output, so the reason is the output. Never the sentence explaining a suspension, never undisclosed metadata, never a case's inputs, and never a prompt or its digest unless the suite declares Disclosure(artifact_body=True). A declared file outside the perimeter is refused when the run is made; one under .digline or .git is refused by the command that would let it cross. Neither is dropped in silence.

Facts

Kind
MCP server
Repo
digline/digline
Group
Uncategorized
Stars
10
License
Apache-2.0
Language
Python
Last push
2026-11-14
Forks
1
MCP Registry
dev.digline/digline-mcp
Homepage
digline.dev
Topics
baseline, ci, evals, github-actions, llm, llm-evaluation, llmops, mcp, prompt-engineering, python, regression-testing, testing, mcp-server, pytest, pytest-plugin

Next on this shelf

  1. 6443Lovabledev.lovable/mcpOfficial MCP server for Lovable, the AI-powered full-stack app builder.10
  2. 6444NYC Charter, Laws and Rulesio.github.BetaNYC/nyc-charter-laws-rulesSearch and read the NYC Charter, Administrative Code and Rules of the City of New York10
  3. 6445io.github.ClawCall-Dev/clawcallio.github.ClawCall-Dev/clawcallReal US phone calls for AI agents: navigates phone menus, waits on hold, returns transcripts.10
  4. 6446io.github.Digilac/simap-mcpio.github.Digilac/simap-mcpMCP server for simap.ch - Swiss public procurement platform10
  5. 6447Funplay Unreal MCPio.github.FunplayAI/funplay-unreal-mcpstdio bridge for the local Unreal Editor MCP server from FunplayAI/funplay-unreal-mcp.10
  6. 6448Awesome Mineflayer MCPio.github.G0Osey99/awesome-mineflayer-mcpControl a Mineflayer Minecraft bot with 120+ MCP tools: move, mine, craft, fight, build, and see.10