AgentOps

DevOps discipline for AI coding agents.

Install · The loop ·
Goals · Try it · Skills ·
Make it yours · Evidence ·
Beyond AgentOps

DevOps makes software delivery repeatable through shared practices, automated
checks and feedback. AgentOps brings that discipline to agentic coding sessions:
paved paths your agents can follow from intent to checked results, with memory
to carry useful context into the next task.

Those paths combine skills with the ao CLI
and your existing engineering tools. Plan clarifies the
expected behavior; Test helps check it;
Validate adds independent judgment where needed.
Skills guide the agent's decisions; executable gates check specific contracts.
Beads preserves the work graph, decisions,
results and project memories. Memory curates supported
findings into linked project knowledge, connecting lessons to the work and
sources behind them. The next session can use that record instead of starting over.

The skills are a kit, and you are expected to change it. Each skill is one
Markdown file of instructions. Keep the ones that fit how you work, rewrite the
ones that almost fit, delete the rest, and add skills you write yourself or find
in other libraries. What you end up with is your own operating model for agents,
which is the reason this project exists. Make it yours shows
how.

Use these paths with Claude Code, Codex, Cursor, OpenCode, Gemini CLI, Pi and
other coding agents, or personal assistants such as OpenClaw and Grok Bot.
Start with one useful task; follow the operational loop
for a change or the goal workflow for work across sessions. When you
need a larger software factory, extend the same practices
with Gas City or tools from the Agentic Coding Flywheel.

Why use AgentOps?

A software factory
makes engineering practices reusable through tools, guidance and repeatable
workflows. AgentOps brings that approach into the coding session: clear intent,
automated checks, recorded decisions and a feedback loop into later work. Pick
the path your task needs and use the same practices when you change agents.

Behavior-driven development (BDD)
starts by agreeing on what the software should do in a concrete situation.
For retries, that could mean: “If a completed job arrives again, return its
result and charge the customer only once.” The same behavior guides the
implementation, its tests and any review. Plan helps you
settle those expectations before the agent starts editing.

Quickstart

Pick one method per agent: a plugin plus npx on the same agent gives you every
skill twice.

claude plugin marketplace add boshu2/agentops
claude plugin install agentops@agentops-marketplace
claude plugin details agentops@agentops-marketplace

codex plugin marketplace add boshu2/agentops
codex plugin add agentops@agentops-marketplace
codex plugin list --json

npx skills@latest add boshu2/agentops

Start a new session so the skills load. Most skills need only your coding
agent; a few use the ao CLI when it is installed. Invocation names
vary by agent: this README shows Claude Code's /agentops:<skill>; Codex uses
$agentops:<skill>.

The operational loop

Each change is shaped, built and checked. Use one fresh review when requested,
when a mistake cannot be cheaply undone, or when no deterministic check covers
the changed behavior. You (or Plan) write intent as
behavior (BDD), using one word per
concept (DDD's
ubiquitous language).
For a system that calls queued work a Job, in Gherkin:

Feature: Job redelivery is idempotent
  A Job is one unit of queued work. Delivering it again never repeats its side effect.

  Scenario: A completed Job is delivered again
    Given Job "J-42" completed and charged the customer $20
    When the worker receives Job "J-42" again
    Then it returns the completed result of "J-42"
    And the customer has been charged $20 exactly once

  Scenario: A Job that failed before charging is delivered again
    Given Job "J-43" failed before charging the customer $20
    When the worker receives Job "J-43" again
    Then Job "J-43" completes
    And the customer has been charged $20 exactly once

The feature defines the domain term once; each scenario has concrete data, one
action and an observable result. Keep scenarios in the issue or conversation;
no .feature file is required.

Enter at the step you need. An existing change can enter at Validate when an
acceptance judgment is needed. Ordinary changes finish on checks and CI. Review
is one round: repair findings and confirm with affected checks; review again only
when you ask. Report NOT_PROVEN with its gaps. The author cannot issue its own
PASS. Merging and releasing follow your repo's rules.

Goals

rpi runs Plan → Implement → checks, with Validate where a
mistake is costly, for one outcome
without check-ins (your agent's permission prompts still apply) and stops at
acceptance, a blocker or a spent limit. Bigger work becomes a goal (it
needs Beads: brew install beads, then bd init in your repo):

1. Interview. One question at a time, each with
   a recommended answer. You settle the outcome, its examples, domain terms,
   non-goals, authority and budget before agents go autonomous.
2. Craft Goal. Returns SAFE_TO_CREATE plus a
   prompt to paste into /goal (Claude Code or Codex), USE_RPI (small enough
   for rpi), or UNSAFE_GOAL plus what's undecided. It creates nothing itself.
3. Navigate each round. Picks a few ready work
   items (beads); each gets one RPI, checks and CI, with one fresh Validate
   when requested, hard to undo, or uncovered by a deterministic check. The goal ends
   ACHIEVED, NOT_ACHIEVED or NEEDS_OPERATOR.

Beads remembers the work. Beads (bd)
keeps the goal and its history outside any conversation. The root epic holds
acceptance; child beads carry the scoped work, design decisions, handoffs and
result notes. Record which checks ran and link any review evidence to the same
bead. That connects the original intent to the checked output, so the next
session can continue from the record instead of reconstructing the chat.
bd ready finds unblocked work; search and history help explain earlier choices.

Beads also has a persistent memory store.
bd remember saves project facts, bd memories searches them, bd recall
retrieves one by key, and bd prime brings stored memories into session context.
The work graph and the memory store serve different needs: one preserves what
happened; the other keeps selected facts easy to find again.

Memory helps you use that material: recall relevant
context, examine recorded decisions and outcomes, or draft a report with links
to its evidence. When you want a reusable project document, it curates supported
findings into reviewed .context/ pages or an external document bundle. Beads
keeps the work record; Git and the original evidence locations keep the code and
check artifacts. Reusable documents link back to those sources.

bd search "redelivery" --status all     # include finished work
bd show <bead-id>                        # intent, notes and relationships
bd history <bead-id>                     # how the record changed
bd remember "Redelivery returns the stored Job result" --key job-redelivery
bd recall job-redelivery                # retrieve the saved project fact

One bead per worker. When the goal delegates, the orchestrator holds the
graph and results, and each worker starts with one bead instead of the
orchestrator's transcript. Reviewers, when needed, start fresh.

bd create "Job redelivery is idempotent" -t epic
bd create "Return the completed result on redelivery" --parent <epic-id>
bd dep add <later-id> <earlier-id>         # real ordering only
bd ready --parent <epic-id>                # the frontier

Navigate shows bd commands; another tracker with status, dependencies and
notes works if you map them. AgentOps never builds a second work index.

Try it

Start read-only in any repo, then swap the Job example for your own change.

# First look (changes nothing)
/agentops:research how does this repo validate input? cite files and lines, change nothing

# One change
/agentops:plan make Job redelivery return the completed result without repeating the side effect
/agentops:implement
/agentops:validate     # when requested, hard to undo, or uncovered by checks; use a new session

# One outcome, end to end
/agentops:rpi make Job redelivery return the completed result without repeating the side effect

# A goal
/agentops:interview make the job worker safe under redelivery, retries and crash recovery
/agentops:craft-goal   # then paste its prompt into /goal

Use the AgentOps Validate skill to judge this finished change.
Original accepted behavior: [issue link or original request text]
Candidate: [commit, branch or working tree; list every changed path]
Author context ID: [task/session ID that made the change]
Checks run: [commands and results]
I opened this new conversation for fresh review. Derive the exact subject
identity at the start and end, inspect every changed path against the original
behavior, and do not modify the candidate. Report PASS, FAIL or NOT_PROVEN with
evidence for each criterion, checked, not_checked, author and reviewer context
IDs, and freshness attestation.

Use the AgentOps Research skill to trace how this repository validates user
input. Follow one path from the input through its checks and tests. Cite the
files and line numbers, explain one edge case, and identify missing coverage.
Name the Research skill file you loaded. Answer here without changing files.

Skills at a glance

All skills are optional. Load one when it answers a specific question. Full
catalog: docs/SKILL-ROUTER.md.

Make it yours

Every skill here is a folder with one SKILL.md: plain instructions an agent
loads when a task calls for them. No skill depends on the full set, and coding
with none of them still works. That makes the library easy to take apart, and
you should. The 29 skills are a starting point for an operating model that fits
your work.

• Start small. Install two or three skills that match work you already do.
  Add another when a real task asks for it.
• Cut what you override. If you or the agent keep ignoring a rule, change
  the rule or remove the skill. npx skills lets you pick skills per agent, and
  ao skills link --skill <name>
  links an exact subset from a checkout.
• Rewrite what almost fits. Fork this repository, edit the SKILL.md and
  install from your fork with the same commands, or link a checkout so every
  agent on your machine reads your edits. A plugin update replaces the installed
  copy, so keep your changes in a repository you control.
• Write your own. When you have explained the same thing to an agent three
  times, it is a skill. Skill Builder drafts the
  package, and tells you when a note in an existing file is enough.
• Mix libraries. Run these next to your company's skills, the
  Agentic Coding Flywheel tools or any other
  library. ao skills link never replaces a skill it did not install.
• Change the workflows too. The operational loop
  and the goal workflow are defaults. Skip the steps your work does
  not need, reorder them, or write your own. The Claude Code workflow scripts in
  workflows/ link into a project with
  ao workflows link.
• Measure what you change. claude plugin eval compares an agent with and
  without a plugin on the same request. To measure one edit, run the old and
  the new version as two plugins. Skill Eval
  covers how to read the result, and Evidence shows the cases this
  repository runs; copy them for your own skills.

Evidence

A skill is a page of instructions, so the test is whether an agent does anything
differently with it installed. AgentOps runs that test on Claude Code's own
evaluator, claude plugin eval: one realistic request per skill, three runs
with the plugin and three without, each answer graded against four or five
criteria taken from the practice the skill teaches. For Test, one criterion is
that a regression test is shown failing without the fix.

On Claude Opus 5.5, measured 2026-10-05:

Fourteen of the 29 skills moved their case by 0.15 or more. Craft Goal, Claude
Exec, Plan, Memory and Skill Builder gained the most. The other 15 made no
measurable difference, and for eight of those the agent already met every
criterion with no plugin.

Read the limits before you quote these numbers:

• One request per skill, three runs, one model. A difference under 0.15 is
  noise.
• The 28 requests in the first row were also used to tune the 3.10
  descriptions, which flatters 3.10. The blind requests in the second row were
  written without sight of the descriptions.
• The criteria come from each skill's own rules. A pass shows the rule landed on
  that request. It says nothing about the outcome of a real task, and an earlier
  coding pilot found no end-to-end
  difference.
• Loading is the weak point. Nine skills did not load on a request they had
  never seen, and Implement does not load on a quick fix. Name the skill when
  you want its rules applied.

The full report has every case,
the method and what was rerun. The cases live in
evals/plugin-eval/: run them against your own
changes, or copy the layout to test your own skills.

Where AgentOps fits

AgentOps grew from applying DevOps experience and established engineering
practice to agents. The Practice Registry records the
lineage; how it works covers responsibilities.

ao CLI (optional)

Most skills need only your coding agent. With ao installed, Validate binds
the exact change it judges to a content manifest; without it, Validate names
the commit and the changed paths.

brew tap boshu2/agentops
brew trust --tap boshu2/agentops
brew install agentops
ao version

With Go installed: go install github.com/boshu2/agentops/cli/cmd/ao@latest.
ao init is optional evidence setup; ao config --show inspects configuration;
ao gate check runs this repository's own gates (mainly for contributors). See the command reference and
installation guide.

Updating and advanced setup

Troubleshooting

Limits

Skills guide agents; installation alone does not enforce their instructions.
A green test suite or an agreeing model can still miss a defect. Missing proof
stays NOT_PROVEN. Saving notes does not establish improved outcomes or
automatic knowledge compounding. See product evidence and limits.

FAQ

Beyond AgentOps

Start with one agent and a paved path. When work needs several agents, a factory
can add scheduling, coordination and runtime supervision. Install AgentOps
skills into its agent runtimes to carry the same behavior, testing and review
practices into each task. The factory runs the agents; your checks and any
required independent judgment establish what the result proves.

Gas City

Gas City, from the organization behind
Beads, provides configurable building
blocks for multi-agent software factories: runtime providers, work routing,
workflow formulas, packs and supervision. AgentOps ships a
Gas City adapter and an
executor integration pack. Use the adapter
to work through Gas City's native coordinator and observe its runs; workers can
use AgentOps skills for implementation, tests and validation.

Agentic Coding Flywheel

Agentic Coding Flywheel is Jeffrey Emanuel's
(Dicklesworthstone) ecosystem of tools
and workflows for operating coding agents. Its
setup project
assembles a multi-agent development environment. You can also adopt individual
tools alongside AgentOps:

For example, use CASS to find how a recurring migration failure was resolved.
Use Memory to examine the supported lesson and curate
project context. If the method belongs in a repeatable path,
Skill Builder can help package it and
Skill Eval can measure whether it helps later work.
This connects session history to guidance you can inspect and improve.

Gas City, the Flywheel and these tools are independent projects. AgentOps ships
its own Gas City adapter and pack; install external tools and their skills from
their authors. Follow their native workflows, integrations and distribution terms.

Contributing

Contributions are welcome: documentation fixes, reproducible bug reports,
tests, CLI improvements and skills. Read the contribution guide;
to work on skills from a checkout, link them with
ao skills link.

Licensed under Apache-2.0.

{
  "name": "agentops-marketplace",
  "owner": {
    "name": "Boden Fuller",
    "email": "fullerbt@users.noreply.github.com"
  },
  "metadata": {
    "description": "Engineering guidance for coding agents: behavior-driven planning, shared domain language, independent validation, and reusable improvements.",
    "version": "3.10.0"
  },
  "plugins": [
    {
      "name": "agentops",
      "description": "Engineering guidance for coding agents: behavior-driven planning, shared domain language, independent validation, and reusable improvements.",
      "version": "3.10.0",
      "source": "./",
      "author": {
        "name": "Boden Fuller",
        "email": "fullerbt@users.noreply.github.com"
      }
    }
  ]
}