foundation-model-skills

A Claude Code plugin marketplace for handing routine work to local Apple frameworks. Extract documents, classify batches, search files, transcribe recordings, translate text and cut out images on an Apple silicon Mac.

Local processing has zero API cost: only your Mac's compute, memory, storage and energy. No API key, subscription or Private Cloud Compute is used by these commands. The calling cloud assistant still spends tokens selecting a skill and reading its results. Save full artifacts locally and return small manifests or selected excerpts to keep that context small.

Plugins

Each plugin is independently installable and bundles the same native fm executable.

Requirements

• Apple silicon Mac running macOS 27 or later.
• Apple Intelligence enabled for extract, classify, condense --explain, and translate --engine model.
• Speech and image segmentation use Apple's model assets. image-cutout downloads its model on first use. transcribe-audio works for the locales fm doctor lists as installed; other locales need --download-assets. These downloads do not upload your input.
• The default Translation engine needs an installed supported language pair. --engine model uses the on-device Foundation Model instead. Hardware, locale and model availability are checked at runtime.

Prebuilt arm64 binaries are included; using the plugins does not require Xcode or Python. NaturalLanguage search falls back to lexical ranking if a sentence embedding is unavailable for the requested language.

Install

/plugin marketplace add goose-farm-future/foundation-model-skills
/plugin install pdf-to-text@foundation-model-skills
/plugin install ocr@foundation-model-skills
/plugin install table-to-csv@foundation-model-skills
/plugin install extract@foundation-model-skills
/plugin install condense@foundation-model-skills
/plugin install classify@foundation-model-skills
/plugin install local-search@foundation-model-skills
/plugin install transcribe-audio@foundation-model-skills
/plugin install translate@foundation-model-skills
/plugin install image-cutout@foundation-model-skills

Install the tasks you use. Each skill description adds a small amount to the assistant's context. Run the executable from any installed plugin to inspect capabilities:

"<plugin-directory>/bin/fm" doctor
"<plugin-directory>/bin/fm" --help

Examples

Here fm means the bundled bin/fm in an installed plugin, not Apple's separate /usr/bin/fm command.

fm table-to-csv statements/ --output tables/
fm extract invoices/ --fields invoice_number,total,due_date --output invoices.json
fm extract contracts/ --schema fields.json --output extracted.json
fm condense build.log --output failures.json --explain
fm classify feedback.txt --lines --labels bug,feature,praise --output labels.json
fm local-search documents/ --query "When does the warehouse lease expire?" --limit 5
fm transcribe-audio meeting.m4a --locale en-AU --format vtt --output meeting.vtt
fm translate strings.json --source en --target es --output translated/
fm translate notes.md --source en --target fr --engine model --output translated-notes/
fm image-cutout product.jpg --point 0.5,0.5 --output product.png --mask mask.png

Extraction schemas are JSON objects mapping 1–30 field names to descriptions:

{
  "invoice_number": "Invoice identifier",
  "total": "Total amount due including currency",
  "due_date": "Payment due date"
}

Extraction returns copied strings or null, evidence quotes, page/line references and explicit conflicts. It rejects values whose quote is not present in the extracted source text. This is a field-extraction schema, not arbitrary JSON Schema or a promise of semantic accuracy.

Use --output for substantial results. JSON commands then print only an artifact path, size and count. Existing output files require --overwrite; table and translation output folders must be new and are published after the whole batch succeeds. Inputs cannot be overwritten by their outputs. extract, classify and condense also accept UTF-8 stdin as -.

For document commands, --pages N-M and --max-pages N explicitly limit PDF processing. The new batch commands process all pages by default; the original ocr and pdf-to-text commands retain their 30-page default. Text extraction, classification and search accept --force-ocr for signed, filled-in or annotated PDFs whose text layer is incomplete. Folders are scanned recursively, skipping hidden files, packages and symlinks.

classify --lines treats each nonempty line as a record. Otherwise classification counts describe document passages; large records may span multiple passages. Custom-label classification uses the general model; topic tagging uses Apple's content-tagging model. Model inputs are counted against the actual context limit and processed in fresh sessions.

Translation supports plain text/Markdown and flat JSON string catalogs. It preserves JSON keys and verifies protected placeholders, code and URLs. Line breaks are retained; unsupported structures and overlong lines fail explicitly. Review translated prose and Markdown formatting before publishing.

Cutout coordinates use a normalized top-left origin after EXIF orientation: --point x,y or --box x,y,width,height. Outputs keep the oriented input's pixel dimensions. Segmentation quality depends on the subject and selection; inspect edge quality when needed.

Local processing and cloud context

These commands do not upload source files or fall back to a hosted model. Apple's framework model assets may be downloaded on first use or with --download-assets. Apple Translation may collect API usage metadata under Apple's platform policies, but it does not send your source or translated content as part of those metrics.

A cloud assistant receives anything it reads from stdout or from a generated file. Keeping a recording or PDF local does not make text copied into the assistant's context private from its cloud provider. For bulk work, return paths, counts, selected evidence and exceptions. Images already pasted into a cloud chat have already been sent there; give the skill a local file path instead.

OCR can misread characters. table-to-csv copies cell text exactly from a PDF's text layer where there is one, but it can miss borderless tables or misalign a label row. Model classification and extraction can be wrong even with valid JSON and real quotes. Apple's model can refuse ordinary passages; extract and classify report those passages instead of failing the run. Search returns ranked source passages, not generated answers or proof that a missing result is absent. Speech output does not identify speakers. Condensation retains exact evidence but does not establish a root cause or preserve a shell command's exit status.

Validation and benchmarks

The new skills are tested locally without paid cloud calls. Local API cost is zero by design; end-to-end cloud-token savings vary with task size and how much output the caller reads.

Earlier PDF/OCR comparisons remain in bench/results/summary.md and the summary comparison. They describe the tested configurations at the time, not measurements of this expanded catalogue. In those fixtures, long PDFs benefited substantially from extraction; single images could cost more once skill overhead was included. Summarisation also lost facts that exact extraction retained.

The SEC tasks were measured against the current catalogue. On three questions about Apple's 77-page FY2025 10-K, Claude cost fell 56–73%, and all 18 runs in both arms answered correctly. Without the skills, Claude sent 18–23 page images to cloud vision per question. With them, no page image left the Mac; Claude read 1–17 KB of extracted lines or passages. The plan-* rows come from earlier runs whose task definitions were not kept.

scripts/build.sh
scripts/prepare-fixtures.sh
python3 tests/test_tasks.py                  # all new commands, local model inference only
tests/smoke.sh                            # packaging + original OCR/PDF checks
claude plugin validate .

Use python3 tests/test_tasks.py --download-assets to allow missing Apple speech/segmentation assets to be installed for integration testing. Otherwise asset-dependent checks are reported as skipped if the model is unavailable. Results and artifacts go under .build/verification/.

The historical cloud comparison is opt-in and incurs cloud usage:

bench/run.py --tasks ops-review-20p --reps 1
bench/report.py

The SEC filing is not committed. SEC_CONTACT=you@example.com scripts/fetch-sec.sh downloads it from EDGAR (which requires a contact email) and renders it to PDF with headless Chrome. bench/sec_retrieval.py then scores local-search on it with no cloud calls: whether a passage containing the exact answer reaches the top 1, 3 or 5 results.

Its plugin list comes from the current marketplace. Do not compare a new catalogue's overhead against old results without rerunning the comparison. The local test suite is the capability check for the eight new tasks; historical cloud tasks remain media-focused.

Development

Building requires Xcode with the macOS 27 SDK. scripts/build.sh selects Xcode or Xcode-beta and bundles the executable only into plugins registered in .claude-plugin/marketplace.json.

src/main.swift           Original image/PDF commands and entry point
src/LocalTasks.swift     CLI parsing, file IO, manifests and token-aware chunking
src/TextTasks.swift      Extraction, classification and log condensation
src/DocumentTasks.swift  Table export and local document search
src/MediaTasks.swift     Speech, translation and image segmentation
plugins/<task>/          Manifest, skill instructions and bundled executable
scripts/                Build, launcher and fixture preparation
tests/                 Local end-to-end verification
bench/                  Historical cloud-token benchmark and evidence

To add a task, implement its command, register its plugin, add concise skill instructions, and add a fixture-backed case to the local test suite. Test behavior and failure paths before claiming support. Measure cloud savings separately if making a numerical savings claim.

The experimental pdf-summary plugin is outside the release catalogue.

License

MIT

{
  "name": "foundation-model-skills",
  "owner": {
    "name": "Goose Farm Future"
  },
  "metadata": {
    "description": "Local Apple-framework skills for documents, extraction, logs, classification, search, speech, translation and image cutouts. No API fees for local processing.",
    "version": "0.3.1"
  },
  "plugins": [
    {
      "name": "ocr",
      "description": "On-device OCR with Apple Vision: Claude reads an image's text without the image leaving your Mac.",
      "version": "0.3.1",
      "source": "./plugins/ocr",
      "category": "productivity",
      "keywords": [
        "ocr",
        "image",
        "screenshot",
        "receipt",
        "on-device"
      ]
    },
    {
      "name": "pdf-to-text",
      "description": "On-device PDF text extraction (PDFKit + Apple Vision OCR): Claude reads the text, not page images, and the PDF stays on your Mac.",
      "version": "0.3.1",
      "source": "./plugins/pdf-to-text",
      "category": "productivity",
      "keywords": [
        "pdf",
        "text-extraction",
        "ocr",
        "on-device"
      ]
    },
    {
      "name": "table-to-csv",
      "description": "Extract tables from PDFs and images into CSV on-device with Apple Vision.",
      "version": "0.3.1",
      "source": "./plugins/table-to-csv",
      "category": "productivity",
      "keywords": [
        "tables",
        "csv",
        "pdf",
        "vision",
        "on-device"
      ]
    },
    {
      "name": "extract",
      "description": "Extract named fields from documents in bulk on-device, with source quotes and missing values.",
      "version": "0.3.1",
      "source": "./plugins/extract",
      "category": "productivity",
      "keywords": [
        "extraction",
        "json",
        "documents",
        "on-device"
      ]
    },
    {
      "name": "condense",
      "description": "Condense long logs and test output locally, preserving exact failures and source lines.",
      "version": "0.3.1",
      "source": "./plugins/condense",
      "category": "productivity",
      "keywords": [
        "logs",
        "tests",
        "errors",
        "on-device"
      ]
    },
    {
      "name": "classify",
      "description": "Classify or tag batches of text on-device with Apple Foundation Models.",
      "version": "0.3.1",
      "source": "./plugins/classify",
      "category": "productivity",
      "keywords": [
        "classification",
        "tags",
        "batch",
        "on-device"
      ]
    },
    {
      "name": "local-search",
      "description": "Search selected local documents with Apple sentence embeddings and return source passages.",
      "version": "0.3.1",
      "source": "./plugins/local-search",
      "category": "productivity",
      "keywords": [
        "search",
        "semantic",
        "documents",
        "on-device"
      ]
    },
    {
      "name": "transcribe-audio",
      "description": "Transcribe audio and video locally into text, timestamped JSON or WebVTT subtitles.",
      "version": "0.3.1",
      "source": "./plugins/transcribe-audio",
      "category": "productivity",
      "keywords": [
        "speech",
        "transcription",
        "subtitles",
        "on-device"
      ]
    },
    {
      "name": "translate",
      "description": "Translate text and JSON string catalogs locally with Apple Translation or Foundation Models.",
      "version": "0.3.1",
      "source": "./plugins/translate",
      "category": "productivity",
      "keywords": [
        "translation",
        "localisation",
        "batch",
        "on-device"
      ]
    },
    {
      "name": "image-cutout",
      "description": "Isolate an image subject locally using a point or bounding box, producing a transparent PNG and mask.",
      "version": "0.3.1",
      "source": "./plugins/image-cutout",
      "category": "productivity",
      "keywords": [
        "segmentation",
        "image",
        "cutout",
        "on-device"
      ]
    }
  ]
}