Orchestra · A self-improving software factory

II. The film

One goal, start to finish, in 60 seconds.

An animated walkthrough of a single Orchestra run. Scrub, pause, or jump to a chapter.

0:00 / 1:00

III. The ensemble

Five players. Strict parts. Fresh context every take.

Each role is a versioned system prompt. Planners never code. Judges never edit. Durable state lives in git and .orchestra/, never in one long conversation.

Planner

default: opus · read-only

Reads the goal, file list, test command and past lessons. Writes the score: a task graph with dependencies and acceptance criteria, plus hidden given/when/then scenarios.

Worker

default: sonnet · own worktree

Implements exactly one task in its own git worktree. Sees the goal, its task, the criteria, learnings and any repair notes. Never sees the hidden scenarios.

Judge

default: opus · edits discarded

Scores the diff 0 to 1 against acceptance criteria and the hidden scenarios. Returns pass, a summary and issues. A reject becomes repair notes for the next attempt.

Librarian

default: sonnet · no tools

After the run, distills repairs and failures into lessons with cited evidence, scoped by domain. The next run starts smarter.

Evolver

default: opus · outside the run loop

Proposes a better harness. A Factory Eval replays past goals against the incumbent. No measured gain, no promotion. It cannot see or edit the frozen scorer.

IV. Features

Built for unattended work you can still trust.

Parallel git worktrees

Every task builds in its own worktree on orchestra/<run>/<task>. Your checkout is never touched. Results land on an integration branch you review and merge yourself.

Hidden holdout scenarios

The judge checks work against scenarios workers cannot read. Enforced by the OS sandbox and a guard hook, not just a prompt.

Self-correcting

Judge rejections, merge conflicts and red post-merge tests all trigger a repair attempt with notes.

Serialized merge queue

Tasks merge one at a time onto orchestra/<run>/integration, with tests after each merge.

Self-evaluating

Every session is recorded in a cost ledger. orchestra stats reports first-pass yield, cost per merge and more, per harness version.

Self-improving, with a seatbelt

The evolver proposes a harness change. A Factory Eval compares it with the incumbent, scored by a frozen scorer it cannot touch. Promotion needs a measured gain, and rollback is one command: orchestra promote <vN>.

Live dashboards

A full-screen terminal board, a read-only web task board with dependency arrows, and a Claude Code statusline.

Real isolation, opt-in

OS sandbox (srt, Claude Code sandbox) or Docker with optional gVisor. Pre-flight self-test aborts if a blocked host is reachable.

Budgets in code

Each run has a spend cap (runBudgetUsd, default $20). Tasks stop when it is reached.

5agent roles with separate context

0build steps (Node runs TypeScript directly)

0runtime dependencies

MITlicensed and open source

V. Quick start

From clone to first merge in four steps.

Needs Node 23.6+, git, and Claude Code with headless auth (orchestra doctor checks it). The demo needs none of the auth.

1
Try the demo. No tokens spent.

Runs the full pipeline with a simulated runner in a temporary repo.
```
git clone https://github.com/deepskpathak/orchestra
cd orchestra
node bin/orchestra.ts demo
```

2

Install it

As a Claude Code plugin, or as a CLI in any terminal.

# Claude Code plugin
claude plugin marketplace add <path-to-orchestra>
claude plugin install orchestra@orchestra-local

# or plain CLI
cd <path-to-orchestra> && npm link

3
Initialise your repo

Creates .orchestra/, seeds harness v1 and detects your test command.
```
cd your-repo
orchestra init
orchestra doctor
```

4

Run a goal, then land it

Results land on an integration branch. You decide when to merge.

orchestra run "Add a --json flag to the export command"

git log main..orchestra/<run>/integration
git merge orchestra/<run>/integration

VI. Docs

The score, part by part.

Short versions here. Every tab links to the full page in the repository.

Filter commands

Inside Claude Code, the plugin adds /orchestra:setup, /orchestra:run, /orchestra:serve, /orchestra:status, /orchestra:stats, /orchestra:bench and /orchestra:evolve. Full reference: README › Commands.

How a run flows

Plan. The planner returns a task graph and hidden scenarios, written to holdout/<run>.json outside every worktree.
Build. One worker per task, each in .orchestra/worktrees/<run>/<task>, cut from the integration tip.
Verify. Your test command runs, then the judge scores against criteria and scenarios.
Repair. A reject becomes repair notes for a fresh worker attempt.
Merge. A serialized queue merges onto orchestra/<run>/integration and re-runs tests.
Learn. The librarian writes lessons for the next run.

Outside the loop

The frozen scorer grades eval replays for promotion. The integrity monitor inspects worker diffs for grading-surface edits, weakened tests and reward hacks. The evolver can neither see nor edit either one.

goal
  │
planner ─► task graph + hidden scenarios
  │
  ├─► worker A (worktree) ─┐
  ├─► worker B (worktree) ─┼─► tests + judge ─► merge queue ─► integration
  └─► worker C (waits A) ──┘        │ fail
                                    └─► repair notes ─► next attempt
  │
librarian ─► learnings (read by the next run)

Read the architecture guide →

The harness is the evolvable part

Role prompts (planner.md, worker.md, judge.md, librarian.md, evolver.md), config.json, and three small optional files: tools.md, style.md, checklist.md.

Each version lives in harness/<vN>/. Versions are append-only.
archive.json records parent, rationale, eval score and status.
The review checklist can only get stricter. passThreshold never goes down.

The loop

orchestra stats       # health per harness version
orchestra learnings   # what this repo taught it
orchestra evolve      # propose, eval, maybe promote
orchestra promote v1  # roll back any time

Evolution reruns past goals, so it costs roughly sample × 2 runs. Tune evolve.sample, evolve.every and evolve.candidates.

Read the self-improvement guide →

All four views read the same source of truth (run.json and events.jsonl), so they always agree.

Full-screen terminal

orchestra watch

Live board, redraws about every 150 ms. Press q to quit.

One-shot board

orchestra status

Snapshots for scripts and chat. --plain or --color.

Web dashboard

orchestra serve

Read-only, localhost port 4317 by default. Diffs, logs, past runs.

Statusline

orchestra statusline

An always-visible summary inside Claude Code.

Read the dashboards guide →

The boundary is the OS, not the hook

Opt in with --sandbox auto|srt|claude or --sandbox docker. Workers can write only to the worktree and temp, cannot read credentials, .env* or the holdout, and reach only the model API plus sandbox.allowedHosts.

Pre-flight self-test aborts if a blocked host or denied path is reachable.
Credentials stripped from worker env. .git/hooks and .git/config read-only.
Only the orchestrator commits, pushes and opens PRs.
--unattended refuses to start without a sandbox backend.

Default is unsandboxed. The src/guard.ts hook is a deny-list backstop that python, node, git remotes or DNS can bypass. Use a sandbox for anything you would not run by hand.

orchestra run "<goal>" --sandbox auto
orchestra run "<goal>" --sandbox docker --gvisor
orchestra run "<goal>" --unattended

Read the isolation guide →

Two plain-JSON layers. Project config (.orchestra/config.json) controls concurrency, tools and budgets. Harness config controls models, retries and thresholds per role, and is versioned and evolvable.

concurrency3

Maximum tasks running in parallel.

testCommandauto

Run in each worktree and on the integration branch. Verification is much stronger with one.

runBudgetUsd20

Spend cap per run.

judgeTiersoff

A cheap judge first, escalating to the strong judge near the threshold.

verification.*off

Judge panel, agentic judge, acceptance contracts, oracles, mutation score.

sandbox.allowedHosts[]

Extra hosts sandboxed workers may reach (the model API is always allowed).

Full configuration reference →

VII. How-tos

How do I…

…see it work without spending tokens?

Use the demo, or point the simulator at your own repo.

orchestra demo
ORCHESTRA_RUNNER=fake orchestra run "Add a health endpoint"

ORCHESTRA_FAKE_STEP_MS sets the simulated step delay (default 150).

…review and land what Orchestra built?

Nothing touches your branch. The run prints exact commands when it finishes.

git log main..orchestra/<run>/integration
git diff main...orchestra/<run>/integration
git merge orchestra/<run>/integration

…run in the background and check in later?

orchestra run "<goal>" --detach
orchestra watch     # live board, q to quit
orchestra status    # one-shot snapshot

…watch a run in the browser?

orchestra run "<goal>" --serve   # dashboard for this run
orchestra serve --port 4317        # or standalone, read-only

…work safely on a repo I don't fully trust?

Use the Docker backend (no network by default) and optionally gVisor. Add hosts like a package registry only if needed.

orchestra run "<goal>" --sandbox docker --gvisor --unattended

…roll back a harness change I don't like?

Versions are append-only, so any one can be made current again.

orchestra stats         # compare versions
orchestra promote v1    # make v1 current again

…see what Orchestra has learned about my repo?

orchestra learnings
orchestra learnings --domain cli
orchestra learnings --domain global

Lessons are plain JSON and Markdown in .orchestra/learnings/. You can read and edit them.

…require a human sign-off on risky tasks?

Turn on oversight.riskTiers. High-risk tasks wait as awaiting_approval until a different person approves, then resume finishes deferred work.

orchestra approve <runId> <taskId> --by <name>
orchestra resume <runId>

…benchmark whether a setting actually helps?

The bench suite runs arms × goals × repeats with hidden checks and paired statistics.

orchestra bench --verify-oracles
orchestra bench --ablations

Benchmark guide →

VIII. Liner notes

What's proven, and what isn't yet.

The full test suite runs against a simulator, no model calls.
Real end-to-end runs with the Claude Code sandbox merged 2 of 2 and 3 of 3 tasks, at $3.04 and $4.52.
Every new capability is opt-in and off by default.

Some sandbox backends (srt, docker, sandbox-exec) have not run a real worker yet.
Most research hypotheses are built but unmeasured.
No harness variant has been promoted from a real gain yet.

Project status Whitepaper Changelog

You bring the goal.
The factory plays it.

One goal, start to finish, in 60 seconds.

Five players. Strict parts. Fresh context every take.

Planner

Worker

Judge

Librarian

Evolver

Built for unattended work you can still trust.

Parallel git worktrees

Hidden holdout scenarios

Self-correcting

Serialized merge queue

Self-evaluating

Self-improving, with a seatbelt

Live dashboards

Real isolation, opt-in

Budgets in code

From clone to first merge in four steps.

Try the demo. No tokens spent.

Install it

Initialise your repo

Run a goal, then land it

The score, part by part.

How a run flows

Outside the loop

The harness is the evolvable part

The loop

Full-screen terminal

One-shot board

Web dashboard

Statusline

The boundary is the OS, not the hook

How do I…

What's proven, and what isn't yet.

Raise the baton.

You bring the goal. The factory plays it.

One goal, start to finish, in 60 seconds.

Five players. Strict parts. Fresh context every take.

Planner

Worker

Judge

Librarian

Evolver

Built for unattended work you can still trust.

Parallel git worktrees

Hidden holdout scenarios

Self-correcting

Serialized merge queue

Self-evaluating

Self-improving, with a seatbelt

Live dashboards

Real isolation, opt-in

Budgets in code

From clone to first merge in four steps.

Try the demo. No tokens spent.

Install it

Initialise your repo

Run a goal, then land it

The score, part by part.

How a run flows

Outside the loop

The harness is the evolvable part

The loop

Full-screen terminal

One-shot board

Web dashboard

Statusline

The boundary is the OS, not the hook

How do I…

What's proven, and what isn't yet.

Raise the baton.

You bring the goal.
The factory plays it.