Architecture rules belong in the harness.

What belongs in an agent harness?

An agent harness is the environment that connects an agent to context, tools, execution and feedback. Harness Engineering designs that environment: which decisions may the agent make, which states can it see, and which checks must a change pass?

The following breakdown is our working model, not an established standard. The layers complement each other; one tool may cover several.

A plugin packages capabilities. A skill describes a workflow or supplies domain knowledge. Both can contribute to a harness. A rule becomes binding when the executing workflow checks it and acts on the result.

Superpowers · GitHub ↗

DATAMIMIC CE: a test-data world from the specification

An empty database leaves many requirements invisible. Models can state relationships, statuses, limits and domain exceptions explicitly. The implementation agent gets concrete cases to work against. The model remains a versionable file that people can review, maintain and extend.

DATAMIMIC CE supports agent authoring through its CLI and an optional MCP adapter. New intent stays in model.dm.json. The scaffold workflow compiles and checks the model, then performs a bounded trial with declared expectations. Structured diagnostics guide the next revision.

verified=true covers declared data expectations. It does not certify a complete specification or correct application behavior. Current Authoring V1 covers file and memstore targets with one level of nested child products. Database targets and more extensive models use the separate XML workflow.

For repeatable checks, version the model, inputs, seed and runtime. CE’s determinism claim is bounded by its documented, tested runtime profiles. A seed alone cannot make arbitrary external sources reproducible.

Example: 240 minutes available, 300 minutes assigned

Suppose the specification forbids overbooking. The test world contains an employee with 240 minutes of available capacity and jobs totaling 300 minutes. The behavior test requires the application to reject that schedule. Separately, ArchKeel checks the declared boundary between scheduling logic and data access.

This is an illustrative scenario, not a DATAMIMIC benchmark executed here. The data exposes the conflict. The expectation defines the correct response.

Turn intent into actionable feedback

1. Derive architecture rules, test states and expected behavior from the specification. Review these constraints before using them for acceptance.
2. Generate the test world and give the agent the relevant contract, scenarios and task.
3. Check data conditions, architecture and runtime behavior. Return specific violations, source locations and unresolved evidence.
4. Let the agent revise and check again against the same accepted constraints. Changes to those constraints need their own review.

The goal is short feedback loops with deterministic checks: deviations become concrete and verifiable. Whether a test-data world measurably improves implementation requires a comparison. If the model, code and tests inherit the same wrong assumption, every check can still pass.

How tight should the boundaries be?

Declare the architecture decisions the agent should not overturn on its own: components, responsibilities, interfaces and allowed dependencies. Leave implementation choices open inside those boundaries.

The constraint comes from the workflow: checks must be required. Changes to the contract, baselines and CI gates need controlled review. If the agent can weaken its own rules, the gate is weak.

What does ArchKeel actually check?

ArchKeel compares static code with declared boundaries. It reports forbidden imports, misplaced types and missing evidence with source locations. Checks run locally without an LLM. Their scope depends on the language profile and contract.

ArchKeel is not an agent sandbox and does not guarantee complete correctness. The harness controls tools and execution. Tests check runtime behavior. People decide which architecture makes sense.

Is an AGENTS.md with good rules enough?

An AGENTS.md gives the agent instructions and context. An executable architecture contract makes supported rules checkable against code. Use both: context before the change and concrete findings afterward.

OpenAI describes this pattern in its Harness Engineering article: check architecture boundaries mechanically and allow implementation choices within them. ArchKeel covers that architecture part. It does not replace the entire harness.

Why are green tests not enough?

A behavior test checks selected inputs and effects. It does not usually check which component imports another or whether static analysis can still resolve a call.

Fewer findings can also mean the analysis sees less. The example below shows one such case with recorded tests and check results.

Report, contract check or change gate?

A local replay checks the supplied records without authenticating them.

CLI reference / exit codes

Focused context for the agent

Main provides a compact architecture summary. The agent can then query components or findings from the saved snapshot. Global verdicts and UNKNOWN remain visible. The query does not check a working tree that has changed since the snapshot.

archkeel report --only architecture --json
archkeel report --input architecture.json --only violations --json

When is the change actually done?

Before the run, define the deliverable and the checks required for acceptance. The agent can write a report. The workflow needs to verify that the linked results exist and belong to the submitted state.

Consider this: tests pass on state A. The agent then changes the code to state B and carries over the old summary. That test run is evidence for A. Affected checks need to run again for B. After a merge, this also applies to the resulting integration state.

ArchKeel supplies architecture evidence. DATAMIMIC checks declared data conditions. Behavior tests check the application. The runner or CI workflow binds these results to the candidate and enforces acceptance rules. A hash identifies an artifact; by itself, it authenticates neither its origin nor a trusted check run.

A reviewer agent can find additional errors. Give it the requirements, diff and actual check output. A second summary is not independent proof. Missing or stale evidence needs to stay visible before acceptance.

Continue across sessions

Keep durable project rules in AGENTS.md, CLAUDE.md or relevant skills. Record the current work in a short handoff: goal, files and snapshot, completed checks, accepted decisions, open issues and one concrete next step.

Test that handoff in a fresh session. Have the agent inspect the referenced files and Git state, run relevant baseline checks and continue the open step. A progress.md can become stale. Its contents need to match the workspace.

Anthropic describes progress notes, Git history and checks at session start as parts of its long-running agent harness. Long-running agent harnesses ↗

Instructions and enforcement have different jobs

Skills and project files explain how to work. Tool permissions, sandboxing and required CI gates determine which actions are possible and which results may proceed. Inspect the actual configuration. A prompt sets neither a hard runtime budget nor process permissions.

The Claude Code documentation gives a concrete limit: Read and Edit deny rules do not cover arbitrary Python or Node processes that open files themselves. Those boundaries need suitable sandbox or operating-system controls. Claude Code permissions ↗

Test the harness too

Use a small, known task to check whether your workflow catches stale test output, an omitted required check and a supported architecture violation. Also try weakening the contract or baseline: the workflow should require the intended review.

Compare harness changes on the same tasks and acceptance criteria across multiple runs. Record accepted outcomes, missed errors, human review time, runtime and costs including retries. Change one component per comparison where possible. Fewer tokens help only when the total work needed for an accepted result goes down.

Anthropic’s guide to agent evals separates the conversation trace from the actual outcome in the environment. That distinction is the basis for acceptance in our engineering harness. Agent evals ↗

Executed / Python

Eight green tests. One newly unresolved call.

A tiny account-adjustment function handles credit and debit in integer units. The refactor replaces explicit branches with dictionary dispatch. Both versions return the same tested values and raise the same KeyError for unknown operation names.

from sample.helpers import credit, debit


def run(key: str, amount: int) -> int:
    if key == 'credit':
        return credit(amount)
    if key == 'debit':
        return debit(amount)
    raise KeyError(key)

from sample.helpers import credit, debit


def run(key: str, amount: int) -> int:
    handlers = {'credit': credit, 'debit': debit}
    return handlers[key](amount)

The analyzer resolves both helper calls and the explicit KeyError constructor before the change. Afterward, handlers[key] is dynamic. These numbers count static call sites, not runtime executions. No rule violation is introduced; the call-resolution regression is enough to reject the candidate.

expectation_fulfilled: FAIL
regression check failed in calls_unresolved: 0->1
regression check failed in unresolved_ratio: 0/3->1/1

From result to finding to source

Open the actual check report and find the regression failures and unresolved_call_changes. The finding names sample.work.run, sample/work.py:6 and handlers[key], with the reason “expression is dynamic”.

Reproduce the result

Download the complete example, extract it and run these commands from its directory. Python 3.12 and Git are required; installation needs network access. The recorded run used CPython 3.12.14 on Linux.

Download source and evidence (.zip)

python3.12 -m venv .venv
.venv/bin/python -m pip install -r requirements.txt
.venv/bin/python -B run.py --output ./replay

The output directory must be new. The runner executes the identical unittest suite on both revisions, records the Git baseline and expectation, and invokes the real ArchKeel CLI. The runner exits 0 when the expected outcomes are verified; the nested ArchKeel check exits 1.

Expected and observed: 8/8 tests before, 8/8 after; calls_unresolved 0 → 1; expectation_fulfilled FAIL. A negative control with an incorrect debit operation fails 5 of 8 tests. The tests catch that real behavior defect.

Read the verdict in its scope

Observation completeness, declared rules and change expectations are separate verdicts. A scan may complete while calls remain unresolved. NOT CHECKED means a check was not performed or could not be completed. Always read diagnostics and exit status together.

ArchKeel checks its own architecture

The Main report from 7 October 2026 contains 123 observed files and 32 components. 48 rules are PASS, four UNKNOWN, none FAIL. The overall result remains UNKNOWN. Explore the same snapshot on the homepage or the continually published report on GitHub Pages.

Historical evidence: 0.8.4 exact-module ownership reports

The older 0.8.4 evidence below remains available for the original module-ownership case. It belongs to neither the current Main report nor the dispatch example.

Version, commands and evidence

Verified on 2 October 2026 against the then-current repository commit. Package metadata: 0.8.5.dev1+g340176715, a development commit after release 0.8.4. This is not a released 0.8.5.

34017671559a19bfe5fd909689400fb5fb0a6317

Who decides the rules?

The agent can propose boundaries and work through findings. The target architecture needs an accountable decision maker. Decide who accepts contract changes and how your workflow handles FAIL and UNKNOWN. Otherwise you document boundaries without enforcing them.