SandBaseSandBase
Start building

Usage · 09

Headless CLI Automation: Bring the Harness into Scripts and CI

Run one fresh persisted task from a controlled workspace, interpret stdout and exits, inspect composition, enforce permissions, and shut down safely.

Reading time
18 minutes
Sources verified

Understand the one-shot contract

The shipped headless profile is an application, not merely Web mode without a page. One invocation creates one fresh persisted Agent through the core registry, submits the positional task as an ordinary user message, waits for whole-agent quiescence, flushes its Session, derives the last non-empty assistant text and final turn/end reason from the owned durable interval, prints, and requests process exit.

It mounts no ApiProxy, Host, HTTP server, Web runtime, or browser client and opens no listening port. It does not resume a named session or accept a prompt over stdin. Use the SDK when a program needs session reuse, event callbacks, structured results, or multiple tasks in one runtime. Use headless when a shell-level one-task contract is enough.

text
process start → profile composition → fresh persisted Agent
→ submit one task → wait for quiescence → flush Session
→ fold owned event interval → stdout/stderr → bounded disposal → exit

Choose the working directory deliberately

The directory from which dsh is invoked becomes the default workspace root. It also supplies applicable AGENTS.md or CLAUDE.md instructions, project .env fallback, and the paths the agent sees in its coding prompt. In CI, cd to the exact checkout before launching; running from a parent monorepo, temporary wrapper directory, or secrets-bearing home changes both context and reachable files.

bash
workspace=/ci/work/acme-service
cd "$workspace"
pwd
test -f package.json
dsh --profile headless "Run the focused tests and summarize failures."

Prefer an isolated checkout or ephemeral worktree. Confirm the revision and clean/dirty state before allowing writes. The default workspace-write preset fences Bash and filesystem mutation to the session workspace and platform temporary roots, but reads, network access, and process visibility are not fully confined. Workspace selection is therefore necessary but not sufficient isolation.

Respect the launcher and app argument boundary

The launcher parses its flags first and stops at the first token it does not recognize. Everything after that boundary is passed verbatim through ctx.cmdlineArgs to the profile application. Headless interprets all positional task words as one joined task and requires non-whitespace content. Launcher flags such as --profile and --patch must appear before task text.

bash
dsh --profile headless --patch ./ci.cordis.yml "inspect this checkout"
dsh --profile headless --help
dsh --help
# The first help is the headless app; the second is launcher help.

The launcher consumes one -- boundary. If an application must literally receive --, pass -- --. Quote task text so shell globbing, variable expansion, or redirection does not silently rewrite it. Do not interpolate untrusted issue text directly into a shell command; pass it as one quoted argument from a process API.

typescript
spawn('dsh', ['--profile', 'headless', task], {
  cwd: workspace,
  stdio: ['ignore', 'pipe', 'pipe'],
  shell: false,
})

Treat stdout and exit status as separate outputs

After the Session flushes, headless writes the last non-empty assistant text to stdout. It exits 0 only when the final durable turn/end reason is completed; every other final reason exits 1. A terminal error reason additionally writes its stable code and message to stderr. Successful runs keep stderr empty. Empty stdout is therefore not itself an exit contract, and polished prose does not override a nonzero status.

bash
result_file=artifacts/dsh-answer.txt
error_file=artifacts/dsh-error.txt
set +e
dsh --profile headless "Review changed files; do not edit." >"$result_file" 2>"$error_file"
status=$?
set -e
printf 'dsh_exit=%s\n' "$status"
test "$status" -eq 0

Keep stdout free for the assistant result and route wrapper diagnostics elsewhere. If a downstream machine needs JSON, do not parse prose heuristically; use the SDK or add an explicitly versioned output adapter. Redact both streams before publishing because model text and terminal errors may contain repository paths or task content.

Inspect composition without executing the task

Use --dump-default-config to print bundle layers only. Use --dump-config to add the profile patch, home-level patch, and each --patch overlay in command order. Dumps include provenance comments, leave !!js expressions unevaluated, report unmatched patch targets on stderr, and do not boot plugins or parse application arguments. A dump carrying task text is rejected.

bash
dsh --profile headless --dump-default-config > artifacts/headless-default.yml
dsh --profile headless --patch ./ci.cordis.yml --dump-config > artifacts/headless-resolved.yml
diff -u artifacts/headless-default.yml artifacts/headless-resolved.yml

Review model adapter, credentials reference, persistence, tools, approval policy, sandbox policy, and executor rows. Remember that a patch replaces the targeted row's complete config rather than deep-merging fields. A config dump can reveal local paths, provider endpoints, executable commands, and credential references even though literal credentials should be absent; handle it as diagnostic data.

Compare the checked-in CI patch with the resolved dump during review, not only after a failure. Home-level patches outrank the profile layer and can make a developer machine behave differently from a clean runner. In reproducible automation, point DSH_HOME at controlled job state or inventory its contents, and hash every overlay. Dumps do not evaluate app-dependent expressions, so a successful dump proves composition syntax and precedence, not successful activation.

Make unattended permission behavior deterministic

A headless process has no natural browser approval channel. New sessions default to workspace-write, which bundles sandbox mode and approval policy for that session. DSH_PERMISSION_MODE changes the process fallback; stored General-settings permissions are aimed at later Web sessions, not an already-running headless task. Verify the actual profile rather than assuming a UI selection carries over.

Unanswerable approval requests fail closed. Design the task and composition so expected actions are pre-authorized within a narrow sandbox and unexpected actions are denied, instead of installing an automatic yes responder. A permission decision answers whether an action may proceed; the sandbox enforces where supported filesystem and subprocess effects can reach. Neither replaces minimal credentials and isolated infrastructure.

text
CI policy
workspace: disposable checkout
preset: workspace-write
expected: reads, tests, edits inside checkout
denied: mutation outside workspace or unanswered approval
credentials: job-scoped, least privilege, no production secrets
network: separately controlled by runner/container

Build a wrapper that preserves evidence

A reliable wrapper fixes cwd, task, profile revision, timeout, environment allowlist, and artifact locations. It captures stdout and stderr separately, preserves the real exit code, and records enough non-secret metadata to reproduce the run. Use an outer supervisor timeout that sends SIGTERM first, allowing the harness to drain, before escalating according to platform policy.

bash
set -euo pipefail
run_dir=artifacts/dsh-$(date -u +%Y%m%dT%H%M%SZ)
mkdir -p "$run_dir"
set +e
timeout --signal=TERM --kill-after=10s 20m \
  dsh --profile headless --patch ./ci.cordis.yml \
  "Run unit tests. Do not access network or modify files." \
  >"$run_dir/stdout.txt" 2>"$run_dir/stderr.txt"
status=$?
set -e
printf '%s\n' "$status" >"$run_dir/exit-code.txt
test "$status" -eq 0

Use the timeout utility available on your CI image; macOS does not ship GNU timeout. Do not log the full environment. Record harness version, git revision, profile and patch hashes, workspace identifier, start/end time, and exit status. Store exported session diagnostics only after redaction and according to retention policy.

Design the task as an automation contract

Prompt text should state the objective, permitted scope, observable success criteria, and prohibited effects. Avoid vague requests such as ‘fix everything’ in a privileged checkout. Name the test command, directories that may change, whether network is permitted, and what evidence the final response should summarize. Prompt restrictions are guidance for the model, so enforce the same boundaries through permissions, sandbox, credentials, and the CI runner.

Separate analysis from mutation when risk is material. A first read-only invocation can produce a plan or diagnostic report. A later job can review that artifact and run a narrowly authorized change in a fresh checkout. Because each invocation is a fresh session, carry only reviewed, non-secret facts forward explicitly; do not assume conversational memory between processes.

text
Objective: run npm test -- --runInBand and explain failures.
Allowed: read repository and write only artifacts/test-report.md.
Forbidden: package installation, network, git operations, source edits.
Success: command exit captured; report names failing test files.
Failure: stop after first infrastructure error; do not retry.

For unavoidable external effects, assign an idempotency key outside the model, use a service account limited to the target, and verify external state before retry. The assistant's final sentence is not a transaction receipt. Durable tool results, CI logs, and the external system's operation id together form the audit trail.

Classify failures before retrying

Argument errors, missing tasks, mutually incompatible dump flags, parse failures, configuration resolution errors, and plugin boot failures exit nonzero before an Agent runs. A model or tool failure can produce a durable non-completed turn, stdout text, stderr error, and exit 1. Signal termination and supervisor timeouts have their own meanings. Separate setup failure, task outcome, and infrastructure interruption in CI reporting.

text
Failure boundary → evidence
argv/usage → launcher or headless help/error; no task session
composition/boot → stderr + nonzero; no Agent completion
model/tool turn → persisted events + final turn/end + exit 1
SIGTERM → graceful supervisor stop, launcher exit 0
SIGINT → interactive interruption, exit 130
forced second signal/kill → disposal may be incomplete

Never blindly retry a task that can mutate files, deploy, publish, charge an API, or contact an external system. The first attempt may have committed a side effect before a later model request failed. Inspect the persisted session and external idempotency record, then resume through an appropriate stateful surface or start a compensating task.

Let shutdown flush and dispose

Normal one-shot completion flushes the Session before requesting exit. The launcher then gives the plugin tree up to five seconds to dispose, including profile watchers and owned resources. SIGTERM is the ordinary supervisor stop and exits 0 after graceful drain; SIGINT reports 130. A second signal forces immediate exit. If normal one-shot disposal is already stuck, the first Ctrl+C escalates immediately.

text
normal: runner flush → appExit(code) → dispose tree ≤5s → process exit
supervisor: SIGTERM → dispose tree ≤5s → exit 0
operator: SIGINT → dispose tree ≤5s → exit 130
second signal → immediate forced exit

A SIGTERM exit of 0 says the supervisor requested an orderly stop; it does not prove the task completed. Preserve the task's own durable outcome when available and let the CI timeout layer report interruption separately. Avoid kill -9 except after the drain budget, because it can leave persistence, subprocesses, or tool-owned resources without orderly cleanup.

Promote automation only after adversarial verification

Test a successful text-only task, an allowed workspace operation, a denied escape, missing credentials, invalid model, provider timeout, tool failure, SIGTERM during execution, stuck disposal, and a task that produces no final text. Confirm there is one fresh persisted session, contiguous durable events, a flushed final turn reason, correct stdout/stderr separation, and expected exit mapping.

text
Release checklist
[ ] cwd is explicit and disposable
[ ] argv passed without shell interpolation
[ ] resolved profile reviewed and pinned
[ ] permission and sandbox limits tested
[ ] secrets and network independently scoped
[ ] stdout, stderr, exit, timeout captured separately
[ ] side effects idempotent or compensatable
[ ] SIGTERM drain and forced-stop behavior tested
[ ] session artifacts redacted and retained appropriately
[ ] retry policy distinguishes committed effects

Begin with read-only repository inspection, then tests, then controlled workspace writes. Add external effects only after the wrapper can prove idempotency and correlate them with durable tool results. Run the same acceptance matrix on every harness upgrade and preserve representative redacted sessions for comparison. Review timeout, token cost, and provider budgets alongside safety boundaries before production promotion. Headless becomes reliable automation when its small contract is preserved, not when a prose answer is treated as an infallible build signal.

Official sources

Continue in this category