Skip to content

Security: TypicalDefender/theta

Security

docs/SECURITY.md

Security Model

Trust model

This local, single-user harness runs untrusted repository code behind a stronger boundary than a host subprocess. The trusted computing base includes the host application, bundle author, base image or Dockerfile, Docker daemon, evaluator runner, hidden tests, and gold patch. Repository code, solver output, candidate files, test processes, provider responses, and model messages are untrusted.

Docker isolates task execution, but the Docker daemon and host kernel remain trusted. This is not a hostile multi-tenant VM boundary. Deployments exposed to container escape, kernel denial-of-service, or mutually distrustful tenants need a dedicated remote worker or microVM.

Container isolation

Every solver and evaluator session uses:

  • a fresh named volume initialized from the exact, clean tracked commit;
  • a read-only root filesystem and bounded tmpfs;
  • a validated numeric identity with non-zero UID and GID;
  • a workspace path disjoint from harness-owned temporary, evaluator, initialization, and capture mounts;
  • --network none;
  • all Linux capabilities dropped and no-new-privileges;
  • CPU, memory, PID, command-time, output, candidate-file, and patch-size limits;
  • no host bind mounts, Docker socket, device, privileged mode, or provider credential environment;
  • deterministic container and volume cleanup.

Host process I/O is multiplexed without reader threads and drained continuously into constant-space head and tail buffers. Output is bounded during capture, not truncated after being written to an unbounded temporary file. Post-exit draining is also bounded so a detached descendant cannot block the driver. Solver-facing stdout and stderr use the bundle's output_bytes limit even when trusted capture has a larger host allowance. Candidate patches are read through patch_bytes + 1; a CLI timeout may reduce the manifest's solver limit but cannot increase it.

The prepared image is inspected once. For a prebuilt image, the declared registry manifest digest must appear in Docker's RepoDigests; that verified manifest is the portable identity persisted by the harness. Every later Docker invocation uses the separately inspected daemon-local image ID rather than the mutable tag. Classic Docker and containerd-backed image stores can expose different .Id values for the same manifest, so the local ID never replaces the portable identity. Dockerfile builds use the inspected image ID for both roles on that daemon and are persisted as local_build / non-portable. They do not provide a resolvable identity on another machine. Repository identity is read from git rev-parse HEAD inside the image; image-authored metadata is not accepted as commit authority.

The initialization container is the only privileged phase. It is networkless, read-only, and receives only CHOWN, DAC_OVERRIDE, and FOWNER so it can copy the verified tree into a named volume and change ownership. Trusted capture mounts that volume read-only. Evaluator assets live in a separate tmpfs and become root-owned and read-only before candidate tests execute.

Hidden-test isolation

The base image and reusable cache never contain bundle evaluation assets. Solver prompts contain only description.md; the bundle directory is not mounted. After solver destruction, the evaluator applies the captured candidate to a new clean volume, applies the hidden test patch, and writes the runner/lists into evaluator-only tmpfs.

Candidate code can inspect tests after evaluation starts because both execute within the evaluator boundary. Hidden inputs remain secret through solver completion; they are not protected from malicious candidate code after materialization. The evaluator provides authoritative measurement, not ongoing test secrecy.

Evaluator report authority

The runner and exact test lists are installed as root-owned, read-only assets, and the harness accepts only the versioned report with the exact declared test IDs. Process exit status and candidate prose never decide the outcome.

This local Docker design does not make the report cryptographically independent of adversarial candidate code. Candidate code and the repository-specific test runner execute inside the same evaluator container and numeric runtime identity; after hidden materialization, malicious code could interfere with its process or writable result directory. The trusted computing base therefore includes the bundle author and evaluator runner, and the harness is intended for task development and honest-but-buggy candidates. A hostile benchmark service would need a separate trusted runner boundary—such as a supervisor/worker protocol or dedicated VM—that receives only authenticated test results from the candidate environment. Root-owned files alone are not claimed as that stronger boundary.

Patch capture

The harness does not rely on a solver-reported diff. It pauses the solver container and uses:

  • immutable Git objects from the verified image;
  • a separate temporary index and object directory;
  • a read-only candidate-volume mount;
  • disabled Git hooks, system/global config, attributes, external diff, and text conversion;
  • allowed Git modes, file-count limits, and binary patch-size limits.

The resulting full-index binary patch is checksummed before it is applied in a fresh evaluator. Candidate patch paths supplied to the stub must be regular, non-symlink files and are size-bounded before use.

Provider and Codex isolation

Models run host-side because authentication and provider networking do not belong in the task container. The flat adapter only serializes three strict function schemas. It rejects parallel calls, repeated call IDs, malformed arguments, excess turns, tool calls, tokens, or elapsed time. Each provider request receives the remaining solver deadline, while the Docker tool surface shares the same wall-time budget.

The Codex adapter requires an API key and a CLI with the expected secure flags. It pipes OPENAI_API_KEY to codex login --with-api-key inside an ephemeral CODEX_HOME; the secret is never placed in argv or the child environment, and the temporary auth file is destroyed with the solver process. Codex then starts with that empty HOME, ignores user configuration and repository rules, uses an empty read-only working directory, disables shell/web/apps/ hooks/goals/plugins/browser/computer/memory/subagent features, and requires a single run-scoped MCP server. That server uses an ephemeral Unix socket and capability token and only publishes run_command, read_file, and apply_patch. JSONL output is audited fail-closed: unknown item types, other servers/tools, malformed events, or incomplete turns fail the solver. Secure- flag preflight and execution spend one monotonic solver deadline; preflight cannot add a second timeout allowance. The MCP transport uses that remaining deadline rather than a fixed timeout, leaving Docker command limits and the outer solver deadline as the authoritative bounds.

Codex can report missing local metadata for an explicitly selected model as an item-level error even when the turn completes successfully. The audit accepts only that exact, nonfatal message shape, records a redacted warning category, and continues to reject every other top-level or item-level error. In particular, an app-server stream-lag notice remains fatal because it says audit events were dropped. This compatibility rule does not expand the server, tool, command, or filesystem allowlists.

Flat-provider credentials remain in the sanitized host client. Codex credentials cross only the login process's stdin and temporary auth file. Neither is written to SQLite, artifacts, bundle files, Docker configuration, prompts, or task processes.

Durable data and redaction

Structured arguments, configurations, summaries, event data, test excerpts, reports, and debug exports pass through key/string redaction. Sensitive key names, bearer values, URL credentials, and the current home path are replaced. Process output is bounded before persistence. Artifact creation uses a fresh UUID-owned private directory, atomic writes, 0600 files, and SHA-256 registration. A failed registration deletes only those new bytes and cannot replace an earlier artifact. Downloads recheck path containment, regular-file status, size, and checksum.

SQLite uses parameterized queries, foreign keys, explicit status constraints, and short transactions. Dynamic SQL only assembles fixed clauses; values stay parameterized. Bundle and artifact paths reject absolute traversal, NULs, backslashes, and symlink aliases.

Local UI boundary

The server binds only to 127.0.0.1, accepts only explicit loopback Host and Origin values, and returns a restrictive content policy, frame denial, no-referrer, and content-type protections. It has no remote deployment mode, authentication, raw terminal, file browser, or arbitrary RPC endpoint. The UI can request only validation, registered solver runs, cooperative cancellation, bounded reads, and registered artifact downloads. Do not expose the port via a public proxy without adding authentication, CSRF protection, tenant isolation, and a stronger execution boundary.

Residual risks

  • A Docker/kernel vulnerability can cross the container boundary.
  • CPU/memory/PID limits do not bound all host I/O pressure or Docker daemon resource usage.
  • Build and image-pull stages use network and trust their declared sources.
  • A trusted base image can itself be malicious.
  • Cooperative cancellation is observed between model/tool boundaries. A provider SDK request already in flight completes or reaches its bounded remaining timeout; the production Codex process group is terminated immediately when polled.
  • Architecture emulation changes performance and can reveal platform-specific failures even with identical functional inputs.

There aren't any published security advisories