Reproducible/deployable environment for LiteLLM proxy/admin server

.gitattributes

@@ -0,0 +1,2 @@
+# SCM syntax highlighting & preventing 3-way merges
+pixi.lock merge=binary linguist-language=YAML linguist-generated=true -diff

.gitignore

@@ -23,3 +23,6 @@ notifiers/*
 
 # This installation's own settings: gateways, credentials, what is available here.
 lab.env
+# pixi environments
+.pixi/*
+!.pixi/config.toml

AGENTS.md

@@ -108,6 +108,46 @@ case to stop the agent if it has not met the users goal such as total tasks or c
 wallclock. `docs/settings.md` has every setting. If their files already indicate these,
 use that and tell them what you set. Otherwise ask, and offer the defaults.
 
+### LiteLLM proxy notes
+
+When LiteLLM fronts an OpenAI-compatible backend for the Claude Agent SDK:
+
+- The SDK sends Anthropic Messages API requests to `/v1/messages`. Set
+  `use_chat_completions_url_for_anthropic_messages: true` so LiteLLM translates them to
+  the backend's Chat Completions API.
+- Keep credentials separate. `LITELLM_MASTER_KEY` authenticates callers to the proxy;
+  `ARGO_API_KEY` (or the backend-specific secret) belongs in the proxy config as the
+  upstream `api_key`. Never commit either secret.
+- Ask which models the user wants campaigns to use. With the repository's current
+  configuration, each model needs a `model_list` entry in `framework/config.yaml`; the
+  proxy exposes the entry's `model_name` as the name the SDK requests and maps it to the
+  backend `litellm_params.model`. Ask for the exact backend model IDs and choose stable,
+  readable aliases. Add one entry per model, reusing the endpoint and environment-backed
+  credential where appropriate; set the selected alias as `AGENT_MODEL` in untracked
+  `lab.env`, and restart the proxy after changing the file.
+- A model not listed in `model_list` is not available through the normal configured proxy
+  route. LiteLLM has pass-through modes, but they are not enabled here because they make
+  model exposure and backend routing less explicit; do not rely on them during setup.
+- A user's `~/.claude/settings.json` can point the SDK directly at another gateway and
+  override the intended route. For a campaign that must use LiteLLM, set
+  `CLAUDE_CONFIG_DIR` in its `run.sh` and provide a campaign-local `claude/settings.json`
+  with the proxy URL and model.
+- Anthropic's `context_management` request field is server-side context-clearing control,
+  not context usage information consumed by the workflow. An OpenAI-compatible backend
+  cannot implement it. Drop only that field with the model's
+  `additional_drop_params: [context_management]`; do not enable global `drop_params`,
+  which can hide other incompatible SDK parameters. `client.get_context_usage()` is a
+  separate CLI query for token reporting and does not require `context_management`.
+- LiteLLM's packaged distribution may include `schema.prisma` without migration files.
+  The local Postgres database must be initialized with `prisma db push --schema=<schema>
+  --skip-generate` after `DATABASE_URL` is exported and before starting the proxy;
+  otherwise dashboard or key-management requests can fail with a missing relation such
+  as `LiteLLM_VerificationToken`.
+- The repository launcher owns this setup: use `pixi run litellm-proxy-start` and
+  `pixi run litellm-proxy-stop`. It uses Postgres port 5433, proxy port 4000, and binds
+  the proxy to `127.0.0.1` by default. Restart the proxy after changing
+  `framework/config.yaml`.
+
 ### 5. The machine
 
 Ask which system. If `systems/<system>.json` exists you have its module line, proxy,

campaigns/example-quick-optimum/run.sh

@@ -14,6 +14,13 @@ export PATH="$HOME/.local/bin:$PATH"
 # What this installation has available -- model gateways, credentials. Anything below
 # overrides it, since a campaign knows its own needs. Settings: docs/settings.md
 . ../../framework/lab_env.sh
+# The user-wide Claude settings can point directly at another gateway. This example
+# uses the LiteLLM proxy and the installation's configured proxy model instead.
+: "${LITELLM_MASTER_KEY:?Set LITELLM_MASTER_KEY to the key used to start LiteLLM}"
+: "${AGENT_MODEL:?Set AGENT_MODEL in lab.env to a model_name in framework/config.yaml}"
+export CLAUDE_CONFIG_DIR="$PWD/claude"
+export ANTHROPIC_BASE_URL="${ANTHROPIC_BASE_URL:-http://127.0.0.1:4000}"
+export ANTHROPIC_API_KEY="$LITELLM_MASTER_KEY"
 export CAMPAIGN="$(basename "$PWD")"
 export USER_NAME="${USER_NAME:-$USER}"
 

docs/llm.md

@@ -30,20 +30,32 @@ SDK's, unchanged.
 format and routes it to any provider it supports. Running it in front of an
 OpenAI-style backend lets a campaign run on that backend through the same SDK path.
 
-Install it in its own environment, since its proxy extra pulls in a large dependency
-set:
+The repository's pixi environment includes LiteLLM, its proxy dependencies, PostgreSQL,
+and Prisma. From the repository root, start the proxy and its local database with:
 
 ```
-python -m venv ~/venvs/litellm && ~/venvs/litellm/bin/pip install "litellm[proxy]" "fastapi<0.140.7"
+pixi install
+pixi run litellm-proxy-start
 ```
 
-The FastAPI cap is needed as of LiteLLM 1.97.0. LiteLLM imports
-`fastapi.dependencies.utils.get_flat_dependant`, which FastAPI removed in 0.140.7, and
-LiteLLM declares `fastapi>=0.136.3,<1.0` — no upper bound below the removal — so an
-uncapped install takes a FastAPI the proxy cannot import. The working range is 0.136.3
-to 0.140.6. Drop the cap once a LiteLLM release no longer needs it; keep it inside that
-range rather than pinning further back, since below 0.136.3 is outside what LiteLLM
-supports.
+This initializes PostgreSQL under `scratch/litellm-postgres`, uses TCP port 5433 for the
+database, and serves the proxy on port 4000. The database is local and gitignored. 
+The first start generates LiteLLM's Prisma client and applies its packaged schema to the
+local database. Stop the proxy cleanly with Ctrl-C in the foreground terminal. The launcher stops
+LiteLLM first and PostgreSQL second. You can also stop both services from another
+terminal with:
+
+```
+pixi run litellm-proxy-stop
+```
+
+Set `LITELLM_MASTER_KEY` and `LITELLM_SALT_KEY` before starting when using this beyond
+local development. The defaults are intentionally only suitable for a local machine.
+Set `ARGO_API_KEY` to the Argo bearer token before starting; the repository's
+`framework/config.yaml` passes it upstream without storing it in Git. Use `LITELLM_PORT`,
+`LITELLM_DB_PORT`, or `LITELLM_PGDATA` to change the proxy port, database port, or
+database directory. The proxy binds to `127.0.0.1` by default; set `LITELLM_HOST`
+deliberately for remote access and use a strong master key.
 
 Write a config naming the upstream model, its endpoint, and the key to reach it:
 
@@ -63,30 +75,31 @@ litellm_settings:
 upstream. Without it LiteLLM translates `/v1/messages` to the OpenAI Responses API,
 and a backend that implements only `/v1/chat/completions` answers 404.
 
-Start the proxy:
+Start the repository-managed proxy:
 
 ```
-~/venvs/litellm/bin/litellm --config config.yaml --port 4000
+pixi run litellm-proxy-start
 ```
 
-Check it before pointing the agent at it:
+Check it before pointing the agent at it. `LITELLM_MASTER_KEY` authenticates the
+Agent SDK to the proxy; the proxy's configured `api_key` authenticates to the upstream:
 
 ```
-curl -s -X POST http://0.0.0.0:4000/v1/messages -H 'content-type: application/json' -H 'x-api-key: <key>' -H 'anthropic-version: 2023-06-01' -d '{"model":"my-model","max_tokens":64,"messages":[{"role":"user","content":"say hi"}]}'
+curl -s -X POST http://127.0.0.1:4000/v1/messages -H 'content-type: application/json' -H "x-api-key: $LITELLM_MASTER_KEY" -H 'anthropic-version: 2023-06-01' -d '{"model":"my-model","max_tokens":64,"messages":[{"role":"user","content":"say hi"}]}'
 ```
 
-Then point the settings at the proxy:
+Then configure the SDK process (or its `~/.claude/settings.json`) with the proxy URL,
+model, and the same proxy master key:
 
-```json
-{
-  "env": {"ANTHROPIC_BASE_URL": "http://0.0.0.0:4000", "ANTHROPIC_API_KEY": "<key>"},
-  "model": "my-model"
-}
+```sh
+export ANTHROPIC_BASE_URL=http://127.0.0.1:4000
+export ANTHROPIC_API_KEY="$LITELLM_MASTER_KEY"
+export AGENT_MODEL=my-model
 ```
 
-LiteLLM passes the caller's credential upstream, so `ANTHROPIC_API_KEY` has to be one
-the backend accepts, not an arbitrary string. Where the backend authenticates by
-username, that username is the value.
+Use `127.0.0.1`, not `0.0.0.0`, as a client URL. Do not put the upstream credential in
+`ANTHROPIC_API_KEY` unless the LiteLLM configuration deliberately uses caller-provided
+credentials instead of its configured `api_key`.
 
 ## Offering several models from one proxy
 
@@ -115,16 +128,15 @@ entry uses the `openai/` handler regardless of who made the model — the handle
 the wire format, not the vendor. Reserve `gemini/` and `anthropic/` for going to those
 vendors directly.
 
-Each person then names the model they want:
+Each person then names the model they want and authenticates to the proxy:
 
 ```json
-{"env": {"ANTHROPIC_BASE_URL": "http://<proxy-host>:4000", "ANTHROPIC_API_KEY": "<their key>"}, "model": "gemini"}
+{"env": {"ANTHROPIC_BASE_URL": "http://<proxy-host>:4000", "ANTHROPIC_API_KEY": "<proxy key>"}, "model": "gemini"}
 ```
 
-LiteLLM passes the caller's credential upstream rather than substituting the one in the
-config, so each person's own key reaches the backend and usage is attributed to them.
-That also means the proxy should only be reachable from where those credentials are
-already trusted.
+Each model's configured `api_key` is used upstream. To attribute upstream usage to each
+caller instead, configure LiteLLM explicitly for caller-provided credentials and limit
+proxy access to the trusted network.
 
 Mixing an Anthropic-native upstream into the same config is untested here.
 `use_chat_completions_url_for_anthropic_messages` applies proxy-wide, so a Claude model

docs/settings.md

@@ -75,6 +75,7 @@ Less often changed.
 | `TASK_DIR` | the campaign directory | where `task.py` is found |
 | `TASK_MODULE` | task | module name within `TASK_DIR` |
 | `CLAUDE_CONFIG_DIR` | `~/.claude` | directory holding the Claude Code `settings.json` that decides which LLM the agent uses. `docs/llm.md` |
+| `AGENT_MODEL` | unset | explicit model name passed to the Agent SDK; normally set as the proxy alias in untracked `lab.env`, otherwise Claude Code's configured model applies |
 | `CLAIM_STALE_SECONDS` | 21600 | before an unfinished claim can be taken over |
 | `ANNOUNCE_POLL` | 2 | seconds between announcement-board checks while waiting |
 

framework/agent.py

@@ -584,7 +584,8 @@ def preflight():
     if note:
         print(f"critic: {note}", flush=True)
     try:
-        CRITIC_MODEL, CRITIC_LABEL = critic.resolve(os.environ.get("ANTHROPIC_MODEL", ""))
+        CRITIC_MODEL, CRITIC_LABEL = critic.resolve(
+            os.environ.get("AGENT_MODEL") or os.environ.get("ANTHROPIC_MODEL", ""))
     except critic.CriticUnavailable as e:
         print(f"preflight FAILED: {e}", flush=True)
         sys.exit(1)
@@ -637,6 +638,9 @@ async def main():
         permission_mode="bypassPermissions",
         system_prompt=system_prompt,
         cwd=SCRIPT_DIR,
+        # The lab configuration supplies an explicit model when campaigns use a shared
+        # LiteLLM proxy; otherwise Claude Code's own configured default still applies.
+        **({"model": os.environ["AGENT_MODEL"]} if os.environ.get("AGENT_MODEL") else {}),
     )
 
     results_file = os.path.join(WORKSPACE_DIR, "results.jsonl")

framework/run_litellm.sh

@@ -0,0 +1,113 @@
+#!/usr/bin/env bash
+# Run LiteLLM with a local Postgres database managed by pixi.
+#
+# Usage:
+#   pixi run litellm-proxy-start  # start PostgreSQL and the proxy
+#   pixi run litellm-proxy-stop   # stop the proxy, then PostgreSQL
+#
+# Press Ctrl-C in the start command to stop the proxy before PostgreSQL.
+# Override LITELLM_PORT, LITELLM_DB_PORT, LITELLM_MASTER_KEY,
+# LITELLM_SALT_KEY, LITELLM_PGDATA, or LITELLM_HOST when needed.
+set -euo pipefail
+
+ROOT_DIR="$(cd "$(dirname "$0")/.." && pwd)"
+PGDATA="${LITELLM_PGDATA:-$ROOT_DIR/scratch/litellm-postgres}"
+PGLOG="${LITELLM_PGLOG:-$ROOT_DIR/scratch/litellm-postgres.log}"
+PIDFILE="${LITELLM_PIDFILE:-$ROOT_DIR/scratch/litellm-proxy.pid}"
+DB_PORT="${LITELLM_DB_PORT:-5433}"
+PROXY_PORT="${LITELLM_PORT:-4000}"
+# Keep the development key local by default. Set LITELLM_HOST deliberately when the
+# proxy must accept remote connections, together with a strong LITELLM_MASTER_KEY.
+PROXY_HOST="${LITELLM_HOST:-127.0.0.1}"
+DB_USER="${LITELLM_DB_USER:-litellm}"
+DB_NAME="${LITELLM_DB_NAME:-litellm}"
+DB_URL="${DATABASE_URL:-postgresql://${DB_USER}@127.0.0.1:${DB_PORT}/${DB_NAME}}"
+
+stop_proxy() {
+    local pid=""
+    if [ -f "$PIDFILE" ]; then
+        pid="$(cat "$PIDFILE")"
+    fi
+    if [ -n "$pid" ] && kill -0 "$pid" 2>/dev/null; then
+        kill -TERM "$pid"
+        for _ in $(seq 1 30); do
+            kill -0 "$pid" 2>/dev/null || break
+            sleep 1
+        done
+        if kill -0 "$pid" 2>/dev/null; then
+            echo "LiteLLM did not stop within 30 seconds" >&2
+            return 1
+        fi
+    fi
+    rm -f "$PIDFILE"
+}
+
+stop_db() {
+    if [ -d "$PGDATA" ] && pg_ctl -D "$PGDATA" status >/dev/null 2>&1; then
+        pg_ctl -D "$PGDATA" -m fast stop
+    fi
+}
+
+if [ "${1:-start}" = "stop" ]; then
+    stop_proxy
+    stop_db
+    exit 0
+fi
+
+if [ "${1:-start}" != "start" ]; then
+    echo "usage: $0 [start|stop]" >&2
+    exit 2
+fi
+
+mkdir -p "$(dirname "$PGDATA")"
+if [ ! -f "$PGDATA/PG_VERSION" ]; then
+    initdb -D "$PGDATA" -U "$DB_USER" --auth=trust --no-instructions
+fi
+
+if pg_ctl -D "$PGDATA" status >/dev/null 2>&1; then
+    if ! pg_isready -h 127.0.0.1 -p "$DB_PORT" >/dev/null 2>&1; then
+        echo "Postgres data directory is running, but not on port $DB_PORT: $PGDATA" >&2
+        exit 1
+    fi
+else
+    pg_ctl -D "$PGDATA" -o "-p $DB_PORT -h 127.0.0.1" -l "$PGLOG" start
+fi
+
+until pg_isready -h 127.0.0.1 -p "$DB_PORT" >/dev/null 2>&1; do
+    sleep 1
+done
+
+if ! psql -h 127.0.0.1 -p "$DB_PORT" -U "$DB_USER" -d postgres -Atqc \
+    "SELECT 1 FROM pg_database WHERE datname = '$DB_NAME'" | grep -q '^1$'; then
+    createdb -h 127.0.0.1 -p "$DB_PORT" -U "$DB_USER" "$DB_NAME"
+fi
+
+export DATABASE_URL="$DB_URL"
+SCHEMA="$(python -c 'import os, litellm; print(os.path.join(os.path.dirname(litellm.__file__), "proxy", "schema.prisma"))')"
+if [ ! -f "$(dirname "$SCHEMA")/client.py" ]; then
+    prisma generate --schema="$SCHEMA"
+fi
+# LiteLLM distributes a Prisma schema rather than migration files. Apply that schema
+# before starting so the proxy never reaches its dashboard with missing tables.
+prisma db push --schema="$SCHEMA" --skip-generate
+
+export LITELLM_MASTER_KEY="${LITELLM_MASTER_KEY:-sk-1234}"
+export LITELLM_SALT_KEY="${LITELLM_SALT_KEY:-sk-local-development-only}"
+export STORE_MODEL_IN_DB="${STORE_MODEL_IN_DB:-True}"
+
+litellm --config "$ROOT_DIR/framework/config.yaml" --host "$PROXY_HOST" --port "$PROXY_PORT" &
+proxy_pid=$!
+printf '%s\n' "$proxy_pid" > "$PIDFILE"
+cleanup() {
+    trap - INT TERM EXIT
+    stop_proxy || true
+    stop_db || true
+}
+trap cleanup INT TERM EXIT
+
+set +e
+wait "$proxy_pid"
+proxy_status=$?
+set -e
+cleanup
+exit "$proxy_status"

lab.env.template

@@ -9,6 +9,11 @@
 #
 # Every setting here is listed with its default in docs/settings.md.
 
+# --- Agent model ----------------------------------------------------------------------
+# The LiteLLM model_name alias campaigns request. Leave empty to let Claude Code's own
+# settings select a model. A LiteLLM-backed campaign can require this explicitly.
+export AGENT_MODEL="${AGENT_MODEL:-}"
+
 # --- A second model, available for reviewing a cycle's write-up (docs/llm.md) --------
 # This says what this installation can reach. Whether a run uses a critic is decided by
 # the campaign or on the command line -- CRITIC_MODEL=auto ./run.sh -- because it costs