From cd9bca4735f9c2c512adf85200bb05d70b07c83d Mon Sep 17 00:00:00 2001 From: Cursor Agent Date: Thu, 13 Aug 2026 07:04:51 +0000 Subject: [PATCH 1/2] docs: persist OSS launch usability, fix, and security handoff docs Cloud Agent /opt/cursor/artifacts is ephemeral per VM and not shared across agents. Keep go-live planning under docs/oss-ux/ so developer agents can read it. Co-authored-by: Venkat SF --- docs/oss-ux/E2E_FIX_PROPOSAL.md | 359 +++++++++++++++++++ docs/oss-ux/OSS_LAUNCH_USABILITY_CRITIQUE.md | 105 ++++++ docs/oss-ux/OSS_SECURITY_REVIEW.md | 236 ++++++++++++ docs/oss-ux/README.md | 11 + 4 files changed, 711 insertions(+) create mode 100644 docs/oss-ux/E2E_FIX_PROPOSAL.md create mode 100644 docs/oss-ux/OSS_LAUNCH_USABILITY_CRITIQUE.md create mode 100644 docs/oss-ux/OSS_SECURITY_REVIEW.md create mode 100644 docs/oss-ux/README.md diff --git a/docs/oss-ux/E2E_FIX_PROPOSAL.md b/docs/oss-ux/E2E_FIX_PROPOSAL.md new file mode 100644 index 0000000..24cb710 --- /dev/null +++ b/docs/oss-ux/E2E_FIX_PROPOSAL.md @@ -0,0 +1,359 @@ +# DeepSQL OSS launch — end-to-end fix proposal + +**Date:** 2026-08-13 +**Scope:** Close every gap from the usability critique so the default path is: + +> install → login → (wizard if needed) → Brain indexes the real schema → Agent answers + +**Success metric for Sunday:** A fresh Docker install + `seed-demo-data.sh`, and a multi-schema Postgres DB, both complete “ask about largest tables / row counts” without auth or “Brain ready but empty” failures. + +**Companion:** Security track is in [`OSS_SECURITY_REVIEW.md`](./OSS_SECURITY_REVIEW.md) (run in parallel; gate “internet multi-user” on S1–S5). + +--- + +## Guiding principles + +1. **One source of truth for “ready.”** Connection green + Brain complete + Agent boot OK must mean the same tables the user expects are queryable by the Agent. +2. **Fail loud, early.** Never burn 6–11 tool steps before saying “MCP token expired.” +3. **Default path is narrow.** Index → optional notes → ask Agent. Advanced jobs stay behind a fold. +4. **Ship in thin vertical slices** that each leave main greener than before. + +--- + +## Workstream map + +| ID | Workstream | Severity | Est. size | Depends on | +|----|------------|----------|-----------|------------| +| **W1** | Agent ↔ DeepSQL MCP auth reliability | Blocker | L | — | +| **W2** | Multi-schema Postgres Brain + coverage gate | Blocker | L | — | +| **W3** | Revive first-run onboarding / setup gate | Blocker | M | W2 for BrainInit step quality | +| **W4** | Brain UI honesty (stages, stepper, jobs) | Blocker | M | W2 for coverage fields | +| **W5** | Unresolved false positives (PK joins) | High | M | — | +| **W6** | Polish (suggestions, title, dbType, dedupe) | Medium | S | — | +| **W7** | Docs + smoke gates for launch | High | S | W1–W4 | + +Parallelize **W1 ∥ W2 ∥ W5 ∥ W6**; then **W3/W4** on top of W2 APIs; finish with **W7**. + +--- + +## W1 — Agent ↔ DeepSQL MCP auth (blocker) + +### Root cause + +Three credential stores diverge: + +| Store | Used by | +|-------|---------| +| Browser session cookie / JWT | UI connection health | +| `~/.config/deepsql/auth.json` | CLI | +| Hermes profile `DEEPSQL_AUTH_TOKEN` (env snapshot in long-lived MCP child) | Agent tools | + +UI “ADMIN” only proves (1). Agent failures are (3) stale/revoked + MCP process not live-reloading env. Logout revokes DB token but leaves disk plaintext. Provisioner often skipped on native runs (`deepsql-agent:8788` DNS). Circuit breaker then reports “unreachable.” + +### Fix design + +**Goal:** Every Agent session starts with a freshly minted (or extended) MCP token that the *running* MCP process can see, and the UI surfaces health before chat. + +#### 1. Token file + live re-read (already half-built in MCP) + +- Provisioner writes `~/.hermes/profiles/u-/deepsql.token` (0600). +- Set `DEEPSQL_TOKEN_FILE` in `mcp_servers.deepsql.env` (keep `DEEPSQL_AUTH_TOKEN` as fallback). +- MCP lib already re-reads file + self-heals on 401 — **wire it in provisioners**. + +**Files:** +`scripts/local-agent-provisioner.py`, Compose agent provision path, `scripts/self-host/setup-agent.sh`, `mcp/deepsql-phase1-lib.js` (verify/tests). + +#### 2. Provision must not be best-effort for Agent tab + +- `AgentBridgeService.ensureProfile` / `callProvisioner`: if provisioner configured, **fail the session bootstrap** on non-2xx (don’t silently return profile name with stale disk token). +- Native AGENTS.md / start scripts: require `AGENT_PROVISION_SECRET` + local provisioner when `SECURITY_AUTH_ENABLED=true`. + +**Files:** +`AgentBridgeService.java`, `AgentBridgeController.java`, `scripts/start-backend.sh` (or docs only if script already supports it). + +#### 3. Revoke → disk cleanup + +- On `revokeAgentTokens`, clear profile token file / blank `DEEPSQL_AUTH_TOKEN` via provisioner hook (or best-effort file delete under known profile path). + +**Files:** +`AuthController.java`, `AgentBridgeService.java`, provisioner. + +#### 4. Agent boot health check (UX) + +- Extend `POST /api/agent/session` (or add `GET /api/agent/health`) to: mint/extend → provision → **probe** `GET /api/connections` with the just-written token. +- `AgentChatPanel` boot: if probe fails, show blocking banner: “Agent cannot reach DeepSQL (auth). Reconnect / check Agent runtime.” Disable send. No chat until green. + +**Files:** +`AgentBridgeController.java`, `src/lib/api/agentClient.js`, `AgentChatPanel.jsx`. + +#### 5. Launch smoke + +- Extend `scripts/self-host/e2e-agent-check.py` (or twin) to fail CI/smoke if Agent tool path 401s. +- Document: plain `deepsql` CLI ≠ Hermes MCP auth. + +### Acceptance + +- [ ] Fresh install: Agent answers “how many customers?” on `demo_shop` without manual token copy. +- [ ] Logout → login: Agent still works (re-mint + token file). +- [ ] Rotating token on disk without Hermes restart: next tool call succeeds (token file). +- [ ] Failed provision: UI shows explicit error before first user message. + +--- + +## W2 — Multi-schema Postgres Brain + coverage gate (blocker) + +### Root cause + +`PostgresIntrospectionProvider` hardcodes `schemaname = 'public'` (and siblings). Brain init progress is **stage-bucket %**, not schema coverage — so 2 public views → COMPLETED 100% while 18 business tables exist. CLI SQL bypasses introspection, so the product feels schizophrenic. + +### Fix design + +#### 1. Default scan = all non-system schemas + +Replace `= 'public'` with: + +```sql +schemaname NOT IN ('pg_catalog','information_schema','pg_toast') +AND schemaname NOT LIKE 'pg_temp_%' +AND schemaname NOT LIKE 'pg_toast_temp_%' +``` + +Always set `TableMetadata.schema` from catalog; use map keys `schema.name` to avoid collisions. + +**Epicenter:** `PostgresIntrospectionProvider.java` (`scanSchema`, `getTablesAndViews`, `getAllTablesWithMetadata`, column/index/FK batches). + +**Also align:** `QueryExecutorService.getPostgreSQLObjects`, `SchemaIntrospectionService`, privilege checks, chat “list tables” prompts that still say public-only, advisor/stats paths that copy the filter. Copy the exclusion pattern already used in Brain classification services. + +#### 2. Optional schema allow-list on connection + +- Add `includedSchemas` (nullable JSON list; null/empty = all non-system). +- Wizard: after successful test, multi-select schemas (`pg_namespace`). +- Brain UI: edit allow-list → Re-initialize. +- Update privilege docs/snippets beyond `GRANT … ON SCHEMA public`. + +**Files:** +`DatabaseConnection` / `ConnectionRequest`, migrations or ddl-auto field, ConnectionWizard, `PrivilegesAccordion.js`, introspection filter `ANY(?)`. + +#### 3. Coverage gate before COMPLETED + +After `SCHEMA_SCAN`: + +- `liveUserTableCount` = countable non-system (or allow-listed) base tables visible to the JDBC user. +- `tablesDiscovered` = what we indexed. +- If `liveUserTableCount > 0 && tablesDiscovered == 0` → **FAILED** with clear message. +- If coverage < threshold (e.g. < 80% of live base tables) → do **not** `markCompleted(100)`; stay in a `NEEDS_ATTENTION` / incomplete state with `coveragePercent`, `schemasScanned`, `skippedSchemas`. +- Init-status API exposes these fields for UI. + +**Files:** +`BrainInitStageExecutor.java`, init-status DTOs, `BackgroundJobsTab.jsx` / Brain enrichment card. + +#### 4. Privileges UX + +After multi-schema scan, surface missing `USAGE`/`SELECT` as Brain warnings (not silent empty). Update wizard grant template to selected schemas. + +### Acceptance + +- [ ] `acme_erp`-shaped DB: Brain lists `crm.*` / `sales.*` / … (or explicit allow-list), not only `pg_stat_statements`. +- [ ] Zero user tables indexed ⇒ cannot show “Complete 100%.” +- [ ] `demo_shop` (public only) still greens. +- [ ] Duplicate table names in two schemas don’t overwrite each other in snapshots. + +--- + +## W3 — First-run onboarding (blocker) + +### Root cause + +`Onboarding.jsx` is complete product work but **unrouted**. Bootstrap/`install.sh` may set `setup.complete=true` early, so that flag ≠ “ready to use.” Login always goes to dashboard. Payload field names in the wizard don’t match `ConnectionRequest`. + +### Fix design + +#### 1. Semantics + +| Concept | Signal | +|---------|--------| +| Admin exists | user table / bootstrap done | +| Product ready | `hasConnections && hasLlmConfig` (and optionally Brain init done for first connection) | +| Wizard complete | `setup.wizard.complete` **or** derive from above — stop using bootstrap’s `setup.complete` alone | + +#### 2. Wire routes + gate + +- `App.jsx`: `/onboarding` protected; `/setup` → `/onboarding`. +- After login (`useAuth.jsx`): if `!hasConnections` → `/onboarding` (primary). Soft-nudge if `!hasLlmConfig`. +- `Login.jsx`: when status shows no connections / incomplete wizard, CTA “Finish setup” → `/onboarding` (and link to README for install-script admins). +- Fix wizard save fields: `connectionName`, `dbType: postgres`, `database` (not `name`/`databaseType`/`databaseName`). + +#### 3. Empty states + +- Sidebar / Home: if zero connections, unlock Docs + onboarding CTA; keep other nav locked with “Add a database.” +- Optional: post-login modal “Load demo_shop” calling seed guidance or deep-link to docs/script output. + +### Acceptance + +- [ ] Fresh admin with no connections lands in wizard, not a locked dashboard. +- [ ] Completing wizard creates connection, configures LLM, kicks Brain init, marks product ready. +- [ ] `/onboarding` no longer blank. + +--- + +## W4 — Brain UI honesty (blocker) + +### Root causes + +1. Frontend `STAGE_ORDER` ≠ backend `InitStage` → grey stages at 100%. +2. Stepper defaults to Initialize / background-jobs and never auto-advances on COMPLETED. +3. Ten scheduled jobs dominate the “teach your business” home. + +### Fix design + +#### 1. Sync stage enum + +Make `BackgroundJobsTab.jsx` `STAGE_ORDER` / labels match `InitStage.java` exactly (`DATA_SAMPLING`, `COLUMN_VALUE_COLLECTION`, `INFERRED_RELATIONSHIPS`, `RAG_EMBEDDING`, `BRAIN_ANALYSIS`, `SEMANTIC_MODELING`, …). On `COMPLETED`, mark all pipeline stages done. + +#### 2. Stepper behavior + +- When init `COMPLETED` and user hasn’t chosen a tab → default to **Add context / schema-context**, not jobs. +- Step 1 shows checkmark when complete; only the active step is highlighted. +- Show coverage line from W2: “Indexed 18/18 tables across 5 schemas” or “Indexed 0/18 — fix grants or schema list.” + +#### 3. Jobs demotion + +- Brain home: enrichment card + next-step CTA only. +- “Scheduled maintenance (N)” collapsed by default; expand for power users. +- Prefer connection-scoped jobs in the summary; globals behind toggle. + +### Acceptance + +- [ ] Complete init ⇒ all listed stages green; stepper on “Add context.” +- [ ] First paint after ready is not a wall of 10 job cards. +- [ ] Incomplete coverage from W2 is visible on the enrichment card. + +--- + +## W5 — Unresolved false positives (high) + +### Root cause + +`KeyColumnAnalysisService.detectAntiPatterns` emits `UNINDEXED_JOIN` when `joinCount >= 5 && indexName == null`, but `enrichWithIndexStats` is a **stub** — never sets `indexName`. Normal PK joins become Unresolved ANTI_PATTERN noise. + +### Fix design + +1. Implement index enrichment via dialect introspection (PK/UK/indexes → `indexName`). +2. Skip `UNINDEXED_*` when column is PK/UK / `TRUE_KEY`. +3. Ambiguity panel: only HIGH actionable types; allow dismiss/acknowledge to stick. +4. Retune copy: “Possible missing index” ≠ “Brain can’t disambiguate.” + +**Files:** `KeyColumnAnalysisService.java`, `SchemaAmbiguityService.java`, `UnresolvedPanel.jsx`. + +### Acceptance + +- [ ] `demo_shop` customers.id / orders.customer_id do **not** appear as Unresolved solely for being join keys with a PK/index. +- [ ] Genuine unindexed join columns still surface. + +--- + +## W6 — Polish (medium) + +| Item | Fix | Files | +|------|-----|-------| +| Hardcoded “bookings” suggestions | Schema-aware chips from top tables / generic fallbacks | `AgentChatPanel.jsx` | +| Title “DBA Agent” | `DeepSQL` (+ optional login badge copy) | `index.html`, `Login.jsx` | +| POSTGRES vs POSTGRESQL | Canonicalize to `postgres` on write via `DatabaseProviderRegistry.getCanonicalName`; display “PostgreSQL” | `CredentialService.java`, `src/lib/dbType.js`, connection UIs | +| Duplicate connections | Dedupe or 409 on same owner+host+port+database; wizard warn | `CredentialService` / `ConnectionController` | +| Login tone | Soften “work email” for OSS or dual copy | `Login.jsx` | + +### Acceptance + +- [ ] Suggestions never mention tables absent from the active connection. +- [ ] Browser tab says DeepSQL. +- [ ] New connection with same host/db doesn’t silently double. + +--- + +## W7 — Docs + launch gates (high) + +1. **In-app Docs:** short “Web UI first hour” — add connection → Brain → Agent (link CLI docs second). +2. **README:** call out multi-schema + Agent health; “Brain ready means indexed coverage.” +3. **Smoke matrix** (must pass on release branch): + +| Check | Command / assertion | +|-------|---------------------| +| Install + login | `install.sh` / existing smoke | +| demo_shop Agent Q&A | e2e-agent-check customer count | +| Multi-schema fixture | mini `crm`+`sales` DB; Brain `tablesDiscovered >= N`; Agent lists those tables | +| Coverage gate | public-only views fixture must **not** report Complete 100% if user tables exist elsewhere | +| Onboarding route | `/onboarding` renders stepper | +| Stage UI | COMPLETED ⇒ no grey pipeline stages | + +4. **Release checklist** in PR template / DocsSection. + +--- + +## Suggested implementation order (PRs) + +```text +PR1 W2a — Postgres non-system schema scan + qualified names (backend) +PR2 W1 — Token file + provision fail-loud + Agent boot probe (backend + scripts + Agent UI) +PR3 W2b — Coverage gate + init-status fields (backend) +PR4 W4 — Stage enum sync + stepper + collapse jobs + coverage UI (frontend) +PR5 W5 — Index enrichment + Unresolved filter (backend + small UI) +PR6 W3 — Onboarding route + auth gate + payload fix + login CTA (frontend + setup semantics) +PR7 W2c — Schema allow-list on connection + wizard (full-stack) +PR8 W6 — Polish (frontend + normalize) +PR9 W7 — Docs + smoke fixtures (scripts + docs) +``` + +**Minimum viable Sunday cut** if time-boxed: **PR1 + PR2 + PR3 + PR4 + PR5 + thin PR6 (route + redirect only) + agent/demo smoke**. Defer allow-list UI (PR7) if default non-system scan is enough; keep privilege messaging in Brain errors. + +--- + +## Combined go-live checklist (UX + security) + +### Must ship (product) + +- [ ] W1 Agent MCP auth reliable + boot banner +- [ ] W2a/b Multi-schema scan + coverage gate +- [ ] W4 Brain stages/stepper/jobs honesty +- [ ] W5 Unresolved PK noise fixed +- [ ] W3 thin: `/onboarding` routed + redirect when no connections + +### Must ship (security) — see `OSS_SECURITY_REVIEW.md` + +- [ ] S1 Kill-session SQLi +- [ ] S2 ACL on apply/kill (and ideally slow-log/growth) +- [ ] S3 Hermes bind `127.0.0.1` +- [ ] S4 Compose: no public DB/Redis; strong DB password; Redis auth +- [ ] S5 Actuator lockdown + JWT fail-closed under prod + +### Launch posture + +| Posture | Extra requirements | +|---------|-------------------| +| **Private single-admin** | Network: only `:3000` public; messaging honest | +| **Internet multi-user** | All Criticals + H4 SET allowlist, H5 SSRF, H6 share password | + +--- + +## Risk notes + +| Risk | Mitigation | +|------|------------| +| Multi-schema scan slows Brain init / embedding cost | Cap concurrency; allow-list; skip system-like schemas later | +| Larger snapshots / RAG volume | Monitor; document; optional “index views” toggle default off for `pg_stat_*` | +| Provisioner required in all envs | Fail closed in UI; make `AGENT_PROVISION_SECRET` mandatory in `.env.example` | +| Breaking clients that assumed public-only | Changelog; allow-list for lock-down | + +--- + +## Out of scope (explicit) + +- Renaming Brain → something else (IA is fine once jobs are demoted). +- Re-enabling `AGENTS_ENABLED` scheduled-agent product. +- Full Flyway adoption. +- Desktop Electron polish. + +--- + +## One-sentence launch bar + +**Ship when: green connection + Brain complete ⇒ Agent can name and query the user’s real tables (including non-`public` schemas) without a human pasting MCP tokens — and dangerous APIs enforce connection ACL.** diff --git a/docs/oss-ux/OSS_LAUNCH_USABILITY_CRITIQUE.md b/docs/oss-ux/OSS_LAUNCH_USABILITY_CRITIQUE.md new file mode 100644 index 0000000..1daaed3 --- /dev/null +++ b/docs/oss-ux/OSS_LAUNCH_USABILITY_CRITIQUE.md @@ -0,0 +1,105 @@ +# DeepSQL OSS launch usability critique + +**Date:** 2026-08-12 +**Build:** `main` @ `3ed7d82` (at time of review) +**Tested as:** `admin@localhost` on Vite `localhost:3000` + backend `:8080` +**Focus:** first-run configure/run simplicity, especially Brain + +**Follow-ups:** +- Fix proposal → [`E2E_FIX_PROPOSAL.md`](./E2E_FIX_PROPOSAL.md) +- Security review → [`OSS_SECURITY_REVIEW.md`](./OSS_SECURITY_REVIEW.md) + +--- + +## Verdict + +**Not Sunday-ready for a smooth OSS “clone → run → ask your DB” experience.** + +The README/Docker self-host path is thoughtfully written, and the **Brain surface on a simple `public`-schema demo (`demo_shop`) is the best part of the product** — clear left-to-right pipeline, good table cards, human review gate. But the core loop an OSS user expects — **connect DB → Brain indexes it → Agent answers** — fails or misleads in ways that will generate Day-1 GitHub issues: + +1. **Agent chat fails** with “unauthorized/unreachable” while the UI shows a healthy `ADMIN` connection. +2. **Multi-schema Postgres Brain is effectively blind** (indexes only `public`, then claims “Complete 100%”). +3. **First-run web wizard is dead code** (`Onboarding.jsx` exists; `/onboarding` is blank; `/setup` redirects to dashboard). +4. Several Brain empty/complete states are **internally inconsistent**, so users cannot trust the green bar. + +Docker `install.sh` may paper over (1) for a fresh install; (2)–(4) are product issues that will hit real databases immediately. + +--- + +## What worked + +| Area | Notes | +|------|--------| +| Login UX | Clean, fast, clear value prop. | +| Nav IA | Agent → Dashboards → Digest → Brain → … is discoverable. | +| Brain pipeline (demo) | “Initialize → Add context → Review → Knowledge base” is the right mental model. | +| Brain on `demo_shop` | 3/3 tables, FACT/DIMENSION/LOOKUP labels, AI blurbs — feels teachable. | +| Connections modal | Usable; Add Connection CTA is obvious. | +| Docs section | Strong CLI/MCP install story. | +| README Quick start | Honest about Docker/buildx, BYO model, bootstrap secrets. | +| CLI | `deepsql whoami` / `connections list` / SQL on `acme_erp` worked. | + +--- + +## Launch blockers (fix before OSS announce) + +### B1. Agent says “unauthorized/unreachable” while UI says connection is fine + +**Repro:** Login → Agent → ask “How many customers…?” on `demo_shop` or `acme_erp`. +**Result:** 6–11 tool steps, then failure. +**Sidebar still shows** `demo_shop ADMIN`. + +This is the **hero feature** of the homepage (“Ask about your database”). Shipping with a green connection badge and a broken Agent loop will dominate launch feedback. + +**UX ask:** Before/while chatting, surface Agent↔DeepSQL auth health explicitly. Do not burn 11 opaque steps first. + +### B2. Brain “Complete” on multi-schema DBs that were barely indexed + +**`acme_erp` reality:** 18 business tables across `crm` / `sales` / `finance` / `inventory` / `hr`. +**Brain/schema API reality:** 2 objects in `public` — `pg_stat_statements` (+ info view). +**Init:** ~1.2s, progress **100%**, copy **“All set! Brain is ready.”** + +CLI can `SELECT` the real schemas; Brain schema scan does not (`PostgresIntrospectionProvider` hardcodes `public`). + +**UX ask:** Never show 100% Complete if indexed table count ≪ live table count. Fail loud. + +### B3. Web first-run wizard is not reachable + +- `src/pages/Onboarding.jsx` exists but `App.jsx` has **no `/onboarding` route**; `/setup` → `/dashboard`. +- Login has **no** first-install CTA (signup is localhost bootstrap fallback only). + +### B4. Misleading Brain stage / job UI + +- Frontend stage keys ≠ backend `InitStage` → grey stages at 100%. +- Initialize stays highlighted after complete. +- Ten jargon-heavy scheduled jobs dominate day-one Brain home. + +--- + +## High-priority polish + +| Issue | Why it hurts | +|------|----------------| +| Hardcoded Agent suggestions (“bookings”) | Wrong for demo/acme; first click feels broken | +| Unresolved ANTI-PATTERN on normal PK joins | Index enrichment stub → false positives | +| Duplicate `demo_shop` + `POSTGRES` vs `POSTGRESQL` | Looks sloppy | +| Document title “DBA Agent” | Branding drift | +| Docs CLI/MCP-first | Missing web “add DB → Brain → Agent” path | + +--- + +## Hello-world results (review session) + +| Action | Result | +|--------|--------| +| Login | Success | +| Brain on `demo_shop` | Success — 3 tables indexed | +| Brain on `acme_erp` | Misleading success — only `pg_stat_statements*` | +| Agent Q&A | **Fail** — unauthorized/unreachable MCP | +| CLI query on `acme_erp` | Success — sees `crm`/`sales`/… | + +--- + +## Launch bar + +**“Brain ready” must mean the Agent can see the same tables the user expects — and the Agent must actually be authenticated when the connection pill is green.** diff --git a/docs/oss-ux/OSS_SECURITY_REVIEW.md b/docs/oss-ux/OSS_SECURITY_REVIEW.md new file mode 100644 index 0000000..1a72b0c --- /dev/null +++ b/docs/oss-ux/OSS_SECURITY_REVIEW.md @@ -0,0 +1,236 @@ +# DeepSQL OSS release — security review + +**Date:** 2026-08-13 +**Scope:** Auth surfaces, secrets/defaults, injection/isolation, self-host attack surface +**Audience:** Ship readiness alongside the usability fix track +**Method:** Code review of `SecurityConfig`, compose, controllers, query policy, Agent/Hermes wiring (spot-verified Critical items) + +**Companion:** Product fix plan → [`E2E_FIX_PROPOSAL.md`](./E2E_FIX_PROPOSAL.md) + +--- + +## Executive verdict + +**Do not market as “safe for multi-user internet exposure” until Criticals are closed.** + +Auth-on-by-default, signup closed, bootstrap localhost+secret, encrypted vault, and cookie HttpOnly are a solid base. The OSS-blocking problems are: + +1. **Missing connection ACL on dangerous APIs** (IDOR → kill sessions / apply indexes on another user’s DB) +2. **SQL injection in kill-session** (`pid` concatenated into SQL) +3. **Hermes `:8787` on `0.0.0.0`** bypasses nginx `auth_request` +4. **Compose publishes Postgres/Valkey/backend** with weak defaults + **unauthenticated fat Actuator** +5. **Blank JWT secret fails open** (ephemeral key) + +Usability agents can keep shipping product fixes; **security Criticals should land before Sunday** (or the launch messaging must be “single-admin, localhost / private network only”). + +--- + +## Severity legend + +| Level | Meaning for OSS | +|-------|-----------------| +| **Critical** | Remote or any-auth’d-user → vault/DB damage; must fix or restrict deployment model | +| **High** | Likely exploit on typical cloud self-host; fix before public announce | +| **Medium** | Defense-in-depth / misconfig footgun; fix soon or document loudly | +| **Low** | Polish / residual | + +--- + +## Critical (must fix or constrain launch) + +### C1. Connection IDOR on dangerous endpoints + +Many controllers take `{connectionId}` and never call `AccessControlService`, while Schema/Chat/Brain correctly do. + +**Confirmed:** `IndexRecommendationController` has **no** `AccessControlService` / `assertCan*` usage. +**Also reported (same pattern):** ActiveQuery kill, LockContention kill, SlowLogSource, GrowthMonitoring (webhooks), SavedQuery, Playbook, Configuration. + +**Impact:** Any logged-in `DEVELOPER` who obtains a connection UUID can apply indexes / kill backends / reconfigure slow-log credentials / fire webhooks on another tenant’s connection. + +**Fix:** Mandatory ACL interceptor or per-controller `assertCanReadConnection` / `assertCanManageConnectionContent`. Apply/kill = manage (or ADMIN). Integration tests: user B → 403 on user A’s id. + +--- + +### C2. SQL injection in session kill + +```193:211:backend/src/main/java/com/dbaagent/service/ActiveQueryService.java +// pid concatenated: +"SELECT pg_terminate_backend(" + pid + ")"; +"KILL QUERY " + pid; +``` + +**Impact:** With C1, crafted `pid` runs attacker SQL on the target DB connection. + +**Fix:** `pid` must match `^[0-9]+$`; bind parameters; ACL first. Same for lock-contention kill. Fix dialect string match (`postgres` vs `POSTGRESQL`) while there. + +--- + +### C3. Hermes webui bound `0.0.0.0:8787` + +```28:28:scripts/self-host/setup-agent.sh +WEBUI_HOST="${HERMES_WEBUI_HOST:-0.0.0.0}" +``` + +Nginx gates `/agent-api/` via `auth_request`; **direct `:8787` does not**. Vite `/agent-api` proxy also has **no** cookie gate (dev only). + +**Impact:** Anyone who can reach `:8787` talks to the Agent/MCP path without DeepSQL login. + +**Fix:** Default `HERMES_WEBUI_HOST=127.0.0.1`; firewall drop WAN 8787; smoke-test refuses open bind; document as hard requirement. Keep nginx `auth_request`. + +--- + +### C4. Compose attack surface + fat Actuator + +| Surface | Issue | +|---------|--------| +| Postgres `:5432` published | Default password `postgres` | +| Valkey `:6379` published | **No password** | +| Backend `:8080` published | Bypasses nginx | +| `/actuator/**` `permitAll` | `health,info,metrics,caches,prometheus` + `show-details=always` | + +**Impact:** Typical “open ports on a cloud VM” install exposes vault DB, cache, metrics, and unauthenticated backend APIs’ recon surface. + +**Fix:** +- Do not publish Postgres/Valkey (internal network only); or `127.0.0.1:` only. +- Generate strong `DB_PASSWORD` in `install.sh` (like JWT/encryption). +- Valkey `--requirepass` + env password. +- Backend port optional / localhost-only; public traffic via `:3000` only. +- Actuator: auth-required except `health`; `show-details=when-authorized`; drop prometheus from public exposure. + +--- + +### C5. Blank `SECURITY_JWT_SECRET` → ephemeral key + +`JwtUtil` warns and generates a random key instead of refusing to start under `prod`. + +**Impact:** “Works” insecurely; multi-replica broken; easy to ship without a real secret. + +**Fix:** Fail closed when `SPRING_PROFILES_ACTIVE=prod` (or always when auth enabled) if secret missing/short (<32 bytes). + +--- + +## High + +| ID | Finding | Fix | +|----|---------|-----| +| **H1** | `SECURITY_AUTH_ENABLED=false` → every request is synthetic ADMIN | Fail boot under prod if false; scrub stale “auth disabled” docs | +| **H2** | CSRF disabled + cookie session | Keep SameSite=Lax; for public HTTPS prefer CSRF token; never `SameSite=None` without CSRF | +| **H3** | Cookie `Secure` defaults **false** | prod: set `SECURITY_COOKIE_SECURE=true` when `APP_PUBLIC_URL` is https | +| **H4** | Read-only policy allows arbitrary `SET` / `USE` preambles | Allowlist safe SETs; block `SET ROLE`, `SESSION AUTHORIZATION`, dangerous `search_path` | +| **H5** | SSRF: webhooks, ES hosts, LLM endpoint test, optional `verifySsl=false` | Deny link-local/metadata/loopback; HTTPS + domain allowlist | +| **H6** | Public dashboard share = unauth read-only SQL on connection | Default require share password; rotate token on revoke; rate-limit | +| **H7** | MCP tokens = full-user PATs (no scopes) | Document; revoke on logout/staff exit; roadmap connection-scoped tokens | +| **H8** | `/auth/internal/token` mint-admin if `INTERNAL_TEST_TOKEN` set | Keep out of `.env.example`; refuse under prod profile | +| **H9** | Index apply DDL outside query policy + no ACL | Same as C1 + confirm + manage ACL | + +--- + +## Medium + +| ID | Finding | Fix | +|----|---------|-----| +| **M1** | `server.error.include-message=always` | `never` / `on-param` in prod | +| **M2** | `spring.jpa.show-sql=true` in prod | Off | +| **M3** | `ENCRYPTION_KEYS` parse errors may echo key material | Redact in exceptions/logs | +| **M4** | SSH `StrictHostKeyChecking=no` | Configurable; default warn/strict for new hosts | +| **M5** | `@CrossOrigin("*")` on some controllers | Remove; rely on global CORS allowlist | +| **M6** | `/admin/bootstrap/link` localhost-only but no secret | Require same bootstrap secret as `/users/admin/bootstrap` | +| **M7** | No gitleaks / secret scanning in CI | Add gitleaks or GitHub secret scanning | +| **M8** | `curl \| bash` Docker/Hermes install | Pin checksums or document trust boundary | +| **M9** | Stale docs: admin/admin, auth-off defaults | Fix before OSS to avoid operator footguns | +| **M10** | Demo seed weak passwords | OK if opt-in + loud “not for production” | +| **M11** | RBAC: VIEWER collapses to DEVELOPER | True read-only role or document “two roles only” | +| **M12** | Vite `/agent-api` ungated | Document “dev only”; never expose Vite publicly | + +--- + +## What’s already in good shape + +- Auth **on** by default; prod profile hardcodes true; signup / setup initialize closed +- Vault credentials encrypted (AES-GCM); encryption key required at startup +- LLM keys encrypted at rest; masked on read +- Chat/MCP/dashboard query paths use `READ_ONLY_ONLY` +- Public share tokens: 192-bit SecureRandom +- Dashboard artifacts: sandboxed iframe (`allow-scripts` only) + strict CSP +- Code archive extract: zip-slip + size caps +- Hermes host toolsets (shell/browser/computer_use) disabled by install scripts +- No committed live API keys in tracked git; `.env` gitignored +- No privileged containers / docker.sock +- CORS defaults localhost-only (not `*`) +- Cookies HttpOnly + SameSite=Lax +- Dependabot + CodeQL present + +--- + +## OSS launch postures (pick one) + +### A. “Private / single-admin self-host” (viable this weekend if Criticals slip) + +Announce as: +- Single trusted admin (or fully trusted team) +- **Not** multi-tenant SaaS +- Bind to private network / Tailscale / SSH tunnel +- Do not publish 5432/6379/8080/8787 + +Still fix **C2** (kill SQLi) and **C1** for apply/kill at minimum — those are bugs, not “deployment choices.” + +### B. “Internet-facing multi-user” (needs Critical + High) + +Require C1–C5, H1–H6, compose bind hardening, Actuator lockdown, Hermes localhost, JWT fail-closed, share-password default. + +--- + +## Recommended fix order (security track) + +```text +S1 Kill-session pid validation + prepared SQL (C2) — hours +S2 ACL on apply / kill / slow-log / growth / saved-query / … (C1) — 1–2 days +S3 Hermes default 127.0.0.1 + smoke assert (C3) — hours +S4 Compose: no public DB/Redis; generate DB_PASSWORD; Redis auth (C4) — hours +S5 Actuator lockdown + JWT fail-closed under prod (C4/C5) +S6 SET preamble allowlist under READ_ONLY (H4) +S7 SSRF guards for webhooks / LLM / ES (H5) +S8 Share password default + token rotate on revoke (H6) +S9 Cookie Secure when HTTPS; prod error/sql logging (H3/M1/M2) +S10 Docs scrub + SECURITY.md + gitleaks (M7/M9) +``` + +Can run **in parallel** with the usability PR train (W1–W7). Do not block Brain/Agent UX PRs on S6–S10; **do** gate merge of “ready for OSS” on S1–S5. + +--- + +## SECURITY.md outline (ship with release) + +1. Threat model: trusted admins vs untrusted multi-tenant (be honest). +2. Required network layout diagram (only `:3000` public). +3. Secrets checklist: JWT, ENCRYPTION_KEY, DB_PASSWORD, Valkey, LLM, AGENT_PROVISION_SECRET. +4. Post-install: disable bootstrap flag; rotate bootstrap secret. +5. MCP tokens = full account access. +6. Public dashboard links = credentials. +7. Reporting channel for vulnerabilities. +8. Explicit non-goals until ACL complete. + +--- + +## Quick post-install smoke (add to `smoke-test.sh`) + +```bash +# Expect 401/404 without cookies: +curl -sf -o /dev/null -w '%{http_code}\n' http://127.0.0.1:8080/api/actuator/prometheus # not 200 if locked +curl -sf http://127.0.0.1:8080/api/auth/me # 401 +curl -sf http://127.0.0.1:3000/agent-api/ # 401 via nginx +# Expect connection refused / filtered from non-loopback perspective in hardened install: +# :5432 :6379 :8787 :8080 not on 0.0.0.0 +ss -lntp | grep -E ':(5432|6379|8787|8080)\b' +``` + +--- + +## Bottom line + +| Question | Answer | +|----------|--------| +| Secrets leaked in git? | **No** live keys found in tracked files | +| Safe default auth model? | **Mostly yes** (auth on, signup closed) | +| Safe multi-user on a public VM? | **Not yet** — IDOR + kill SQLi + open Hermes/DB/Redis/Actuator | +| Sunday possible? | **Yes as private single-admin** if S1–S2 land; **internet multi-user needs S1–S5 + H4–H6** | diff --git a/docs/oss-ux/README.md b/docs/oss-ux/README.md new file mode 100644 index 0000000..b301b27 --- /dev/null +++ b/docs/oss-ux/README.md @@ -0,0 +1,11 @@ +# OSS launch handoff docs + +These docs are the durable home for OSS go-live planning. **Do not rely on `/opt/cursor/artifacts/`** — that path is ephemeral per Cloud Agent VM and is not shared with other agents. + +| Doc | Purpose | +|-----|---------| +| [`OSS_LAUNCH_USABILITY_CRITIQUE.md`](./OSS_LAUNCH_USABILITY_CRITIQUE.md) | E2E usability findings (Brain, Agent, onboarding) | +| [`E2E_FIX_PROPOSAL.md`](./E2E_FIX_PROPOSAL.md) | Product fix workstreams W1–W7 + PR order | +| [`OSS_SECURITY_REVIEW.md`](./OSS_SECURITY_REVIEW.md) | Security Criticals/Highs + S1–S10 track | + +Other Cloud Agents: read these paths from the repo (this branch or after merge to `main`). From 195e6a0f78eb2a463f1c7d2e77248ac335fe6c29 Mon Sep 17 00:00:00 2001 From: Cursor Agent Date: Thu, 13 Aug 2026 07:06:06 +0000 Subject: [PATCH 2/2] docs: point Cloud Agents at docs/oss-ux for durable OSS handoff Note that /opt/cursor/artifacts is ephemeral per VM and not cross-agent. Co-authored-by: Venkat SF --- AGENTS.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/AGENTS.md b/AGENTS.md index 342b4e3..cdc5bee 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -262,3 +262,6 @@ only covers cloud-specific, non-obvious caveats. deadlock on an `ALTER TABLE`. Test env vars are documented in `CLAUDE.md` (Testing). - `npm run lint` currently reports many pre-existing warnings/errors in the repo; that is the baseline, not a setup failure. +- **`/opt/cursor/artifacts/` is ephemeral and agent-scoped.** It is wiped on new Cloud Agent + VMs and is **not** shared with other agents. Durable OSS go-live handoff lives in + [`docs/oss-ux/`](docs/oss-ux/) (usability critique, E2E fix proposal, security review).