serve: mirror describe-batch + unload-table, expose job idempotency (#657)

[padak]

Fixes #657.

Three MEDIUM gaps from the 0.89.0 serve audit. Each mirror deliberately differs from its CLI command in one place — wherever the CLI option names the caller's filesystem, which over serve is the server's.

1. storage describe-batch had no route at all

The whole bulk-documentation path was CLI-only. POST /storage/describe-batch/{project} takes the buckets / tables / columns sections inline in the body (plus branch_id) instead of a --from-file path.

The issue noted the validation core was already factored out. It was factored out of the file reader, not of the write loop, so this splits both:

• parse_describe_batch_file → parse_describe_batch_document(raw, source_label) + a thin YAML-loading wrapper.
• describe_batch → _apply_describe_batch(alias, parsed, ...), shared with the new describe_batch_document.

Both surfaces therefore run literally the same rules and the same write loop. Two copies would drift, and the half that drifted would be the one letting a malformed section reach the write loop.

One typing choice worth a reviewer's eye: the body's three sections are declared Any, not dict[str, str]. The stricter annotation looks better and is worse — pydantic would reject a wrong shape first with its generic 422, discarding exactly what #645 built: the offending key, its actual type, and a copy-pasteable example. Typing it loose is what lets the shared validator be the one that speaks.

Error-shape note: a malformed document answers 422 (HTTP_ERROR) where the CLI exits 2 (INVALID_ARGUMENT) — same message, different code. It is a malformed request body, which is the class of problem FastAPI already answers 422 for on this surface, and the alternatives were worse: a global ValueError handler would turn genuine bugs into 400s, and KeboolaApiError maps to 502. Documented in docs/web-server.md and gotchas.md so a caller branches on the status here rather than the code. Unifying validation error codes across the two surfaces is a broader cleanup, not this PR.

2. storage unload-table had no mirror

POST /storage/tables/{project}/{table_id}/unload, with download hard-wired False and no output_path: writing to "the caller's disk" means writing to the server's. The response carries the Storage file_id, and GET /storage/files/{project}/{file_id}/download fetches the bytes.

That is also the only shape that works for file_type: "parquet", whose export is sliced and has no single-file download even on the CLI.

The body is optional — an unload with no options is the common case, and requiring {} would be noise.

3. POST /jobs/{p}/run dropped client-side idempotency

JobService.run_job has taken idempotency_key / force_rerun since #427, and retrying a POST over HTTP is the canonical case that store was built for — so the REST mirror omitting them left the caller who needs them most without them. Both are now body fields. Omitting them produces a byte-identical call to the pre-change one (a test asserts exactly that).

Dedup is scoped to the served config dir, i.e. per machine, same as on the CLI.

The bonus ask went elsewhere

GET /permissions/show is in #682 (issue #655) rather than here: a read-only view of the policy belongs with the enforcement it describes, not on its own.

Testing

make check green: 6137 passed, 12 skipped. ty clean (the one remaining diagnostic is the pre-existing scripts/hatch_build.py unresolved import). Lint, format, command-sync, version-gates, sentinel-guards, error-codes and the endpoint-reference gate all pass; docs/web-server-endpoints.md regenerated and committed.

20 new tests in tests/test_server_missing_mirrors.py, service layer mocked — what is under test is the router → service contract, not Keboola behaviour:

• describe-batch — sections reach the service as one document; an omitted section stays None rather than becoming an invented {}; a shape error is 422 carrying the CLI's message; a per-item API failure is a 200 with errors, not a 422 (shape is a usage error, an API refusal is not).
• unload — an exact-kwargs assertion, so a future edit that starts passing output_path fails rather than quietly writing to the server's disk; body-optional; all options threaded; and a greedy-{table_id:path} case proving /unload is not swallowed as part of out.c-my.deeply.dotted.
• job run — both fields reach run_job; omitting them preserves the pre-change call.
• shared validator — parametrised, asserting the inline path rejects the same four malformed documents and accepts the same four empty ones as the file path, and that source_label names the body rather than a filename.

Merge-order note

#682 (issue #655, open) adds an app-wide permission firewall whose completeness test requires every route to have a ROUTE_OPERATIONS entry. Whichever of the two lands second needs three entries added:

("POST", "/storage/describe-batch/{project}"): "storage.describe-batch",
("POST", "/storage/tables/{project}/{table_id:path}/unload"): "storage.unload-table",

(POST /jobs/{p}/run is already mapped — this PR only adds fields to its body.) Both operations already exist in OPERATION_REGISTRY. The completeness test names each missing route, so the failure is self-explaining rather than mysterious.

No version bump, no changelog entry (per CONTRIBUTING: those belong to the release PR). New behaviour gated with the literal (since vNEXT) placeholder.

Doc surfaces

docs/web-server.md (new "CLI options that a REST mirror deliberately drops" table, plus the stale "mirrors still considered missing" sentence), docs/web-server-endpoints.md (regenerated), and gotchas.md. No CLI command added, renamed, or removed, so CLAUDE.md's command list, context.py, keboola-expert.md and SKILL.md need no change.