UN-4008 [FEAT] Add the unstract CLI to run extractions and API deployments from the terminal - #2
Open
chandrasekharan-zipstack wants to merge 46 commits into
Open
UN-4008 [FEAT] Add the unstract CLI to run extractions and API deployments from the terminal#2chandrasekharan-zipstack wants to merge 46 commits into
chandrasekharan-zipstack wants to merge 46 commits into
Conversation
Wheel skeleton for the `unstract` console script: Click app with the
whisper / docstudio / config groups, and the three cross-cutting layers
every command will sit on.
- config: named profiles resolved flag > env > profile > default, with
`env:` indirection so the file records where a secret lives rather than
the secret, 0600 writes, deployment aliases, and `config doctor`
reporting where each setting resolved from without echoing a value.
- output: one JSON envelope {ok, data, error, meta} on stdout for success
and failure alike, so parsing never depends on TTY detection; table and
raw are opt-in renderings, diagnostics go to stderr.
- errors: the exit-code table as a stable API, retry policy that never
retries a 4xx, redaction, and undeclared statuses reported verbatim
rather than guessed.
- poll: transport-agnostic --wait loop reading terminal state from the
response body rather than the HTTP status, never sleeping past the
deadline, echoing the job handle on timeout so work resumes instead of
being resubmitted, and persisting a one-shot result before the read is
acknowledged.
No transport yet: the clients own HTTP. Tests are offline and need no
credentials.
Flags for an operation come from the spec the published client is generated from, intersected with what that client's signature actually accepts: a spec parameter the frozen client cannot name would raise TypeError at the call rather than reach the API, so it is not offered. Two rules keep the derivation honest. Every option defaults to None, meaning absent, so an unpassed flag is not sent and the client or server default applies rather than a value pinned here. And only None is treated as absent: 0, false and "" are choices a caller made and travel to the request. Help text has three sources in order: the overlay, the spec, and the client method's own docstring, which is the only one that describes the parameters today. The overlay carries what a generated spec cannot express -- allowed values, short flags, wording -- in TOML read with the stdlib.
Thirteen commands: whisper extract/status/retrieve/detail/highlights/usage and its four webhook commands, plus deployment run and status. Each one holds only what a spec cannot say -- which parameter is the argument, which the CLI owns, and how a result is polled for. The CLI runs the poll loop for both products rather than using the loop one client ships, so --wait, --interval, --timeout and the handle-returned-on- timeout behaviour are the same everywhere. Deployment runs are queued (timeout=0) so a request does not hold a connection open for the length of the job. Line-highlight scaling is arithmetic on a reply rather than a request, so it is folded into the command that fetches the metadata. Failures converge on one envelope: LLMWhisperer raises with a status code, the deployment client returns one, and both become a CLIError with an exit code and a hint. A result that can be read only once is written to disk before it is printed.
--discover answers what --help answers, as JSON, in three tiers: groups names the products, summary adds their commands, full adds every flag with its type, choices and default plus the exit-code table -- enough to construct a call without a second round trip. A caller starts cheap and drills down. Every tier is read back from Click itself, so a described command cannot drift from the one the parser accepts, and discovery reads no configuration: it is how a caller learns what exists, so it has to work before anything is set up. config doctor --probe adds the second diagnostic question -- does the resolved key work -- to the one it already answered offline, where it resolves from. LLMWhisperer is checked against its usage endpoint. A deployment has no side-effect-free endpoint to call, so its entry reports that the settings resolve and says plainly that nothing was verified.
The vendored specs and the pinned clients move independently, so a refreshed spec can declare a parameter the published client has no argument for. Such a parameter is dropped rather than offered and rejected at the call, and dropping it silently is the failure this pins: the gap is written down per operation, so widening it is a decision rather than an accident.
Two failures a live call found and no offline test could. The metadata arrives as a named object carrying the coordinate list under `raw`, while the client's geometry takes the bare list, so no line was ever scaled. And a line the service has no geometry for is reported as all zeros, whose page height is a divisor in that scaling: it raised ZeroDivisionError out of the client, which the entry point does not catch, so the command printed a traceback with an empty stdout. Such a line now gets no box.
Three follow-ups to the command surface. Both client pins move forward, and the six extraction parameters and three status parameters they gained appear as flags with no line written here -- which is what deriving from the specs was for. The contract test's unreachable set shrinks to what the clients own rather than lack: the URL-in-body flag and the execution id read from the endpoint URL. --base-url, --api-key and (for deployments) --org-id sit on the product group and fill the flag tier of flag > env > profile > default, which the loader already supported but nothing populated. A key given on the command line warns: it lands in shell history and in the process list. The 406 hint is scoped to deployments. A whisper result read twice comes back as a 400 whose body says so, and translating on that prose would break the moment the wording changes -- the service's own message already says what happened, and it is passed through verbatim.
`deployment status` derived --include-metadata, --include-metrics and --include-extracted-text from the spec, collected them into **params, and never passed them to the client. The command succeeded and the payload parsed, so a dropped flag was indistinguishable from a working one. The poll loop behind `deployment run --wait` had the same hole, which made a waited run return less than the identical flags returned without --wait. Both now forward what was asked for, and the parameters the status endpoint does not accept are filtered out rather than sent. Tests cover each flag in both polarities, since a flag silently dropped is exactly what the offline suite missed. Alongside: - `config doctor` no longer reports an `org_id` setting for LLMWhisperer, which has none. It always read as unresolved and there was no way to resolve it. - The deployment probe reports `ok: null`, not `ok: true`. Nothing is called, so there is no verdict; `true` beside `checked: false` reads as a live check that passed. `resolved` carries what is actually known. - The 406 hint pointed at --save, which does not exist on the command that emits the hint. It now names the command that has it. - A 400 carries a hint. The service can answer 400 with an empty error body, in which case the message was a synthesised fallback and there was nothing else to go on. Adds RUNBOOK.md: install, moving the client pins, the live-gate checklist, and the release steps. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014f9oEEYspPH4fmPULTnLkJ
Waiting returns the result and nothing else: the extracted text, or the deployment's structured output. Neither names the job, so a caller who waited had no handle to correlate against the service, quote in a bug report, or use for a follow-up call. Without --wait the handle is the entire payload, so the identity appeared and disappeared depending on a flag. Both waited paths now carry it in `meta` -- the whisper hash and the execution id -- leaving `data` exactly as it was. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014f9oEEYspPH4fmPULTnLkJ
`--save` exists to protect a read the service serves exactly once, and it
was the flag that lost the data: the write ran after the acknowledging
read, raised `OSError` through an entry point that does not catch it, and
left an empty stdout with the extraction gone. The target is now proven
writable before anything destructive runs, the write goes through a
temporary file so a full disk cannot truncate the previous copy, and a
write that fails anyway raises with the payload attached under its own
exit code -- by that point the envelope carries the only copy left.
Also on the one-shot path: a waited extract read the result with a bare
`.get("extraction")` where the sibling command falls back to the whole
payload, so a response shaped any other way printed `ok: true, data: null`
for a document that had been processed and billed. Both now read it the
same way, and a genuinely empty result is a failure rather than a silent
success.
Redaction was an opt-in keyword argument that only the success path
passed, so every error envelope and every stderr summary went out with
the key in it -- four times on stdout in the reproduced case. Credentials
are now registered where they resolve and scrubbed by every emitter, and
`CLIError.details` is redacted structurally rather than at each call site.
Three more places where a failure was reported as a success: the
standalone status commands ignored a finished-and-failed execution inside
an HTTP 200, the poll loop treated an unreadable body as progress and then
blamed the timeout on a job it never confirmed was running, and any status
outside 4xx/5xx mapped to exit 0 while printing `ok: false`.
Verified by mutation -- moving the save after the print, dropping the
registry, dropping the details redaction and dropping the status check
each fail the suite now, and none of them did before.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014f9oEEYspPH4fmPULTnLkJ
A CLI whose output shape depends on whether a terminal is attached is a CLI whose scripts break when they move from a shell to CI. This drops the isatty question entirely: the default is a table, in a terminal and in a pipe alike, and anything that parses the output asks for `-o json`. An explicit `-o` is the last word. The environment picks the default and nothing more, so the same `-o json` invocation renders the same bytes wherever it runs -- which is the property a caller is actually relying on. Coding agents are the exception worth making: they set a marker in the environment, and there the default becomes json rather than making every call carry a flag. `--agent yes|no` settles it either way. Every envelope now carries `meta.contract_version`, and `--discover full` publishes what a consumer has to do to hold up its end: ignore unknown fields, refuse a version above the one it was written against. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014f9oEEYspPH4fmPULTnLkJ
Nothing bounded a stalled connection: the deployment client is untimed and its api_timeout is an execution mode the backend reads, not a socket timeout. --transport-timeout sets one. Unset by default, so a run that would have hung still hangs rather than starting to fail in a way no existing script expects. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014f9oEEYspPH4fmPULTnLkJ
Ctrl-C came back as exit 1 with nothing on stdout, which reads to a supervisor as a failed command worth retrying -- the one thing that must not happen to a run the user deliberately stopped. It now exits 130, the value every shell already reads that way, and prints the same envelope as any other failure. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014f9oEEYspPH4fmPULTnLkJ
Overlay, spec and client docstring can each describe a flag. No spec parameter carries a description today, so the order between them is unexercised until one does, which is exactly when an inversion would ship unnoticed.
The vendored LLMWhisperer spec was several revisions behind and now declares enums the CLI was hand-listing. The two had already diverged: --mode rejected three modes the service accepts and --output-mode two, and nothing would have reported it. Read the enum off the spec, keep the overlay for narrowing one on purpose, and drop the descriptions' own value lists for the same reason their default sentences are dropped. `highlights` gains a `mode` query parameter that the published client has no argument for, so it joins the parameters the CLI cannot reach.
A sentence-shaped match ends at the first period, so "Defaults to 0.3." was left in the help beside the default rendered from the signature. Strip each restated sentence with its own end-anchored pass instead.
The pinned clients predated the fix that stops an omitted optional parameter being sent as the string "None", so a CLI built on them sent it. The derived surface is byte-identical across the move; neither signature changed.
Each of these restated the line below it, or described a prior state that is no longer there to check against. Keep the reason, drop the narration.
Copies one organization's resources into another by calling the client's orchestrator directly. Two endpoints with a key each, which no single profile describes, so both are flags and both keys come from the environment. Also moves the client pin forward to pick up the status path-prefix fix. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014f9oEEYspPH4fmPULTnLkJ
The vendored copy was several iterations behind the one the pinned client is generated from, so the CLI's help, its parameter set and what --discover publishes all described an older service contract. The flag snapshot is the check that makes a resync safe: every other contract assertion reads the spec on both sides of its comparison, so a spec that loses a parameter loses the flag and the expectation with it.
…tatus A transport error was translated into a CLIError outside the poll loop, where the handle no longer exists, so the caller was left to resubmit a document the service had already processed and billed. Translating at the call keeps the loop's own context; the loop attaches the handle itself for anything the caller did not translate. `whisper status` reported a failed extraction as a success, its sibling in the other product having already been fixed: both read the body, not the status code.
Four failures the CLI reported as successes or as something vaguer than it knew: - a server-reported error inside a 2xx got the catch-all exit code, which is the least informative one for the most interesting failure this API has; - `config doctor` printed its own findings and exited 0, so a setup script branching on it read a broken configuration as a working one; - a deployment alias pointing at an unset environment variable fell back to the profile's organisation and key, running against a tenant nobody named; - a webhook's auth token was echoed verbatim. The restated-default stripper was also greedy to the end of the string, so a description whose value list came first lost every sentence after it.
The command that writes into a live organisation had none of its own behaviour pinned. Its table output -- the one a person gets, and the only output path that did not go through the emitter -- scrubbed by hand and was run by no test, while the test that claimed a platform key never reaches stdout passed with the registration deleted. Rendered output now goes out through the same path as every envelope, and a key planted in a report is asserted not to survive it. Also: --on-name-conflict decides what is written into the target and is now asserted to arrive; skipped documents are counted at the top of the payload, because skipping is not fatal and a caller reading the exit code alone would never learn a document did not move; `config doctor` resolves each deployment alias the way a run does, instead of listing names its docstring implies it checked; a failed retrieve is pinned to carry the handle; the restated-default stripper ends at its own sentence rather than at the end of the text; and the groups tier lists leaf commands apart from groups, which a consumer walks differently.
The status endpoint's own query parameters are forwarded now, and a deployment URL that carries no derivable prefix is polled where the service said rather than at a rebuilt path.
The notes carry the console-script collision, the behaviours a script would otherwise discover by being surprised, and the service version a custom page separator needs. The pin moves to a documentation-only commit.
The envelope shape is documented in the README and published by --discover; greeting every --help with it buries the two things a reader is there for.
A .unstract.toml found by upward search comes from whatever checkout the user happens to be standing in. It may still select a profile, set org_id and define deployment aliases; api_key and base_url are withheld, with a warning, and reported as withheld by config doctor. Named explicitly with --config or $UNSTRACT_CONFIG, the same file is honoured in full. Also point a first-time user at where keys are minted, from config init, from doctor and from the README, and ship an on-prem profile shape. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014f9oEEYspPH4fmPULTnLkJ
The discovered config path is written to as well as read from, so a symlinked .unstract.toml let a repository redirect config set and config init --force onto any file it named. The upward search now skips a symlinked candidate, and the write opens with O_NOFOLLOW so a symlink at the target is a clear error rather than a truncation. Also: the config group reports the file's warnings instead of dropping them, doctor answers for a withheld deployment-alias key the way it does for a product one, trust is derived from the path rather than from how the loader was called, and the README says plainly that routing stays repo-controllable. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014f9oEEYspPH4fmPULTnLkJ
A spec resync that narrows an enum, changes a type or moves a default left the gate green while the CLI began rejecting a value it used to take. The snapshot now carries the whole parameter surface, and the failure names the flags that moved. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014f9oEEYspPH4fmPULTnLkJ
chandrasekharan-zipstack
marked this pull request as ready for review
August 17, 2026 16:05
chandrasekharan-zipstack
changed the title
chandrasekharan-zipstack
marked this pull request as draft
August 18, 2026 07:34
`config init` writes a placeholder for every setting only the user can supply. An empty string satisfied `require`, so a request went out with a hole in it instead of failing with a message naming the setting. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BFSunNN6RKRA1xo6kWkztx
Both restated the README for an audience that has neither shipped nor operated this yet. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BFSunNN6RKRA1xo6kWkztx
Install and dev commands go through uv, matching how the project is built and tested. The exit-code table says it is this CLI's own convention and names the enum it copies, and a test now fails when the two disagree. The credential section says a literal key works and why `env:` is the default. `clone` reads as the operator command it is, so an agent does not reach for it unasked, and the connection flags are named as the top tier of the resolution chain. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BFSunNN6RKRA1xo6kWkztx
Every comment that ran to three or more lines narrated the decision rather than naming it. Each is now one or two lines that hold up without the discussion they came from. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BFSunNN6RKRA1xo6kWkztx
The help named the products but not what they are for, so a first reader (or an agent) had to run something to find out what was possible. It now says what each product does and states the json envelope, the exit codes and `--discover` in one paragraph. The command list under it is printed by Click, so the prose does not repeat it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BFSunNN6RKRA1xo6kWkztx
chandrasekharan-zipstack
marked this pull request as ready for review
August 18, 2026 13:28
Author
|
@greptileai review this PR. If its not 5/5, help suggest what will make it 5/5 |
|
| Filename | Overview |
|---|---|
| src/unstract_cli/config.py | Implements config discovery, trust filtering, resolution, and owner-only atomic persistence. |
| src/unstract_cli/core/poll.py | Implements shared deadline-aware polling and safely staged result persistence. |
| src/unstract_cli/app.py | Defines the root Click application, global resolution options, product groups, and discovery entry point. |
| src/unstract_cli/commands/whisper_cmd.py | Exposes LLMWhisperer extraction, retrieval, status, usage, highlight, and webhook operations. |
| src/unstract_cli/commands/docstudio_cmd.py | Exposes Document Studio deployment execution and status operations using shared polling behavior. |
| .github/workflows/release.yml | Adds manually dispatched versioning, validation, package publication, tagging, and GitHub release automation. |
Flowchart
%%{init: {'theme': 'neutral'}}%%
flowchart TD
A[CLI invocation] --> B[Click command parsing]
B --> C[Resolve flags, environment, and profile]
C --> D{Command group}
D -->|Whisper| E[LLMWhisperer client]
D -->|Docstudio| F[Deployment client]
D -->|Config| G[Local config operations]
D -->|Clone| H[Clone orchestrator]
E --> I[Shared polling and persistence]
F --> I
G --> J[Standard output envelope]
H --> J
I --> J
Reviews (7): Last reviewed commit: "fix: say why a config in an unwritable d..." | Re-trigger Greptile
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
Both writes could be steered or observed by another local user with write access to the containing directory, or read access to an existing config file. `persist` wrote through a predictable `<target>.tmp` sibling opened with plain "w". Anyone able to write the save directory could pre-plant that name as a symlink and have the write truncate the file it pointed at. It now writes through an exclusively created, unpredictably named temporary file in the same directory. `save_config` passed 0600 to `os.open`, but that mode applies only when the call creates the file. Rewriting an existing group- or world-readable config left the old mode in place for the duration of the write, so a freshly written literal credential was readable by anyone who could already read the file until the trailing `chmod`. The descriptor is now narrowed before any content goes through it, which also removes the path-based `chmod` that followed. Both are pinned by a test that fails if either guard is removed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014f9oEEYspPH4fmPULTnLkJ
chandrasekharan-zipstack
requested review from
hari-kuriakose,
jaseemjaskp and
ritwik-g
August 19, 2026 08:01
uv brings its own interpreter, so the CLI's Python floor stops being the installer's problem.
Both clients are on PyPI, so the pins are exact versions rather than commits: the CLI derives its flags and help from their signatures, so a client that moves changes the CLI's surface and each release re-pins on purpose. The vendored specs are refreshed to the commits those releases were generated from, and `provenance.json` now records where each one came from so a copy taken from anywhere else fails a test instead of deriving flags the released client cannot carry. What the refresh moves, and why: * `mode` is gone from five LLMWhisperer operations, so the highlights command no longer has a parameter it cannot reach. * `extract` declares the deprecated `page_seperator` spelling, which the client also accepts. A deprecated parameter is now skipped, or the same value would have had two flags. * `--mode pdf_to_images` is gone: that mode is its own operation. * Several LLMWhisperer defaults are no longer reported, because the client stopped pinning them and lets the server choose.
Mirrors the two Python clients: a manual dispatch bumps the version, lints, tests, builds, publishes with `uv publish` through a PyPI Trusted Publisher, and only then commits the bump, tags it and cuts the GitHub release -- so a failure anywhere before the publish leaves main untouched. The version now lives only in `__version__`; pyproject reads it through hatch, so the bump edits one file. `version_bump: none` publishes what is already in the repo, which is what a first release needs: the convention here is that the committed version is the last released one, and nothing has been released yet.
`pre_release` now publishes `<version>rcN` rather than flagging the release on GitHub alone: the rc number counts up from the tags already published for that target version, so repeat dispatches give rc2, rc3, and the committed `__version__` is left alone because it names the last stable release, not a candidate for the next one. Promoting is the same dispatch with `pre_release` off, which bumps and commits the version, tags it and publishes it stable.
Every one of these is something a real call reached and a test did not. Discovery published Click's "no default" sentinel as the string `Sentinel.UNSET` -- fourteen flags on the newest Click the pin allows, each reading as a value a caller could send back. The marker stopped being `None` in 8.2 and is not exported, so it is read off a bare option and tracks whichever version is installed. CI now runs the suite a second time against the newest resolvable Click, because `uv run` resolves from the lockfile and an install in the wild does not. A required option was given `default=None`, which from Click 8.2 counts as a value the caller supplied, so `required` was never enforced: `whisper highlights <hash>` with no line range escaped as a raw TypeError from inside the client. Requiredness now follows the spec rather than the client signature -- a signature without a default says only that the call cannot omit the argument, which the command answers by supplying one -- and `--extract-all-lines` stands in for a line range, as the API allows and the client's positional argument does not. The CLI read only `LLMWHISPERER_BASE_URL` while the published client reads `LLMWHISPERER_BASE_URL_V2`, and `UNSTRACT_DEPLOYMENT_KEY` while the deployment client reads `UNSTRACT_API_DEPLOYMENT_KEY`. An environment set up for a client therefore left the CLI on its built-in default, which is production. Both client spellings are now honoured after the CLI's own. `--no-wait` handed the acknowledgement to the result finisher, so `--output raw` printed `null` for a started, billed execution and the handle appeared only on the path that did not need it. An ack now reports the handle in both places. Two `retryable` flags said the opposite of the truth: a poll timeout, the one failure a caller is meant to come back from, said false, while a host name that does not resolve said true. The unresolvable host is now named and reported as final; every other connection failure stays retryable. Also: `--discover` answered in whatever format `-o` asked for, including a wrapped table, though it is the machine-readable description and its own contract tells callers to pass `-o json`; it now always answers as JSON. And it described only leaf commands, omitting the root's `-o` and the connection flags each product group carries, which is a description of a call nobody can make.
`config set` opened the real file with O_TRUNC and tightened its mode afterwards, so anything that failed between the two -- an fchmod the filesystem refuses, a full disk, a value that will not serialise -- left the user with an empty config and no way back to the one they had. It is now written to a temporary file and renamed into place. The rename is atomic, so the previous config survives every failure, and `mkstemp` creates the temporary unpredictably named and 0600, which is the mode the config lands at: a guessable sibling in a shared directory is a symlink waiting to be planted, and a mode widened until after the write is a window in which the new credential is readable. Replacing a symlink would quietly turn a deliberate one into a regular file, so that is still refused rather than followed.
Comment on lines
+321
to
+326
| handle_fd, name = tempfile.mkstemp(dir=target.parent, suffix=".tmp") | ||
| tmp = Path(name) | ||
| try: | ||
| with os.fdopen(handle_fd, "wb") as fh: | ||
| tomli_w.dump(doc, fh) | ||
| os.replace(tmp, target) |
There was a problem hiding this comment.
If a selected config file is writable but its parent directory does not permit creating or renaming entries, mkstemp or os.replace fails, causing config set and config init --force to reject an update that could previously write the existing file directly.
Prompt To Fix With AI
This is a comment left during a code review.
Path: src/unstract_cli/config.py
Line: 321-326
Comment:
**Config updates require directory writes**
If a selected config file is writable but its parent directory does not permit creating or renaming entries, `mkstemp` or `os.replace` fails, causing `config set` and `config init --force` to reject an update that could previously write the existing file directly.
---
For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.
There was a problem hiding this comment.
Accurate, but not taken deliberately — leaving this open for a human to close or overrule.
The only way to accept a write here is to fall back to writing the file in place, which is the O_TRUNC path an earlier round on this PR flagged as a blocker: it empties the existing file before the replacement is written, so any failure after that point destroys a working config. That risk does not go away when the directory is unwritable, it gets worse, because there is then nowhere to stage a copy or leave a backup beside it.
There is no version that has both. Atomic replacement needs the directory by definition. Opening without O_TRUNC and calling ftruncate after the write closes the empty-file window but leaves a mixed old/new document if the write dies partway, which is a corrupt config rather than an absent one.
The case is also narrow and no longer silent. The default config path has a user-owned parent, so it needs an explicit --config pointing into a directory that forbids creating entries, and that now raises a ConfigError naming the directory and the reason rather than surfacing a bare PermissionError. Failing loudly and changing nothing is the intended behaviour.
Replacing the file rather than overwriting it is what keeps a failed write from destroying a working config, and a rename needs the directory even when the file itself is writable. Writing in place when the directory refuses would put the truncate back exactly where recovery is hardest, so the case is reported instead, naming the directory.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
unstract— one command that takes a document to structured JSON: LLMWhisperer extraction, an API deployment run, and the polling in between.unstract config init | list | get | set | doctorunstract whisper extract | status | retrieve | detail | highlights | usage,whisper webhook create | get | update | deleteunstract docstudio deployment run | statusunstract --discover groups | summary | fullWhy
Both products are reachable today only from Python. This makes them scriptable from a shell and drivable by an agent: one output envelope, one exit-code table, and a JSON description of the whole surface so a caller can construct a command without a second round trip.
How
flag > env > profile > default. Discovery is--config→$UNSTRACT_CONFIG→ a project-local.unstract.tomlfound by searching upward (stopping at$HOME) →~/.unstract/config.toml. Values may indirect throughenv:VAR, so a config file can be committed without a key in it. Files are written 0600. Deployments are named aliases inheriting org and key from their profile..unstract.tomlmay not supplyapi_keyorbase_url. A checkout the user did not write would otherwise hand the CLI a key or point it at another host; a path the user names explicitly is a deliberate choice and is honoured. Withheld values are kept and written back to their own file, never carried into another one. Config writes useO_NOFOLLOW, so a planted symlink cannot redirectconfig setonto some other file. Routing (org_id,api_name, profile selection) stays repo-controllable by design.{ok, data, error, meta}on stdout, in JSON by default whether or not stdout is a TTY, so a script gets the same bytes as a terminal.--output tablewraps rather than truncates.--timeout 30returns at 30s — and a timeout carries the handle out so a caller can resume rather than restart.TypeErrorat the call rather than reach the API, so it is not offered, andtests/test_contract.pyrecords which ones those are so the gap widens on purpose or not at all. An unpassed flag is not sent, so the server default applies rather than one pinned here; onlyNonecounts as absent, so0,falseand""travel.--discoveranswers what--helpanswers, as JSON, read back from Click itself, so a described command cannot drift from the one the parser accepts.fulladds every flag with its type, choices and default plus the exit-code table.Can this PR break any existing features
No. New repository, nothing depends on it yet, and it is not published. It reads the two clients through their public APIs only.
Notes on Testing
230 tests, offline by design: no network and no credentials — the clients are replaced at the factory, so what is asserted is which arguments a command hands the client, and what a caller sees on stdout and in the exit code. CI runs ruff and pytest, then the suite a second time against the newest
clickthe pin allows —uv runresolves from the lockfile, and an install in the wild does not. Live round trips are a manual pre-release step.Related Issues or PRs
Pins
Built on
unstract-client==1.6.0andllmwhisperer-client==2.9.0, both from PyPI. The vendored specs are byte-identical copies of the ones those releases were generated from, recorded with their source commit and sha256 insrc/unstract_cli/specs/provenance.jsonand checked bytests/test_specs.py. Neither client installs a console script, sounstractis this CLI's alone;unstract-cliremains as a second name.Release
.github/workflows/release.yml— a manual dispatch that bumps the version, lints, tests, builds, publishes to PyPI withuv publishthrough a Trusted Publisher, and only then commits the bump, tags it and cuts the GitHub release. The version lives only insrc/unstract_cli/__init__.py, read through hatch, so the bump edits one file, and the committed value names the last stable release.pre_releasepublishes a PEP 440 release candidate: the target version is computed as usual, thenrcNis appended, counting up from the rc tags already published for that target, and__version__is left alone because a candidate is not a release. Promoting is the same dispatch withpre_releaseoff.version_bump: nonetargets the version already in the repo, so the first cut isnone + pre_release→0.1.0rc1, andnonealone promotes it to0.1.0.Owner-side setup, none of which exists yet on this repo:
unstract-cli, OwnerZipstack, Repository nameunstract-cli, Workflow namerelease.yml, Environment name blank (the workflow declares no environment).PUSH_TO_MAIN_APP_ID(Actions variable) andPUSH_TO_MAIN_APP_PRIVATE_KEY(Actions secret) must reach this repo. Neither exists at repo level here; both are presumably org-level, and an org-level secret still has to listunstract-cliamong the repositories it is visible to.Zipstack/unstract-cli— the workflow requests a token scoped to that repository by name.🤖 Generated with Claude Code
https://claude.ai/code/session_014f9oEEYspPH4fmPULTnLkJ