SBCL librarian runtime by BatmanAoD · Pull Request #58 · rigetti/libquil

BatmanAoD · 2026-08-13T17:50:26Z

Updates sbcl-librarian and adopt its new conventions.

The motivation here is to get libquil working on Apple Silicon, but the approach required rearchitecting to account for upstream changes in quilc, specifically updates to the sbcl-librarian FFI layer.

The change was generated by Claude Code, and the design decisions are documented in the REARCHITECTURE file.

Use branch-refs to rely on three upstream PRs: quil-lang/sbcl-librarian#91, quil-lang/magicl#222, and quil-lang/quilc#933. Each has a corresponding issue in this repo, and comments in the source code citing those issues.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

Moves off the 2023 sbcl-librarian pin, which also unblocks modern SBCL. libquil is now the generated C bindings only; the Lisp image lives in a core loaded by the libsbcl_librarian runtime, whose constructor initializes Lisp when it is loaded. The explicit init(core) call is gone, as is libquil's hand-rolled error handling. libquil keeps shipping a core rather than the FASL bundles that CREATE-FASL-LIBRARY-CMAKE-PROJECT produces. FASL bundles are re-loaded on every startup, re-running load-time code, and cl-quil resolves stdgates.quil through ASDF:SYSTEM-RELATIVE-PATHNAME at load time -- so a FASL build only runs where quilc's source tree sits at the path recorded when it was built. A core evaluates that once, at build time. build-image.lisp emits three things from one image: libquil's bindings, the runtime's bindings, and the core exporting both sets of callables. The Makefile builds the runtime too, so plain 'make' still yields a usable artifact. Errors move to the runtime's get_error_message/lisp_err_t. libquil redefines default-error-map so ordinary bad input reports as LISP_ERR_FAILURE rather than an internal bug, handling T rather than CL:ERROR because cl-quil signals conditions that are not subtypes of ERROR. REARCHITECTURE.md records the decisions. Needs three sbcl-librarian fixes, on its fix-secondary-system-bundles branch. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

- build.yml builds SBCL $SBCL_VERSION (2.6.7) with --with-sb-linkable-runtime, replacing the sbcl-2.2.4 build that failed in make-host-1 ('undefined variable: SB-VM::END') because a current host SBCL cannot cross-compile a 2022 target. sbcl-librarian's repo and ref are workflow inputs, since libquil needs three fixes that are not upstream yet. - The artifact is now the bindings library, its header and the whole runtime directory: libsbcl_librarian, libquil.core, libsbcl.so and the runtime headers. install.sh and the release archives follow, with libquil.core installed beside libsbcl_librarian because the runtime finds its core relative to itself. - The examples drop their init(core) calls and -pagezero_size, link the runtime, and use get_error_message/lisp_err_t. - The redundant quickload steps before make are gone; build-image.lisp does it. Verified by simulating package -> zip -> install and running the libquil-sys suite against the installed layout. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

magicl looks for Homebrew's reference LAPACK first and otherwise falls back to a bare liblapack.dylib, which on macOS resolves to Accelerate's. Neither works: the reference build returns incorrect eigenvectors on arm64, failing with 'Could not find diagonalizer for matrix ... after 16 attempts', and Accelerate's LAPACK predates 3.3 so routines quilc calls are missing, failing with 'The alien function zuncsd_ is undefined'. OpenBLAS is correct and complete. Loading it first means its symbols are the ones that resolve, and because SBCL records loaded shared objects in the core and reloads them at startup, the choice is baked into the artifact instead of depending on what the loader happens to find. Verified against a stock magicl checkout with Homebrew's lapack installed. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

Cutting a prerelease from a branch produced a tag pointing at the wrong code. knope 0.10/0.11 create the GitHub release against the default branch, so the version bump landed on the branch while the tag landed on main -- and the publish job, which checks out whatever that tag resolves to, then tried to republish main's already-released version: error: crate libquil-sys@0.4.2 already exists on crates.io index Adopt the setup rigetti-pyo3 arrived at (36a1850, 9b5235c, 3505971): - knope 0.23, which tags the ref being released and takes --prerelease-label, so the separate prerelease workflow in knope.toml is no longer needed; - the ref decides what gets cut: main releases, anything else prereleases; - check out ${{ github.ref }} so a release acts on the dispatched branch; - pass the token to the Release step through GITHUB_TOKEN, which is how it authenticates; - dry-run the release on pull requests, so a broken config is visible before it is dispatched. The publish job now checks out the release tag explicitly, so it always publishes exactly what was tagged rather than whatever the target commitish points at. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

… root Two problems with installing a release: magicl loads BLAS and LAPACK at runtime under their unversioned names, libblas.so and liblapack.so, so a missing one does not surface at install or link time: it surfaces much later, in the middle of compiling a program. Check for them before anything is downloaded, and name the ones that are missing. The unversioned names are also why a runtime-only package is not enough -- Debian's libblas3 provides libblas.so.3 and no unversioned symlink -- so say that in the requirements, which is the part users get wrong. Drop libz from the requirements while there: nothing in the release artifacts refers to it. Container images commonly run as root with no sudo installed, where every sudo call here fails with "sudo: command not found" even though nothing needs elevating. That is how CI installs libquil, so the installer was unusable there. Resolve sudo once: empty when already root, sudo when available, and a clear error when neither. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

Applies the reviewable changes from the review of #58 and records the answers to the questions raised in PR-58-REVIEW-ANSWERS.md. - Share the SBCL/Quicklisp install and Lisp-dependency clone between the Linux and macOS build jobs (scripts/install-sbcl.sh, scripts/clone-lisp-deps.sh). - install.sh: unify the library search paths into one array, discover the Homebrew prefix rather than hardcoding it, test the OS explicitly instead of via IS_LINUX, require root rather than shelling out to sudo, and announce the quarantine removal. - README: document that the -dev packages are required, narrow the "any package manager" claim, update the install commands for the root requirement, and replace the removed libquil_error_t/libquil_error() API reference with lisp_err_t/get_error_message(). - Trim reviewer-facing comments that do not earn their place in the tree, and clarify the SBCL_LIBRARIAN_CORE_NAME and D4 "upstream" wording. - REARCHITECTURE: drop the point-in-time status checklist. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

…karounds magicl picked a BLAS/LAPACK by searching a fixed list of paths that did not include OpenBLAS, so libquil had to force the choice from outside: the build image loaded OpenBLAS via load-shared-object before magicl could look, and CI uninstalled Homebrew's lapack and symlinked OpenBLAS over liblapack.dylib. Both were workarounds for magicl having no way to say "use OpenBLAS". magicl now searches for OpenBLAS by name, honours MAGICL_LAPACK_PATH, and verifies at load that the backend it chose is complete and computes correctly (quil-lang/magicl#221, PR #222). CI tracks rigetti/magicl at backend-selection-and-validation, parameterised as MAGICL_REPO/MAGICL_REF alongside the sbcl-librarian pin, until that lands upstream. So all three workarounds go. A bad backend now fails the build with a message naming the library, rather than yielding an artifact that is quietly wrong -- which is strictly better than what the preload did, since the preload only ensured OpenBLAS was *available*, never that it was what magicl used. Verified by loading libquil against the new magicl on a machine that still has the miscompiled Homebrew lapack 3.12.1 bottle installed: magicl selects OpenBLAS for both BLAS and LAPACK unaided. Unchanged: the backend is still fixed at build time, because SBCL records loaded shared objects in the core and reopens them at startup. That is also why install.sh still checks the user's machine for BLAS and LAPACK. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

PROCESS-PROTOQUIL and STRIP-FINAL-HALT-RESPECTING-REWIRINGS lived in quilc's *application*, which libquil does not depend on, so libquil carried a copy of both. quil-lang/quilc#933 moves them into the library as src/analysis/process-protoquil.lisp and exports the transform from CL-QUIL.FRONTEND, which is where they belong. So the vendored copy goes -- 85 lines, and with it a maintenance hazard: two transforms registered under the same name would have collided once the library defined one. Both call sites now name CL-QUIL.FRONTEND:PROCESS-PROTOQUIL rather than a bare 'PROCESS-PROTOQUIL. The libquil package only uses CL, so the unqualified symbol read as LIBQUIL::PROCESS-PROTOQUIL and would no longer have matched any registered transform. QUILC_REPO/QUILC_REF join the sbcl-librarian and magicl pins, tracking rigetti/quilc at 896-protoquil-transform until #933 merges. That branch is three commits ahead of the commit libquil pinned and contains it, so this is a fast-forward rather than a version bump. Verified against the branch locally: libquil loads, and compiling CNOT 0 1 as protoquil yields 8 instructions with the final HALT stripped, circuit definitions cleared, and statistics computed. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

Each of the three forks libquil temporarily depends on now carries an inline TODO naming the libquil issue that tracks moving back to a released upstream: sbcl-librarian #60 (quil-lang/sbcl-librarian#91) quilc #61 (quil-lang/quilc#933) magicl #62 (quil-lang/magicl#222) On both the REPO and REF lines, so that grepping for TODO turns up every line that has to change rather than only half of each pair. Replaces the previous ad-hoc markers, one of which read "TODO(#60-adjacent)" and named no issue at all. Values are unchanged; the additions are YAML comments. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

libquil pinned quil-lang/qvm at 4617625 (2022-12-03) in the first CI commit, with no recorded reason. qvm is close to dormant -- no release since 1.17.2 in 2021, and five commits in the three years since that revision -- so a commit pin is the only option, but there was no reason for it to be that commit. Moves to 1c4ca60, the head of master, which picks up the rename of the abstracts dependency and the move to clos-encounters. Current cl-quil uses clos-encounters too, so this removes a divergence between the two halves of what libquil loads into one image. Verified locally: libquil loads against the new pin, and its qvm paths behave -- a Bell state gives |amplitude| 0.7071 on |00> and |11>, ten shots of X 0 all measure 1, and FSIM runs. The C examples under examples/qvm are libquil's only qvm tests and were not run: they need a built artifact, and this machine cannot produce a working one (its SBCL core and linkable runtime come from different builds). The checks above exercise the same code paths from Lisp instead. CI is the real test. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

qvm is the one dependency pinned to a bare commit rather than a branch or a tag, because it has no release newer than 1.17.2 (2021). quil-lang/qvm#330 asks for one; when it exists this pin becomes a tag. Unlike the other three TODOs, this cites an upstream issue rather than a libquil one: there is no libquil-side decision waiting, only an upstream release. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

The long-form answers have all been acted on, and their reasoning now lives in commit messages and in comments at the point of use, which is where a reader will actually look for it. Keeping a third copy here meant three places to keep in step, and two of them had already drifted. What is left is what the file is now for: the four dependency pins and what each is blocked on, the two decisions still outstanding, what CI covers that local testing could not, and a comment ready to post on the PR. 295 lines to 86. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

Several passages read as a changelog -- "no longer published", "this replaces the pre-0.4 API", a section on which LAPACK to avoid and why. Someone arriving at the repo has no prior version to be told about, so those are gone. What is left states what is required and what is supported. The Homebrew reference-LAPACK caveat goes with them. magicl now prefers OpenBLAS and verifies its backend at load, so the caveat described a hazard a reader cannot encounter by following these instructions. Clarifies that make.sh, make-shared-library.sh and install.sh in the build-from-source section are SBCL's, not libquil's -- SBCL has an install.sh of its own, and libquil's is now scripts/install.sh, so the collision was worth naming. Adds the clone step, since the snippet has to run inside an SBCL checkout and never said so. Also records that the sbcl building libquil and the libsbcl.so it links must come from the same *build*, not merely the same version: SBCL stamps a build ID into both, and a mismatch surfaces only at runtime as "core was built for runtime ... but this is ...". The Makefile's error message gave the same version-only advice and now says the same thing. Installer URLs follow scripts/install.sh to its new location. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

The macOS job passed on 9799967 and both jobs pass on the current head, so the "not verified locally" caveats can be replaced with what CI actually covers -- which is everything that could not be checked here: - the C examples under examples/qvm and examples/quilc run in the Test step on both jobs; - the macOS job builds SBCL from source through scripts/install-sbcl.sh, exercising the shared-script refactor rather than the cached path; - magicl's new load-time backend check passes on both runners. Also makes the draft comment self-contained: it referred to "the table above", which will not exist once it is pasted into GitHub, so it carries its own. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

The message blamed Homebrew's refusal to run as root, which is misleading: the very next line already handles that by running brew as SUDO_USER. What the guard actually catches is the absence of anyone to drop to -- a root session that was not entered through sudo. Kept rather than removed, but the comment now says why it barely matters: brew is not on root's default PATH, so a root login normally fails the "Homebrew was not found" check just above instead. It earns its place only by turning what would otherwise be a bare "SUDO_USER: unbound variable" from set -u into something actionable. Also drops the "sudo bash install.sh" hint, which named the wrong path after the move to scripts/ and duplicated the invocation the README documents. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

QUILC_COMPILE_PROTOQUIL writes the compilation metadata back through an argument, which needs SB-ALIEN:DEREF, which cannot be applied to the `(* t)' that sbcl-librarian's :POINTER maps to. libquil worked around that by hand-writing the DEFINE-ALIEN-CALLABLE and letting it redefine the one DEFINE-API had just generated -- which SBCL signals a continuable error for, so the build carried a handler that took the CONTINUE restart on any error whose text mentioned "redefine alien callable". None of that was necessary. DEFINE-TYPE is generic over the C and SB-ALIEN spellings of a type; it simply was not exported (fixed in quil-lang/sbcl-librarian#91). A :POINTER-OUT type spelled `void*' to C and `(* (* t))' to Lisp expresses the argument directly, so DEFINE-API generates the callable correctly and there is nothing to redefine. Three things follow: - the 31-line hand-written callable goes; - WITH-ALIEN-REDEFINITION-ALLOWED goes, along with a handler that would have swallowed the CONTINUE restart of any similarly-worded error; - `(ql:quickload :libquil)` now loads unaided, so the CI pre-flight check removed earlier could be restored if wanted. Error handling improves too: the generated callable runs the full DEFAULT-ERROR-MAP rather than the open-coded handler-bind the override carried, and since src/libquil.lisp precedes src/quilc/api.lisp in a :serial t system, that is libquil's redefined map. The C spelling stays `void*'. `void**' would document the contract better, but C does not implicitly convert `T**' to `void**', so every caller passing `&handle' would take an -Wincompatible-pointer-types warning -- confirmed by compiling the example's call pattern against both. The published header is therefore unchanged but for a parameter name, `metata_ptr' to `metadata_ptr'. Verified: libquil loads with a bare quickload where it previously aborted; the generated header matches the committed one; the example's call compiles without diagnostics; and compiling CNOT 0 1 as protoquil overwrites a sentinel in the out-parameter with a handle that dereferences to the 9-entry metadata table. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

…ds it Requiring root was incidental. It followed from installing to /usr/local, not from anything the install does -- and consumers can already point at an arbitrary location, since libquil-sys reads LIBQUIL_SRC_PATH and LIBQUIL_LIB_PATH. So --prefix (or LIBQUIL_PREFIX) chooses where to install, and a prefix under $HOME needs no sudo. The root check is gone. In its place the script creates the target directories and writes a temporary file into each, which is the question actually worth asking: `id -u` says nothing about whether the write will succeed, and neither does -w, which ignores ACLs, reports true on a read-only mount, and is always true for root. A non-default prefix is on no default search path, so on finishing the script prints the LIBQUIL_SRC_PATH, LIBQUIL_LIB_PATH and DYLD/LD_LIBRARY_PATH values that make it usable. The two libquil-sys variables differ because the headers sit a directory deeper than the libraries. Verified without sudo: installing 0.4.0-rc.0 into a scratch prefix lays out the same seven files, leaves no probe behind, and building qcs-sdk-rust's libquil example with only the printed variables compiles through quilc and simulates on the QVM. Pointing --prefix at /usr/local as an ordinary user now reports which directory it cannot write rather than a bare root demand. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

Building from source told you how to build and then stopped. The layout the artifacts have to end up in was recorded only in scripts/install.sh, which installs a *release* -- so anyone building from source, which is the only route on an unsupported platform, had to reverse-engineer it. `make install` writes that layout, honouring PREFIX the same way the installer honours --prefix, and prints the LIBQUIL_SRC_PATH / LIBQUIL_LIB_PATH / library path variables when the prefix is not /usr/local. Verified that `make install PREFIX=...` produces a file list identical to what scripts/install.sh lays down, and that building qcs-sdk-rust's libquil example against it compiles through quilc and simulates on the QVM. README picks up both: --prefix in the installer section, with its own heading so the note above can link to it, and `make install` after the artifact list. The old claim that the installer "must be run as root" is now conditional, because it is. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

The set of files that make up an installed libquil was written out in five places: both build.yml package steps, both release.yml zip commands, install.sh, the Makefile install recipe, and the README. Three of those predate this branch; the rearchitecture grew the list from four files to seven, and I added the last two. Adding a file meant remembering all of them, and release.yml failed quietly when you did not -- a new file was simply absent from the archive, surfacing only when someone installed it. install.sh now names the list once, resolving each file from either an unpacked release or a build tree, where the runtime sits in runtime/. That gives it a --from DIR mode, and `make install` becomes a delegation: install: all scripts/install.sh --from . --prefix "$(PREFIX)" so the layout, the prefix handling, the quarantine clearing and the post-install hint have one implementation rather than two. install.sh stays self-contained, which it has to be -- it is run as `curl ... | bash` with no repository present, so it cannot source a helper. release.yml zips the directory the build job assembled instead of listing its contents, which removes the silent-omission failure and the per-platform duplication with it. The README drops its copy of the list. It is documentation for someone installing libquil, not an inventory, and it went stale the moment the runtime moved. Two fixes fell out of the consolidation. --from no longer requires curl and unzip, which it does not use. And it no longer rejects unsupported architectures: that check exists to pick a release asset, so applying it to a local build turned away exactly the platforms building from source is for. Verified that `make install`, `--from` an unpacked release, and downloading 0.4.0-rc.1 all produce identical trees, that qcs-sdk-rust's libquil example compiles and simulates against the `make install` output, and that a missing file, `--from` with a version, and `--from` a non-directory each report what is wrong. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

… build intermediate Both build jobs assembled artifacts/libquil with the same three commands, differing only in the shared-library suffix. That is now scripts/package-artifact.sh, alongside install-sbcl.sh and clone-lisp-deps.sh. It also fixes a regression from the previous commit. Those steps ended with `cp runtime/*`, which includes sbcl_librarian.c -- the generated C that produced libsbcl_librarian, of no use to a consumer. release.yml used to name the archive's contents explicitly, so the stray file was excluded by accident rather than by intent; switching to `zip -r` would have started shipping it. The script copies everything from runtime/ except the generated C, so what lands in the directory is what belongs in the archive. Verified against a real build tree: the packaged directory holds the same seven files 0.4.0-rc.1 shipped and no .c, installing from it with --from produces the same tree as every other path, and qcs-sdk-rust's libquil example compiles and simulates against the result. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

…eview), remove references to it

Shadow53

Reviewed just the diff since my last review. A few small things, but approving since they are minor.

libquil 0.4.0 is released, so the pins can come off the fork and the branch: LIBQUIL_VERSION 0.4.0-rc.1 -> 0.4.0 LIBQUIL_INSTALL_REF branch -> tag v0.4.0 LIBQUIL_INSTALL_REPO BatmanAoD/libquil -> rigetti/libquil LIBQUIL_RELEASE_REPO goes entirely; the installer defaults to rigetti/libquil. The fork it pointed at is archived and read-only. Two things had to change with them. install.sh moved to scripts/install.sh, so fetching the old path from a 0.4.0 tag would 404. And the separate apt and brew steps that installed BLAS, LAPACK and libffi are gone: --install-deps does that now, from the same list libquil itself declares, rather than this workflow guessing with `libffi\\* libblas\\* liblapack\\*`. The URL was checked rather than assumed: the tagged installer returns 200 and does carry --install-deps. Still pinned rather than tracking latest, because install.sh has to be fetched from a ref that has scripts/install.sh, and rigetti/libquil#58 is not merged, so main does not. The TODO says so. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

BatmanAoD and others added 13 commits November 27, 2023 10:18

Add job for Apple Silicon MacOS build

110093e

Merge branch 'rigetti:main' into patch-1

6522083

Claude gets it working, maybe

07373f9

fix README notes on aarch64

32e3558

GHA fixes

975064e

Drop Intel macOS support

b659ea1

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

small fixes

ff1fb63

chore: prepare release 0.4.0-rc.0

7db4bfd

This was referenced Aug 13, 2026

Update for libquil on modern sbcl-librarian rigetti/libquil-sys#68

Merged

Use sbcl-librarian runtime #57

Closed