Auto-provision FastTree so macOS doesn't need conda at all

README.md

@@ -14,7 +14,7 @@ That's it for most users -- CANDy's default toolchain is fully bundled:
 
 - **Clustering**: [MMseqs2](https://github.com/soedinglab/MMseqs2) -- auto-downloaded and cached on first use (no conda needed). On Linux/macOS this just works. On **Windows**, MMseqs2's clustering workflows internally need a POSIX shell; the official Windows build handles this itself by installing a small helper (`busybox`) the first time it runs, which may ask for administrator permission **once** -- never again after that. (This mirrors upstream: MMseqs2's own docs list WSL as the recommended Windows path and this static build as the fallback for anyone who can't use WSL.)
 - **MSA**: [FAMSA](https://github.com/refresh-bio/FAMSA) via [`pyfamsa`](https://github.com/althonos/pyfamsa) -- a real pip dependency, runs in-process, no download needed.
-- **Phylogenetics**: [VeryFastTree](https://github.com/citiususc/veryfasttree) via [`veryfasttree`](https://github.com/citiususc/veryfasttree-python) -- also a real pip dependency, no download needed. **Except on Apple Silicon Macs**: `veryfasttree` has no `macOS arm64` wheel at all (as of 4.0.4.1) and its from-source build fails on stock macOS (an upstream OpenMP-detection bug). CANDy detects this automatically and defaults `--tree-tool` to [FastTree](http://www.microbesonline.org/fasttree/) instead there -- see below.
+- **Phylogenetics**: [VeryFastTree](https://github.com/citiususc/veryfasttree) via [`veryfasttree`](https://github.com/citiususc/veryfasttree-python) -- also a real pip dependency, no download needed. **Except on Apple Silicon Macs**: `veryfasttree` has no `macOS arm64` wheel at all (as of 4.0.4.1), so CANDy skips it there entirely and defaults `--tree-tool` to [FastTree](https://github.com/morgannprice/fasttree) instead, which auto-downloads (Linux/Windows) or auto-compiles from a single dependency-free C file (macOS) on first use, cached afterward -- no conda needed there either. See below.
 
 ### Apple Silicon (M1/M2/M3/M4) setup
 
@@ -70,21 +70,22 @@ Either way, verify before running a real job:
 python3 -c "import platform; print(platform.machine())"   # should print "arm64", not "x86_64"
 ```
 
-**2. `--tree` fails to install or build `veryfasttree` (e.g. a CMake/OpenMP compiler error).** This is unrelated to Rosetta -- `veryfasttree` simply has no `macOS arm64` wheel at all, for any Python version, so it always falls back to a from-source build there, and that build fails on stock macOS due to an upstream bug (`find_package(OpenMP)` fails, since Apple's Clang has no OpenMP support out of the box, and `veryfasttree`'s CMake fallback for that case is itself broken). **You shouldn't normally hit this**: CANDy detects a Mac without a working native `veryfasttree` build and automatically defaults `--tree-tool` to `fasttree` instead, which *does* have a real `arm64` conda-forge/bioconda build (no compiling anything). That needs the bundled conda environment (a real, one-time dependency for this one platform):
+**2. `--tree` fails to install or build `veryfasttree` (e.g. a CMake/OpenMP compiler error).** This is unrelated to Rosetta -- `veryfasttree` simply has no `macOS arm64` wheel at all, for any Python version, so `pip`/`uv` would otherwise try to build it from source there, and that build fails on stock macOS due to an upstream bug (`find_package(OpenMP)` fails, since Apple's Clang has no OpenMP support out of the box, and `veryfasttree`'s CMake fallback for that case is itself broken). **You shouldn't hit this at all as of `candy-cazyme` 3.0.4+**: on a Mac without a working native `veryfasttree` build, CANDy both skips it as an install-time dependency entirely (so nothing tries to build it) and defaults `--tree-tool` to `fasttree` instead -- which auto-downloads a precompiled binary (Linux/Windows) or, on macOS specifically (no precompiled binary is published upstream), auto-compiles one from a single dependency-free C source file using whatever C compiler is already on your machine (Xcode Command Line Tools' `clang`, already present on essentially every real Mac). No conda needed. This happens once and is cached, the same way MMseqs2 auto-downloads itself.
+
+If that auto-compile step ever fails (e.g. genuinely no compiler on PATH), the error message tells you to run `xcode-select --install`, or you can still fall back to the bundled conda environment:
 
 ```bash
 conda env create -f environment.yml
 conda activate candy
-candy GH173 --email you@example.com --tree   # --tree-tool defaults to fasttree here automatically
 ```
 
-If you'd rather force `veryfasttree` anyway (e.g. you've solved the OpenMP build issue yourself), pass `--tree-tool veryfasttree` explicitly.
+If you'd rather force `veryfasttree` anyway (e.g. you've solved the OpenMP build issue yourself), `pip install veryfasttree` explicitly and pass `--tree-tool veryfasttree`.
 
 If you'd rather use the original CD-HIT/MAFFT/FastTree tools instead (e.g. to reproduce results bit-for-bit against the published notebook), `environment.yml` provides CD-HIT and FastTree (`conda env create -f environment.yml && conda activate candy`, then `--clustering-software cd-hit --tree-tool fasttree`); MAFFT isn't included there (no `osx-arm64` build -- see above) and needs a separate install, e.g. `brew install mafft` on Intel Mac/Linux, then `--alignment-tool mafft`.
 
 To also enable automated Gemini-based domain-name curation, see [Domain-name curation](#domain-name-curation) below.
 
-If you'd rather not have CANDy download anything automatically (e.g. air-gapped environments), set `CANDY_NO_AUTO_DOWNLOAD=1` -- clustering will then require `mmseqs`/`cd-hit` already on PATH.
+If you'd rather not have CANDy download anything automatically (e.g. air-gapped environments), set `CANDY_NO_AUTO_DOWNLOAD=1` -- clustering will then require `mmseqs`/`cd-hit`, and `--tree-tool fasttree` will require `FastTree`, already on PATH.
 
 ## Usage
 

environment.yml

@@ -8,8 +8,11 @@
 #   - fasttree: phylogenetics alternative to VeryFastTree (--tree-tool fasttree).
 #               This is also CANDy's *default* on a Mac where VeryFastTree has
 #               no working native build (Apple Silicon, or an x86_64 Python
-#               under Rosetta translation) -- see the README's Apple Silicon
-#               setup section.
+#               under Rosetta translation) -- but CANDy auto-downloads/compiles
+#               its own copy of FastTree there when needed (see
+#               candy.phylogenetics.fasttree_download), so this conda package
+#               is only needed as a fallback (e.g. no C compiler available) or
+#               if you specifically want the conda-provided build instead.
 # MAFFT (the alternative to FAMSA, --alignment-tool mafft) is deliberately
 # NOT included here: bioconda has no osx-arm64 build for it, which would
 # break `conda env create` on Apple Silicon entirely -- exactly the platform

src/candy/phylogenetics/fasttree.py

@@ -3,31 +3,26 @@
 import logging
 from pathlib import Path
 
-from candy.external_tools import MissingDependencyError, find_binary, run_tool
+from candy.external_tools import run_tool
+from candy.phylogenetics.fasttree_download import resolve_fasttree_binary
 
 logger = logging.getLogger(__name__)
 
-# Different packagers ship this binary under different casings
-# (bioconda historically used 'FastTree', some distros use 'fasttree').
-_CANDIDATE_NAMES = ["FastTree", "fasttree"]
-
 
 class FastTreeBuilder:
+    """Phylogenetics via FastTree.
+
+    Prefers a ``FastTree``/``fasttree`` already on PATH (e.g. from the
+    bundled conda environment); otherwise transparently downloads a
+    precompiled binary (Linux/Windows) or compiles one from source
+    (macOS, which has no precompiled binary upstream) and caches it -- see
+    :mod:`candy.phylogenetics.fasttree_download`.
+    """
+
     name = "fasttree"
 
     def build_tree(self, alignment_fasta: Path, output_newick: Path) -> Path:
-        binary = self._locate_binary()
+        binary = resolve_fasttree_binary()
         logger.info("Building phylogenetic tree with FastTree.")
         run_tool([binary, str(alignment_fasta)], stdout_path=output_newick)
         return output_newick
-
-    def _locate_binary(self) -> str:
-        for candidate in _CANDIDATE_NAMES:
-            path = find_binary(candidate)
-            if path is not None:
-                return path
-        raise MissingDependencyError(
-            f"Required external tool 'FastTree' was not found on PATH "
-            f"(tried: {', '.join(_CANDIDATE_NAMES)}). Install it via the bundled conda "
-            "environment: `conda env create -f environment.yml && conda activate candy`."
-        )

src/candy/phylogenetics/fasttree_download.py

@@ -0,0 +1,172 @@
+"""Locate, download, or compile a FastTree binary automatically.
+
+FastTree became CANDy's default ``--tree-tool`` on Macs where VeryFastTree
+has no working native build (see :mod:`candy.platform_utils`), but until
+now the only way to actually get a ``FastTree`` binary was the bundled
+conda environment -- a real, unwelcome extra dependency for anyone who
+doesn't already have conda installed (confirmed by a real user hitting
+exactly that wall).
+
+Upstream (https://github.com/morgannprice/fasttree) publishes precompiled
+binaries for Linux and Windows directly in the repository, and FastTree.c
+itself is a single, dependency-free C file (no CMake, no OpenMP requirement
+for the plain single-threaded build) that compiles in about a second with
+whatever C compiler is already on the machine -- Xcode Command Line Tools
+on macOS, build-essential-equivalent on Linux. That makes a "just pip
+install and run" story possible here too: on first use, if a binary isn't
+already on PATH, fetch a precompiled one (Linux/Windows) or fetch the
+source and compile it locally (macOS, which has no precompiled binary
+upstream), then cache the result and reuse it on every later run.
+
+Pinned to a specific commit for reproducibility; downloads only ever come
+from the official morgannprice/fasttree GitHub repository over HTTPS, and
+the whole thing can be disabled with the CANDY_NO_AUTO_DOWNLOAD environment
+variable (falling back to requiring `FastTree` on PATH, e.g. via the
+bundled conda environment).
+"""
+
+from __future__ import annotations
+
+import logging
+import os
+import platform
+import stat
+import subprocess
+from pathlib import Path
+
+import requests
+from platformdirs import user_cache_dir
+
+from candy.external_tools import MissingDependencyError, find_binary
+
+logger = logging.getLogger(__name__)
+
+FASTTREE_REF = "a5a2723ea1e64faf3da7ea514521cfa348891add"
+_RAW_BASE_URL = f"https://raw.githubusercontent.com/morgannprice/fasttree/{FASTTREE_REF}"
+
+NO_AUTO_DOWNLOAD_ENV = "CANDY_NO_AUTO_DOWNLOAD"
+
+_COMPILER_CANDIDATES = ["cc", "clang", "gcc"]
+_COMPILE_FLAGS = ["-O3", "-fopenmp-simd", "-funsafe-math-optimizations", "-march=native"]
+
+
+def _cache_dir() -> Path:
+    return Path(user_cache_dir("candy")) / "fasttree" / FASTTREE_REF
+
+
+def _binary_name() -> str:
+    return "FastTree.exe" if platform.system() == "Windows" else "FastTree"
+
+
+def _cached_binary() -> Path | None:
+    candidate = _cache_dir() / _binary_name()
+    return candidate if candidate.is_file() else None
+
+
+def _download(url: str, dest: Path) -> None:
+    dest.parent.mkdir(parents=True, exist_ok=True)
+    with requests.get(url, stream=True, timeout=120) as response:
+        response.raise_for_status()
+        with open(dest, "wb") as f:
+            for chunk in response.iter_content(chunk_size=1 << 20):
+                f.write(chunk)
+
+
+def _make_executable(path: Path) -> None:
+    if platform.system() != "Windows":
+        path.chmod(path.stat().st_mode | stat.S_IXUSR | stat.S_IXGRP | stat.S_IXOTH)
+
+
+def _provision_precompiled(cache_dir: Path) -> Path:
+    asset = "FastTree.exe" if platform.system() == "Windows" else "FastTree"
+    binary_path = cache_dir / _binary_name()
+    logger.info(
+        "Downloading FastTree (%s) for phylogenetics -- this happens once and is cached at %s",
+        FASTTREE_REF[:12], cache_dir,
+    )
+    _download(f"{_RAW_BASE_URL}/{asset}", binary_path)
+    _make_executable(binary_path)
+    return binary_path
+
+
+def _find_compiler() -> str | None:
+    for candidate in _COMPILER_CANDIDATES:
+        path = find_binary(candidate)
+        if path:
+            return path
+    return None
+
+
+def _provision_by_compiling(cache_dir: Path) -> Path:
+    compiler_path = _find_compiler()
+    if compiler_path is None:
+        raise MissingDependencyError(
+            "No C compiler was found on PATH to build FastTree from source (tried: "
+            f"{', '.join(_COMPILER_CANDIDATES)}). On macOS, install Xcode Command Line Tools: "
+            "`xcode-select --install`. Alternatively, install FastTree yourself (e.g. via "
+            "`conda env create -f environment.yml`) or set CANDY_NO_AUTO_DOWNLOAD=1 and provide it "
+            "on PATH another way."
+        )
+
+    cache_dir.mkdir(parents=True, exist_ok=True)
+    source_path = cache_dir / "FastTree.c"
+    binary_path = cache_dir / _binary_name()
+    logger.info(
+        "No precompiled FastTree is available for macOS -- compiling it from source with %s "
+        "(one-time, cached at %s).",
+        compiler_path, cache_dir,
+    )
+    _download(f"{_RAW_BASE_URL}/FastTree.c", source_path)
+
+    result = subprocess.run(
+        [compiler_path, *_COMPILE_FLAGS, "-o", str(binary_path), str(source_path), "-lm"],
+        capture_output=True,
+        text=True,
+    )
+    if result.returncode != 0 or not binary_path.is_file():
+        raise MissingDependencyError(
+            f"Compiling FastTree from source failed:\nstdout:\n{result.stdout}\nstderr:\n{result.stderr}"
+        )
+    _make_executable(binary_path)
+    return binary_path
+
+
+def resolve_fasttree_binary() -> str:
+    """Return a path to a usable FastTree binary, downloading/compiling one if needed.
+
+    Resolution order: PATH, then the local cache, then (unless disabled via
+    the CANDY_NO_AUTO_DOWNLOAD env var) automatic provisioning for the
+    current platform -- a precompiled download on Linux/Windows, or a local
+    from-source compile on macOS (where no precompiled binary is published
+    upstream).
+    """
+    for name in ("FastTree", "fasttree"):
+        on_path = find_binary(name)
+        if on_path:
+            return on_path
+
+    cached = _cached_binary()
+    if cached:
+        return str(cached)
+
+    if os.environ.get(NO_AUTO_DOWNLOAD_ENV):
+        raise MissingDependencyError(
+            "FastTree was not found on PATH and automatic download is disabled "
+            f"({NO_AUTO_DOWNLOAD_ENV} is set). Install it yourself (e.g. via "
+            "`conda env create -f environment.yml`) or unset that variable."
+        )
+
+    cache_dir = _cache_dir()
+    try:
+        if platform.system() == "Darwin":
+            binary = _provision_by_compiling(cache_dir)
+        else:
+            binary = _provision_precompiled(cache_dir)
+    except requests.RequestException as exc:
+        raise MissingDependencyError(
+            f"Failed to download FastTree automatically: {exc}. Install it yourself (e.g. via "
+            "`conda env create -f environment.yml`) or check your network connection. You can also "
+            f"set {NO_AUTO_DOWNLOAD_ENV}=1 to disable this download attempt."
+        ) from exc
+
+    return str(binary)

tests/test_alignment_and_phylogenetics.py

@@ -18,16 +18,16 @@ def test_mafft_aligner_invokes_binary_with_stdout_redirect(tmp_path):
     assert kwargs["stdout_path"] == output_fasta
 
 
-def test_fasttree_builder_falls_back_to_lowercase_binary_name(tmp_path):
+def test_fasttree_builder_uses_resolved_binary(tmp_path):
+    # Binary resolution itself (PATH, cache, auto-download/compile) is
+    # tested in tests/test_fasttree_download.py; this just verifies
+    # FastTreeBuilder wires the resolved path through to run_tool correctly.
     alignment = tmp_path / "aligned.fasta"
     output = tmp_path / "tree.nwk"
 
-    def fake_find_binary(name):
-        return "/usr/bin/fasttree" if name == "fasttree" else None
-
-    with patch("candy.phylogenetics.fasttree.find_binary", side_effect=fake_find_binary), patch(
-        "candy.phylogenetics.fasttree.run_tool"
-    ) as mock_run:
+    with patch(
+        "candy.phylogenetics.fasttree.resolve_fasttree_binary", return_value="/usr/bin/fasttree"
+    ), patch("candy.phylogenetics.fasttree.run_tool") as mock_run:
         FastTreeBuilder().build_tree(alignment, output)
 
     args, kwargs = mock_run.call_args