[Bug]: Discovery tests fail when global config.yaml exists — isolate pytest from merged config

[rosspeili]

Affected Component

Tests / CI

Skill ID (if applicable)

No response

Bug Description

Two framework tests assume legacy skill-root order (SKILLWARE_SKILL_PATH → cwd ./skills/ → bundled):

• tests/test_discovery.py::test_get_skill_roots_order_env_project_bundled
• tests/test_loader.py::test_resolve_skill_prefers_env_over_cwd

Since #246, when global or project YAML config exists, discovery uses configured mode with default paths.resolution.order: project → external → bundled.

After operator setup (e.g. skillware mail signature init from #292), a user-level config.yaml exists. Local pytest tests/ then uses configured mode and those two tests fail (458 pass, 2 fail). CI stays green on a clean home directory.

Not caused by #292/#293 skill code — test isolation + assertions that only match legacy mode.

Why it matters

• Developers get false “2 failed” after normal mail/config init.
• Shadowing policy: configured mode lets project ./skills/ override SKILLWARE_SKILL_PATH; tests should lock both legacy and configured behavior.
• Versioning: test/docs fix only; no runtime semver change unless we deliberately change [Feat]: CLI config and three-tier skill path resolution (global + project) #246 order (out of scope).

Steps to Reproduce

1. Clone repo; pip install -e ".[dev,all]".

2. Run skillware mail signature init (writes global config.yaml).

3. From repo root:

   python -m pytest tests/test_discovery.py::test_get_skill_roots_order_env_project_bundled tests/test_loader.py::test_resolve_skill_prefers_env_over_cwd -q

4. Failures: project tier listed/resolved before external/env.

Control (should pass): set SKILLWARE_CONFIG_DIR to an empty temp directory, then run the same two tests.

Error Logs

AssertionError: assert <SkillRootTier.PROJECT: 'project'> == <SkillRootTier.EXTERNAL: 'external'>
  tests/test_discovery.py:46

AssertionError: assert 'from_cwd' == 'from_env'
  tests/test_loader.py:398

Environment

Windows 11 · Python 3.13 · editable clone · global config.yaml after skillware mail signature init · full suite otherwise 458/460 pass

[rosspeili]

Suggested fix:

Ideal path (minimal, matches current repo state):

1. Isolate config in affected tests — monkeypatch.setenv("SKILLWARE_CONFIG_DIR", str(tmp_path / "empty")) + clear_config_cache() so real global YAML is not loaded.
2. Split by mode — keep legacy tests (no config → external first); add configured-mode tests with minimal config.yaml asserting project → external → bundled ([Feat]: CLI config and three-tier skill path resolution (global + project) #246 contract).
3. Shared fixture — e.g. isolated_skillware_config in tests/conftest.py for tests hitting get_skill_roots() / SkillLoader.load_skill().
4. Docs — note in docs/TESTING.md: local full suite must not depend on operator config.yaml; CI is authoritative.
5. Do not change default resolution.order without a separate RFC — affects skill shadowing and [Feat]: CLI config and three-tier skill path resolution (global + project) #246 behavior.

[rosspeili]

Hey @wdwd200, please go ahead.

[wdwd200]

Thanks! I noticed PR #303 is already linked to this issue. Would you still like me to work on an alternative fix, or should I pick another issue to avoid duplicating work?

[rosspeili]

Hey @wdwd200 since #303 is unresponsive to the review, and #311 has not claimed the issue, you can go ahead and try to solve this. I will assign to you for now. And of course you are more than welcome to pick other issues that interest you or you have a strong opinion about. <3

[wdwd200]

Thanks! I’ll take this on.

I’ll first review #303 and #311 to understand the existing approaches and the review feedback, then work on a clean fix that addresses the issue without duplicating the same problems.

I’ll reach out here if I run into anything unclear.

[rosspeili]

@wdwd200 appreciate it, and yeah go ahead. My only intial tip is to read contributing guide and ai native workflow. Everything else should be easy to test and evaluate based on the flows suggested. For anything else, at your dispossal. <3