Classify plugin startup import failures (#993) - #14824
Open
RonnyPfannschmidt wants to merge 1 commit into
Open
Conversation
RonnyPfannschmidt
force-pushed
the
plugin-import-error-exit-codes-993
branch
5 times, most recently
from
August 26, 2026 22:12
434b9ab to
363aea6
Compare
RonnyPfannschmidt
force-pushed
the
plugin-import-error-exit-codes-993
branch
from
August 27, 2026 07:02
363aea6 to
baff0df
Compare
RonnyPfannschmidt
marked this pull request as ready for review
August 27, 2026 07:20
RonnyPfannschmidt
force-pushed
the
plugin-import-error-exit-codes-993
branch
from
August 27, 2026 07:21
baff0df to
875f10f
Compare
nicoddemus
requested changes
Aug 27, 2026
A plugin failing to load during startup escaped `_main` as an unhandled exception: Python printed a raw traceback and exited 1, which is indistinguishable from EXIT_TESTSFAILED. Meanwhile a conftest.py failing to import already returned EXIT_USAGEERROR, which is the inconsistency pytest-dev#993 was filed about. Split the failure into the two things it can actually mean: - the plugin cannot be found at all -- pytest was pointed at something which is not there, so this is a usage error (exit 4), matching what conftest.py import failures already do. - the plugin was found but raised while importing -- including a missing transitive dependency and a broken pytest11 entry point -- which is a defect in the plugin rather than a misuse of pytest, so it is reported as an internal error (exit 3). The plugin traceback is preserved in both the report and the exception chain (PluginImportFailure is always raised `from` the original error); losing it was the main objection to the earlier attempt in pytest-dev#7290. Side effects: - pytest.main() now returns these exit codes instead of propagating the exception to its caller, matching its documented contract. - a bare `raise ImportError` in a plugin no longer crashes pytest's own internals with `IndexError: tuple index out of range` from `e.args[0]`. conftest.py import failures are deliberately left alone and keep returning exit 4. Co-Authored-By: Claude Opus 5 <ai@anthropic.com> Co-Authored-By: Claude Fable 5 <ai@anthropic.com> Co-Authored-By: Claude Code <ai@anthropic.com>
RonnyPfannschmidt
force-pushed
the
plugin-import-error-exit-codes-993
branch
from
August 27, 2026 19:57
875f10f to
98fcf26
Compare
Member
Author
|
might make sense to wait 1-2 majro releases before putting this in as is |
Member
|
Why wait? Going into the next major release should be fine? |
Member
Author
|
Sorry I meant feature release I recall we have some new deprecation |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #993.
On
maina plugin failing to import at startup escapes_mainas a plainImportError: raw traceback, exit 1 — colliding withEXIT_TESTSFAILEDby accident — andpytest.main()raises instead of returning its documented exit code. A bare arglessImportErroradditionally crashes pytest's owne.args[0]handling withIndexError.#7290 stalled on whether a plugin failing under a conftest's
pytest_pluginsis a conftest problem or a plugin problem. This PR splits on what actually went wrong instead of on which mechanism asked:-p,pytest_plugins,PYTEST_PLUGINS) → misuse →USAGE_ERROR(4), same as a failingconftest.pyimportpytest11entry point → plugin defect →INTERNAL_ERROR(3), traceback preserved in the report and on the exception chain (raise ... from) (losing it was the other objection to Issue 993 - improve handling exceptions in conftest #7290)The split keys on
ModuleNotFoundError.namematching the requested spec or a parent package, so a plugin whose ownimport foofails is not blamed on the user.conftest.pyimport failures stay at 4, unchanged.-p/pytest_plugins/PYTEST_PLUGINSconftest.pyfails to importpytest.main()in-processImportErrorDraft because the classification is a judgement call and exit codes are observable API:
INTERNAL_ERROR(3) right for a broken third-party plugin, or does this warrant a new exit code?wrap_sessionalready uses 3 for plugin exceptions mid-session.conftest.pyfollow the same split? Left alone here to keep the blast radius small.load_setuptools_entrypointsoverride widens scope — but without it a broken installed plugin still exits 1 with a raw traceback.Changelog:Filed asbugfixorbreaking?breakingper review.Tests: new
TestStartupPluginImportErrorscovers every entry path plus both regressions; six existing tests updated fromImportErrortoUsageError/PluginImportFailure. Full suite and pre-commit (incl. mypy) pass locally.🤖 Written by Claude Code (Claude Fable 5); reviewed and submitted by @RonnyPfannschmidt.