fix(cli): clarify exception diagnostics#2602
Open
darion-yaphet wants to merge 1 commit into
Open
Conversation
Consolidate the CLI diagnostic plan, implementation, and test hardening into one reviewable change. The CLI now reports phase and target context for broad failure paths while preserving existing fail-fast behavior for real setup failures and warning-only behavior for optional best-effort work. The workflow unit tests also avoid discovering real local agent CLIs, so developer machines with tools such as gemini installed do not hang pytest during metadata-only assertions. Constraint: CLI setup failures must remain fail-fast, while optional preset and cleanup paths should continue with clear warnings. Rejected: Replace broad handlers across the whole codebase in one pass | too broad for a targeted CLI diagnostic fix Rejected: Add runtime timeouts to workflow agent dispatch | dispatch may legitimately be long-running and the observed hang was test isolation Confidence: high Scope-risk: moderate Directive: Keep future best-effort CLI warnings tied to the failed phase and target so users can diagnose setup state. Tested: uvx ruff check src/; uv run pytest tests/integrations/test_cli.py -v; uv run pytest tests/test_workflows.py::TestCommandStep::test_step_override_integration tests/test_workflows.py::TestPromptStep::test_execute_with_step_integration tests/test_workflows.py::TestPromptStep::test_execute_with_model -vv; uv run pytest Not-tested: Real Nacos/PG/Redis-style external service failure injection; real interactive workflow dispatch against installed gemini CLI
Contributor
There was a problem hiding this comment.
Pull request overview
Improves Spec Kit CLI diagnostics by standardizing exception detail formatting and adding phase/target context for both fatal and best-effort failure paths, plus hardens workflow step tests to avoid accidentally invoking real agent CLIs on developer machines.
Changes:
- Add internal helpers to format exception details and phase/target labels, and use them in
init,integration install/switch/upgradeerror and warning output. - Improve best-effort warning messages (e.g., optional preset install, extension artifact cleanup) to include actionable context while still continuing execution.
- Update workflow step tests to patch CLI detection (
shutil.which) so locally installed CLIs (e.g.,gemini) don’t get executed during pytest.
Show a summary per file
| File | Description |
|---|---|
src/specify_cli/__init__.py |
Introduces _cli_error_detail, _cli_phase_label, _print_cli_warning and applies them to key CLI exception paths for clearer diagnostics. |
tests/integrations/test_cli.py |
Adds regression tests asserting the new diagnostic formatting and phase/target reporting behaviors. |
tests/test_workflows.py |
Patches CLI detection in step tests to prevent unintended real CLI invocation/hangs. |
Copilot's findings
Tip
Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
- Files reviewed: 3/3 changed files
- Comments generated: 0
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Improves CLI error diagnostics for broad exception and best-effort paths.
This PR keeps the existing fail-fast behavior for real setup failures, but makes user-facing errors more actionable by including the operation phase, target
integration/preset/workflow, and underlying exception detail where available. It also keeps optional best-effort failures as warnings with clearer context, so users
can understand what failed and what continued.
It also fixes workflow tests that could accidentally invoke a real local agent CLI, such as
gemini, causinguv run pytestto hang on developer machines where thatCLI is installed.
Testing
uv run specify --helpuv run pytestuvx ruff check src/Results:
uvx ruff check src/passeduv run pytestpassed:2917 passed, 34 skipped, 19 warningsAI Disclosure
Used AI assistance to identify broad CLI exception-handling paths, implement clearer diagnostics, add regression coverage, and run/fix local verification.