Context
The test suite has grown to ~1200 tests across 4 tiers (unit, integration, live, quick). Full mise run check takes ~3 minutes and check:full ~4-5 minutes. As we add more providers (Spicerack, MTGGoldfish, etc.), test count and runtime keep climbing. Time for a periodic audit.
Areas to Review
1. Test runtime profiling
- Run
pytest --durations=30 to identify the slowest tests
- Are any unit tests doing unnecessary work (e.g., redundant fixture setup, repeated cache clearing)?
- Can slow service tests share fixtures more efficiently?
2. Redundancy between tiers
- Do integration tests (
tests/integration/) duplicate coverage already in provider unit tests?
- Do live tests (
tests/live/) overlap with integration tests beyond what's needed for real-world regression?
- Are there unit tests that test the same code path through slightly different inputs without adding value?
3. Provider test patterns
- Each new provider (EDHREC, Moxfield, Spicerack, MTGGoldfish) follows the same test pattern. Are we testing boilerplate (e.g.,
ToolError wrapping, response_format variants) redundantly across providers?
- Could shared parametrized fixtures reduce duplication for common patterns (lifespan init, error handling, attribution lines)?
4. Fixture overhead
conftest.py clears all service caches via autouse fixture. Is the clearing list growing unwieldy?
- Are HTML/JSON fixture files larger than needed? Could we trim captured fixtures to only the fields tests actually assert on?
5. Workflow test efficiency
- Workflow tests use
AsyncMock — these should be fast. Are any accidentally doing real work?
test_workflow_server.py tests tool registration — does this overlap with test_orchestrator_e2e.py?
6. testmon effectiveness
- Is
pytest-testmon correctly detecting affected tests, or is it re-running too many?
- Are there test files that testmon always marks as affected due to broad imports?
Outcome
- Identify and remove redundant tests
- Speed up the slowest tests
- Document any test patterns that should be standardized
- Target: keep
mise run check under 2 minutes as the suite grows
Priority
Low — not blocking anything, but good hygiene to do periodically as the project scales.
Context
The test suite has grown to ~1200 tests across 4 tiers (unit, integration, live, quick). Full
mise run checktakes ~3 minutes andcheck:full~4-5 minutes. As we add more providers (Spicerack, MTGGoldfish, etc.), test count and runtime keep climbing. Time for a periodic audit.Areas to Review
1. Test runtime profiling
pytest --durations=30to identify the slowest tests2. Redundancy between tiers
tests/integration/) duplicate coverage already in provider unit tests?tests/live/) overlap with integration tests beyond what's needed for real-world regression?3. Provider test patterns
ToolErrorwrapping,response_formatvariants) redundantly across providers?4. Fixture overhead
conftest.pyclears all service caches via autouse fixture. Is the clearing list growing unwieldy?5. Workflow test efficiency
AsyncMock— these should be fast. Are any accidentally doing real work?test_workflow_server.pytests tool registration — does this overlap withtest_orchestrator_e2e.py?6. testmon effectiveness
pytest-testmoncorrectly detecting affected tests, or is it re-running too many?Outcome
mise run checkunder 2 minutes as the suite growsPriority
Low — not blocking anything, but good hygiene to do periodically as the project scales.