Skip to content

Allow xc-testing to provide more details #19

@johnrogers

Description

@johnrogers

Hi there,

Many thanks for building the plugin, it mostly works great.

However, I have a lot of tests and every single time it starts by using xc-testing, it doesn't get enough information about why tests failed, and therefore has to revert to using xcodebuild commands anyway.

Example:

{
       "success": false,
       "error": "Tests failed",
       "details": "2026-01-01 12:48:31.015 xcodebuild[78241:7467539] [MT] IDETestOperationsObserverDebug: 87.305 elapsed -- Testing started completed.\n2026-01-01 12:48:31.015 xcodebuild[78241:7467539] [MT]
     IDETestOperationsObserverDebug: 0.000 sec, +0.000 sec -- start\n2026-01-01 12:48:31.015 xcodebuild[78241:7467539] [MT] IDETestOperationsObserverDebug: 87.305 sec, +87.305 sec -- end\n** TEST FAILED **"
}

∴ Thinking…

It says TEST FAILED but no specific failures were shown. Let me run all tests and capture more details to see what's failing.

Now let me build and run tests:

⏺ plugin:xclaude-plugin:xc-build - xcode_build (MCP)(scheme: "App", destination: "platform=iOS Simulator,name=iPhone Air")
⎿  {
"success": true,
"data": {
… +5 lines (ctrl+o to expand)
⎿  Running PostToolUse hooks… (1/2 done)

⏺ plugin:xclaude-plugin:xc-testing - xcode_test (MCP)(scheme: "App", destination: "platform=iOS Simulator,name=iPhone Air")
⎿  {
"success": false,
"error": "Tests failed",
… +3 lines (ctrl+o to expand)
⎿  Running PostToolUse hooks… (1/2 done)

⏺ Bash(xcodebuild test -scheme App -destination 'platform=iOS Simulator,name=iPhone Air' -only-testing:AppTests 2>&1 | grep -E "(Test Case|passed|failed|error:|F…) timeout: 5m 0s
⎿  Test case 'Tests/test()' passed on 'Clone 1 of iPhone Air - App (79564)' (4.000 seconds)
Test case 'Tests/test()' passed on 'Clone 1 of iPhone Air - App (79564)' (4.000 seconds)
Test case 'Tests/test()' passed on 'Clone 1 of iPhone Air - App (79564)' (4.000 seconds)
… +47 lines (ctrl+o to expand)
⎿  Running PostToolUse hooks… (1/2 done)

⏺ Bash(xcodebuild test -scheme App -destination 'platform=iOS Simulator,name=iPhone Air' -only-testing: AppTests 2>&1 | grep -E "(FAILED|error:|✗|Assertion Failed…) timeout: 5m 0s
⎿  ** TEST FAILED **
⎿  Running PostToolUse hooks… (1/2 done)

⏺ Bash(xcodebuild test -scheme App -destination 'platform=iOS Simulator,name=iPhone Air' 2>&1 | grep -B5 -A5 "failed|FAILED|error:" | tail -60) timeout: 5m 0s
⎿  Running…
ctrl+b to run in background

Is there a way to allow Claude to discover and specify that more lines of details should be provided, so that it doesn't have to revert to using Xcode CLI tools anyway?

The result of the above is that it takes several tries (each time Claude greps/tails more and more lines) to get the actual failures, taking a lot of time. I'm having to run the tests manually and paste in the failures a lot of the time.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions