dotnet-watch tests failing

[dsplaisted]

Build Information

Build: https://dev.azure.com/dnceng-public/cbb18261-c48f-4abb-8651-8cdcb5474649/_build/results?buildId=629827
Build error leg or test failing: dotnet-watch.Tests.dll.1.WorkItemExecution
Pull request: #39888

Error Message

##[error].packages/microsoft.dotnet.helix.sdk/8.0.0-beta.24177.1/tools/Microsoft.DotNet.Helix.Sdk.MultiQueue.targets(89,5): error : (NETCORE_ENGINEERING_TELEMETRY=Test) Work item dotnet-watch.Tests.dll.1 in job 82728a56-9430-4356-ab82-1c21670792a7 has failed

{
  "ErrorMessage": "",
  "ErrorPattern": "Work item dotnet-watch\\.Tests\\.dll\\.1 in job [a-z0-9\\-]+ has failed",
  "BuildRetry": true,
  "ExcludeConsoleLog": true
}

Known issue validation

Build: 🔎 https://dev.azure.com/dnceng-public/public/_build/results?buildId=629827
Error message validated: [Work item dotnet-watch\.Tests\.dll\.1 in job [a-z0-9\-]+ has failed]
Result validation: ✅ Known issue matched with the provided build.
Validation performed at: 4/4/2024 9:51:10 AM UTC

Report

Build	Definition	Step Name	Console log	Pull Request
1317286	dotnet/sdk	🟣 Run FullFramework Tests	Log	#53219
1317281	dotnet/sdk	Run FullFramework Tests	Log	#53067
1317173	dotnet/sdk	🟣 Run TestBuild Tests	Log	#52802
1317140	dotnet/sdk	🟣 Run TestBuild Tests	Log	#52956
1316726	dotnet/sdk	🟣 Run TestBuild Tests	Log	#52930
1316128	dotnet/sdk	🟣 Run TestBuild Tests	Log	#52930
1315189	dotnet/sdk	🟣 Run FullFramework Tests	Log	#53205
1314674	dotnet/sdk	🟣 Run FullFramework Tests	Log	#53170
1314175	dotnet/sdk	🟣 Run TestBuild Tests	Log	#53192
1314090	dotnet/sdk	🟣 Run FullFramework Tests	Log	#53188
1313948	dotnet/sdk	🟣 Run TestBuild Tests	Log	#53196
1313827	dotnet/sdk	🟣 Run TestBuild Tests	Log	#52930
1313720	dotnet/sdk	🟣 Run TestBuild Tests	Log	#53192
1313852	dotnet/sdk	🟣 Run TestBuild Tests	Log	#53181
1313699	dotnet/sdk	🟣 Run TestBuild Tests	Log	#52930
1313483	dotnet/sdk	🟣 Run FullFramework Tests	Log	#52956
1313249	dotnet/sdk	🟣 Run TestBuild Tests	Log	#52930
1312943	dotnet/sdk	🟣 Run FullFramework Tests	Log	#52937
2914068	dotnet-sdk	Run TestBuild Tests	Log	#58431
1312752	dotnet/sdk	🟣 Run FullFramework Tests	Log	#53186
2914067	dotnet-sdk	Run TestBuild Tests	Log	#58430
1312479	dotnet/sdk	🟣 Run FullFramework Tests	Log	#53175
1312297	dotnet/sdk	🟣 Run TestBuild Tests	Log	#53181
1310452	dotnet/sdk	🟣 Run TestBuild Tests	Log	#53123
1311959	dotnet/sdk	🟣 Run TestBuild Tests	Log	#52930
1297889	dotnet/sdk	🟣 Run TestBuild Tests	Log	#52956
1311288	dotnet/sdk	🟣 Run TestBuild Tests	Log	#52930
1311210	dotnet/sdk	🟣 Run TestBuild Tests	Log	#53175
1311309	dotnet/sdk	🟣 Run FullFramework Tests	Log	#52938
1310265	dotnet/sdk	🟣 Run TestBuild Tests	Log	#53108
1310172	dotnet/sdk	🟣 Run FullFramework Tests	Log	#52930
1310237	dotnet/sdk	🟣 Run TestBuild Tests	Log	#53142
1309877	dotnet/sdk	🟣 Run TestBuild Tests	Log	#53158
1309927	dotnet/sdk	🟣 Run TestBuild Tests	Log	#53159
1309895	dotnet/sdk	🟣 Run TestBuild Tests	Log	#53108
1309560	dotnet/sdk	🟣 Run TestBuild Tests	Log	#52930
1309203	dotnet/sdk	🟣 Run TestBuild Tests	Log	#53150
1309115	dotnet/sdk	🟣 Run TestBuild Tests	Log	#53136
1308856	dotnet/sdk	🟣 Run TestBuild Tests	Log	#52648
1308569	dotnet/sdk	🟣 Run FullFramework Tests	Log	#52930
1308564	dotnet/sdk	🟣 Run FullFramework Tests	Log	#53138
1308346	dotnet/sdk	🟣 Run TestBuild Tests	Log	#52648
1307940	dotnet/sdk	🟣 Run TestBuild Tests	Log	#52802
1307835	dotnet/sdk	🟣 Run TestBuild Tests	Log	#53136
1307679	dotnet/sdk	🟣 Run TestBuild Tests	Log	#52930
1307724	dotnet/sdk	🟣 Run TestBuild Tests	Log	#53127
1307700	dotnet/sdk	🟣 Run TestBuild Tests	Log	#53108
1307838	dotnet/sdk	🟣 Run TestBuild Tests	Log	#53056
1307497	dotnet/sdk	🟣 Run TestBuild Tests	Log	#52802
1307478	dotnet/sdk	🟣 Run TestBuild Tests	Log	#53123
1307467	dotnet/sdk	🟣 Run TestBuild Tests	Log	#53056
1307300	dotnet/sdk	🟣 Run FullFramework Tests	Log	#53119
1307008	dotnet/sdk	🟣 Run FullFramework Tests	Log	#52648
1306828	dotnet/sdk	🟣 Run TestBuild Tests	Log	#53096
1306836	dotnet/sdk	🟣 Run TestBuild Tests	Log	#53130
1306834	dotnet/sdk	Run TestBuild Tests	Log	#53129
1306345	dotnet/sdk	🟣 Run TestBuild Tests	Log	#52930
1306242	dotnet/sdk	🟣 Run FullFramework Tests	Log	#53127
1305261	dotnet/sdk	🟣 Run TestBuild Tests	Log	#53119
1304711	dotnet/sdk	🟣 Run TestBuild Tests	Log	#53116
1304162	dotnet/sdk	🟣 Run TestBuild Tests	Log	#53113
1304163	dotnet/sdk	🟣 Run TestBuild Tests	Log	#52648
1304161	dotnet/sdk	🟣 Run TestBuild Tests	Log	#53112
1304118	dotnet/sdk	🟣 Run TestBuild Tests	Log	#53105
1303317	dotnet/sdk	🟣 Run TestBuild Tests	Log	#52648
1303417	dotnet/sdk	🟣 Run TestBuild Tests	Log	#53095
1303284	dotnet/sdk	🟣 Run TestBuild Tests	Log	#53105
1303221	dotnet/sdk	🟣 Run TestBuild Tests	Log	#53104
1302726	dotnet/sdk	🟣 Run TestBuild Tests	Log	#52581
1302834	dotnet/sdk	🟣 Run TestBuild Tests	Log	#53100
1302463	dotnet/sdk	🟣 Run TestBuild Tests	Log	#53056
1302572	dotnet/sdk	🟣 Run TestBuild Tests	Log	#53100
2908674	dotnet-sdk	Run TestBuild Tests	Log	#58174
1302206	dotnet/sdk	🟣 Run TestBuild Tests	Log	#53056
1300908	dotnet/sdk	🟣 Run TestBuild Tests	Log	#52347
1301137	dotnet/sdk	🟣 Run TestBuild Tests	Log	#53086
1302013	dotnet/sdk	🟣 Run TestBuild Tests	Log	#53095
2908612	dotnet-sdk	Run TestBuild Tests	Log	#58174
1301702	dotnet/sdk	🟣 Run TestBuild Tests	Log	#52648
2908332	dotnet-sdk	Run TestBuild Tests	Log	#58156
1301172	dotnet/sdk	🟣 Run FullFramework Tests	Log	#52648
1300925	dotnet/sdk	🟣 Run TestBuild Tests	Log	#53086
1300062	dotnet/sdk	🟣 Run TestBuild Tests	Log	#52930
1299936	dotnet/sdk	🟣 Run TestBuild Tests	Log	#52960
1299924	dotnet/sdk	🟣 Run TestBuild Tests	Log	#52648
1299395	dotnet/sdk	🟣 Run TestBuild Tests	Log	#52960
1299531	dotnet/sdk	🟣 Run TestBuild Tests	Log	#53029
1299376	dotnet/sdk	🟣 Run TestBuild Tests	Log	#52648
1299171	dotnet/sdk	🟣 Run TestBuild Tests	Log	#52930
1299192	dotnet/sdk	Run TestBuild Tests	Log	#53072
1298061	dotnet/sdk	🟣 Run TestBuild Tests	Log	#52960
1298099	dotnet/sdk	🟣 Run TestBuild Tests	Log	#53029
1298033	dotnet/sdk	🟣 Run TestBuild Tests	Log	#52930
1297796	dotnet/sdk	🟣 Run TestBuild Tests	Log	#52960
1297789	dotnet/sdk	🟣 Run TestBuild Tests	Log	#53059
1297666	dotnet/sdk	🟣 Run TestBuild Tests	Log	#53029
1297617	dotnet/sdk	🟣 Run TestBuild Tests	Log	#52581
1296816	dotnet/sdk	🟣 Run TestBuild Tests	Log	#53044
1296541	dotnet/sdk	🟣 Run TestBuild Tests	Log	#52960
1296526	dotnet/sdk	🟣 Run TestBuild Tests	Log	#53048
Displaying 100 of 191 results

Summary

24-Hour Hit Count	7-Day Hit Count	1-Month Count
6	52	181

[marcpopMSFT]

@tmat we still see this fairly frequently. Any luck? Here's a recent one from a codeflow PR:
https://helixre107v0xdeko0k025g8.blob.core.windows.net/dotnet-sdk-refs-pull-41616-merge-58511284b77c40778a/dotnet-watch.Tests.dll.1/1/console.418de7ca.log?helixlogtype=result

[tmat]

No luck yet. This might help: https://github.com/dotnet/sdk/blob/d45e40d665e647804de03d2b4eec2aae9c488cb5/test/dotnet-watch.Tests/xunit.runner.json

It will print out test info every 20 seconds a test is running.

[marcpopMSFT]

@tmat another failure this morning shows a bunch of build processes for watch apps still running... https://helixre107v0xd1eu3ibi6ka.blob.core.windows.net/dotnet-sdk-refs-pull-41958-merge-84357342ee154a5088/dotnet-watch.Tests.dll.2/1/console.3dddc463.log?helixlogtype=result

[tmat]

Looks like that's in 8.0 branch. Porting test diagnostics from 9: #41963

[Forgind]

This has been making one of my PRs fail today, so I looked into it a bit. Based on when this issue was created and which tests are failing, I suspect this PR:
#39618

I asked jasonmalinowski whether the MSBuildWorkspace is supposed to support F# because I'd found a code snippet that suggested that it shouldn't, and he said it doesn't and never did. Did that PR make us use MSBuildWorkspace in a different way than we had before?

[Forgind]

The changes to HotReloadDotNetWatcher look most suspicious to me. It looks like a substantial change to how we call MSBuild.