The Operation will be canceled. The next steps may not contain expected logs.

[tarekgh]

Build

https://dev.azure.com/dnceng-public/cbb18261-c48f-4abb-8651-8cdcb5474649/_build/results?buildId=696281

Build leg reported

Build / browser-wasm linux Release LibraryTests / Send to Helix

Pull Request

dotnet/runtime#103009

Known issue core information

Fill out the known issue JSON section by following the step by step documentation on how to create a known issue

 {
    "ErrorMessage" : "The Operation will be canceled. The next steps may not contain expected logs.",
    "BuildRetry": false,
    "ErrorPattern": "",
    "ExcludeConsoleLog": false
 }

@dotnet/dnceng

Release Note Category

• Feature changes/additions
• Bug fixes
• Internal Infrastructure Improvements

Release Note Description

Additional information about the issue reported

No response

Known issue validation

Build: 🔎 https://dev.azure.com/dnceng-public/public/_build/results?buildId=696281
Error message validated: [The Operation will be canceled. The next steps may not contain expected logs.]
Result validation: ✅ Known issue matched with the provided build.
Validation performed at: 6/4/2024 8:08:48 PM UTC

Report

Build	Definition	Step Name	Console log	Pull Request
1288212	dotnet/maui	Run Integration Tests - Build windows	Log	dotnet/maui#33985
1287884	dotnet/runtime	Send to Helix	Log
2900411	dotnet-roslyn	Test	Log
2900472	dotnet-aspire	🟣Build native packages	Log
2900385	dotnet-runtime	Install workload using artifacts	Log
2900386	dotnet-runtime	Install workload using artifacts	Log
1287814	dotnet/runtime	Publish Build Assets	Log	dotnet/runtime#123990
2900349	dotnet-roslyn	Test	Log
2900354	dotnet-efcore	Send job to helix	Log
2900254	dotnet-arcade-services	Run E2E tests	Log
1287651	dotnet/runtime	Send to Helix	Log	dotnet/runtime#124220
1287764	dotnet/runtime	Send tests to Helix (Unix)	Log	dotnet/runtime#124142
1287831	dotnet/runtime	Send to Helix	Log
2900310	dotnet-performance	Run performance job script	Log
1287808	dotnet/aspnetcore	Run Quarantined Tests	Log
2900292	dotnet-sdk	Generate delegation SAS Token for dotnetbuilds/internal	Log	#57745
1287762	dotnet/runtime	Send tests to Helix (Unix)	Log
2900291	dotnet-sdk	Checkout dotnet-sdk@refs/pull/57745/merge to s	Log	#57745
1287770	dotnet/runtime	Send to Helix	Log
1287690	dotnet/runtime	Send to Helix	Log	dotnet/runtime#124176
2900260	dotnet-sdk	Run Tests in Helix	Log	#57746
2900259	dotnet-sdk	Run Tests in Helix	Log	#57745
1287560	dotnet/maui	Controls SoftInput,Stepper,Switch,SwipeView	Log	dotnet/maui#33945
2900239	dotnet-dotnet	Initialize containers	Log
1287700	dotnet/maui	Run Integration Tests - Build windows	Log	dotnet/maui#33982
1287693	dotnet/maui	Install .NET	Log
1287531	dotnet/runtime	Send to Helix	Log
2900162	dotnet-arcade-services	Run E2E tests	Log
1287654	dotnet/runtime	Build product	Log	dotnet/runtime#123933
1287508	dotnet/runtime	Publish Build Assets	Log	dotnet/runtime#122638
2900204	dotnet-efcore	Send job to helix	Log
1286988	dotnet/maui	Run Integration Tests - Build windows	Log	dotnet/maui#33977
2900033	dotnet-runtime	Send ZipArchiveFuzzer to OneFuzz	Log
1287438	dotnet/runtime	Zip	Log	dotnet/runtime#123992
2900119	dotnet-aspnetcore	Run build.cmd helix target	Log
2900117	dotnet-aspnetcore	Run build.cmd helix target	Log
2900059	dotnet-arcade-services	Run E2E tests	Log
1287485	dotnet/maui	Controls Layout	Log
2899965	dotnet-efcore	Send job to helix	Log
1287350	dotnet/runtime	Send to Helix	Log
1287257	dotnet/maui	Controls Editor,Effects,FlyoutPage,Focus,Fonts,Frame,Gestures,GraphicsView	Log	dotnet/maui#33346
1287230	dotnet/sdk	🟣 Run TestBuild Tests	Log	dotnet/sdk#52932
1287352	dotnet/runtime	Send job to Helix (Unix)	Log
2899869	dotnet-sdk	Run AoT Tests	Log	#57694
1287254	dotnet/maui	Controls (API 30) CollectionView	Log	dotnet/maui#33953
2899888	dotnet-runtime	Send job to Helix (Windows)	Log
2899887	dotnet-runtime	Send job to Helix (Windows)	Log
1287216	dotnet/aspnetcore	Run Quarantined Tests	Log
1286665	dotnet/runtime	Send tests to Helix (Unix)	Log
2899878	dotnet-sdk	Run Tests in Helix	Log	#57700
2899870	dotnet-sdk	Run TestBuild Tests	Log	#57714
2899877	dotnet-sdk	Run Tests in Helix	Log	#57702
1287171	dotnet/maui	Controls Layout	Log
1287170	dotnet/maui	Controls Layout	Log
2899761	dotnet-sdk	Run AoT Tests	Log	#57694
1286956	dotnet/runtime	Send to Helix	Log
2899826	dotnet-roslyn	Test	Log
2899815	dotnet-efcore	Send job to helix	Log
2899795	dotnet-dotnet-monitor	Build	Log
2899788	dotnet-sdk	Run Tests in Helix	Log	#57702
2899789	dotnet-sdk	Run Tests in Helix	Log	#57700
1274111	dotnet/roslyn	Test	Log	dotnet/roslyn#82230
2899777	dotnet-runtime	Send job to Helix (Unix)	Log
2899773	dotnet-roslyn	Test	Log
1286972	dotnet/sdk	🟣 Run TestBuild Tests	Log	dotnet/sdk#52792
2899760	dotnet-sdk	Run AoT Tests	Log	#57714
2899627	dotnet-sdk	Build	Log	#57694
2899740	dotnet-aspnetcore	Secure Supply Chain Analysis (auto-injected by policy)	Log	#57720
2899692	dotnet-dotnet	Build (centOSStreamContainer)	Log
1286881	dotnet/aspnetcore	Run Quarantined Tests	Log	dotnet/aspnetcore#65377
1286990	dotnet/runtime	Send job to Helix (Unix)	Log
1286943	dotnet/runtime	Send tests to Helix (Windows)	Log	dotnet/runtime#124202
1286053	dotnet/runtime	Send to Helix	Log	dotnet/runtime#124179
2899652	dotnet-aspnetcore	Run build.cmd helix target	Log
2899655	dotnet-aspnetcore	Run build.cmd helix target	Log
2899674	dotnet-efcore	Send job to helix	Log
1286080	dotnet/runtime	Send to Helix	Log
2899642	dotnet-aspnetcore	Run build.cmd helix target	Log	#57697
2899664	dotnet-performance	Send job to Helix (Windows)	Log
2899637	dotnet-roslyn	Test	Log
1286046	dotnet/runtime	Send tests to Helix (Unix)	Log	dotnet/runtime#124178
2899604	dotnet-runtime	Send job to Helix (Windows)	Log
2899626	dotnet-sdk	Run AoT Tests	Log	#57714
1286755	dotnet/machinelearning	Run Helix Tests	Log	dotnet/machinelearning#7578
2899510	dotnet-sdk	Secure Supply Chain Analysis (auto-injected by policy)	Log	#57694
2899597	dotnet-roslyn	Test	Log
2899595	dotnet-roslyn	Test	Log
2899592	dotnet-roslyn	Test	Log
1286070	dotnet/runtime	Send tests to Helix (Unix)	Log
2899561	dotnet-roslyn	Test	Log
1286065	dotnet/runtime	Send tests to Helix (Unix)	Log
2899499	dotnet-runtime	Send job to Helix (Windows)	Log
2899543	dotnet-roslyn	Test	Log
1286579	dotnet/aspnetcore	Run Quarantined Tests	Log
2899548	dotnet-aspnetcore	Run build.cmd helix target	Log	#57696
2899545	dotnet-aspnetcore	Run build.cmd helix target	Log	#57456
2899547	dotnet-aspnetcore	Run build.cmd helix target	Log	#57695
2899525	dotnet-roslyn	Test	Log
2899511	dotnet-sdk	Run AoT Tests	Log	#57707
2899500	dotnet-dotnet	Build	Log	#57704
Displaying 100 of 2700 results

Summary

24-Hour Hit Count	7-Day Hit Count	1-Month Count
93	482	2700

[dkurepa]

These appear to be timeout cancellations, and most of them are happening on the same runtime leg that uses OSX.1200.ARM64.Open, looking at grafana, they do indeed appear to be quite high. Will check with the team on what we can do about this

[dkurepa]

For pipelines experiencing timeouts on osx.1200.arm64.open we recommend using osx.14.arm64.open, as this queue has a lot more machines

[dkurepa]

There is also a number of Publish Using Darc timeouts, these are in 99% of cases AzDo Api issues. During the Publish Using Darc job, we queue up a Maestro Promotion pipeline, that downloads the artifacts from the build calling it, and publishes them where they need to go. While publishing is happening, the build pipeline keeps busy waiting for the promotion pipeline to be done, checking the status of it every 60 seconds. In cases where this times out, the build most likely did get promoted, the AzDo API just kept telling us the wrong thing

[ivanpovazan]

@dkurepa for context, the same timeouts are also caught with this tracking #1883
where I asked about potentially bumping the queues to osx.13.amd/arm64, would that suffice?
(bumping to osx.14 is not an option right now as we still want testing on osx.13 which is the minimum supported OS version)

[CarnaViire]

For the "known issues" folks: the timeout here is reported twice, making BA red despite the tracking issue. E.g.:

1️⃣ dotnet/runtime#117255

2️⃣ dotnet/runtime#117800

cc @AlitzelMendez