Skip to content

Commit 2eae688

Browse files
committed
Disable ONNX Runtime thread pool spin-waiting
Set session.intra_op.allow_spinning=0 and session.inter_op.allow_spinning=0 to prevent ORT worker threads from busy-waiting between inference calls. This eliminates idle CPU burn without affecting inference latency.
1 parent f9af273 commit 2eae688

1 file changed

Lines changed: 2 additions & 0 deletions

File tree

src/ort-utils/ort-session-utils.cpp

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -27,6 +27,8 @@ int createOrtSession(filter_data *tf)
2727
Ort::SessionOptions sessionOptions;
2828

2929
sessionOptions.SetGraphOptimizationLevel(GraphOptimizationLevel::ORT_ENABLE_ALL);
30+
sessionOptions.AddConfigEntry("session.intra_op.allow_spinning", "0");
31+
sessionOptions.AddConfigEntry("session.inter_op.allow_spinning", "0");
3032
if (tf->useGPU != USEGPU_CPU) {
3133
sessionOptions.DisableMemPattern();
3234
sessionOptions.SetExecutionMode(ExecutionMode::ORT_SEQUENTIAL);

0 commit comments

Comments
 (0)