setInterOpNumThreads method

void setInterOpNumThreads(
  1. int numThreads
)

Sets the number of inter-op threads used by ONNX Runtime.

IMPORTANT: This controls NATIVE C++ threads inside ONNX Runtime, NOT Dart isolates!

Inter-op threads are used to parallelize execution between independent operators in the computation graph. For example, if two operators don't depend on each other, they can run simultaneously on different threads.

This is completely independent of Dart isolates:

  • To run multiple inferences in parallel, use runOnceAsync() or runParallelAsync()
  • Those methods create separate Dart isolates (separate memory spaces)
  • This setting only affects threading within each isolate's ONNX session

Default is 0 (uses all available CPU cores). Set to 1 to disable inter-op parallelism and run operators sequentially.

Implementation

void setInterOpNumThreads(int numThreads) {
  final statusPtr = OrtEnv.instance.ortApiPtr.ref.SetInterOpNumThreads
      .asFunction<
          bg.OrtStatusPtr Function(
              ffi.Pointer<bg.OrtSessionOptions>, int)>()(_ptr, numThreads);
  OrtStatus.checkOrtStatus(statusPtr);
}