ENH: AdaptiveCpp added to SuperBuild Mac and Linux

[xskere]

AdaptiveCpp has been added to the superbuild. By default it's not added to the build, but by setting EXTENSION_BUILDS_ADAPTIVE_CPP to ON, AdaptiveCpp will build, but only on Mac or Linux, otherwise it will be omitted.

Re #292

[cpinter]

It looks good to me. @ferdymercury if you have any comments or requests please let us know. I'll wait for your feedback before clicking merge. Thanks!

[ferdymercury]

Looks great to me. Thanks.

The only remaining question would be the one raised by @lassoan on whether we can also support Windows (not important to me).

The point here was to support SYCL. If I understand correctly, for Windows, since you mentioned that AdaptiveCpp only works with gcc, we could instead use Intel's OneApi DPC compiler. https://www.intel.com/content/www/us/en/developer/tools/oneapi/dpc-compiler.html which natively supports SYCL compilation.

Would that be doable?

I guess we would need something like:

if (UNIX)
find_package(AdaptiveCpp REQUIRED)
else()
find_package(IntelDPCPP REQUIRED)
endif()

Windows example:
https://www.intel.com/content/www/us/en/docs/dpcpp-cpp-compiler/developer-guide-reference/2023-0/use-cmake-with-the-compiler.html

[cpinter]

A single compiler needs to be used on the factory machine so that the components are compatible with each other, so I don't see this possible.

Maybe if you explained your motivation behind including AdaptiveCpp and IPOPT in SlicerRT and the kind of audience for which these are useful, @lassoan would have a better idea about the needs. Thanks!

(FYI we are exploring the possibility of using Ipopt via the pre-compiled DLLs on Windows. I don't want to include binary files into the repo, but we'll suggest a solution if using the DLLs work)

[ferdymercury]

Sure.

For AdaptiveCpp, this is the idea: we are developing a fast library GPU proton dose calculator using SYCL that we want to integrate with SlicerRT. This library has let's say 10 targets, and one of them uses SYCL.

The correct way of dealing with this in CMake is:

add_library(Target1)
add_library(Target2)
add_library(Target10)
add_sycl_to_target(Target10)

The last statement is a CMake macro that properly uses the SYCL compiler for that target. The rest of targets are compiled with standard GCC.

This is nothing new, the same happens when you use CUDA and CMake. You define (with slightly different statements) which targets use CUDA acceleration, the other ones are compiled with standard GCC. One does not need to also compile all the others with CUDA nvcc compiler even if it's posible.

This is the situation with Linux at least, not sure about Windows.

So... to embed our new tool in SlicerRT, we need a compiler that is SYCL-capable, and we do not mind if it's intel DPC or adaptivecpp or any other. The challenge here might be that it's a rather modern feature that only some recent compilers have. . The audience is people doing forward treatment planning in proton therapy, and comparisons between analytical and Monte carlo dosimetry.

Concerning IPOPT, it's a completely different (opposite) problem. It's an external very old library. We want to embed it with SlicerRT for fast inverse treatment planning optimization. Here the challenge is the opposite: rather old Fortran stuff and old configure/make scripts. The audience is people doing or learning inverse treatment planning in radiotherapy.

[cpinter]

Thanks a lot @ferdymercury ! What I don't know for example, is how prevalent it is using Linux among researchers working with proton therapy.

In any case @xskere already managed to include the DLLs in a simple way (CMake downloads it), so it should work. He confirmed that headers can be included without an issue on Windows, but I'd rather test it somehow, executing the code. Do you have a simple example we could run on Windows? Thank you!

[ferdymercury]

Thanks, makes sense.

What about this example? https://github.com/coin-or/Ipopt/tree/stable/3.14/examples/Cpp_example
If needed we could port it to CMake rather than Makefile and make it a CTest ?

Among researchers... well researchers in hospitals probably most have Windows, researchers outside hospitals have 60% Unix systems

[cpinter]

Thank you! The example seems to be for Ipopt. I'll copy it to that ticket for reference. Any suggestions for AdaptiveCpp?

[cpinter]

It seems that I am mixing up the two. Hang on :)

[ferdymercury]

For AdaptiveCpp, there could be this example: https://github.com/AdaptiveCpp/AdaptiveCpp/blob/develop/examples/CMakeLists.txt

Or if you want something more simple, we have done something by hand ourselves, just in the CMakeLists:

find_package(AdaptiveCpp CONFIG REQUIRED)
# If needed I could search for the equivalent call for IntelDPCpp for Windows

find_package(CLI11 CONFIG REQUIRED) # header-only library  argparse lib https://github.com/CLIUtils/CLI11 or sudo apt install libcli11-dev

add_executable(basic_SYCL_test basic_SYCL_test.cpp)
target_link_libraries(basic_SYCL_test PRIVATE CLI11::CLI11)
add_sycl_to_target(TARGET basic_SYCL_test)

add_test(
    NAME basic_cpu_test
    COMMAND basic_SYCL_test
    --DeviceName cpu
)

add_test(
    NAME basic_gpu_test
    COMMAND basic_SYCL_test
    --DeviceName gpu
)

and then a file basic_SYCL_test.cpp

/**
 * @file
 * @brief Basic test to check that SYCL is running correctly on the selected device ("gpu", "cpu")
 */

#include <CLI/CLI.hpp>
#include <iostream>
#include <string>
#include <sycl/sycl.hpp>

/**
 * @brief Basic function to free memory that was allocated during the test
 * @param q The sycl queue used
 * @param testArrayOnDevice The unsigned int array used for testing
 * @param submitFlag Error flag to signal unsuccessful submiting
 * @param copyFlag Error flag to signal unsuccessful copying
 * @param arithmeticFlag Error flag to signal unsuccessful arithmetics
 */
void cleanup(sycl::queue &q, unsigned *testArrayOnDevice, bool *submitFlag, bool *copyFlag,
             bool *arithmeticFlag)
{
  if (testArrayOnDevice)
    sycl::free(testArrayOnDevice, q);
  if (submitFlag)
    sycl::free(submitFlag, q);
  if (copyFlag)
    sycl::free(copyFlag, q);
  if (arithmeticFlag)
    sycl::free(arithmeticFlag, q);
}

/**
 * @brief main
 * @param argc Argument counts
 * @param argv Variables
 * @return EXIT_FAILURE if error, EXIT_SUCCESS if ok
 */
int main(int argc, char **argv)
{
  CLI::App app{"Simple ctest to check if SYCL is functioning correctly"};
  std::string deviceName = "gpu";
  app.add_option<std::string>("-d,-D,--DeviceName", deviceName,
                              "The name of the device to run test on, Accepts: gpu or cpu");
  CLI11_PARSE(app, argc, argv);

  // Create SYCL queue
  sycl::queue q;
  if (deviceName == "cpu") {
    q = sycl::queue{sycl::multi_cpu_selector_v};

    if (q.get_device().is_cpu() == false) {
      std::cerr << deviceName << " SYCL fail: Unable to create sycl queue" << std::endl;
      return EXIT_FAILURE;
    }
  } else if (deviceName == "gpu") {
    q = sycl::queue{sycl::multi_gpu_selector_v};

    if (q.get_device().is_gpu() == false) {
      std::cerr << deviceName << " SYCL fail: Unable to create sycl queue" << std::endl;
      return EXIT_FAILURE;
    }
  } else {
    std::cerr << "Unknown device name: " << deviceName << ", Accepted values: cpu, gpu"
              << std::endl;
    return EXIT_FAILURE;
  }

  // Define a simple incrementing array that will be used for testing
  const unsigned arraySize = 10000;
  unsigned testArray[arraySize];
  for (unsigned i = 0; i < arraySize; i++)
    testArray[i] = i;

  // Copy to Device
  unsigned *testArrayOnDevice = sycl::malloc_device<unsigned>(arraySize, q);
  q.memcpy(testArrayOnDevice, testArray, sizeof(unsigned) * arraySize);

  // A flag that remains false until the queue is submitted successful
  bool *submitFlag = sycl::malloc_shared<bool>(1, q);
  submitFlag[0] = false; // Assumed fail until proven pass

  // A flag that becomes false if copying to device fails
  bool *copyFlag = sycl::malloc_shared<bool>(1, q);
  copyFlag[0] = true; // Assumed pass until proven fail

  // A flag that becomes false if basic arithmetic test on device fails
  bool *arithmeticFlag = sycl::malloc_shared<bool>(1, q);
  arithmeticFlag[0] = true; // Assumed passed until proven fail

  q.wait();

  q.submit([&](sycl::handler &h) {
     h.parallel_for(sycl::range<1>(arraySize), [=](sycl::id<1> idx) {
       submitFlag[0] = true;
       if (testArrayOnDevice[idx[0]] != idx[0])
         copyFlag[0] = false;

       // Test very basic arithmetic equation x = 2 * (x + 3)
       testArrayOnDevice[idx[0]] += 3;
       testArrayOnDevice[idx[0]] *= 2;
       if (testArrayOnDevice[idx[0]] != 2 * (idx[0] + 3))
         arithmeticFlag[0] = false;
     });
   }).wait();

  if (submitFlag[0] == false) {
    std::cerr << deviceName << " SYCL fail: Unsuccessfully submitted to device" << std::endl;
    cleanup(q, testArrayOnDevice, submitFlag, copyFlag, arithmeticFlag);
    return EXIT_FAILURE;
  }
  if (copyFlag[0] == false) {
    std::cerr << deviceName << " SYCL fail: Copy from host to device unsuccessful" << std::endl;
    cleanup(q, testArrayOnDevice, submitFlag, copyFlag, arithmeticFlag);
    return EXIT_FAILURE;
  }
  if (arithmeticFlag[0] == false) {
    std::cerr << deviceName << " SYCL fail: basic arithmetics on device unsuccessful" << std::endl;
    cleanup(q, testArrayOnDevice, submitFlag, copyFlag, arithmeticFlag);
    return EXIT_FAILURE;
  }

  q.memcpy(testArray, testArrayOnDevice, sizeof(unsigned) * arraySize).wait();
  for (unsigned i = 1; i < arraySize; i++) {
    if (testArray[i] != 2 * (i + 3)) {
      std::cerr << deviceName << " SYCL fail: Copy from device to host unsuccessful" << std::endl;
      cleanup(q, testArrayOnDevice, submitFlag, copyFlag, arithmeticFlag);
      return EXIT_FAILURE;
    }
  }

  cleanup(q, testArrayOnDevice, submitFlag, copyFlag, arithmeticFlag);
  return EXIT_SUCCESS;
}

[ferdymercury]

To my needs, it looks perfect and can be merged, thanks a lot!

Unimportant for me:
The remaining question is whether the same basic_SYCL_Ctest can be run in Windows via find_package(IntelDPCPP) or if that would mess up with the rest of the SlicerRt Build system.

[xskere]

Unimportant for me: The remaining question is whether the same basic_SYCL_Ctest can be run in Windows via find_package(IntelDPCPP) or if that would mess up with the rest of the SlicerRt Build system.

I attempted to use this and have not managed to make it work. I tried with both IntelDPCPP (which is apparently being deprecated soon) and IntelSYCL which was the recommended alternative. There are incompatibilities with MSVC and it does not work due to them.

[ferdymercury]

IntelSYCL which was the recommended alternative. There are incompatibilities with MSVC and it does not work due to them.

Thanks. Right, it's IntelSYCL now. I managed to use IntelSYCL on Linux successfully some time ago (installing oneAPI on Linux), but yeah no idea about MSVC. So in my opinion let's just skip Windows

[ferdymercury]

I managed to use IntelSYCL on Linux successfully some time ago (installing oneAPI on Linux), but yeah no idea about MSVC. So in my opinion let's just skip Windows

Just for future reference, in case someone using Linux wants later to swap AdaptiveCpp with IntelSYCL compiler, our SYCL-basic test compiles and runs perfectly fine on Alma9 after tuning the CMakeLists a bit:

cmake_minimum_required(VERSION 3.25)

# This must go BEFORE project-call or enable_language-call https://stackoverflow.com/a/45934279/7471760
# The compiler of other upper projects won't be affected
# Unlike in AdaptiveCpp, where you do not override the CMAKE_C(XX)_COMPILER, find_package(IntelSYCL) is stupid and fails if you don't do this, which defeats the whole purpose of the statement add_sycl_to_target
# So we are forced to do:
set(CMAKE_C_COMPILER "/opt/intel/oneapi/2025.0/bin/icx") 
set(CMAKE_CXX_COMPILER "/opt/intel/oneapi/2025.0/bin/icpx")

project(MySYCLtest) # Do not move this above _COMPILER https://stackoverflow.com/a/45934279/7471760

find_package(CLI11 CONFIG REQUIRED) # header-only library  argparse lib https://github.com/CLIUtils/CLI11 or sudo apt install libcli11-dev

# Find SYCL (and oneMKL packages)
find_package(IntelSYCL CONFIG REQUIRED HINTS /opt/intel/oneapi/2025.0/lib/cmake/IntelSYCL/)
# find_package(MKL CONFIG REQUIRED HINTS /opt/intel/oneapi/2025.0/lib/cmake/mkl/)

add_executable(basic_SYCL_test basic_SYCL_test.cpp)

# Link libraries
target_link_libraries(basic_SYCL_test PRIVATE CLI11::CLI11) # MKL::MKL

# Intel find_package is shitty, it does not have a proper CMake target taking care of link directories, so we need to do it by hand by seeing what /opt/intel/oneapi/setvars.sh does (which we do not want to call to not pollute the rest of the environment). In theory, all these things should be taken care of by add_sycl_to_target, but that seems only to work with Adaptivecpp
target_link_directories(basic_SYCL_test PRIVATE "/opt/intel/oneapi/2025.0/lib" )
set(INTEL_SYCL_LD_LIBRARY_PATH "LD_LIBRARY_PATH=/opt/intel/oneapi/tcm/1.2/lib:/opt/intel/oneapi/umf/0.9/lib:/opt/intel/oneapi/tbb/2022.0/env/../lib/intel64/gcc4.8:/opt/intel/oneapi/pti/0.10/lib:/opt/intel/oneapi/mkl/2025.0/lib:/opt/intel/oneapi/ipp/2022.0/lib:/opt/intel/oneapi/debugger/2025.0/opt/debugger/lib:/opt/intel/oneapi/compiler/2025.0/opt/compiler/lib:/opt/intel/oneapi/compiler/2025.0/lib")

# Add SYCL support
add_sycl_to_target(TARGET basic_SYCL_test SOURCES basic_SYCL_test.cpp)

# Add tests
enable_testing()
add_test(
  NAME IntelSYCL_basic_cpu_test
  COMMAND basic_SYCL_test --DeviceName cpu
)
set_property(TEST IntelSYCL_basic_cpu_test PROPERTY ENVIRONMENT ${INTEL_SYCL_LD_LIBRARY_PATH})
add_test(
  NAME IntelSYCL_basic_gpu_test
  COMMAND basic_SYCL_test --DeviceName gpu
)
set_property(TEST IntelSYCL_basic_gpu_test PROPERTY ENVIRONMENT ${INTEL_SYCL_LD_LIBRARY_PATH})

[ferdymercury]

Thanks!