[SYCL][E2E] Add logic to react to REQUIRED/UNSUPPORTED in `build-…

…only` (#16725)

Adds logic for evaluating `REQUIRES`/`UNSUPPORTED` statements on
`build-only` mode by "ignoring" features in these statements that do not
affect the compilation (Everything other than OS, triple, and
SDK/libraries).

More precisely, the ability to ignore features is implemented by
extending the boolean expressions to use a third boolean state -
`ignore`. If a particular sub-expression includes an `ignore`, and if
its result could be changed by setting that `ignore` to either `true` or
`false`, then the result of that sub-expression is set to `ignore`. For
example `ignore || true = true`, but `ignore || false = ignore`, this is
because in the first example there would be no way to set the ignore
such that the result is anything other than `true`, while in the second
example the result is dependent on the "actual" value of the `ignore`.

If the resulting value of a `REQUIRES` predicate is `ignore` we
interpret that as `true` (The requirements were met), on the other hand
for `UNSUPPORTED` predicates we would interpret `ignore` as `false`
instead (The unsupported criteria was not met).

The triples that can be used for a given test are then selected by
evaluating the `REQUIRES`/`UNSUPPORTED` with the set of features + the
target feature corresponding to each triple (mirroring the way we select
devices), while ignoring all features that do not affect compilation.

The target features available are the following: `target-spir`,
`target-amd`, `target-nvidia`, and `target-native_cpu`. Each of these
map to a triple.

Similarly to how `XFAIL` is handled when using multiple devices if we
are compiling for multiple triples, and a single triple is marked as
`XFAIL`, then it is treated as unsupported instead.

The available target triples in `build-only` mode is determined through
the use of the `sycl_build_targets` lit param (i.e., `--param
sycl_build_targets=spir;amd`). This is currently set to only `spir`, as
the changes in test markup included in this pr do not take into account
building for `nvptx` and `amdgcn` triples. In `run-only` and `full` mode
the available triples are determined via the available devices.

sycl/test-e2e/AmdNvidiaJIT/kernel_and_bundle.cpp

@@ -1,6 +1,5 @@
 // UNSUPPORTED: windows
-// REQUIRES: cuda || hip
-// REQUIRES: build-and-run-mode
+// REQUIRES: target-nvidia || target-amd
 
 // This test relies on debug output from a pass, make sure that the compiler
 // can generate it.

sycl/test-e2e/Basic/windows_version_agnostic_sycl_lib.cpp

@@ -1,5 +1,4 @@
 // REQUIRES: windows
-// REQUIRES: build-and-run-mode
 
 // RUN: %clangxx --driver-mode=cl /std:c++17 /EHsc %sycl_include -I%opencl_include_dir %s -o %t.out /link /defaultlib:%sycl_static_libs_dir/sycl.lib
 // RUN: %{run} %t.out

sycl/test-e2e/Compression/no_zstd_warning.cpp

@@ -1,5 +1,4 @@
 // using --offload-compress without zstd should throw an error.
 // REQUIRES: !zstd
-// REQUIRES: build-and-run-mode
 // RUN: not %{build} %O0 -g --offload-compress %S/Inputs/single_kernel.cpp -o %t_compress.out 2>&1 | FileCheck %s
 // CHECK: '--offload-compress' option is specified but zstd is not available. The device image will not be compressed.

sycl/test-e2e/DeviceLib/math_fp64_windows_test.cpp

@@ -1,5 +1,4 @@
 // REQUIRES: aspect-fp64, windows
-// REQUIRES: build-and-run-mode
 
 // DEFINE: %{mathflags} = %if cl_options %{/clang:-fno-fast-math%} %else %{-fno-fast-math%}
 

sycl/test-e2e/DeviceLib/math_windows_test.cpp

@@ -1,5 +1,4 @@
 // REQUIRES: windows
-// REQUIRES: build-and-run-mode
 
 // TODO: Add hypotf case back when the missing symbol is fixed.
 

sycl/test-e2e/E2EExpr.py

@@ -0,0 +1,167 @@
+from lit.BooleanExpression import BooleanExpression
+
+
+class E2EExpr(BooleanExpression):
+    build_specific_features = {
+        "build-and-run-mode",
+        "target-spir",
+        "target-nvidia",
+        "target-amd",
+        "target-native_cpu",
+        "any-target-is-spir",
+        "any-target-is-nvidia",
+        "any-target-is-amd",
+        "any-target-is-native_cpu",
+        "linux",
+        "system-linux",
+        "windows",
+        "system-windows",
+        "enable-perf-tests",
+        "preview-breaking-changes-supported",
+        "has_ndebug",
+        "ocloc",
+        "opencl-aot",
+        "opencl_icd",
+        "cm-compiler",
+        "xptifw",
+        "level_zero_dev_kit",
+        "cuda_dev_kit",
+        "zstd",
+        "vulkan",
+        "true",
+        "false",
+    }
+
+    def __init__(self, string, variables, build_only_mode, final_unknown_value):
+        BooleanExpression.__init__(self, string, variables)
+        self.build_only_mode = build_only_mode
+        self.unknown = False
+        self.final_unknown_value = final_unknown_value
+
+    @staticmethod
+    def evaluate(string, variables, build_only_mode, final_unknown_value=True):
+        """
+        string: Expression to evaluate
+        variables: variables that evaluate to true
+        build_only_mode: if true enables unknown values
+        final_unknown_value: final boolean result if evaluation results in `unknown`
+        """
+        try:
+            parser = E2EExpr(
+                string, set(variables), build_only_mode, final_unknown_value
+            )
+            return parser.parseAll()
+        except ValueError as e:
+            raise ValueError(str(e) + ("\nin expression: %r" % string))
+
+    def parseMATCH(self):
+        token = self.token
+        BooleanExpression.parseMATCH(self)
+        if token not in self.build_specific_features and self.build_only_mode:
+            self.unknown = True
+        else:
+            self.unknown = False
+        if self.value and self.unknown:
+            raise ValueError(
+                'Runtime feature "' + token + '" evaluated to True in build-only'
+            )
+
+    def parseAND(self):
+        self.parseNOT()
+        while self.accept("&&"):
+            left = self.value
+            left_unknown = self.unknown
+            self.parseNOT()
+            right = self.value
+            right_unknown = self.unknown
+            self.value = left and right
+            # Unknown if both are unknown or if one is true and the other is unknown
+            self.unknown = (
+                (left_unknown and right_unknown)
+                or (left_unknown and right)
+                or (left and right_unknown)
+            )
+
+    def parseOR(self):
+        self.parseAND()
+        while self.accept("||"):
+            left = self.value
+            left_unknown = self.unknown
+            self.parseAND()
+            right = self.value
+            right_unknown = self.unknown
+            self.value = left or right
+            # Unknown if both are unknown or if one is false and the other is unknown
+            self.unknown = (
+                (left_unknown and right_unknown)
+                or (left_unknown and not right)
+                or (not left and right_unknown)
+            )
+
+    def parseAll(self):
+        self.token = next(self.tokens)
+        self.parseOR()
+        self.expect(BooleanExpression.END)
+        return self.final_unknown_value if self.unknown else self.value
+
+
+import unittest
+
+
+class TestE2EExpr(unittest.TestCase):
+    def test_basic(self):
+        BuildOnly = True
+        BuildAndRun = False
+        RequiresDirective = True
+        UnsupportedDirective = False
+        RegularEval = lambda expr, features: E2EExpr.evaluate(
+            expr, features, BuildAndRun
+        )
+        RequiresBuildEval = lambda expr, features: E2EExpr.evaluate(
+            expr, features, BuildOnly, RequiresDirective
+        )
+        UnsupportedBuildEval = lambda expr, features: E2EExpr.evaluate(
+            expr, features, BuildOnly, UnsupportedDirective
+        )
+        # Non build-only expressions should work the same
+        self.assertTrue(RegularEval("linux", {"linux", "rt_feature"}))
+        self.assertTrue(RegularEval("rt_feature", {"linux", "rt_feature"}))
+        self.assertFalse(
+            RegularEval("rt_feature1 && rt_feature2", {"linux", "rt_feature1"})
+        )
+        # build-only expressions with no unknowns should work the same
+        self.assertTrue(UnsupportedBuildEval("linux", {"linux"}))
+        self.assertFalse(RequiresBuildEval("linux && windows", {"linux"}))
+        self.assertTrue(UnsupportedBuildEval("!(windows || zstd)", {"linux"}))
+        # build-only expressions where unknown affects the resulting value
+        self.assertTrue(RequiresBuildEval("rt_feature", {}))
+        self.assertFalse(UnsupportedBuildEval("rt_feature", {}))
+        self.assertFalse(UnsupportedBuildEval("!rt_feature", {}))
+        self.assertTrue(RequiresBuildEval("windows || rt_feature", {"linux"}))
+        self.assertFalse(UnsupportedBuildEval("windows || rt_feature", {"linux"}))
+        self.assertTrue(RequiresBuildEval("linux && rt_feature", {"linux"}))
+        self.assertFalse(UnsupportedBuildEval("linux && rt_feature", {"linux"}))
+        self.assertTrue(RequiresBuildEval("linux && !(zstd || rt_feature)", {"linux"}))
+        self.assertFalse(
+            UnsupportedBuildEval("linux && !(zstd || rt_feature)", {"linux"})
+        )
+        # build-only expressions where unknown does not affect the resulting value
+        self.assertTrue(RequiresBuildEval("linux || rt_feature", {"linux"}))
+        self.assertTrue(UnsupportedBuildEval("linux || rt_feature", {"linux"}))
+        self.assertFalse(RequiresBuildEval("windows && rt_feature", {"linux"}))
+        self.assertFalse(UnsupportedBuildEval("windows && rt_feature", {"linux"}))
+        self.assertFalse(
+            RequiresBuildEval("linux && (vulkan && rt_feature)", {"linux"})
+        )
+        self.assertFalse(
+            UnsupportedBuildEval("linux && (vulkan && rt_feature)", {"linux"})
+        )
+        # runtime feature is present in build-only
+        with self.assertRaises(ValueError):
+            RequiresBuildEval("rt_feature", {"rt_feature"})
+        with self.assertRaises(ValueError):
+            UnsupportedBuildEval("rt_feature", {"rt_feature"})
+
+
+if __name__ == "__main__":
+    unittest.main()

sycl/test-e2e/EnqueueNativeCommand/custom-command-cuda.cpp

@@ -1,7 +1,6 @@
 // RUN: %{build} -Wno-error=deprecated-declarations -o %t.out %cuda_options
 // RUN: %{run} %t.out
 // REQUIRES: cuda, cuda_dev_kit
-// REQUIRES: build-and-run-mode
 
 #include <cuda.h>
 

sycl/test-e2e/EnqueueNativeCommand/custom-command-hip.cpp

@@ -2,8 +2,7 @@
 // we should set this with some variable instead.
 // RUN: %{build} -Wno-error=deprecated-pragma -o %t.out -I%rocm_path/include -L%rocm_path/lib -lamdhip64
 // RUN: %{run} %t.out
-// REQUIRES: hip
-// REQUIRES: build-and-run-mode
+// REQUIRES: target-amd
 
 #include <iostream>
 #include <sycl/backend.hpp>

sycl/test-e2e/EnqueueNativeCommand/custom-command-multiple-dev-cuda.cpp

@@ -1,5 +1,4 @@
 // REQUIRES: cuda, cuda_dev_kit
-// REQUIRES: build-and-run-mode
 // RUN: %{build} -o %t.out %cuda_options
 // RUN: %{run} %t.out
 

sycl/test-e2e/HostInteropTask/interop-task-cuda-buffer-migrate.cpp

@@ -1,5 +1,4 @@
 // REQUIRES: cuda, cuda_dev_kit
-// REQUIRES: build-and-run-mode
 //
 // RUN: %{build} -o %t.out %cuda_options
 // RUN: %{run} %t.out

sycl/test-e2e/HostInteropTask/interop-task-cuda.cpp

@@ -1,7 +1,6 @@
 // RUN: %{build} -o %t.out %cuda_options
 // RUN: %{run} %t.out
 // REQUIRES: cuda, cuda_dev_kit
-// REQUIRES: build-and-run-mode
 
 #include <iostream>
 #include <sycl/backend.hpp>

sycl/test-e2e/HostInteropTask/interop-task-hip.cpp

@@ -2,8 +2,7 @@
 // we should set this with some variable instead.
 // RUN: %{build} -Wno-error=deprecated-pragma -Wno-error=deprecated-declarations -o %t.out -I%rocm_path/include -L%rocm_path/lib -lamdhip64
 // RUN: %{run} %t.out
-// REQUIRES: hip
-// REQUIRES: build-and-run-mode
+// REQUIRES: target-amd
 
 #include <iostream>
 #include <sycl/backend.hpp>

sycl/test-e2e/OptionalKernelFeatures/is_compatible/is_compatible_amdgcn.cpp

@@ -1,5 +1,4 @@
 // REQUIRES: hip, opencl, gpu, cpu
-// REQUIRES: build-and-run-mode
 
 // RUN: %clangxx -fsycl -Xsycl-target-backend=amdgcn-amd-amdhsa --offload-arch=gfx906 -fsycl-targets=amdgcn-amd-amdhsa %S/Inputs/is_compatible_with_env.cpp -o %t.out
 

sycl/test-e2e/OptionalKernelFeatures/is_compatible/is_compatible_nvptx64.cpp

@@ -1,5 +1,4 @@
 // REQUIRES: cuda, opencl, gpu, cpu
-// REQUIRES: build-and-run-mode
 
 // RUN: %clangxx -fsycl -fsycl-targets=nvptx64-nvidia-cuda %S/Inputs/is_compatible_with_env.cpp -o %t.out
 

sycl/test-e2e/OptionalKernelFeatures/is_compatible/is_compatible_several_targets.cpp

@@ -1,8 +1,7 @@
 // REQUIRES: ocloc, any-device-is-level_zero, any-device-is-gpu, any-device-is-cpu
-// REQUIRES: build-and-run-mode
 
 // RUN: %clangxx -fsycl -fsycl-targets=spir64_gen -Xsycl-target-backend=spir64_gen "-device *" %S/Inputs/is_compatible_with_env.cpp -o %t.out
 
 // RUN: env ONEAPI_DEVICE_SELECTOR=opencl:cpu %{run-unfiltered-devices} not %t.out
 // RUN: env ONEAPI_DEVICE_SELECTOR=opencl:gpu %{run-unfiltered-devices} %t.out
-// RUN: env ONEAPI_DEVICE_SELECTOR=level_zero:gpu %{run-unfiltered-devices} %t.out
+// RUN: env ONEAPI_DEVICE_SELECTOR=level_zero:gpu %{run-unfiltered-devices} %t.out

sycl/test-e2e/OptionalKernelFeatures/is_compatible/is_compatible_spir64.cpp

@@ -1,5 +1,4 @@
 // REQUIRES: cuda, opencl, gpu, cpu
-// REQUIRES: build-and-run-mode
 
 // RUN: %clangxx -fsycl -fsycl-targets=spir64 %S/Inputs/is_compatible_with_env.cpp -o %t.out
 

sycl/test-e2e/OptionalKernelFeatures/is_compatible/is_compatible_spir64_fpga.cpp

@@ -1,5 +1,4 @@
 // REQUIRES: opencl-aot, accelerator, gpu, cpu
-// REQUIRES: build-and-run-mode
 
 // RUN: %clangxx -fsycl -fsycl-targets=spir64_fpga %S/Inputs/is_compatible_with_env.cpp -o %t.out
 

sycl/test-e2e/OptionalKernelFeatures/is_compatible/is_compatible_spir64_gen.cpp

@@ -1,5 +1,4 @@
 // REQUIRES: ocloc, gpu, level_zero, cpu
-// REQUIRES: build-and-run-mode
 
 // RUN: %clangxx -fsycl -fsycl-targets=spir64_gen -Xsycl-target-backend "-device *" %S/Inputs/is_compatible_with_env.cpp -o %t.out
 

sycl/test-e2e/OptionalKernelFeatures/is_compatible/is_compatible_spir64_x86_64.cpp

@@ -1,5 +1,4 @@
 // REQUIRES: opencl-aot, cpu, gpu, level_zero
-// REQUIRES: build-and-run-mode
 
 // RUN: %clangxx -fsycl -fsycl-targets=spir64_x86_64 %S/Inputs/is_compatible_with_env.cpp -o %t.out
 

sycl/test-e2e/Regression/compile_on_win_with_mdd.cpp

@@ -1,5 +1,4 @@
 // REQUIRES: windows
-// REQUIRES: build-and-run-mode
 
 // RUN: %clangxx --driver-mode=cl -fsycl /MDd -c %s -o %t.obj
 // RUN: %clangxx --driver-mode=cl -fsycl %t.obj -Wno-unused-command-line-argument -o %t.out

sycl/test-e2e/Regression/fsycl-host-compiler-win.cpp

@@ -1,7 +1,6 @@
 // RUN: %{build} -fsycl-host-compiler=cl -DDEFINE_CHECK -fsycl-host-compiler-options="-DDEFINE_CHECK /std:c++17 /Zc:__cplusplus" -o %t.exe
 // RUN: %{run} %t.exe
 // REQUIRES: windows
-// REQUIRES: build-and-run-mode
 //
 //==------- fsycl-host-compiler-win.cpp - external host compiler test ------==//
 //

sycl/test-e2e/Regression/msvc_crt.cpp

@@ -3,7 +3,6 @@
 // RUN: %{build} /MDd -o %t2.exe
 // RUN: %{run} %t2.exe
 // REQUIRES: system-windows, cl_options
-// REQUIRES: build-and-run-mode
 //==-------------- msvc_crt.cpp - SYCL MSVC CRT test -----------------------==//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.

sycl/test-e2e/Regression/multiple-targets.cpp

@@ -2,18 +2,17 @@
 // It tests if the target triples can be specified with any order.
 // The test is repeated for per_kernel device code splitting.
 //
-// REQUIRES: cuda || hip || native_cpu
-// REQUIRES: build-and-run-mode
-// RUN: %clangxx -fsycl -fsycl-targets=%{sycl_triple},spir64 %if any-device-is-hip %{ %{hip_arch_opts} %} -o %t1.out %s
+// REQUIRES: (target-nvidia || target-amd || target-native_cpu) && any-target-is-spir
+// RUN: %clangxx -fsycl -fsycl-targets=%{sycl_triple},spir64 %if any-target-is-amd %{ %{hip_arch_opts} %} -o %t1.out %s
 // RUN: %{run} %t1.out
 //
-// RUN: %clangxx -fsycl -fsycl-targets=spir64,%{sycl_triple} %if any-device-is-hip %{ %{hip_arch_opts} %} -o %t2.out %s
+// RUN: %clangxx -fsycl -fsycl-targets=spir64,%{sycl_triple} %if any-target-is-amd %{ %{hip_arch_opts} %} -o %t2.out %s
 // RUN: %{run} %t2.out
 //
-// RUN: %clangxx -fsycl -fsycl-targets=%{sycl_triple},spir64 %if any-device-is-hip %{ %{hip_arch_opts} %} -fsycl-device-code-split=per_kernel -o %t3.out %s
+// RUN: %clangxx -fsycl -fsycl-targets=%{sycl_triple},spir64 %if any-target-is-amd %{ %{hip_arch_opts} %} -fsycl-device-code-split=per_kernel -o %t3.out %s
 // RUN: %{run} %t3.out
 //
-// RUN: %clangxx -fsycl -fsycl-targets=spir64,%{sycl_triple} %if any-device-is-hip %{ %{hip_arch_opts} %} -fsycl-device-code-split=per_kernel -o %t4.out %s
+// RUN: %clangxx -fsycl -fsycl-targets=spir64,%{sycl_triple} %if any-target-is-amd %{ %{hip_arch_opts} %} -fsycl-device-code-split=per_kernel -o %t4.out %s
 // RUN: %{run} %t4.out
 
 #include <sycl/detail/core.hpp>

sycl/test-e2e/SpecConstants/2020/non_native/cuda.cpp

@@ -1,5 +1,4 @@
 // REQUIRES: cuda
-// REQUIRES: build-and-run-mode
 
 // RUN: %clangxx -fsycl -fsycl-targets=nvptx64-nvidia-cuda %S/Inputs/common.cpp -o %t.out
 // RUN: %{run-unfiltered-devices} env ONEAPI_DEVICE_SELECTOR="cuda:*" %t.out