Grcuda 61 cleanup for release 1 (#19)

* adding changelog, removed unused thread manager

* fixed install.sh, now using env variables to retrieve the absolute path to grcuda.jar

* added curl to install script

* added license to demo

* removed unused cuda code; added license to benchmarks

* added license to tests

* added license to functions and libraries

* fixed removal of thread manager breaking build

* added more updated licenses

* added license to runtime files

* Updated changelog

* fixed typo

* added grcuda-data info to readme

* udpated tracking of grcuda-data

* temporarily removed submodule grcuda-data

* readded grcuda-data submodule

* tracking master?

* updated grcuda-data tracking

* Added the possibility to send execution times to the frontend

* Display execution times in race mode

* clarified streamattach in changelog

Co-authored-by: Guido Walter Di Donato <[email protected]>
Co-authored-by: Francesco Sgherzi <[email protected]>

.gitmodules

@@ -1,4 +1,4 @@
 [submodule "grcuda-data"]
 	path = grcuda-data
-	url = https://github.com/AlbertoParravicini/grcuda-data.git 
+	url = https://github.com/AlbertoParravicini/grcuda-data.git
 	branch = master

CHANGELOG.md

@@ -0,0 +1,63 @@
+# 2021-09-30, Release 1
+
+## API Changes
+
+* Added option to specify arguments in NFI kernel signatures as `const`
+    * The effect is the same as marking them as `in` in the NIDL syntax
+    * It is not strictly required to have the corresponding arguments in the CUDA kernel marked as `const`, although that's recommended
+    * Marking arguments as `const` or `in` enables the async scheduler to overlap kernels that use the same read-only arguments
+
+## New asynchronous scheduler
+
+* Added a new asynchronous scheduler for GrCUDA, enable it with `--experimental-options --grcuda.ExecutionPolicy=async`
+    * With this scheduler, GPU kernels are executed asynchronously. Once they are launched, the host execution resumes immediately
+    * The computation is synchronized (i.e. the host thread is stalled and waits for the kernel to finish) only once GPU data are accessed by the host thread
+    * Execution of multiple kernels (operating on different data, e.g. distinct DeviceArrays) is overlapped using different streams
+    * Data transfer and execution (on different data, e.g. distinct DeviceArrays) is overlapped using different streams
+    * The scheduler supports different options, see `README.md` for the full list
+    * It is the scheduler presented in "DAG-based Scheduling with Resource Sharing for Multi-task Applications in a Polyglot GPU Runtime" (IPDPS 2021)
+
+* Enabled partial support for cuBLAS and cuML in the aync scheduler
+    * **Known limitation:** functions in these libraries work with the async scheduler, although they still run on the default stream (i.e. they are not asynchronous)
+    * They do benefit from prefetching
+* Set TensorRT support to experimental
+    * TensorRT is currently not supported on CUDA 11.4, making it impossible to use along a recent version of cuML
+    * **Known limitation:** due to this incompatibility, TensorRT is currently not available on the async scheduler 
+
+## New features
+
+* Added generic AbstractArray data structure, which is extended by DeviceArray, MultiDimDeviceArray, MultiDimDeviceArrayView, and provides high-level array interfaces
+* Added API for prefetching 
+    * If enabled (and using a GPU with architecture newer or equal than Pascal), it prefetches data to the GPU before executing a kernel, instead of relying on page-faults for data transfer. It can greatly improve performance
+* Added API for stream attachment
+    * Always enabled in GPUs with with architecture older than Pascal, and the async scheduler is active. With the sync scheduler, it can be manually enabled
+    * It restricts the visibility of GPU data to the specified stream
+    * In architectures newer or equal than Pascal it can provide a small performance benefit
+* Added `copyTo/copyFrom` functions on generic arrays (Truffle interoperable objects that expose the array API)
+    * Internally, the copy is implemented as a for loop, instead of using CUDA's `memcpy`
+    * It is still faster than copying using loops in the host languages, in many cases, and especially if host code is not JIT-ted
+    * It is also used for copying data to/from DeviceArrays with column-major layout, as `memcpy` cannot copy non-contiguous data 
+
+## Demos, benchmarks and code samples
+
+* Added demo used at SeptembeRSE 2021 (`demos/image_pipeline_local` and `demos/image_pipeline_web`)
+    * It shows an image processing pipeline that applies a retro look to images. We have a local version and a web version that displays results a in web page
+* Added benchmark suite written in Graalpython, used in "DAG-based Scheduling with Resource Sharing for Multi-task Applications in a Polyglot GPU Runtime" (IPDPS 2021)
+    * It is a collection of complex multi-kernel benchmarks meant to show the benefits of asynchronous scheduling.
+
+## Miscellaneosus
+
+* Added dependency to `grcuda-data` submodule, used to store data, results and plots used in publications and demos.
+* Updated name "grCUDA" to "GrCUDA". It looks better, doesn't it?
+* Added support for Java 11 along with Java 8
+* Added option to specify the location of cuBLAS and cuML with environment variables (`LIBCUBLAS_DIR` and `LIBCUML_DIR`)
+* Refactored package hierarchy to reflect changes to current GrCUDA (e.g. `gpu -> runtime`)
+* Added basic support for TruffleLogger
+* Removed a number of existing deprecation warnings
+* Added around 800 unit tests, with support for extensive parametrized testing and GPU mocking
+* Updated documentation 
+    * Bumped GraalVM version to 21.2
+    * Added scripts to setup a new machine from scratch (e.g. on OCI), plus other OCI-specific utility scripts (see `oci_setup/`)
+    * Added documentation to setup IntelliJ Idea for GrCUDA development
+    * Added documentation about Python benchmark suite
+    * Added documentation on asynchronous scheduler options

LICENSE

@@ -1,4 +1,6 @@
-Copyright (c) 2019, NVIDIA CORPORATION. All rights reserved.
+Copyright (c) 2019, 2020, NVIDIA CORPORATION. All rights reserved.
+Copyright (c) 2019, 2020, Oracle and/or its affiliates. All rights reserved.
+Copyright (c) 2020, 2021, NECSTLab, Politecnico di Milano. All rights reserved.
 
 Redistribution and use in source and binary forms, with or without
 modification, are permitted provided that the following conditions
@@ -11,6 +13,12 @@ are met:
  * Neither the name of NVIDIA CORPORATION nor the names of its
    contributors may be used to endorse or promote products derived
    from this software without specific prior written permission.
+ * Neither the name of NECSTLab nor the names of its
+   contributors may be used to endorse or promote products derived
+   from this software without specific prior written permission.
+ * Neither the name of Politecnico di Milano nor the names of its
+   contributors may be used to endorse or promote products derived
+   from this software without specific prior written permission.
 
 THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS ``AS IS'' AND ANY
 EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE

README.md

@@ -282,12 +282,15 @@ graalpython -m ginstall install numpy;
 
 4. **Run GrCUDA Unit tests** using 
 
-```bash
+```
 mx unittest com.nvidia
 # To run a specific test, you can use
 mx unittest com.nvidia.grcuda.test.BuildKernelTest#testBuildKernelwithNFILegacytSignature
 ```
 
+5. **Setup the grcuda-data sumbodule**
+The `grcuda-data` repository is used as a `git` submodule to store data, results, and plots for demos, benchmarks, and publications. You will need this submodule to run the full benchmark suite, and some of the demos. To setup the submodule, follow this [`README`](https://github.com/AlbertoParravicini/grcuda-data/tree/master).
+
 ### Setup your IDE
 
 To develop GrCUDA, you will greatly benefit from having an IDE that allows jumping between symbols and debugging individual tests. 
@@ -296,6 +299,7 @@ Here, we explain how to setup IntelliJ Idea.
   1. `mx ideinit` from `$GRCUDA_HOME`, to setup the IDE
   2. Open Idea and select *"open project"*, then open GrCUDA
   3. See this [guide](https://github.com/graalvm/mx/blob/master/docs/IDE.md) to configure the syntax checker
+     * `File -> Settings -> Plugins -> Marketplace -> Search "Eclipse Code Formatter" and install it`
   4. In IntelliJ Idea, install the Python plugin with `Settings -> Plugin -> Search "Python"`, then do `Project Structure -> SDKs -> Create a new Python 3.8 Virtual Environment`, it is used by `mx`
   5. Select the right JVM. It should select automatically your `$JAVA_HOME`. Othewise, `Project Structures -> Modules -> Set the Module SDK (under Dependencies)` of `mx` and submodules to your Java SDK (e.g. `11`). You can pick either the `labsjdk` or `graalvm`.
       * This is also given by the `configure` option if you try to build the project in IntelliJ Idea before setting these options. Set your project Java SDK (e.g. `11`) for those missing modules
@@ -304,16 +308,18 @@ Here, we explain how to setup IntelliJ Idea.
 
           a. For those packages (look at the log to find them), manually specify a more recent SDK (e.g. `11`) as you did in step above. If you get errors of missing symbols, follow IntelliJ's hints and export the requested packages
 
-          b. Remove the exports. `File -> Settings -> Build ... -> Java Compiler`, then remove all the `--export` flags.
+          b. Remove the exports. `File -> Settings -> Build ... -> Compiler -> Java Compiler`, then remove all the `--export` flags.
   7. To run tests:
 
       a. Go to `Run (top bar) -> Edit Configurations -> Edit configuration templates -> Junit`
 
       b. (Not always necessary) By default, Idea should use your `env`. If not, make sure to have the same. Update the `PATH` variable so that it can find `nvcc`, and export `$GRAAL_HOME`. See `setup_machine_from_scratch.sh` to find all the environment variables.
 
-      c. Modify the template Junit test configuration adding `-Djava.library.path="$GRAAL_HOME/lib` (in Java 11) to the VM options to find `trufflenfi`
+      c. Modify the template Junit test configuration adding `-Djava.library.path="$GRAAL_HOME/lib"` (in Java 11) to the VM options to find `trufflenfi`
 
       d. In IntelliJ Idea, `Run -> Edit Configurations`. Create a new JUnit configuration set to `All in package` with `com.nvidia.grcuda` as module and `com.nvidia.grcuda.test` selected below. Add `-Djava.library.path="$GRAAL_HOME/lib"` (or your version of GraalVM) if it's not already in VM options. Specify the SDK by setting the GraalVM JRE in e.g. `$GRAAL_HOME`, if not specified already.   
+
+      e. If you change something in GrCUDA, rebuild it with `./install.sh` before running tests. That's because tests that use the GPU load the `.jar` in `$GRAAL_HOME`, which is updated by `./install.sh`    
 
 ## Execute performance tests using Graalpython
 

demos/image_pipeline_local/array_copy_performance_test.js

@@ -1,3 +1,32 @@
+// Copyright (c) 2021, NECSTLab, Politecnico di Milano. All rights reserved.
+
+// Redistribution and use in source and binary forms, with or without
+// modification, are permitted provided that the following conditions
+// are met:
+//  * Redistributions of source code must retain the above copyright
+//    notice, this list of conditions and the following disclaimer.
+//  * Redistributions in binary form must reproduce the above copyright
+//    notice, this list of conditions and the following disclaimer in the
+//    documentation and/or other materials provided with the distribution.
+//  * Neither the name of NECSTLab nor the names of its
+//    contributors may be used to endorse or promote products derived
+//    from this software without specific prior written permission.
+//  * Neither the name of Politecnico di Milano nor the names of its
+//    contributors may be used to endorse or promote products derived
+//    from this software without specific prior written permission.
+
+// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS ``AS IS'' AND ANY
+// EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+// IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+// PURPOSE ARE DISCLAIMED.  IN NO EVENT SHALL THE COPYRIGHT OWNER OR
+// CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
+// EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+// PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
+// PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
+// OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
 const System = Java.type("java.lang.System");
 const cu = Polyglot.eval('grcuda', 'CU')
 const { assert } = require("console");

demos/image_pipeline_local/cuda/image_pipeline.cu

@@ -1,3 +1,32 @@
+// Copyright (c) 2021, NECSTLab, Politecnico di Milano. All rights reserved.
+
+// Redistribution and use in source and binary forms, with or without
+// modification, are permitted provided that the following conditions
+// are met:
+//  * Redistributions of source code must retain the above copyright
+//    notice, this list of conditions and the following disclaimer.
+//  * Redistributions in binary form must reproduce the above copyright
+//    notice, this list of conditions and the following disclaimer in the
+//    documentation and/or other materials provided with the distribution.
+//  * Neither the name of NECSTLab nor the names of its
+//    contributors may be used to endorse or promote products derived
+//    from this software without specific prior written permission.
+//  * Neither the name of Politecnico di Milano nor the names of its
+//    contributors may be used to endorse or promote products derived
+//    from this software without specific prior written permission.
+
+// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS ``AS IS'' AND ANY
+// EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+// IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+// PURPOSE ARE DISCLAIMED.  IN NO EVENT SHALL THE COPYRIGHT OWNER OR
+// CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
+// EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+// PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
+// PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
+// OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
 #include "image_pipeline.cuh"
 
 //////////////////////////////

demos/image_pipeline_local/cuda/image_pipeline.cuh

@@ -1,3 +1,32 @@
+// Copyright (c) 2021, NECSTLab, Politecnico di Milano. All rights reserved.
+
+// Redistribution and use in source and binary forms, with or without
+// modification, are permitted provided that the following conditions
+// are met:
+//  * Redistributions of source code must retain the above copyright
+//    notice, this list of conditions and the following disclaimer.
+//  * Redistributions in binary form must reproduce the above copyright
+//    notice, this list of conditions and the following disclaimer in the
+//    documentation and/or other materials provided with the distribution.
+//  * Neither the name of NECSTLab nor the names of its
+//    contributors may be used to endorse or promote products derived
+//    from this software without specific prior written permission.
+//  * Neither the name of Politecnico di Milano nor the names of its
+//    contributors may be used to endorse or promote products derived
+//    from this software without specific prior written permission.
+
+// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS ``AS IS'' AND ANY
+// EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+// IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+// PURPOSE ARE DISCLAIMED.  IN NO EVENT SHALL THE COPYRIGHT OWNER OR
+// CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
+// EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+// PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
+// PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
+// OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
 #pragma once
 #include <chrono>
 #include <iostream>

demos/image_pipeline_local/cuda/main.cpp

@@ -1,3 +1,32 @@
+// Copyright (c) 2021, NECSTLab, Politecnico di Milano. All rights reserved.
+
+// Redistribution and use in source and binary forms, with or without
+// modification, are permitted provided that the following conditions
+// are met:
+//  * Redistributions of source code must retain the above copyright
+//    notice, this list of conditions and the following disclaimer.
+//  * Redistributions in binary form must reproduce the above copyright
+//    notice, this list of conditions and the following disclaimer in the
+//    documentation and/or other materials provided with the distribution.
+//  * Neither the name of NECSTLab nor the names of its
+//    contributors may be used to endorse or promote products derived
+//    from this software without specific prior written permission.
+//  * Neither the name of Politecnico di Milano nor the names of its
+//    contributors may be used to endorse or promote products derived
+//    from this software without specific prior written permission.
+
+// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS ``AS IS'' AND ANY
+// EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+// IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+// PURPOSE ARE DISCLAIMED.  IN NO EVENT SHALL THE COPYRIGHT OWNER OR
+// CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
+// EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+// PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
+// PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
+// OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
 #include <string>
 #include <iostream>
 #include <ctime>    // For time()

demos/image_pipeline_local/cuda/opencv_interface.cpp

@@ -1,3 +1,31 @@
+// Copyright (c) 2021, NECSTLab, Politecnico di Milano. All rights reserved.
+
+// Redistribution and use in source and binary forms, with or without
+// modification, are permitted provided that the following conditions
+// are met:
+//  * Redistributions of source code must retain the above copyright
+//    notice, this list of conditions and the following disclaimer.
+//  * Redistributions in binary form must reproduce the above copyright
+//    notice, this list of conditions and the following disclaimer in the
+//    documentation and/or other materials provided with the distribution.
+//  * Neither the name of NECSTLab nor the names of its
+//    contributors may be used to endorse or promote products derived
+//    from this software without specific prior written permission.
+//  * Neither the name of Politecnico di Milano nor the names of its
+//    contributors may be used to endorse or promote products derived
+//    from this software without specific prior written permission.
+
+// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS ``AS IS'' AND ANY
+// EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+// IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+// PURPOSE ARE DISCLAIMED.  IN NO EVENT SHALL THE COPYRIGHT OWNER OR
+// CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
+// EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+// PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
+// PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
+// OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
 
 #include "opencv_interface.hpp"