Merge branch 'master' into devel

docs/deploy-im-dashboard.md

@@ -57,12 +57,29 @@ These are the steps:
     In this panel you can specify the number of Working Nodes (WNs) of the
     cluster together with the computational requirements for each node. We
     leave the default values.
+    - Number of WNs in the oscar cluster: Number of working nodes.
+    - Number of CPUs for the front-end node: Number of CPUs in the primary node.
+    - Amount of Memory for the front-end node: RAM in the primary node.
+    - Flavor name of the front-end node. Only required in case of special flavors i.e. with GPUs: Type of instance that will be selected in the front node.
+    - Number of CPUs for the WNs: number of CPUs per working node.
+    - Amount of Memory for the WNs: RAM per working node.
+    - Flavor name of the WNs. Only required in case of special flavors i.e. with GPUs: Type of instance that will be selected in the working nodes.
+    - Size of the extra HD added to the instance: Extra memory in the primary node.
     ![template-hw](images/im-dashboard/im-dashboard-02.png)
 
     In this panel, specify the passwords to be employed to access the
     Kubernetes Web UI (Dashboard), to access the OSCAR web UI and to access
     the MinIO dashboard. These tokens can also be used for programmatic access
     to the respective services.
+
+    - Access Token for the Kubernetes admin user: It is the token to connect to the Dashboard of Kubernetes.
+    - OSCAR password: password to OSCAR.
+    - MinIO password 8 characters min.: password to MinIO.
+    - Email to be used in the Lets Encrypt issuer: It is an Email linked with the certificates in case the user has any questions.
+    - ID of the user that creates the infrastructure: unique identifier. Do not touch.
+    - VO to support: It supports OIDC log in. If there is nothing, only can connect the user who deploys, in case there is a VO, it can be the user who deploys and all people in the VO.
+    - Flag to add NVIDIA support: if you want to use NVIDIA.
+    - Flag to install Apache YuniKorn: if you are going to use YuniKorn.
     ![template-param](images/im-dashboard/im-dashboard-03.png)
 
     Now, choose the Cloud provider. The ID specified when creating the Cloud

docs/faq.md

@@ -15,4 +15,21 @@ You can check if you are using the correct default cluster with the following co
 
 and set a new default cluster with the following command:
 
-`oscar-cli cluster default -s CLUSTER_ID`
+`oscar-cli cluster default -s CLUSTER_ID`
+
+- **How do I use a secret image?**
+
+In case it is required the use of secret images, you should create a [secret with the docker login configuration](https://kubernetes.io/docs/tasks/configure-pod-container/pull-image-private-registry/#registry-secret-existing-credentials) with a structure like this:
+
+```
+apiVersion: v1
+kind: Secret
+metadata:
+  name: dockersecret
+  namespace: oscar-svc
+data:
+  .dockerconfigjson: {base64 .docker/config.json}
+type: kubernetes.io/dockerconfigjson
+```
+
+Apply the file through kubectl into the Kubernetes OSCAR cluster to create the secret. To use it in OSCAR services, you must add the secret name (`dockersecret` in this example) in the definition of the service, using the API or a FDL, under the `image_pull_secrets` parameter, or through the "Docker secret" field in OSCAR-UI.

examples/body-pose-detection/README.md

@@ -0,0 +1,21 @@
+# Body Pose Invocation asynchronous invocation
+
+This example uses the pre-trained classification model by DEEP-Hybrid-DataCloud
+[Body Pose Detection](https://marketplace.deep-hybrid-datacloud.eu/modules/deep-oc-posenet-tf.html)
+and is prepared to be used with asynchronous invocations.
+
+
+In order to invoke the function, first you have to do is create a service,
+either by the OSCAR UI or by using the FDL within the following command.
+
+``` sh
+oscar-cli apply body-pose-detection-async.yaml
+```
+
+Once the service is created you can make the invocation with the following
+command, which will store the output on a minio bucket.
+
+``` sh
+oscar-cli service put-file body-pose-detection-async minio images/001.jpg body-pose-detection-async/input/001.jpg
+```
+

examples/body-pose-detection/body-pose-detection-async.yaml

@@ -0,0 +1,14 @@
+functions:
+  oscar:
+  - oscar-cluster:
+     name: body-pose-detection-async
+     memory: 2Gi
+     cpu: '1.0'
+     image: deephdc/deep-oc-posenet-tf
+     script: script.sh
+     input:
+     - storage_provider: minio.default
+       path: body-pose-detection-async/input
+     output:
+     - storage_provider: minio.default
+       path: body-pose-detection-async/output

examples/body-pose-detection/script.sh

@@ -0,0 +1,6 @@
+#!/bin/bash
+
+IMAGE_NAME=`basename "$INPUT_FILE_PATH"`
+OUTPUT_IMAGE="$TMP_OUTPUT_DIR/"
+
+deepaas-predict -i "$INPUT_FILE_PATH" -ct application/zip -o $OUTPUT_IMAGE

examples/compss/README.md

@@ -0,0 +1,47 @@
+# COMPSS with OSCAR
+
+## Dockerfile
+
+The first step is the creation of the Docker image
+The Docker image has to start from an image allocated in the [compss profile at dockerhub](https://hub.docker.com/u/compss)
+with all the dependencies
+
+``` Docker
+FROM compss/compss:{version}
+```
+
+Copy the program inside the Docker container:
+
+- C/C++ Applications should be the entire project folder (the compilation will create the binary in execution time) compss_build_app
+- Java Application, the `.jar` file should be copy
+- In Python Applications, introduce all the code
+
+## Script
+
+Create the name of the output file and save it into the `$OUTPUT_FILE` variable.
+
+``` bash
+FILE_NAME=`basename "$INPUT_FILE_PATH" | cut -f 1 -d '.'`
+OUTPUT_FILE="$TMP_OUTPUT_DIR/$FILE_NAME.txt"
+```
+
+Some services receive a compressed file that needs to be uncompressed, while others require parsing the input file. Once the input is parsed, the ssh server needs to be initialized.
+
+``` bash
+/etc/init.d/ssh start
+```
+
+Finally, run COMPSs. Select the command more appropriate according to your language program.
+C programs need to be built first with `compss_build_app increment`.
+Redirect the output to `$OUTPUT_FILE`.
+
+``` bash
+runcompss --pythonpath=$(pwd) --python_interpreter=python3 {path_to_the_python_program.py} {input_variables}  > $OUTPUT_FILE
+runcompss  --classpath={path_to_the_jar.jar} {in_java_folder/MainClass} {input_variables} > $OUTPUT_FILE
+runcompss --lang=c --project=./xml/templates/project.xml  master/{name_program} {input_variables} > $OUTPUT_FILE
+```
+
+### Output warning redirect
+
+When COMPSs starts the execution, a warning message will appear in the logs.
+This message can be ignored. `WARNING: COMPSs Properties file is null. Setting default values`

examples/compss/c/Dockerfile

@@ -0,0 +1,2 @@
+FROM compss/compss:latest
+COPY increment /opt/increment

examples/compss/c/README.md

@@ -0,0 +1,29 @@
+# Increment in C with COMPSS and OSCAR
+
+The source code can be found in [github](https://github.com/bsc-wdc/tutorial_apps/tree/stable/c/increment)
+This example works by introducing four numbers: the first is the number of times to increase the counters the other three are the counters.
+The return will be the three counters incremented by the first number.
+
+The first step in this example is to parse the input file and set the arguments:
+
+```bash
+file=$(cat $INPUT_FILE_PATH)
+incrementNumber=$(echo "$file" | cut -f 1 -d ';')
+counter1=$(echo "$file" | cut -f 2 -d ';')
+counter2=$(echo "$file" | cut -f 3 -d ';')
+counter3=$(echo "$file" | cut -f 4 -d ';')
+```
+
+In C language, it is necessary to change the directory and compile the project in a run time:
+
+``` bash
+cd /opt/increment
+compss_build_app increment
+```
+
+The compilation process will create a folder with the name `master`.
+This folder contains binary program:
+
+``` bash
+runcompss --lang=c --project=./xml/templates/project.xml  master/increment $incrementNumber $counter1 $counter2  $counter3 > $OUTPUT_FILE
+```

examples/compss/c/buildAndPush.sh

@@ -0,0 +1,3 @@
+#!/bin/bash
+docker build -t ghcr.io/grycap/increment-compss-c .
+docker push ghcr.io/grycap/increment-compss-c

examples/compss/c/increment/Readme

@@ -0,0 +1,31 @@
+This is the Readme for:
+Increment
+
+[Name]: Increment
+[Contact Person]: [email protected]
+[Access Level]: public
+[License Agreement]: Apache2
+[Platform]: COMPSs
+
+[Body]
+== Description ==
+Increment is an application that takes three different values and increases them a number of given times. The purpose of this application is to show parallelism between the different increments. 
+
+== Execution instructions ==
+Usage:
+runcompss --lang=c /home/compss/tutorial_apps/c/increment/master/increment <N> <initValue1> <initValue2> <initValue3>
+
+where:
+		 * - N: Number of times to increase the counters
+		 * - initValue1: Initial value for counter 1
+                 * - initValue2: Initial value for counter 2
+                 * - initValue3: Initial value for counter 3
+
+
+== Execution Example ==
+runcompss --lang=c /home/compss/tutorial_apps/c/increment/master/increment 10 1 2 3
+
+
+== Build ==
+compss_build_app increment
+

examples/compss/c/increment/increment-functions.cc

@@ -0,0 +1,54 @@
+/*
+ *  Copyright 2002-2015 Barcelona Supercomputing Center (www.bsc.es)
+ *
+ *  Licensed under the Apache License, Version 2.0 (the "License");
+ *  you may not use this file except in compliance with the License.
+ *  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ *  Unless required by applicable law or agreed to in writing, software
+ *  distributed under the License is distributed on an "AS IS" BASIS,
+ *  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ *  See the License for the specific language governing permissions and
+ *  limitations under the License.
+ */
+#include<iostream>
+#include<fstream>
+#include<string>
+#include"increment.h"
+
+
+void increment(file fileName) {
+    std::cout << "INIT TASK" << std::endl;
+    std::cout << "Param: " << fileName << std::endl;
+
+    int value;
+    // Read value
+    std::ifstream fis(fileName);
+    if (fis.is_open()) {        
+        if (fis >> value) {
+            fis.close();
+        } else {
+            std::cerr << "[ERROR] Unable to read final value" << std::endl;
+            fis.close();
+        }
+        fis.close();
+    } else {
+        std::cerr << "[ERROR] Unable to open file" << std::endl;
+    }
+
+    // Increment
+    std::cout << "INIT VALUE: " << value << std::endl;
+    std::cout << "FINAL VALUE: " << ++value << std::endl;
+
+    // Write new value
+    std::ofstream fos (fileName);
+    if (fos.is_open()) {
+        fos << value << std::endl;
+        fos.close();
+    } else {
+        std::cerr << "[ERROR] Unable to open file" << std::endl;
+    }
+    std::cout << "END TASK" << std::endl;
+}