docs/input-validators (#464)

* docs/input-validators * Update content/docs/05.developer-guide/04.inputs.md Co-authored-by: Loïc Mathieu <[email protected]> * Update content/docs/05.developer-guide/04.inputs.md Co-authored-by: Loïc Mathieu <[email protected]> * Update content/docs/05.developer-guide/04.inputs.md Co-authored-by: Loïc Mathieu <[email protected]> * Update content/docs/05.developer-guide/04.inputs.md Co-authored-by: Loïc Mathieu <[email protected]> * Update content/docs/05.developer-guide/04.inputs.md Co-authored-by: Loïc Mathieu <[email protected]> --------- Co-authored-by: Loïc Mathieu <[email protected]>
kestra-io · Aug 25, 2023 · 4f5daa2 · 4f5daa2 · vercel · Aug 25, 2023
1 parent f5ae58c
commit 4f5daa2
Show file tree

Hide file tree

Showing 2 changed files with 123 additions and 54 deletions.
diff --git a/content/docs/05.developer-guide/04.inputs.md b/content/docs/05.developer-guide/04.inputs.md
@@ -2,28 +2,30 @@
 title: Inputs
 ---
 
-A flow can be parameterized using inputs. Flow inputs will be available in variables inside the execution context and can be used during the flow execution to customize its tasks properties.
+A flow can be parameterized using inputs to allow multiple executions of the same flow, each with different input values. Flow inputs are stored as variables within the flow execution context and can be accessed within the flow using the syntax `{{ inputs.parameter_name }}`. 
 
-A good example of the usage of an input is when an identifier is needed (ex: storeId, paymentId) to define the path of an uploaded file.
+You can use inputs to make your tasks more dynamic. For instance, you can use an input to dynamically define the path of a file that needs to be processed within a flow.
+
+You can inspect the input values in the **Overview** tab of the **Execution** page.
 
-Flow inputs can be seen in the **Overview** tab of the **Execution** page.
 
 ## Declaring inputs
 
-You can declare as many inputs as necessary for any flow. Inputs can be **required** or **not**.
-If an input is required, the flow cannot start if the input is not provided during the creation of the execution.
+You can declare as many inputs as necessary for any flow. Inputs can be **required** or **optional**.
+
+If an input is required, we recommend using the `defaults` property to set default values. The flow cannot start if the input is not provided during the creation of the Execution.
 
 Every input will be parsed during the creation of the execution, and any invalid inputs will deny the creation of the execution.
 
 ::alert{type="warning"}
 If the execution is **not created** due to invalid or missing inputs, no execution will be found on the list of executions.
 ::
 
-Examples:
+Below is an example flow using several inputs:
 
 ```yaml
-id: my-flow
-namespace: io.kestra.docs
+id: inputs
+namespace: dev
 
 inputs:
   - name: string
@@ -61,87 +63,142 @@ inputs:
 ```
 
 ## Input types
-Inputs can be of multiple types:
-
-- `STRING`: No control is done (no parsing); it can be any string.
-- `INT`: Must be a valid integer (without any decimals).
-- `FLOAT`: Must be a valid float (with any decimals).
-- `BOOLEAN`: Must be `true` or `false` as strings.
-- `DATETIME`: Must be a valid full [ISO 8601](https://en.wikipedia.org/wiki/ISO_8601) date and time with the timezone expressed in UTC from a text string such as `2007-12-03T10:15:30.00Z`.
-- `DATE`: Must be a valid full [ISO 8601](https://en.wikipedia.org/wiki/ISO_8601) date without the timezone from a text string such as `2007-12-03`.
-- `TIME`: Must be a valid full [ISO 8601](https://en.wikipedia.org/wiki/ISO_8601) time without the timezone from a text string such as `10:15:30`.
-- `DURATION`: Must be a valid full [ISO 8601](https://en.wikipedia.org/wiki/ISO_8601) duration from a text string such as `PT5M6S`.
-- `FILE`: Must be a file sent as `Content-Type: multipart/form-data` with `Content-Disposition: form-data; name="files"; filename="my-file"`, where `my-file` is the name of the input.
-- `JSON`: Must be a valid JSON string and will be converted to typed form.
-- `URI`: Must be a valid URI and will be kept as a string.
-
-All the inputs of type `FILE` will be automatically uploaded to Kestra's [internal storage](../08.architecture.md#the-internal-storage) and available for all tasks. After upload, the input variable will contain a fully qualified URL of the form `kestra:///.../.../` that will be automatically handled by Kestra and can be used as is by tasks.
+
+Inputs in Kestra are strongly typed and validated before starting the flow execution. 
+
+Here is a list of supported data types:
+
+- `STRING`: it can be any string value — inputs of type STRING are passed to the execution in its raw format without parsing; for additional validation, you can add a custom regex `validator` to allow only specific string patterns
+- `INT`: must be a valid integer value (without any decimals)
+- `FLOAT`: must be a valid float value (with decimals)
+- `BOOLEAN`: must be `true` or `false` passed as strings
+- `DATETIME`: must be a valid full [ISO 8601](https://en.wikipedia.org/wiki/ISO_8601) date and time with the timezone expressed in UTC format; pass input of type DATETIME in a string format following the pattern `2042-04-02T04:20:42.00Z`
+- `DATE`: must be a valid full [ISO 8601](https://en.wikipedia.org/wiki/ISO_8601) date without the timezone from a text string such as `2042-12-03`
+- `TIME`: must be a valid full [ISO 8601](https://en.wikipedia.org/wiki/ISO_8601) time without the timezone from a text string such as `10:15:30`
+- `DURATION`: must be a valid full [ISO 8601](https://en.wikipedia.org/wiki/ISO_8601) duration from a text string such as `PT5M6S`
+- `FILE`: must be a file sent as `Content-Type: multipart/form-data` with `Content-Disposition: form-data; name="files"; filename="my-file"`, where `my-file` is the name of the input
+- `JSON`: must be a valid JSON string and will be converted to a typed form
+- `URI`: must be a valid URI and will be kept as a string.
+
+All inputs of type `FILE` will be automatically uploaded to Kestra's [internal storage](../08.architecture.md#the-internal-storage) and accessible to all tasks. After the upload, the input variable will contain a fully qualified URL of the form `kestra:///.../.../` that will be automatically managed by Kestra and can be used as-is within any task.
 
 ## Input Properties
-These are the properties available for all input types:
 
-- `name`: The input name to be used with variables.
-- `required`: If the input is required. If required and no default value or no input is provided, the execution will not be created.
-- `defaults`: The default value if no input is provided. Must be a string.
-- `description`: A markdown description to document the input.
+Below is a list of properties available to all inputs regardless of their types:
+
+- `name`: the input parameter name — this property is important as it's used to reference the input variables in your flow, e.g. `{{ inputs.user }}` will reference the input parameter named `user`
+- `type`: the data type of the input parameter, as described in the previous section
+- `required`: whether the input is required or optional; if required is set to true and no default value is configured and also no input is provided at runtime, the execution will not be created as Kestra cannot know what value to use
+- `defaults`: the default value that will be used if no custom input value is provided at runtime; this value must be provided as a string and will be coerced to the desired data type specified using the `type` property
+- `description`: a markdown description to document the input.
+
 
 ## Input validation
 
-In addition to the standard validation rules explained above, some input types can be configured with validation rules that will be enforced at execution time.
+Kestra validates the `type` of each input. In addition to the type validation, some input types can be configured with validation rules that will be enforced at execution time.
+
+- `STRING`: a `validator` property allows the addition of a validation [regex](https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/util/regex/Pattern.html).
+- `INT`: `min` and `max` properties allow the addition of minimum and maximum value ranges
+- `FLOAT`: `min` and `max` properties allow the addition of minimum and maximum ranges
+- `DURATION`: `min` and `max` properties allow the addition of minimum and maximum ranges
+- `DATE`: `after` and `before` properties help you ensure that the input value is within the allowed date range
+- `TIME`: `after` and `before` properties help you ensure that the input value is within the allowed time range
+- `DATETIME`: `after` and `before` properties help you ensure that the input value is within the allowed date range.
+
+## Example: using input validators in your flows
+
+To ensure that your input value is within a certain `integer` value range, you can use the `min` and `max` properties. Similarly, to ensure that your string input matches a regex pattern, you can provide a custom regex `validator`. The following flow demonstrates how this can be accomplished:
+
+```yaml
+id: regex-input
+namespace: dev
+
+inputs:
+  - name: age
+    type: INT
+    defaults: 42
+    required: false
+    min: 18
+    max: 64
+
+  - name: user
+    type: STRING
+    defaults: student
+    required: false
+    validator: ^student(\d+)?$
+
+tasks:
+  - id: validator
+    type: io.kestra.core.tasks.log.Log
+    message: User {{ inputs.user }}, age {{ inputs.age }}
+```
+
+The `age` input must be within a valid range between `min` and `max` integer value. Specifically, we specify that this input will be by default set to 42, but it can be overwritten at runtime to a value between 18 and 64. If you attempt to execute the flow with the `age` input set to 17 or 65, the validation will fail and the execution won't start.
 
-- `STRING`: A `validator` property allows the addition of a validation [regex](https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/util/regex/Pattern.html).
-- `INT`: `min` and `max` properties allow the addition of minimum and maximum bounds.
-- `FLOAT`: `min` and `max` properties allow the addition of minimum and maximum bounds.
-- `DATETIME`: `after` and `before` properties allow the addition of minimum and maximum bounds.
-- `DATE`: `after` and `before` properties allow the addition of minimum and maximum bounds.
-- `TIME`: `after` and `before` properties allow the addition of minimum and maximum bounds.
-- `DURATION`: `min` and `max` properties allow the addition of minimum and maximum bounds.
+Similarly, the Regex expression `^student(\d+)?$` is used to validate that the input argument `user` of type STRING follows the following pattern:
+- `^student`: this part of the regex asserts that the string must begin with the lowercase string value `student`
+- `\d`: this part of the regex matches any digit (0-9)
+- `+`: this part of the regex asserts that there is one or more of the preceding token (i.e., one or more digits are allowed after the value `student`)
+- `()?`: the parentheses group the digits together, and the question mark makes the entire group optional — this means that the digits after the word `student` are optional
+- `$`: this part of the regex asserts the end of the string. This ensures that the string doesn't contain any additional characters after the optional digits.
+
+With this pattern:
+- "student" would be a match
+- "student123" would be a match
+- "studentabc" would not be a match because "abc" isn't a sequence of digits
+- "student123abc" would not be a match because no more characters are allowed after the sequence of "students" with the following numbers.
+
+
+Try running this flow with various inputs or adjust the regex pattern to see how the input validation works.
 
 
 ## Nested Inputs
 
 If you use a `.` inside the name of an input, the input will be nested.
 
-For example, when you declare the input with the following:
+Consider the input with the following configuration:
 
 ```yaml
   - name: nested.string
     type: STRING
     required: false
 ```
 
-You can reach it with: `{{ inputs.nested.string }}`. This is a convenient way to handle strong typing on input (with validation) without using raw JSON that will not be validated.
+You can access that input value using `{{ inputs.nested.string }}`. This syntax provides a convenient type validation of nested inputs without using raw JSON that would not be validated (JSON-type input values are passed as strings).
 
 
 ## Using input value in a flow
 
-Every input is available with dynamic variables such as: `{{ inputs.NAME }}` or `{{ inputs[NAME] }}`.
+Every input is available with dynamic variables such as: `{{ inputs.name }}` or `{{ inputs['name'] }}`.
+
+For example, if you declare the following inputs:
 
-For example, if you declare these inputs:
 ```yaml
 inputs:
   - name: my-file
     type: FILE
-  - name: my-value
+  - name: my-string
     type: STRING
     required: true
 ```
 
-You can use the value of the input `my-value` inside dynamic task properties with `{{ inputs['my-value'] }}`. See [variables basic usage](./03.variables/02.basic-usage.md).
+You can use the value of the input `my-string` inside dynamic task properties with `{{ inputs['my-string'] }}`. See [variables basic usage](./03.variables/02.basic-usage.md).
 
-## Set inputs at flow execution
 
-When you execute a flow with some inputs, you must set all inputs (unless optional or with a default value) to be able to create the execution.
+## Set input values at flow execution
 
-Let's take this example that defines multiple inputs:
+When you execute a flow with inputs, you must set all inputs (unless optional or with a default value) to be able to create the execution.
+
+Let's consider the following example that defines multiple inputs:
 
 ```yaml
 id: kestra-inputs
-namespace: io.kestra.test
+namespace: dev
 
 inputs:
   - name: string
     type: STRING
+    defaults: hello
   - name: optional
     type: STRING
     required: false
@@ -155,20 +212,23 @@ inputs:
     type: FILE
 ```
 
+Here, the inputs `{{ inputs.string }}` and `{{ inputs.optional }}` can be skipped because the `string` input has a default value and the `optional` input is not required. All other inputs must be specified at runtime.
+
+
 ### Set inputs from the web UI
 
-When creating an execution from the web UI, you must set the inputs in the generated form. Kestra's UI will generate a dedicated form based on your flow definition.
+When creating an execution from the web UI, you must set the inputs in the UI form. Kestra's UI will generate a dedicated form based on your inputs definition. For example, datetime input proprties will have a date picker.
 
-The input form for the inputs above looks like this:
+The input form for the inputs above looks as follows:
 
 ![Flow inputs](/docs/developer-guide/inputs/inputs.jpg)
 
-Once the inputs are set, you can trigger an execution of the flow. The inputs will be available as variables for the flow's tasks.
+Once the inputs are set, you can trigger an execution of the flow. 
 
 
-### Set inputs with curl
+### Set inputs when executing the flow using the `curl` command
 
-To create an execution with these inputs using `curl`, you can use the following command:
+To create an execution with these inputs using the `curl` command, you can use the following command:
 
 ```bash
 curl -v "http://localhost:8080/api/v1/executions/trigger/io.kestra.test/kestra-inputs" \
@@ -178,13 +238,13 @@ curl -v "http://localhost:8080/api/v1/executions/trigger/io.kestra.test/kestra-i
     -F optional="an optional string"  \
     -F int=1  \
     -F float=1.255  \
-    -F instant="2020-01-14T23:00:00.000Z" \
+    -F instant="2023-12-24T23:00:00.000Z" \
     -F "files=@/tmp/128M.txt;filename=file"
 ```
 
 All files must be sent as multipart form data named `files` with a header `filename=my-file` which will be the name of the input.
 
-### Set inputs programmatically in Python
+### Set inputs when executing the flow in Python
 
 To create an execution with these inputs in Python, you can use the following script:
 

diff --git a/content/docs/15.migrations/templates.md b/content/docs/15.migrations/templates.md
@@ -5,12 +5,21 @@ title: Deprecation of Templates
 Templates are marked as deprecated and disabled by default starting from the 0.11.0 release. Please use subflows instead. You can re-enable them with this [configuration](../09.administrator-guide/01.configuration/05.others.md).
 
 
-## Why are templates are being deprecated?
+## Why templates are deprecated
 
 1. Subflows are more powerful — subflows provide the same functionality as templates while simultaneously being more flexible than templates. For instance,  `inputs` are not allowed in a template because a template is only a list of tasks that get copied to another flow that references it. In contrast, when invoking a subflow, you can parametrize it with custom parameters. This way, subflows allow you to define workflow logic once and invoke it in other flows with custom parameters.
 2. Subflows are more transparently reflected in the topology view and don't require copying tasks.
 
 
+If you are using templates and you are not ready to migrate to subflows yet, add the following Kestra configuration option to still be able to use them:
+
+```yaml
+kestra:
+  templates:
+    enabled: true
+```
+
+
 ---
 
 ## Templates :warning: