Merge pull request #96 from kinode-dao/da/more-scripts

piping and writing scripts

src/apis/terminal.md

@@ -1,4 +1,7 @@
 # Terminal API
+It is extremely rare for an app to have direct access to the terminal api.
+Normally, the terminal will be used to call scripts, which will have access to the process in question.
+For documentation on using, writing, publishing, and composing scripts, see the [terminal use documentation](../terminal.md), or for a quick start, the [script cookbook](../cookbook/writing_scripts.md).
 
 The Kinode terminal is broken up into two segments: a Wasm app, called `terminal:terminal:sys`, and a runtime module called `terminal:distro:sys`.
 The Wasm app is the central area where terminal logic and authority live.
@@ -21,7 +24,7 @@ For example, `hi:terminal:sys` can be shortened to just `hi` as in: `hi default-
 
 The other most commonly used script is `m:terminal:sys`, or just `m` - which stands for `Message`. `m` let's you send a request to any node or application like so:
 ```bash
-m john.os@proc:pkg:pub {"foo":"bar"}
+m john.os@proc:pkg:pub '{"foo":"bar"}'
 ```
 
 Note that if your process has the ability to message the `terminal` app, then that process can call any script - of which there may be many on a machine, so we cannot possibly write all of them down in this document.

src/cookbook/writing_scripts.md

@@ -7,9 +7,9 @@ When writing a script, you cannot control the `OnExit` behavior like you can wit
 - Scripts are registered in the `scripts.json` file instead of the `manifest.json` file
 
 ## Writing a Script
-Let's look at the simplest possible script: `echo`, which takes in an argument, and prints it out again:
+Consider the simplest possible script: `echo`, which takes in an argument, and prints it out again:
 ```rust
-use kinode_process_lib::{await_next_request_body, call_init, println, Address};
+use kinode_process_lib::{await_next_request_body, call_init, println, Address, Response};
 
 wit_bindgen::generate!({
     path: "wit",
@@ -27,22 +27,24 @@ fn init(_our: Address) {
         return;
     };
 
-    println!(
-        "{}",
-        String::from_utf8(args).unwrap_or("echo: error".into())
-    );
+    let _ = Response::new().body(args).send();
 }
 ```
-From writing applications, this should look very familiar - the imports, `wit_bindge::generate!`, `call_init!`, `init(our: Address)`, etc. are all exactly the same.
+From writing applications, this should look very familiar - the imports, `wit_bindgen::generate!`, `call_init!`, `init(our: Address)`, etc. are all exactly the same.
 The first unique thing about scripts is that we will have no `loop` where we `await_message`.
 Instead, our initial arguments will come from a single message from the terminal - which we get by calling `await_next_message_body()`.
 Next, all we do is `String`ify the message body, and print it out.
 
 Arbitrary logic can be put below `await_next_message_body` - just like an app, you can fire-off a number of requests, choose to await their responses, handle errors, etc. just like normal.
 
+In this case, `echo` sends a `Response` containing the arguments passed in.
+The `Response` enables composition of scripts via [piping](../terminal.md#piping-and-composing-scripts), so that the output of `echo` can serve as the input to the next program.
+Not every script needs to end with a `Response`.
+It would be valid to simply `println` the `args` and terminate, but this would mean that the script has no "return value" to compose with other scripts via pipes.
+
 ## Publishing a Script
-Unlike processes accociated with a long-running application, which will be put into the `manifest.json`, scripts must be registered in a separate `scripts.json` file.
-While very similar, there are a few important differences; let's take a look at an example that could live in your packages `pkg/scripts.json` file:
+Unlike processes associated with a long-running application, which will be put into the `manifest.json`, scripts must be registered in a separate `scripts.json` file.
+While very similar, there are a few important differences; here's an example that could live in your packages `pkg/scripts.json` file:
 ```json
 {
     "echo.wasm": {
@@ -58,7 +60,7 @@ This `scripts.json` file corresponds to a package which publishes a single scrip
 The keys of this object are the process paths inside of the `pkg/` folder.
 The name of the script will be the file path, with `.wasm` taken off.
 The object that `echo.wasm` points to is very similar to `manifest.json`, with a few things removed, and `root` has been added:
-- `root` means that all the capabilities held by the `terminal:terminal:sys` are passed to this script. This is rarely needed.
+- `root` means that all the capabilities held by the `terminal:terminal:sys` are passed to this script (this is powerful, and rarely needed)
 - `public`: same as `manfiest.json` - corresponds to whether or not other processes can message `echo.wasm` without the messsaging cap
 - `requestNetworking`: same as `manfiest.json` - corresponds to whether or not this script will need to send messaages over the network
 - `requestCapabilities`: same as `manifest.json` - a list of capabilities that will be granted to this script on startup (NOTE if you have `root`, there is no reason to populate `requestCapabilities` as well)

src/terminal.md

@@ -1,9 +1,9 @@
 # Terminal
 
-## Basic Usage and Utilities
+## Commands
 
 All commands in the terminal are calling scripts - a special kind of process.
-KinodeOS comes pre-loaded with a number of scripts useful for debugging and everyday use.
+Kinode OS comes pre-loaded with a number of scripts useful for debugging and everyday use.
 These scripts are fully named `<SCRIPT>:terminal:sys` e.g `hi:terminal:sys`, but the distro [aliases](#alias---alias-a-script-name) these to short names, in this case just `hi`, for convenience.
 
 
@@ -77,3 +77,85 @@ Example:
 ```
 
 For more information on writing your own scripts, see the [cookbook](./cookbook/writing_scripts.md).
+
+## Packaging Scripts with `scripts.json`
+For your scripts to be usable by the terminal, you must include a `pkg/scripts.json` file.
+The JSON object in `scripts.json` describes the configuration for each script in your package.
+Each top-level key represents the path of the WASM module in your package, usually just `"myscript.wasm"`, `"echo.wasm"`, etc.
+
+The value for each module is an object that specifies the configuration for that particular module.
+The object can contain the following fields:
+
+- `root` (Boolean): Indicates whether the script has "root" privileges - meaning whether it gets *every* capability that the terminal has (not necessarily every capability in existence on your machine)
+- `public` (Boolean): Determines if the script is publicly accessible by other processes
+- `requestNetworking` (Boolean): Specifies whether the script will get networking capabilities
+- `requestCapabilities` (Array): An array that lists the capabilities requested by the script. Each element in the array can be either a string or an object. The string represents a `ProcessId` that this script will be able to message. When an object is used, it specifies a different kind of capability from `issuer` with `params` as an arbitrary json object.
+- `grantCapabilities` (Array): An array `ProcessId`s as strings which represent which processes will be able to message this script back
+
+Modules may not necessarily use all these fields. For instance, "m.wasm" only uses root, public, and requestNetworking, omitting requestCapabilities and grantCapabilities.
+
+### Example
+This is a `scripts.json` that publishes a single script, `hi`, which doesn't receive `root` capabilities, is not `public`, can send messages over the network, will receive the capability to message `net:distro:sys`, and gives `net:distro:sys` the ability to message it back:
+```json
+{
+    "hi.wasm": {
+        "root": false,
+        "public": false,
+        "requestNetworking": true,
+        "requestCapabilities": [
+            "net:distro:sys"
+        ],
+        "grantCapabilities": [
+            "net:distro:sys"
+        ]
+    }
+}
+```
+
+## Piping and Composing Scripts
+### Using Pipes in the Terminal
+The terminal lets you pipe the previous command's output into another script.
+For exapmle:
+```bash
+echo default-router-1.os Hello! |1 hi
+```
+Since the `echo` script just returns its arguments, we are piping the string `default-router-1.os Hello!` into `hi`, so the end result is just 
+```bash
+hi default-router-1.os Hello!
+```
+This is exactly the same as a unix pipe except for the number that comes after the pipe.
+This number after the pipe represents timeout in seconds.
+In this case: wait for `echo` to respond, and if it does not after 1 second, abort.
+Any `u64` value is a valid timeout value.
+
+### Writing Your Script to be Pipe-able
+Not all scripts are pipable.
+For a script's output to be pipeable you must take the output and `.send()` it in a `Response`.
+The [cookbook](./cookbook/writing_scripts.md) describes how every script begins with a `Request` that contains arguments in the `body`.
+If you want the outputs of your script to be composable via pipes, you must terminate your script with a `Response` to this initial request.
+As a simple example, let's look at the `echo` code:
+```rust
+use kinode_process_lib::{await_next_request_body, call_init, println, Address, Response};
+
+wit_bindgen::generate!({
+    path: "wit",
+    world: "process",
+    exports: {
+        world: Component,
+    },
+});
+
+call_init!(init);
+
+fn init(_our: Address) {
+    let Ok(args) = await_next_request_body() else {
+        println!("echo: failed to get args, aborting");
+        return;
+    };
+
+    let _ = Response::new().body(args).send();
+}
+```
+
+All we do here is get the args, and send a response back with the exact same args.
+If instead of sending a `Response` we simply `println`d the arguments out, then we wouldn't be able to compose `echo` with pipes.