Skip to content

Commit

Permalink
Merge pull request #68 from uqbar-dao/hf/addition-edits
Browse files Browse the repository at this point in the history
add more edits
  • Loading branch information
edgaruncentered authored Jan 18, 2024
2 parents bc69021 + 365522a commit b58b9a5
Show file tree
Hide file tree
Showing 14 changed files with 69 additions and 49 deletions.
2 changes: 1 addition & 1 deletion src/SUMMARY.md
Original file line number Diff line number Diff line change
Expand Up @@ -68,7 +68,7 @@
- [KV API](./apis/kv.md)
- [Net API](./apis/net.md)
- [`kinode.wit`](./apis/kinode_wit.md)
- [SQLITE API](./apis/sqlite.md)
- [SQLite API](./apis/sqlite.md)
- [Terminal API](./apis/terminal.md)
- [VFS API](./apis/vfs.md)
- [Websocket API](./apis/websocket_authentication.md)
2 changes: 1 addition & 1 deletion src/apis/http_client.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# HTTP Client API

*See also: docs.rs for HTTP Client part of [process_lib](../process_stdlib/overview.md)*
See also: [docs.rs for HTTP Client part of `process_lib`](https://docs.rs/kinode_process_lib/0.0.0-reserved/kinode_process_lib/http/index.html).

**Note: Most processes will not use this API directly. Instead, they will use the [`process_lib`](./process_stdlib/overview.md) library, which papers over this API and provides a set of types and functions which are much easier to natively use. This is mostly useful for re-implementing this module in a different client or performing niche actions unsupported by the library.**

Expand Down
2 changes: 1 addition & 1 deletion src/apis/http_server.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# HTTP Server API

*See also: docs.rs for HTTP Server part of process_lib*
See also: [docs.rs for HTTP Server part of `process_lib`](https://docs.rs/kinode_process_lib/0.0.0-reserved/kinode_process_lib/http/index.html).

**Note: Most processes will not use this API directly. Instead, they will use the [`process_lib`](./process_stdlib/overview.md) library, which papers over this API and provides a set of types and functions which are much easier to natively use. This is mostly useful for re-implementing this module in a different client or performing niche actions unsupported by the library.**

Expand Down
7 changes: 4 additions & 3 deletions src/apis/kv.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
### KV API

Useful helper functions can be found in the [kinode_process_lib](https://github.com/uqbar-dao/process_lib)
Useful helper functions can be found in the [`kinode_process_lib`](../process_stdlib/overview.md).
More discussion of databases in Kinode can be found [here](./databases).

#### Creating/Opening a database

Expand All @@ -19,8 +20,8 @@ let key = b"hello";
let value= b"world";

let returnvalue = kv.set(&key, &value, None)?;
// The third argument None is for tx_id.
// You can group sets and deletes and commit them later.
// The third argument None is for tx_id.
// You can group sets and deletes and commit them later.
```

#### Get
Expand Down
5 changes: 3 additions & 2 deletions src/apis/sqlite.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
### SQLITE API
### SQLite API

Useful helper functions can be found in the [kinode_process_lib](https://github.com/uqbar-dao/process_lib)
Useful helper functions can be found in the [`kinode_process_lib`](../process_stdlib/overview.md).
More discussion of databases in Kinode can be found [here](./databases).

#### Creating/Opening a database

Expand Down
42 changes: 21 additions & 21 deletions src/cookbook/file_transfer.md
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
# File Transfer

This entry will teach you to build a simple file transfer app, allowing nodes to download files from a public directory.
It will use the vfs to read and write files, and will spin up worker processes for the transfer.
It will use the [vfs](../apis/vfs.md) to read and write files, and will spin up worker processes for the transfer.

This guide assumes a basic understanding of Kinode process building, some familiarity with `kit`, requests and responses, and some knowledge of rust syntax.
This guide assumes a basic understanding of Kinode process building, some familiarity with [`kit`](../kit/kit.md), requests and responses, and some knowledge of rust syntax.

## Contents

Expand All @@ -16,7 +16,10 @@ This guide assumes a basic understanding of Kinode process building, some famili

## Start

First, initialize a new project with `kit new file_transfer`
First, initialize a new project with
```
kit new file_transfer
```

Here's a clean template so you have a complete fresh start:

Expand Down Expand Up @@ -64,7 +67,7 @@ impl Guest for Component {
}
```

Before delving into the code, you can handle the capabilities you need to request at spawn, these will be messaging capabilities to "net:distro:sys" (as you'll want to talk to other nodes), and one to "vfs:distro:sys" as you'll want to talk to the filesystem.
Before delving into the code, you can handle the capabilities you need to request at spawn, these will be messaging capabilities to `"net:distro:sys"` (as you'll want to talk to other nodes), and one to `"vfs:distro:sys"` as you'll want to talk to the filesystem.

`pkg/manifest.json`

Expand Down Expand Up @@ -118,7 +121,8 @@ pub struct FileInfo {
}
```

You can handle these messages cleanly by modifying the handle message function slightly, it will match on whether a message is a request or a response, the errors get thrown to the main loop automatically with the `?` after the await_message() function.
You can handle these messages cleanly by modifying the handle message function slightly.
It will match on whether a message is a request or a response, the errors get thrown to the main loop automatically with the `?` after the `await_message()` function.

```rust
use kinode_process_lib::{
Expand Down Expand Up @@ -267,13 +271,12 @@ Thu 1/11 13:14 response from node2.os@file_transfer:file_transfer:template.os: {

Now the fun part, downloading/sending files!

You could handle all of this within the file_transfer process, but you can also spin up another process, a worker, that handles the downloading/sending and then sends progress updates back to the main file_transfer.

You could handle all of this within the `file_transfer` process, but you can also spin up another process, a worker, that handles the downloading/sending and then sends progress updates back to the main `file_transfer`.
This way you can download several files downloading at the same time without waiting for one to finish.

Start by defining some types.

You'll need a request that tells our main process to spin up a worker, requesting the node you're downloading from to do the same. Also, a progress report would be nice!
You'll need a request that tells our main process to spin up a worker, requesting the node you're downloading from to do the same.
Also, a progress report would be nice!

```rust
#[derive(Serialize, Deserialize, Debug)]
Expand All @@ -286,7 +289,7 @@ pub enum TransferRequest {

Now, a request to downoad a file will result in a respose to the requesting process to download the file using a worker.

Add a simple Start and Done variant, so you'll know when the worker has successfully been spawned and initialized.
Add a simple `Start` and `Done` variant, so you'll know when the worker has successfully been spawned and initialized.

```rust
#[derive(Serialize, Deserialize, Debug)]
Expand Down Expand Up @@ -316,13 +319,10 @@ pub enum WorkerRequest {
}
```

Workers will take an `Inititialize` request from their own node, that either tells them they're a receiver or a sender based on if they have a target worker `Option<Address>`.

Progress reports are sent back to the main process, which you can then pipe them through as websocket updates to the frontend.

To enable spawning, import the `spawn` function from the `process_lib`.

The only additional part you need to handle in the transfer app is the Download request you've added.
- Workers will take an `Inititialize` request from their own node, that either tells them they're a receiver or a sender based on if they have a target worker `Option<Address>`.
- Progress reports are sent back to the main process, which you can then pipe them through as websocket updates to the frontend.
- To enable spawning, import the `spawn` function from the `process_lib`.
- The only additional part you need to handle in the transfer app is the Download request you've added.

`TransferRequest::Download` will handle 2 cases:

Expand Down Expand Up @@ -396,7 +396,7 @@ This makes adding more features later on very simple.
Now, the actual worker.
Add this bit by bit:

First, because when you spawn your worker you give it `our_capabilities()` (i.e. it has the same capabilities as the parent process), the worker will have the ability to message both `"net:distro:sys"` and `"vfs:distro:sys".
First, because when you spawn your worker you give it `our_capabilities()` (i.e. it has the same capabilities as the parent process), the worker will have the ability to message both `"net:distro:sys"` and `"vfs:distro:sys"`.
As it's also within the same package, you can simply open the `files_dir` without issue.

```rust
Expand Down Expand Up @@ -460,7 +460,7 @@ impl Guest for Component {
}
```

The `handle_message` function will handle 3 types: the requests Initialize, Chunk and Size.
The `handle_message` function will handle three `WorkerRequest` variants: the requests `Initialize`, `Chunk` and `Size`.

`WorkerRequest::Initialize` runs once, received from the spawner:

Expand Down Expand Up @@ -553,8 +553,8 @@ fn handle_message(

So upon `Initialize`, you open the existing file or create an empty one. Then, depending on whether the worker is a sender or receiver, you take one of two options:

- if receiver, save the File to your state, and then send a Started response to parent.
- if sender, get the file's length, send it as Size to the `target_worker`, and then chunk the data, loop, read into a buffer and send to `target_worker`.
- if receiver, save the `File` to your state, and then send a Started response to parent.
- if sender, get the file's length, send it as `Size` to the `target_worker`, and then chunk the data, loop, read into a buffer and send to `target_worker`.

`WorkerRequest::Chunk` will look like this:

Expand Down
4 changes: 2 additions & 2 deletions src/cookbook/manage_child_processes.md
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
# Spawning and Managing Child Processes

In Kinode OS, a "parent" process can create additional processes, known as "children".
In Kinode OS, a "parent" process can create additional processes, known as "children" (also discussed [here](../processes.md#spawning-child-processes)).
These child processes are particularly useful for handling intensive tasks (referred to as "workers") that require long computation times without hindering the performance of the main application.
They are also beneficial for segregating distinct logical components.
Each process is its own subdirectory within the package.
E.g., for Rust processes, each is its own Rust project, complete with a separate Cargo.toml file.
E.g., for Kinode processes written in Rust, each is its own Rust project, complete with a separate Cargo.toml file.

Your package's file structure might resemble the following:

Expand Down
11 changes: 6 additions & 5 deletions src/databases.md
Original file line number Diff line number Diff line change
@@ -1,14 +1,13 @@
# Databases

The runtime currently comes preloaded with 2 databases, a key value (RocksDB) and sqlite.

These can be created, accessed, and shared amongst processes.
The runtime currently provides key-value databases via RocksDB, and relational databases via SQLite.
Processes can create independent databases using wrappers over these libraries, and can read, write, and share these databases with other processes.
The APIs for doing so you can find here: [KV](./apis/kv.md) and [SQLite](./apis/sqlite.md).

Similarly to files in the VFS, they are accessed by `package_id` and a `db` name.
Capabilities to read and write can be shared with other processes; processes within a given package have access by default.

All examples are using the [kinode_process_lib](https://github.com/uqbar-dao/process_lib) functions.
All examples are using the [`kinode_process_lib`](./process_stdlib/overview.md) functions.

## Usage

Expand Down Expand Up @@ -57,4 +56,6 @@ println!("rows: {}", rows.len());
- [SQLite API](./apis/sqlite.md)
- [RocksDB](https://github.com/rust-rocksdb/rust-rocksdb)
- [SQLite](https://www.sqlite.org/docs.html)
- [kinode_process_lib](https://github.com/uqbar-dao/process_lib)
- [`kinode_process_lib` book entry](./process_stdlib/overview.md)
- [`kinode_process_lib` docs.rs](https://docs.rs/kinode_process_lib)
- [`kinode_process_lib`](https://github.com/uqbar-dao/process_lib)
2 changes: 1 addition & 1 deletion src/kit/kit.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
These documents describe some ways you can use these tools, but do not attempt to be completely exhaustive.
You are encouraged to make use of the `--help` flag, which can be used for the top-level `kit`:

```bash
```
$ kit --help
Development toolkit for Kinode OS
Expand Down
2 changes: 1 addition & 1 deletion src/kit/new.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ Currently, three languages are supported: `rust` (the default), `python`, and `j
Two templates are currently supported: `chat`, a simple chat application, and `fibonacci`, which computes Fibonacci numbers.
In addition, some subset of these templates also have a UI-enabled version.

The following table describes the matrix of "Exists/Has UI-enabled version" for each template/language combination:
The following table describes the matrix of ["Exists/Has UI-enabled version"](#existshas-ui-enabled-vesion) for each template/language combination:

### Exists/Has UI-enabled vesion

Expand Down
8 changes: 2 additions & 6 deletions src/my_first_app/chapter_1.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ cargo install --git https://github.com/uqbar-dao/kit

## Creating a New Kinode Package Template

The `kit` toolkit has a [variety of features](https://github.com/uqbar-dao/kit).
The `kit` toolkit has a [variety of features](../kit/kit.md).
One of those tools is `new`, which creates a template for an Kinode package.
The `new` tool takes two arguments: a path to create the template directory and a name for the package:

Expand Down Expand Up @@ -156,10 +156,6 @@ $ cat my_chat_app/pkg/metadata.json
Here, the `publisher` is some default value, but for a real package, this field should contain the NDNS id of the publishing node.
The `publisher` can also be set with a `kit new --publisher` flag.

### `src/lib.rs`

TODO

## Building the Package

To build the package, use the `kit build` tool.
Expand Down Expand Up @@ -286,7 +282,7 @@ and replying, from the other terminal:
```

Messages can also be injected from the outside.
From a bash terminal, use `uqdev inject-message`, like so:
From a bash terminal, use `kit inject-message`, like so:

```bash
kit inject-message my_chat_app:my_chat_app:template.os '{"Send": {"target": "fake2.os", "message": "hello from the outside world"}}'
Expand Down
4 changes: 2 additions & 2 deletions src/my_first_app/chapter_2.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ Tight feedback loops when building: very important.

## Starting from Scratch

If you want to hit the ground running, you can take the template code or the [chess tutorial](../chess_app/start.md) and start hacking away.
If you want to hit the ground running, you can take the template code or the [chess tutorial](../chess_app/chess_engine.md) and start hacking away.
Here, you'll start from scratch and learn about every line of boilerplate.

The last chapter explained packages, the package manifest, and metadata.
Expand Down Expand Up @@ -77,7 +77,7 @@ fn my_init_fn(our: Address) {
}
```

See [kinode.wit](./apis/kinode_wit.md) for more details on what is imported by the WIT bindgen macro.
See [kinode.wit](../apis/kinode_wit.md) for more details on what is imported by the WIT bindgen macro.
These imports are the necessary "system calls" for talking to other processes and runtime components in Kinode OS.

Run
Expand Down
1 change: 1 addition & 0 deletions src/my_first_app/chapter_5.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ This can be added to your package metadata.json like so:
Next, review all the data in [`pkg/manifest.json`](./chapter_1.md#pkgmanifestjson) and [`pkg/metadata.json`](./chapter_1.md#pkgmetadatajson).
The `package` field in `metadata.json` determines the name of the package.
The `publisher` field determines the name of the publisher (you!).

**Note: you *can* set any publisher name you want, but others will be able to verify that you are the publisher by comparing the value in this field with a signature attached to the entry in a (good) app store or package manager, so it's a good idea to put *your node identity* here.**

Once you're ready to share, it's quite easy.
Expand Down
26 changes: 23 additions & 3 deletions src/process_stdlib/overview.md
Original file line number Diff line number Diff line change
@@ -1,11 +1,31 @@
# Overview

The [process standard library](https://github.com/uqbar-dao/process_lib) is the easiest way to write Rust apps on Kinode OS.
Documentation can be found [here](https://docs.rs/kinode_process_lib), and the crate lives [here](https://crates.io/crates/kinode_process_lib).

Note that the process lib is under heavy ongoing development.
This means that the best way to use it is by selecting a tag or commit hash from the GitHub repo, and using that as the version in your `Cargo.toml` file.
See [here](https://github.com/uqbar-dao/process_lib/releases) for a list of published versions.

In your `Cargo.toml` file, use a version tag like this:
```toml
kinode_process_lib = { git = "https://github.com/uqbar-dao/process_lib.git", tag = "v0.5.4-alpha" }
```

To use a specific commit hash, use this:
```toml
kinode_process_lib = { git = "https://github.com/uqbar-dao/process_lib.git", rev = "5305453" }
```

Make sure to use a recent version of the process_lib while the system is in alpha and very active development.

The major version of the process_lib will always match the major version of Kinode OS.
Since the current major version of both is 0, breaking changes can occur at any time.
Once the major version reaches 1, breaking changes will only occur between major versions.
As is, developers may have to update the version of process_lib they use as they update Kinode OS.

Since Kinode apps use the [WebAssembly Component Model](https://component-model.bytecodealliance.org/), they are built on top of a WIT (Wasm Interface Type) package.
This interface contains the core types and functions that are available to all Kinode apps, and these are automatically generated in Rust when building a Wasm app.
However, the types themselves are unwieldy to use directly, and runtime modules present APIs that can be drastically simplified by using helper functions and types in the process standard library.

[link to the crate]

[link to the crate-docs]
Almost all code examples in this book make use of the process lib. For specific examples of its usage, check out the [docs](https://docs.rs/kinode_process_lib) or just follow the tutorials later in this book.

0 comments on commit b58b9a5

Please sign in to comment.