From 40c6393982caf5a62e479467ad8f657f7603fda3 Mon Sep 17 00:00:00 2001 From: Trent Houliston Date: Sun, 1 Sep 2024 21:37:50 +1000 Subject: [PATCH] Start making a new docs. Lots of this is copilot and needs more help/more copilot and some isn't filled in. Still nto 100% sure about the structure either. --- docs2/Overview.md | 0 docs2/components/PowerPlant.md | 0 docs2/components/Reaction Tasks.md | 6 + docs2/components/Reactions.md | 10 ++ docs2/components/Reactor.md | 2 + docs2/concepts/Bind.md | 41 ++++++ docs2/concepts/Emit.md | 7 ++ docs2/concepts/Get.md | 117 +++++++++++++++++ docs2/concepts/Group.md | 59 +++++++++ docs2/concepts/Inline.md | 77 ++++++++++++ docs2/concepts/Pool.md | 99 +++++++++++++++ docs2/concepts/Postcondition.md | 60 +++++++++ docs2/concepts/Precondition.md | 65 ++++++++++ docs2/concepts/Serialization.md | 46 +++++++ docs2/concepts/Time.md | 56 +++++++++ docs2/dsl/DSL.md | 5 + docs2/dsl/Emit.md | 0 docs2/dsl/Extending.md | 0 docs2/dsl/words/Always.md | 1 + docs2/dsl/words/Buffer.md | 0 docs2/dsl/words/Every.md | 0 docs2/dsl/words/Group.md | 0 docs2/dsl/words/IO.md | 0 docs2/dsl/words/Idle.md | 0 docs2/dsl/words/Inline.md | 0 docs2/dsl/words/MainThread.md | 0 docs2/dsl/words/Network.md | 0 docs2/dsl/words/Once.md | 0 docs2/dsl/words/Optional.md | 0 docs2/dsl/words/Pool.md | 0 docs2/dsl/words/Priority.md | 0 docs2/dsl/words/Shutdown.md | 0 docs2/dsl/words/Single.md | 0 docs2/dsl/words/Startup.md | 0 docs2/dsl/words/Sync.md | 0 docs2/dsl/words/TCP.md | 0 docs2/dsl/words/Trigger.md | 0 docs2/dsl/words/UDP.md | 0 docs2/dsl/words/Watchdog.md | 0 docs2/dsl/words/With.md | 0 docs2/engine/Co-Messaging.md | 1 + docs2/engine/Fission.md | 1 + docs2/engine/Fusion.md | 1 + docs2/extension/Extension.md | 3 + docs2/network/NUClearNet.md | 115 +++++++++++++++++ docs2/system/Clock.md | 0 docs2/system/Scheduler.md | 5 + docs2/system/Start Up.md | 195 +++++++++++++++++++++++++++++ docs2/system/Tracing.md | 4 + 49 files changed, 976 insertions(+) create mode 100644 docs2/Overview.md create mode 100644 docs2/components/PowerPlant.md create mode 100644 docs2/components/Reaction Tasks.md create mode 100644 docs2/components/Reactions.md create mode 100644 docs2/components/Reactor.md create mode 100644 docs2/concepts/Bind.md create mode 100644 docs2/concepts/Emit.md create mode 100644 docs2/concepts/Get.md create mode 100644 docs2/concepts/Group.md create mode 100644 docs2/concepts/Inline.md create mode 100644 docs2/concepts/Pool.md create mode 100644 docs2/concepts/Postcondition.md create mode 100644 docs2/concepts/Precondition.md create mode 100644 docs2/concepts/Serialization.md create mode 100644 docs2/concepts/Time.md create mode 100644 docs2/dsl/DSL.md create mode 100644 docs2/dsl/Emit.md create mode 100644 docs2/dsl/Extending.md create mode 100644 docs2/dsl/words/Always.md create mode 100644 docs2/dsl/words/Buffer.md create mode 100644 docs2/dsl/words/Every.md create mode 100644 docs2/dsl/words/Group.md create mode 100644 docs2/dsl/words/IO.md create mode 100644 docs2/dsl/words/Idle.md create mode 100644 docs2/dsl/words/Inline.md create mode 100644 docs2/dsl/words/MainThread.md create mode 100644 docs2/dsl/words/Network.md create mode 100644 docs2/dsl/words/Once.md create mode 100644 docs2/dsl/words/Optional.md create mode 100644 docs2/dsl/words/Pool.md create mode 100644 docs2/dsl/words/Priority.md create mode 100644 docs2/dsl/words/Shutdown.md create mode 100644 docs2/dsl/words/Single.md create mode 100644 docs2/dsl/words/Startup.md create mode 100644 docs2/dsl/words/Sync.md create mode 100644 docs2/dsl/words/TCP.md create mode 100644 docs2/dsl/words/Trigger.md create mode 100644 docs2/dsl/words/UDP.md create mode 100644 docs2/dsl/words/Watchdog.md create mode 100644 docs2/dsl/words/With.md create mode 100644 docs2/engine/Co-Messaging.md create mode 100644 docs2/engine/Fission.md create mode 100644 docs2/engine/Fusion.md create mode 100644 docs2/extension/Extension.md create mode 100644 docs2/network/NUClearNet.md create mode 100644 docs2/system/Clock.md create mode 100644 docs2/system/Scheduler.md create mode 100644 docs2/system/Start Up.md create mode 100644 docs2/system/Tracing.md diff --git a/docs2/Overview.md b/docs2/Overview.md new file mode 100644 index 00000000..e69de29b diff --git a/docs2/components/PowerPlant.md b/docs2/components/PowerPlant.md new file mode 100644 index 00000000..e69de29b diff --git a/docs2/components/Reaction Tasks.md b/docs2/components/Reaction Tasks.md new file mode 100644 index 00000000..e9903fcb --- /dev/null +++ b/docs2/components/Reaction Tasks.md @@ -0,0 +1,6 @@ +- What is a reaction task +- How are they created +- When and where will they run + - Inline + - Thread Pool + - Priority diff --git a/docs2/components/Reactions.md b/docs2/components/Reactions.md new file mode 100644 index 00000000..25b1f5cb --- /dev/null +++ b/docs2/components/Reactions.md @@ -0,0 +1,10 @@ +- What is a reaction +- What do they do +- How to create a reaction +- When a reaction will run + + - Has the required data + - Has a bind event trigger + +- Bind and unbind +- Disable diff --git a/docs2/components/Reactor.md b/docs2/components/Reactor.md new file mode 100644 index 00000000..07db6f31 --- /dev/null +++ b/docs2/components/Reactor.md @@ -0,0 +1,2 @@ +- What is a reactor +- What they should do diff --git a/docs2/concepts/Bind.md b/docs2/concepts/Bind.md new file mode 100644 index 00000000..79557e87 --- /dev/null +++ b/docs2/concepts/Bind.md @@ -0,0 +1,41 @@ +# Bind + +## Overview + +In NUClear, the "bind" concept is a fundamental mechanism that allows functions to be executed when specific conditions are met. This is achieved by associating (or binding) functions to specific types or events. When the specified type or event occurs, the bound function is triggered and executed. This concept is essential for creating reactive systems where actions are performed in response to changes or events in the system. + +## How Bind Works + +The bind concept is implemented through various DSL (Domain Specific Language) words that define the conditions under which functions should be executed. These DSL words encapsulate the logic for binding functions to specific types or events, making it easier to create complex reactive behaviors. + +Example: Trigger DSL Word +One example of a DSL word that implements the bind concept is the Trigger word. The Trigger word allows reactions to be triggered based on the emission of a specific type of data. + +Usage +Here are some examples of how the Trigger DSL word can be used to bind functions to events: + +Binding to a Specific Data Type + +```cpp +on>().then([](const DataType& data) { + // Handle the received data +}); +``` + +In this example, the function is bound to the emission of DataType. When data of this type is emitted, the function is triggered and executed. + +Binding with Runtime Arguments +In this example, the function is bound to the emission of DataType and can also take runtime arguments. This allows for more dynamic and flexible reactions based on runtime conditions. + +```cpp +on>(runtime_argument).then([](const DataType& data) { + // Handle the received data with the runtime argument +}); +``` + +## Implementation + +The Trigger DSL word is implemented in the src/dsl/word/Trigger.hpp file. It provides methods for binding functions to specific data types, including: + +bind: Binds the function to the specified data type. +These methods encapsulate the logic for handling data emissions and provide a simple interface for binding functions to these events. diff --git a/docs2/concepts/Emit.md b/docs2/concepts/Emit.md new file mode 100644 index 00000000..9c1e4e6c --- /dev/null +++ b/docs2/concepts/Emit.md @@ -0,0 +1,7 @@ +- Talk about emit scopes +- Talk about how emit works +- Talk about multiple scopes + +- Talk about emit being unique pointers so it can't be modified after emit + +- Talk about emit shared and when/how it should be used diff --git a/docs2/concepts/Get.md b/docs2/concepts/Get.md new file mode 100644 index 00000000..b2ea6595 --- /dev/null +++ b/docs2/concepts/Get.md @@ -0,0 +1,117 @@ +# Get + +The `Get` concept in NUClear's DSL (Domain Specific Language) is used to retrieve data required for a reaction. +It allows a reaction to specify dependencies on certain types of data, which will be provided when the reaction is executed. + +Each of the items returned by a `get` call must be `truthy` for the reaction to execute, that is they must evaluate to `true` when cast to a boolean. +If any of the items are `falsy`, the reaction will not execute. + +When these arguments are passed into the reaction function, not all of them need to be present. +If an argument is not requested by the reaction function, it not be passed in. +However, even if the argument is not requested, the get call will still occur and the result must be `truthy`. + +## How does Get work + +The `Get` concept works by defining a `get` function within a DSL word. +This function is called when the reaction is executed, and it retrieves the necessary data. +The retrieved data is then passed to the reaction function as arguments. + +Typically implementers of the `Get` concept will use a data store to retrieve the data. +Or they may use a thread local variable that is set by another part of the system before the reaction is created. + +The type which is requested by a reaction can either be the type returned by the `get` or the type which results from dereferencing the type returned by the `get`. +For example, if the `get` returns a `std::shared_ptr`, the type requested by the reaction can be `T`. + +## Example of a DSL word that implements Get + +One example of a DSL word that implements the Get concept is the With word. +The With word retrieves the data of a specific type from NUClear's cache. + +```cpp +namespace NUClear { +namespace dsl { + namespace word { + template + struct With { + public: + template + static std::shared_ptr get() { + return store::ThreadStore>::value == nullptr + ? store::DataStore::get() + : *store::ThreadStore>::value; + } + }; + } +} +} +``` + +## Optional Parameters + +The `Optional` keyword in NUClear's DSL (Domain Specific Language) is used to mark certain fields as optional in a `get` call. This allows the reaction function to execute even if some of the optional data is missing. + +When using the `Optional` keyword, the return type of the `get` call is always "truthy", meaning it evaluates to `true` when cast to a boolean. This ensures that the reaction can execute even when it is missing some data. + +By marking certain fields as optional, the reaction function can choose to use only the relevant arguments and ignore the rest. This is particularly useful when dealing with complex data structures where only a subset of the data is needed. + +The `Optional` keyword ensures that the reaction function can still execute and perform its intended functionality, even if some of the optional data is not present. + +### Example of Optional Parameters + +In this example, we will demonstrate how to use the `Optional` keyword in conjunction with the `With` word in NUClear's DSL. + +Let's say we have a reaction that needs to retrieve two types of data: `DataA` and `DataB`. However, `DataB` is optional and the reaction should still execute even if it is not present. + +```cpp +on, Optional>>().then([this](const DataA& a, const std::shared_ptr& b) { + // b will be null if DataB has not been emitted +}); +``` + +## Transients in Get + +The concept of transients in the context of `Get` refers to data that is only temporarily available. +When a type is marked as transient, the system caches the last copy of the data provided. +If no new data is available, the cached data is used instead. + +This commonly occurs when the data is only available while the associated reaction is executing. +For example, if you were to use `on>` to receive a UDP packet, the data would be transient. +Then if the reaction were triggered by a `Trigger` message, the previous data for the UDP packet would be provided. + +### Flagging types as transient + +To mark a type as transient, you need to define a trait that specifies that the type is transient. + +For example, consider a type `TransientMessage` that represents a transient message. +The `TransientMessage` type is an example of a transient type. +It is marked as transient using the `is_transient` trait. + +```cpp +namespace NUClear { +namespace dsl { + namespace trait { + template <> + struct is_transient : std::true_type {}; + } +} +} +``` + +In the reaction, the `Get` for the TransientMessage retrieves the data, and if it is not available, it returns an empty message. +When the reaction is executed, the last known value of the TransientMessage is used if the data is not freshly available. +If no data has been received yet, the reaction will not execute. + +```cpp +struct TransientGetter : NUClear::dsl::operation::TypeBind { + template + static TransientMessage get(NUClear::threading::ReactionTask& task) { + auto raw = NUClear::dsl::operation::CacheGet::get(task); + if (raw == nullptr) { + return {}; + } + return *raw; + } +}; +``` + +This ensures that the reaction can still execute even if the transient data is not freshly available, by using the last known value. diff --git a/docs2/concepts/Group.md b/docs2/concepts/Group.md new file mode 100644 index 00000000..cd6e830c --- /dev/null +++ b/docs2/concepts/Group.md @@ -0,0 +1,59 @@ +# Group + +The `Group` concept in NUClear is designed to ensure that across multiple threads, the concurrency of tasks flagged with a specific group is controlled. +This is particularly useful in scenarios where certain tasks should not run concurrently to avoid race conditions or to manage shared resources efficiently. + +## How it Works + +When a task is flagged with a `Group`, it is assigned to a specific group identified by a group type and a maximum concurrency level. +The Group concept ensures that no more than the specified number of tasks from the same group can run concurrently. +This is achieved through a combination of task queuing and locking mechanisms. + +## Example: Sync + +The `Sync` is a special case of the `Group` concept where the concurrency level is set to one. +This means that across multiple threads and pools, only a single reaction will be executing at the same time for the specified group. +This is particularly useful for ensuring that tasks that modify shared state do not run concurrently, thereby avoiding race conditions. + +## Usage + +To use the `Group` concept, you can specify it in the DSL of a reaction. For example: + +```cpp +struct MyGroup { + static constexpr int max_concurrency = 2; +}; + +on, Group>() +``` + +In this example, at most two tasks from the `MyGroup` group can run concurrently. + +For the `Sync` case: + +```cpp +on, Sync>() +``` + +In this example, only one task from the MySyncGroup group can run at any given time. + +## Implementation Details + +The `Group` concept is implemented using a combination of descriptors and locks. +Each group has a descriptor that specifies its name and maximum concurrency level. +When a task is flagged with a group, it is added to a queue managed by the group. +The group uses a locking mechanism to ensure that no more than the specified number of tasks are running concurrently. + +### Group Descriptor + +The GroupDescriptor struct holds the name and maximum concurrency level of a group: + +```cpp +struct GroupDescriptor { + GroupDescriptor(std::string name, const int& thread_count) + : name(std::move(name)), thread_count(thread_count) {} + + std::string name; + int thread_count; +}; +``` diff --git a/docs2/concepts/Inline.md b/docs2/concepts/Inline.md new file mode 100644 index 00000000..25df13e2 --- /dev/null +++ b/docs2/concepts/Inline.md @@ -0,0 +1,77 @@ +# Inline + +The `Inline` concept in NUClear allows you to control whether a reaction should be executed inline or in its target thread pool. This can be useful for optimizing performance and ensuring that certain tasks are executed in a specific manner. + +## What is an inline task + +An inline task is a reaction that is executed in the same thread as the emitter, rather than being scheduled in a separate thread pool. This can reduce the overhead of context switching and improve performance for lightweight tasks. + +## How are they created + +Inline tasks are created using the `emit` function or by specifying the `Inline::ALWAYS` keyword in the reaction definition. + +## When will a task run inline + +A task will run inline based on the following conditions: + +### Inline emit + +Using `emit` will attempt to run the emitted task inline. However, this can be overridden by the reaction's inline level. + +### Inline::NEVER + +The `Inline::NEVER` keyword can be used in the reaction definition to ensure that the reaction is never executed inline, even if it is emitted with `Scope::INLINE`. + +```cpp +on, Inline::NEVER>().then([](const SimpleMessage& message) { + // This reaction will never run inline +}); +``` + +### Inline::ALWAYS + +The `Inline::ALWAYS` keyword can be used in the reaction definition to ensure that the reaction is always executed inline, even if it is not emitted with `Scope::INLINE`. + +```cpp +on, Inline::ALWAYS>().then([](const SimpleMessage& message) { + // This reaction will always run inline +}); +``` + +### Groups + +Inlining will never happen if the group lock prevents it from being inlined. This ensures that tasks that require synchronization are not executed inline, which could lead to race conditions or other concurrency issues. + +## Example + +Here is an example of how to use the `Inline` concept in a NUClear reactor: + +```cpp +class TestReactor : public NUClear::Reactor { +public: + TestReactor(std::unique_ptr environment) : Reactor(std::move(environment)) { + + on, Inline::ALWAYS>().then([](const SimpleMessage& message) { + // This reaction will always run inline + }); + + on, Inline::NEVER>().then([](const SimpleMessage& message) { + // This reaction will never run inline + }); + + on>().then([](const SimpleMessage& message) { + // This reaction will run inline based on the emit scope + }); + + emit(std::make_unique("Inline Message")); + } +}; +``` + +In this example, the first reaction will always run inline, the second reaction will never run inline, and the third reaction will run inline based on the emit scope. + +## Summary + +The `Inline` concept provides fine-grained control over how reactions are executed in NUClear. By using `Inline::NEVER` and `Inline::ALWAYS`, you can ensure that reactions are executed in the desired manner. Additionally, the `emit` function allows you to emit tasks inline, but this can be overridden by the reaction's inline level. Finally, inlining will never happen if the group lock prevents it from being inlined, ensuring that tasks requiring synchronization are not executed inline. + +This documentation provides a comprehensive overview of the `Inline` concept, including how it works, how it can be controlled, and examples of its usage. diff --git a/docs2/concepts/Pool.md b/docs2/concepts/Pool.md new file mode 100644 index 00000000..1bdf304a --- /dev/null +++ b/docs2/concepts/Pool.md @@ -0,0 +1,99 @@ +# Pool + +## What is a Pool + +A Pool in NUClear is a thread pool that manages the execution of tasks. It allows for concurrent execution of tasks across multiple threads, improving the efficiency and performance of your application. + +## How does a Pool work + +A Pool works by maintaining a set of threads that are used to execute tasks. When a task is submitted to the Pool, it is queued and executed by one of the available threads. If no threads are available, the task waits until a thread becomes free. + +## How to make a Pool + +To create a custom thread pool in NUClear, you use the `Pool` DSL. +The `Type` struct defines the attributes of the thread pool, such as the number of threads, whether the pool participates in idle, and whether it persists after shutdown. + +### Pool Concepts + +#### Pool Size + +The size of the thread pool is determined by the `thread_count` attribute in the `Type` struct. +This specifies the number of threads that will be created for the pool. + +Example: + +```cpp +struct MyPool { + static constexpr int thread_count = 4; +}; +``` + +#### Persistent Pool + +The persistent pool concept, referred to as "ignore shutdown" in the code, is controlled by the `continue_on_shutdown` attribute. +If set to `true`, the pool will continue to accept tasks even after the system is shutting down. + +Example: + +```cpp +struct PersistentPool { + static constexpr bool continue_on_shutdown = true; +}; +``` + +#### Idle Participation + +The `counts_for_idle` attribute determines whether the threads in the pool count towards the system's idle state. +If set to `true`, the pool's threads will be considered when determining if the system is idle. + +Example: + +```cpp +struct IdlePool { + static constexpr bool counts_for_idle = true; +}; +``` + +### Default Thread Pool + +The default thread pool is created with a thread count specified at runtime when the `PowerPlant` is instantiated. +This allows for flexibility in configuring the number of threads based on the application's needs. + +Example: + +```cpp +NUClear::PowerPlant powerplant(config, argc, argv); +``` + +### MainThread Pool + +The `MainThread` pool uses the thread that started the `PowerPlant`, which is typically the main thread. This is particularly useful in scenarios involving computer graphics, such as OpenGL, where certain functions must be executed on the main thread. + +Example: + +```cpp +on, MainThread>() { + // This code will run on the main thread +} +``` + +### Example Usage + +Here is an example of how to create and use a custom thread pool in NUClear: + +```cpp +struct CustomPool { + static constexpr int thread_count = 2; + static constexpr bool counts_for_idle = false; + static constexpr bool continue_on_shutdown = true; +}; + +on, Pool>() { + // This code will run in the custom thread pool +} +``` + +In this example, a custom thread pool named `CustomPool` is created with 2 threads, does not count towards idle, and continues to accept tasks after shutdown. +The `on, Pool>()` syntax specifies that the reaction should run in this custom thread pool. + +By understanding and utilizing these concepts, you can effectively manage the concurrency and performance of your NUClear application. diff --git a/docs2/concepts/Postcondition.md b/docs2/concepts/Postcondition.md new file mode 100644 index 00000000..44f5af52 --- /dev/null +++ b/docs2/concepts/Postcondition.md @@ -0,0 +1,60 @@ +# Postcondition + +A **postcondition** in NUClear is a condition that is executed after a reaction has run. +Postconditions are used to perform cleanup or additional actions once the main reaction logic has completed. + +## How it Works + +Postconditions are defined as static methods within the DSL (Domain Specific Language) words and are executed after the reaction's main callback has finished. They are useful for tasks such as unbinding reactions, logging, or other post-reaction activities. + +Postconditions will only run if the reaction itself runs. If a `Precondition` fails or a `Get` returns a falsey value, the postconditions will not be executed. + +## Usage + +To use a postcondition, you need to include it in the DSL chain when defining a reaction. The postcondition will be executed each time the reaction's main callback completes. + +### Example: Using the `Once` Postcondition + +The `Once` postcondition ensures that a reaction only runs once by unbinding the reaction after it has executed. + +#### Code Example + +```cpp +#include + +struct TestReactor : public NUClear::Reactor { + struct SimpleMessage { + SimpleMessage(int run) : run(run) {} + int run = 0; + }; + + TestReactor(std::unique_ptr environment) : Reactor(std::move(environment)) { + + // This reaction will only run once + on, Once>().then([this](const SimpleMessage& msg) { + std::cout << "Once Trigger executed " << msg.run << std::endl; + }); + + // This reaction will run multiple times + on>().then([this](const SimpleMessage& msg) { + std::cout << "Normal Trigger Executed " << msg.run << std::endl; + if (msg.run < 10) { + emit(std::make_unique(msg.run + 1)); + } + else { + powerplant.shutdown(); + } + }); + + on().then([this] { + emit(std::make_unique(0)); + }); + } +}; +``` + +In this example, the `Once` postcondition is used to ensure that the reaction to `SimpleMessage` only runs once. After the reaction runs, the `Once` postcondition unbinds the reaction, preventing it from running again. + +## Summary + +Postconditions in NUClear provide a way to perform additional actions after a reaction has executed. By using postconditions like `Once`, you can ensure that reactions only run under certain conditions, improving the control and flexibility of your application. Remember that postconditions will only run if the reaction itself runs, so if a `Precondition` fails or a `Get` returns a falsey value, the postconditions will not be executed. diff --git a/docs2/concepts/Precondition.md b/docs2/concepts/Precondition.md new file mode 100644 index 00000000..1e6a907b --- /dev/null +++ b/docs2/concepts/Precondition.md @@ -0,0 +1,65 @@ +# Precondition + +A **precondition** in NUClear is a condition that must be met for a reaction to execute. If the precondition evaluates to `true`, the reaction will proceed as normal. +If it evaluates to `false`, the reaction will be skipped. + +## How it Works + +Preconditions are used to control the execution of reactions based on specific criteria. +They are defined as static methods within the DSL (Domain Specific Language) words and are checked before the reaction is executed. If any precondition returns `false`, the reaction will not run. + +## Usage + +To use a precondition, you need to include it in the DSL chain when defining a reaction. The precondition will be evaluated each time the reaction is triggered. + +### Example: Using the `Single` Precondition + +The `Single` precondition ensures that only one instance of the associated reaction can execute at any given time. +If the reaction is triggered while an existing task for this reaction is either in the queue or still executing, the new task request will be ignored. + +#### Code Example + +```cpp +#include + +struct TestReactor : public NUClear::Reactor { + TestReactor(std::unique_ptr environment) : Reactor(std::move(environment)) { + + on, Single>().then([this] { + // This code will only run if no other instance of this reaction is currently running + std::cout << "Single reaction executed" << std::endl; + }); + } +}; +``` + +In this example, the `Single` precondition is used to ensure that the reaction to `SomeEvent` only runs if no other instance of the same reaction is currently running. + +### Defining Custom Preconditions + +You can also define your own preconditions by creating a struct with a static `precondition` method. This method should return a `bool` indicating whether the reaction should run. + +#### Custom Precondition Example + +```cpp +struct CustomPrecondition { + template + static bool precondition(NUClear::threading::ReactionTask& task) { + // Custom logic to determine if the reaction should run + return some_condition_is_met(); + } +}; + +// Usage in a reaction +on, CustomPrecondition>().then([this] { + // This code will only run if CustomPrecondition returns true + std::cout << "Custom precondition met, reaction executed" << std::endl; +}); +``` + +In this example, `CustomPrecondition` is a user-defined precondition that checks a custom condition before allowing the reaction to execute. + +## Summary + +Preconditions in NUClear provide a powerful way to control the execution of reactions based on specific criteria. +By using preconditions like `Single`, you can ensure that reactions only run under certain conditions, improving the robustness and predictability of your application. diff --git a/docs2/concepts/Serialization.md b/docs2/concepts/Serialization.md new file mode 100644 index 00000000..9ef3d400 --- /dev/null +++ b/docs2/concepts/Serialization.md @@ -0,0 +1,46 @@ +### Serialization in the NUClear System + +Serialization in the NUClear system is designed to convert data into a format that can be easily stored or transmitted and then reconstructed later. This is particularly useful for sending data over a network or saving it to a file. NUClear provides built-in support for serializing various types of data, including Plain Old Data (POD) types and iterables of POD types. + +#### How Serialization Types Work + +NUClear uses a template-based approach to serialization. The primary template is `Serialise`, where `T` is the type to be serialized, and `Check` is a type trait used to specialize the serialization process for different types. The `Serialise` template provides three static methods that must be implemented for each type: + +- `static std::vector serialise(const T& in)`: Converts the object `in` into a vector of bytes. +- `static T deserialise(const std::vector& in)`: Reconstructs an object of type `T` from a vector of bytes. +- `static uint64_t hash()`: Returns a hash value that uniquely identifies the type `T`. + +#### Built-in Serialization for POD Types + +NUClear provides built-in serialization for POD types, which are simple data types like integers, floats, and plain structs. The serialization process for these types is straightforward: + +1. **Serialization**: The data is converted into a sequence of bytes, taking into account the endianness and alignment requirements of the target system. +2. **Deserialization**: The byte sequence is converted back into the original data type. + +For example, to serialize an integer, the system converts the integer into its binary representation and stores it in a byte vector. When deserializing, the binary data is read from the byte vector and converted back into an integer. + +#### Serialization for Iterables of POD Types + +NUClear also supports serialization for iterables of POD types, such as arrays, vectors, and lists. The process involves serializing each element of the iterable individually and then combining the results into a single byte vector. During deserialization, the byte vector is split back into individual elements, which are then deserialized into their original types. + +#### Using Serialization in NUClear + +To use serialization in NUClear, you typically don't need to interact with the `Serialise` template directly. Instead, you use higher-level constructs provided by the NUClear framework, such as the `Network` or `UDP` DSL words, which internally handle the serialization and deserialization of data. + +For example, to send a message over the network, you might use the `Network` DSL word: + +```cpp +on>>().then([this](const MyMessage& msg) { + // Process the received message +}); +``` + +In this example, `MyMessage` is a user-defined type that has been specialized for serialization. The `Network` DSL word automatically serializes the message before sending it and deserializes it upon receipt. + +#### Custom Serialization + +If you need to serialize a custom type that is not a POD type or an iterable of POD types, you can specialize the `Serialise` template for your type. This involves implementing the three static methods (`serialise`, `deserialise`, and `hash`) to define how your type should be converted to and from a byte vector. + +### Conclusion + +Serialization in the NUClear system is a powerful feature that allows you to easily convert data into a format suitable for storage or transmission. With built-in support for POD types and iterables of POD types, as well as the ability to define custom serialization for more complex types, NUClear provides a flexible and efficient serialization mechanism that integrates seamlessly with its other features. diff --git a/docs2/concepts/Time.md b/docs2/concepts/Time.md new file mode 100644 index 00000000..96986662 --- /dev/null +++ b/docs2/concepts/Time.md @@ -0,0 +1,56 @@ +### Understanding Time in the NUClear System + +The NUClear system provides a robust and flexible way to manage time within your application. This document will guide you through the various time-related features available in NUClear, ensuring you understand how to use them effectively. + +#### NUClear::clock Type + +The `NUClear::clock` type is a specialized clock used within the NUClear system. It is based on the `std::chrono::system_clock` by default, but it can be overridden to use other clock types if needed. The `NUClear::clock` provides several functionalities: + +- **Current Time**: You can get the current time using `NUClear::clock::now()`. +- **Adjusting the Clock**: The clock can be adjusted using `NUClear::clock::adjust_clock`. +- **Setting the Clock**: You can set the clock to a specific time using `NUClear::clock::set_clock`. +- **Real-Time Factor (RTF)**: The clock supports a real-time factor, which allows you to control the speed at which time progresses. + +#### TimeTravel in the NUClear System + +TimeTravel is a powerful feature in NUClear that allows you to manipulate the system clock and the rate at which time passes. This is particularly useful for testing and simulation purposes. The `NUClear::message::TimeTravel` struct provides the following actions: + +- **RELATIVE**: Adjusts the clock and moves all chrono tasks with it. +- **ABSOLUTE**: Adjusts the clock to a target time and leaves chrono tasks where they are. +- **NEAREST**: Adjusts the clock to as close to the target as possible without skipping any chrono tasks. + +Example usage: + +```cpp +emit(std::make_unique( + NUClear::clock::time_point(adjustment), rtf, action)); +``` + +#### ChronoTasks + +ChronoTasks are tasks scheduled to run at specific times or intervals. They are managed by the `ChronoController` and can be influenced by TimeTravel. There are two primary ways to use ChronoTasks: + +1. **Emit Delay**: Schedule a task to run after a specific delay. +2. **On Every**: Schedule a task to run at regular intervals. + +Example of scheduling a ChronoTask: + +```cpp +emit(std::make_unique( + [](NUClear::clock::time_point&) { + // Task code here + return false; // Return true to reschedule + }, + NUClear::clock::now() + std::chrono::seconds(10), // Schedule 10 seconds from now + 1)); +``` + +TimeTravel influences ChronoTasks by adjusting their scheduled times based on the selected action (RELATIVE, ABSOLUTE, NEAREST). + +#### Usage Clock + +The usage clock records CPU thread time, providing insights into how much CPU time is being consumed by different parts of your application. This can be useful for performance monitoring and optimization. + +### Conclusion + +Understanding how time works in the NUClear system allows you to leverage its powerful features for scheduling tasks, simulating different time scenarios, and monitoring performance. By using `NUClear::clock`, TimeTravel, and ChronoTasks, you can create flexible and efficient applications that handle time-based operations with ease. diff --git a/docs2/dsl/DSL.md b/docs2/dsl/DSL.md new file mode 100644 index 00000000..fee06381 --- /dev/null +++ b/docs2/dsl/DSL.md @@ -0,0 +1,5 @@ +- Talk about the DSL +- Show some examples +- Explain how things interact +- Talk about how words implement concepts +- Talk about how concepts are implemented in code diff --git a/docs2/dsl/Emit.md b/docs2/dsl/Emit.md new file mode 100644 index 00000000..e69de29b diff --git a/docs2/dsl/Extending.md b/docs2/dsl/Extending.md new file mode 100644 index 00000000..e69de29b diff --git a/docs2/dsl/words/Always.md b/docs2/dsl/words/Always.md new file mode 100644 index 00000000..4a39af26 --- /dev/null +++ b/docs2/dsl/words/Always.md @@ -0,0 +1 @@ +- Mention that shutdown and always is broken or fix it diff --git a/docs2/dsl/words/Buffer.md b/docs2/dsl/words/Buffer.md new file mode 100644 index 00000000..e69de29b diff --git a/docs2/dsl/words/Every.md b/docs2/dsl/words/Every.md new file mode 100644 index 00000000..e69de29b diff --git a/docs2/dsl/words/Group.md b/docs2/dsl/words/Group.md new file mode 100644 index 00000000..e69de29b diff --git a/docs2/dsl/words/IO.md b/docs2/dsl/words/IO.md new file mode 100644 index 00000000..e69de29b diff --git a/docs2/dsl/words/Idle.md b/docs2/dsl/words/Idle.md new file mode 100644 index 00000000..e69de29b diff --git a/docs2/dsl/words/Inline.md b/docs2/dsl/words/Inline.md new file mode 100644 index 00000000..e69de29b diff --git a/docs2/dsl/words/MainThread.md b/docs2/dsl/words/MainThread.md new file mode 100644 index 00000000..e69de29b diff --git a/docs2/dsl/words/Network.md b/docs2/dsl/words/Network.md new file mode 100644 index 00000000..e69de29b diff --git a/docs2/dsl/words/Once.md b/docs2/dsl/words/Once.md new file mode 100644 index 00000000..e69de29b diff --git a/docs2/dsl/words/Optional.md b/docs2/dsl/words/Optional.md new file mode 100644 index 00000000..e69de29b diff --git a/docs2/dsl/words/Pool.md b/docs2/dsl/words/Pool.md new file mode 100644 index 00000000..e69de29b diff --git a/docs2/dsl/words/Priority.md b/docs2/dsl/words/Priority.md new file mode 100644 index 00000000..e69de29b diff --git a/docs2/dsl/words/Shutdown.md b/docs2/dsl/words/Shutdown.md new file mode 100644 index 00000000..e69de29b diff --git a/docs2/dsl/words/Single.md b/docs2/dsl/words/Single.md new file mode 100644 index 00000000..e69de29b diff --git a/docs2/dsl/words/Startup.md b/docs2/dsl/words/Startup.md new file mode 100644 index 00000000..e69de29b diff --git a/docs2/dsl/words/Sync.md b/docs2/dsl/words/Sync.md new file mode 100644 index 00000000..e69de29b diff --git a/docs2/dsl/words/TCP.md b/docs2/dsl/words/TCP.md new file mode 100644 index 00000000..e69de29b diff --git a/docs2/dsl/words/Trigger.md b/docs2/dsl/words/Trigger.md new file mode 100644 index 00000000..e69de29b diff --git a/docs2/dsl/words/UDP.md b/docs2/dsl/words/UDP.md new file mode 100644 index 00000000..e69de29b diff --git a/docs2/dsl/words/Watchdog.md b/docs2/dsl/words/Watchdog.md new file mode 100644 index 00000000..e69de29b diff --git a/docs2/dsl/words/With.md b/docs2/dsl/words/With.md new file mode 100644 index 00000000..e69de29b diff --git a/docs2/engine/Co-Messaging.md b/docs2/engine/Co-Messaging.md new file mode 100644 index 00000000..cb158914 --- /dev/null +++ b/docs2/engine/Co-Messaging.md @@ -0,0 +1 @@ +Talk about co-messaging and how it works in NUClear diff --git a/docs2/engine/Fission.md b/docs2/engine/Fission.md new file mode 100644 index 00000000..35573f71 --- /dev/null +++ b/docs2/engine/Fission.md @@ -0,0 +1 @@ +Talk about the NUClear fission engine. diff --git a/docs2/engine/Fusion.md b/docs2/engine/Fusion.md new file mode 100644 index 00000000..476fec65 --- /dev/null +++ b/docs2/engine/Fusion.md @@ -0,0 +1 @@ +Talk about the NUClear fusion engine and how it works diff --git a/docs2/extension/Extension.md b/docs2/extension/Extension.md new file mode 100644 index 00000000..d6f26b16 --- /dev/null +++ b/docs2/extension/Extension.md @@ -0,0 +1,3 @@ +- Talk about how NUClear is built on extensions +- Show some examples of extensions +- Show how new extensions can be created diff --git a/docs2/network/NUClearNet.md b/docs2/network/NUClearNet.md new file mode 100644 index 00000000..3edf966f --- /dev/null +++ b/docs2/network/NUClearNet.md @@ -0,0 +1,115 @@ +# NUClear Network Documentation + +## Overview + +The NUClear network is a UDP-based mesh network designed for efficient communication between nodes. It supports both reliable and unreliable message delivery, multicast, and unicast communication. This document provides an overview of how the NUClear network operates, including its key features and usage. + +## UDP Basis of the Network + +The NUClear network uses UDP (User Datagram Protocol) as its underlying transport mechanism. UDP is chosen for its low-latency and connectionless nature, making it suitable for real-time communication. However, UDP does not guarantee message delivery, order, or integrity, so NUClear implements additional mechanisms to handle these aspects. + +## Mesh Network Formation + +The NUClear network forms a mesh network where each node can communicate directly with any other node. This decentralized approach ensures robustness and scalability. Nodes discover each other through periodic announce packets and maintain connections by monitoring activity. + +## Announce Packets + +### Announce Rates + +Nodes periodically send announce packets to inform others of their presence. The default announce interval is 500 milliseconds. These packets contain the node's name and other metadata. + +### Unicast Responses from New Peers + +When a node receives an announce packet from a new peer, it responds with a unicast announce packet to establish a bidirectional connection. This ensures that both nodes are aware of each other and can start communicating. + +## Packet Targets + +### Groups + +Nodes can be organized into groups for targeted communication. A group is identified by a unique name, and nodes can join multiple groups. Messages can be sent to specific groups, ensuring that only relevant nodes receive them. + +### All Group + +The "all" group is a special group that includes all nodes in the network. Messages sent to the "all" group are broadcast to every node, making it useful for announcements and global messages. + +## Sequence Diagrams + +### New Peer Joins the Network + +```mermaid +sequenceDiagram + participant Peer A + participant Peer B + Peer A->>All: Announce Packet + All->>Peer B: Announce Packet + Peer B->>Peer A: Unicast Announce Packet + Peer A->>Peer B: Unicast Announce Packet +``` + +### Peer Leaves the Network + +```mermaid +sequenceDiagram + participant Peer A + participant Peer B + Peer A->>All: Leave Packet + All->>Peer B: Leave Packet + Peer B->>Peer A: Acknowledgement (optional) +``` + +### Reliable Packet (Single Packet) + +```mermaid +sequenceDiagram + participant Peer A + participant Peer B + Peer A->>Peer B: Data Packet (Reliable) + Peer B->>Peer A: ACK Packet +``` + +### Reliable Packet (Split) + +```mermaid +sequenceDiagram + participant Peer A + participant Peer B + Peer A->>Peer B: Data Packet 1 (Reliable) + Peer A->>Peer B: Data Packet 2 (Reliable) + Peer B->>Peer A: ACK Packet for Packet 1 + Peer B->>Peer A: ACK Packet for Packet 2 +``` + +### Unreliable Packet (Single Packet) + +```mermaid +sequenceDiagram + participant Peer A + participant Peer B + Peer A->>Peer B: Data Packet (Unreliable) +``` + +### Unreliable Packet (Split) + +```mermaid +sequenceDiagram + participant Peer A + participant Peer B + Peer A->>Peer B: Data Packet 1 (Unreliable) + Peer A->>Peer B: Data Packet 2 (Unreliable) +``` + +### Packet to a Group of Peers + +```mermaid +sequenceDiagram + participant Peer A + participant Group + Peer A->>Group: Data Packet (Group) + Group->>Peer B: Data Packet + Group->>Peer C: Data Packet + Group->>Peer D: Data Packet +``` + +## Conclusion + +The NUClear network provides a robust and flexible communication framework for distributed systems. By leveraging UDP and implementing additional reliability mechanisms, it ensures efficient and reliable message delivery. Understanding the network's operation and features will help users effectively utilize it in their applications. diff --git a/docs2/system/Clock.md b/docs2/system/Clock.md new file mode 100644 index 00000000..e69de29b diff --git a/docs2/system/Scheduler.md b/docs2/system/Scheduler.md new file mode 100644 index 00000000..56f745fb --- /dev/null +++ b/docs2/system/Scheduler.md @@ -0,0 +1,5 @@ +- What is the scheduler +- How does it decide what to run +- When are tasks submitted +- Are they accepted +- Scheduler flows diff --git a/docs2/system/Start Up.md b/docs2/system/Start Up.md new file mode 100644 index 00000000..3fcade53 --- /dev/null +++ b/docs2/system/Start Up.md @@ -0,0 +1,195 @@ +TrentHouliston: @workspace Please prepare a document describing the progression of execution stages in NUClear works. This is for documentation for a user of the NUClear system so describe it from the perspective of someone using the system rather than for implementers of the system. Ensure you have sufficient detail so that a new user of the system will understand what feature are available to them and will know how to use them. + +Topics that should be covered are as follows, but also include any other topics that you think are important to cover for describing how NUClear works. + +- Single threaded main sequence where: + - `PowerPlant` is created + - `Reactors are installed +- Powerplant Start Up + - Single threaded main thread + - Startup messages are emitted Inline +- Multithreaded execution phase + - system operates an normal + - tasks executed as normal on thread pools +- Shutdown sequence + - shutdown message emitted at lowest priority + - new non inline tasks blocked on normal pools + - all tasks still allowed on persistent pools + - main thread waits for non persistent pools to finish all tasks and the threads terminate + - persistent pool threads are told to shutdown once they exhaust all tasks + - main thread returns from powerplant.start(); + +Additionally please make diagrams using the mermaid diagram format to illustrate the concepts you are explaining. +Specifically ensure you have a sequence diagram showing the interaction between the main thread and the thread pools at each stage of execution. + +Workspace: # NUClear System Execution Stages + +This document describes the progression of execution stages in the NUClear system from the perspective of a user. It provides sufficient detail to help new users understand the available features and how to use them effectively. + +## Single Threaded Main Sequence + +### PowerPlant Creation + +The first step in setting up a NUClear system is creating a [`PowerPlant`](command:_github.copilot.openSymbolFromReferences?%5B%22PowerPlant%22%2C%5B%7B%22uri%22%3A%7B%22%24mid%22%3A1%2C%22fsPath%22%3A%22%2FUsers%2Ftrenthouliston%2FCode%2FNUClear%2Fdocs%2Fcomponents.rst%22%2C%22external%22%3A%22file%3A%2F%2F%2FUsers%2Ftrenthouliston%2FCode%2FNUClear%2Fdocs%2Fcomponents.rst%22%2C%22path%22%3A%22%2FUsers%2Ftrenthouliston%2FCode%2FNUClear%2Fdocs%2Fcomponents.rst%22%2C%22scheme%22%3A%22file%22%7D%2C%22pos%22%3A%7B%22line%22%3A3%2C%22character%22%3A61%7D%7D%2C%7B%22uri%22%3A%7B%22%24mid%22%3A1%2C%22fsPath%22%3A%22%2FUsers%2Ftrenthouliston%2FCode%2FNUClear%2Fdocs%2Fstartup.rst%22%2C%22external%22%3A%22file%3A%2F%2F%2FUsers%2Ftrenthouliston%2FCode%2FNUClear%2Fdocs%2Fstartup.rst%22%2C%22path%22%3A%22%2FUsers%2Ftrenthouliston%2FCode%2FNUClear%2Fdocs%2Fstartup.rst%22%2C%22scheme%22%3A%22file%22%7D%2C%22pos%22%3A%7B%22line%22%3A16%2C%22character%22%3A12%7D%7D%2C%7B%22uri%22%3A%7B%22%24mid%22%3A1%2C%22fsPath%22%3A%22%2FUsers%2Ftrenthouliston%2FCode%2FNUClear%2Fdocs%2Fstartup.rst%22%2C%22external%22%3A%22file%3A%2F%2F%2FUsers%2Ftrenthouliston%2FCode%2FNUClear%2Fdocs%2Fstartup.rst%22%2C%22path%22%3A%22%2FUsers%2Ftrenthouliston%2FCode%2FNUClear%2Fdocs%2Fstartup.rst%22%2C%22scheme%22%3A%22file%22%7D%2C%22pos%22%3A%7B%22line%22%3A104%2C%22character%22%3A10%7D%7D%2C%7B%22uri%22%3A%7B%22%24mid%22%3A1%2C%22fsPath%22%3A%22%2FUsers%2Ftrenthouliston%2FCode%2FNUClear%2Fdocs2%2Fsystem%2FStart%20Up.md%22%2C%22external%22%3A%22file%3A%2F%2F%2FUsers%2Ftrenthouliston%2FCode%2FNUClear%2Fdocs2%2Fsystem%2FStart%2520Up.md%22%2C%22path%22%3A%22%2FUsers%2Ftrenthouliston%2FCode%2FNUClear%2Fdocs2%2Fsystem%2FStart%20Up.md%22%2C%22scheme%22%3A%22file%22%7D%2C%22pos%22%3A%7B%22line%22%3A5%2C%22character%22%3A5%7D%7D%2C%7B%22uri%22%3A%7B%22%24mid%22%3A1%2C%22fsPath%22%3A%22%2FUsers%2Ftrenthouliston%2FCode%2FNUClear%2Fsrc%2FEnvironment.hpp%22%2C%22external%22%3A%22file%3A%2F%2F%2FUsers%2Ftrenthouliston%2FCode%2FNUClear%2Fsrc%2FEnvironment.hpp%22%2C%22path%22%3A%22%2FUsers%2Ftrenthouliston%2FCode%2FNUClear%2Fsrc%2FEnvironment.hpp%22%2C%22scheme%22%3A%22file%22%7D%2C%22pos%22%3A%7B%22line%22%3A33%2C%22character%22%3A6%7D%7D%2C%7B%22uri%22%3A%7B%22%24mid%22%3A1%2C%22fsPath%22%3A%22%2FUsers%2Ftrenthouliston%2FCode%2FNUClear%2Fsrc%2FPowerPlant.cpp%22%2C%22external%22%3A%22file%3A%2F%2F%2FUsers%2Ftrenthouliston%2FCode%2FNUClear%2Fsrc%2FPowerPlant.cpp%22%2C%22path%22%3A%22%2FUsers%2Ftrenthouliston%2FCode%2FNUClear%2Fsrc%2FPowerPlant.cpp%22%2C%22scheme%22%3A%22file%22%7D%2C%22pos%22%3A%7B%22line%22%3A46%2C%22character%22%3A0%7D%7D%2C%7B%22uri%22%3A%7B%22%24mid%22%3A1%2C%22fsPath%22%3A%22%2FUsers%2Ftrenthouliston%2FCode%2FNUClear%2Fsrc%2FPowerPlant.hpp%22%2C%22external%22%3A%22file%3A%2F%2F%2FUsers%2Ftrenthouliston%2FCode%2FNUClear%2Fsrc%2FPowerPlant.hpp%22%2C%22path%22%3A%22%2FUsers%2Ftrenthouliston%2FCode%2FNUClear%2Fsrc%2FPowerPlant.hpp%22%2C%22scheme%22%3A%22file%22%7D%2C%22pos%22%3A%7B%22line%22%3A65%2C%22character%22%3A6%7D%7D%5D%5D "Go to definition"). The [`PowerPlant`](command:_github.copilot.openSymbolFromReferences?%5B%22PowerPlant%22%2C%5B%7B%22uri%22%3A%7B%22%24mid%22%3A1%2C%22fsPath%22%3A%22%2FUsers%2Ftrenthouliston%2FCode%2FNUClear%2Fdocs%2Fcomponents.rst%22%2C%22external%22%3A%22file%3A%2F%2F%2FUsers%2Ftrenthouliston%2FCode%2FNUClear%2Fdocs%2Fcomponents.rst%22%2C%22path%22%3A%22%2FUsers%2Ftrenthouliston%2FCode%2FNUClear%2Fdocs%2Fcomponents.rst%22%2C%22scheme%22%3A%22file%22%7D%2C%22pos%22%3A%7B%22line%22%3A3%2C%22character%22%3A61%7D%7D%2C%7B%22uri%22%3A%7B%22%24mid%22%3A1%2C%22fsPath%22%3A%22%2FUsers%2Ftrenthouliston%2FCode%2FNUClear%2Fdocs%2Fstartup.rst%22%2C%22external%22%3A%22file%3A%2F%2F%2FUsers%2Ftrenthouliston%2FCode%2FNUClear%2Fdocs%2Fstartup.rst%22%2C%22path%22%3A%22%2FUsers%2Ftrenthouliston%2FCode%2FNUClear%2Fdocs%2Fstartup.rst%22%2C%22scheme%22%3A%22file%22%7D%2C%22pos%22%3A%7B%22line%22%3A16%2C%22character%22%3A12%7D%7D%2C%7B%22uri%22%3A%7B%22%24mid%22%3A1%2C%22fsPath%22%3A%22%2FUsers%2Ftrenthouliston%2FCode%2FNUClear%2Fdocs%2Fstartup.rst%22%2C%22external%22%3A%22file%3A%2F%2F%2FUsers%2Ftrenthouliston%2FCode%2FNUClear%2Fdocs%2Fstartup.rst%22%2C%22path%22%3A%22%2FUsers%2Ftrenthouliston%2FCode%2FNUClear%2Fdocs%2Fstartup.rst%22%2C%22scheme%22%3A%22file%22%7D%2C%22pos%22%3A%7B%22line%22%3A104%2C%22character%22%3A10%7D%7D%2C%7B%22uri%22%3A%7B%22%24mid%22%3A1%2C%22fsPath%22%3A%22%2FUsers%2Ftrenthouliston%2FCode%2FNUClear%2Fdocs2%2Fsystem%2FStart%20Up.md%22%2C%22external%22%3A%22file%3A%2F%2F%2FUsers%2Ftrenthouliston%2FCode%2FNUClear%2Fdocs2%2Fsystem%2FStart%2520Up.md%22%2C%22path%22%3A%22%2FUsers%2Ftrenthouliston%2FCode%2FNUClear%2Fdocs2%2Fsystem%2FStart%20Up.md%22%2C%22scheme%22%3A%22file%22%7D%2C%22pos%22%3A%7B%22line%22%3A5%2C%22character%22%3A5%7D%7D%2C%7B%22uri%22%3A%7B%22%24mid%22%3A1%2C%22fsPath%22%3A%22%2FUsers%2Ftrenthouliston%2FCode%2FNUClear%2Fsrc%2FEnvironment.hpp%22%2C%22external%22%3A%22file%3A%2F%2F%2FUsers%2Ftrenthouliston%2FCode%2FNUClear%2Fsrc%2FEnvironment.hpp%22%2C%22path%22%3A%22%2FUsers%2Ftrenthouliston%2FCode%2FNUClear%2Fsrc%2FEnvironment.hpp%22%2C%22scheme%22%3A%22file%22%7D%2C%22pos%22%3A%7B%22line%22%3A33%2C%22character%22%3A6%7D%7D%2C%7B%22uri%22%3A%7B%22%24mid%22%3A1%2C%22fsPath%22%3A%22%2FUsers%2Ftrenthouliston%2FCode%2FNUClear%2Fsrc%2FPowerPlant.cpp%22%2C%22external%22%3A%22file%3A%2F%2F%2FUsers%2Ftrenthouliston%2FCode%2FNUClear%2Fsrc%2FPowerPlant.cpp%22%2C%22path%22%3A%22%2FUsers%2Ftrenthouliston%2FCode%2FNUClear%2Fsrc%2FPowerPlant.cpp%22%2C%22scheme%22%3A%22file%22%7D%2C%22pos%22%3A%7B%22line%22%3A46%2C%22character%22%3A0%7D%7D%2C%7B%22uri%22%3A%7B%22%24mid%22%3A1%2C%22fsPath%22%3A%22%2FUsers%2Ftrenthouliston%2FCode%2FNUClear%2Fsrc%2FPowerPlant.hpp%22%2C%22external%22%3A%22file%3A%2F%2F%2FUsers%2Ftrenthouliston%2FCode%2FNUClear%2Fsrc%2FPowerPlant.hpp%22%2C%22path%22%3A%22%2FUsers%2Ftrenthouliston%2FCode%2FNUClear%2Fsrc%2FPowerPlant.hpp%22%2C%22scheme%22%3A%22file%22%7D%2C%22pos%22%3A%7B%22line%22%3A65%2C%22character%22%3A6%7D%7D%5D%5D "Go to definition") is the core of the system, responsible for managing reactors and their communications. + +```cpp +NUClear::Configuration config; +NUClear::PowerPlant plant(config); +``` + +### Reactor Installation + +Once the [`PowerPlant`](command:_github.copilot.openSymbolFromReferences?%5B%22PowerPlant%22%2C%5B%7B%22uri%22%3A%7B%22%24mid%22%3A1%2C%22fsPath%22%3A%22%2FUsers%2Ftrenthouliston%2FCode%2FNUClear%2Fdocs%2Fcomponents.rst%22%2C%22external%22%3A%22file%3A%2F%2F%2FUsers%2Ftrenthouliston%2FCode%2FNUClear%2Fdocs%2Fcomponents.rst%22%2C%22path%22%3A%22%2FUsers%2Ftrenthouliston%2FCode%2FNUClear%2Fdocs%2Fcomponents.rst%22%2C%22scheme%22%3A%22file%22%7D%2C%22pos%22%3A%7B%22line%22%3A3%2C%22character%22%3A61%7D%7D%2C%7B%22uri%22%3A%7B%22%24mid%22%3A1%2C%22fsPath%22%3A%22%2FUsers%2Ftrenthouliston%2FCode%2FNUClear%2Fdocs%2Fstartup.rst%22%2C%22external%22%3A%22file%3A%2F%2F%2FUsers%2Ftrenthouliston%2FCode%2FNUClear%2Fdocs%2Fstartup.rst%22%2C%22path%22%3A%22%2FUsers%2Ftrenthouliston%2FCode%2FNUClear%2Fdocs%2Fstartup.rst%22%2C%22scheme%22%3A%22file%22%7D%2C%22pos%22%3A%7B%22line%22%3A16%2C%22character%22%3A12%7D%7D%2C%7B%22uri%22%3A%7B%22%24mid%22%3A1%2C%22fsPath%22%3A%22%2FUsers%2Ftrenthouliston%2FCode%2FNUClear%2Fdocs%2Fstartup.rst%22%2C%22external%22%3A%22file%3A%2F%2F%2FUsers%2Ftrenthouliston%2FCode%2FNUClear%2Fdocs%2Fstartup.rst%22%2C%22path%22%3A%22%2FUsers%2Ftrenthouliston%2FCode%2FNUClear%2Fdocs%2Fstartup.rst%22%2C%22scheme%22%3A%22file%22%7D%2C%22pos%22%3A%7B%22line%22%3A104%2C%22character%22%3A10%7D%7D%2C%7B%22uri%22%3A%7B%22%24mid%22%3A1%2C%22fsPath%22%3A%22%2FUsers%2Ftrenthouliston%2FCode%2FNUClear%2Fdocs2%2Fsystem%2FStart%20Up.md%22%2C%22external%22%3A%22file%3A%2F%2F%2FUsers%2Ftrenthouliston%2FCode%2FNUClear%2Fdocs2%2Fsystem%2FStart%2520Up.md%22%2C%22path%22%3A%22%2FUsers%2Ftrenthouliston%2FCode%2FNUClear%2Fdocs2%2Fsystem%2FStart%20Up.md%22%2C%22scheme%22%3A%22file%22%7D%2C%22pos%22%3A%7B%22line%22%3A5%2C%22character%22%3A5%7D%7D%2C%7B%22uri%22%3A%7B%22%24mid%22%3A1%2C%22fsPath%22%3A%22%2FUsers%2Ftrenthouliston%2FCode%2FNUClear%2Fsrc%2FEnvironment.hpp%22%2C%22external%22%3A%22file%3A%2F%2F%2FUsers%2Ftrenthouliston%2FCode%2FNUClear%2Fsrc%2FEnvironment.hpp%22%2C%22path%22%3A%22%2FUsers%2Ftrenthouliston%2FCode%2FNUClear%2Fsrc%2FEnvironment.hpp%22%2C%22scheme%22%3A%22file%22%7D%2C%22pos%22%3A%7B%22line%22%3A33%2C%22character%22%3A6%7D%7D%2C%7B%22uri%22%3A%7B%22%24mid%22%3A1%2C%22fsPath%22%3A%22%2FUsers%2Ftrenthouliston%2FCode%2FNUClear%2Fsrc%2FPowerPlant.cpp%22%2C%22external%22%3A%22file%3A%2F%2F%2FUsers%2Ftrenthouliston%2FCode%2FNUClear%2Fsrc%2FPowerPlant.cpp%22%2C%22path%22%3A%22%2FUsers%2Ftrenthouliston%2FCode%2FNUClear%2Fsrc%2FPowerPlant.cpp%22%2C%22scheme%22%3A%22file%22%7D%2C%22pos%22%3A%7B%22line%22%3A46%2C%22character%22%3A0%7D%7D%2C%7B%22uri%22%3A%7B%22%24mid%22%3A1%2C%22fsPath%22%3A%22%2FUsers%2Ftrenthouliston%2FCode%2FNUClear%2Fsrc%2FPowerPlant.hpp%22%2C%22external%22%3A%22file%3A%2F%2F%2FUsers%2Ftrenthouliston%2FCode%2FNUClear%2Fsrc%2FPowerPlant.hpp%22%2C%22path%22%3A%22%2FUsers%2Ftrenthouliston%2FCode%2FNUClear%2Fsrc%2FPowerPlant.hpp%22%2C%22scheme%22%3A%22file%22%7D%2C%22pos%22%3A%7B%22line%22%3A65%2C%22character%22%3A6%7D%7D%5D%5D "Go to definition") is created, reactors can be installed. Reactors are the components that define the behavior of the system. + +```cpp +plant.install(); +plant.install(); +``` + +## PowerPlant Start Up + +### Single Threaded Main Thread + +The [`PowerPlant`](command:_github.copilot.openSymbolFromReferences?%5B%22PowerPlant%22%2C%5B%7B%22uri%22%3A%7B%22%24mid%22%3A1%2C%22fsPath%22%3A%22%2FUsers%2Ftrenthouliston%2FCode%2FNUClear%2Fdocs%2Fcomponents.rst%22%2C%22external%22%3A%22file%3A%2F%2F%2FUsers%2Ftrenthouliston%2FCode%2FNUClear%2Fdocs%2Fcomponents.rst%22%2C%22path%22%3A%22%2FUsers%2Ftrenthouliston%2FCode%2FNUClear%2Fdocs%2Fcomponents.rst%22%2C%22scheme%22%3A%22file%22%7D%2C%22pos%22%3A%7B%22line%22%3A3%2C%22character%22%3A61%7D%7D%2C%7B%22uri%22%3A%7B%22%24mid%22%3A1%2C%22fsPath%22%3A%22%2FUsers%2Ftrenthouliston%2FCode%2FNUClear%2Fdocs%2Fstartup.rst%22%2C%22external%22%3A%22file%3A%2F%2F%2FUsers%2Ftrenthouliston%2FCode%2FNUClear%2Fdocs%2Fstartup.rst%22%2C%22path%22%3A%22%2FUsers%2Ftrenthouliston%2FCode%2FNUClear%2Fdocs%2Fstartup.rst%22%2C%22scheme%22%3A%22file%22%7D%2C%22pos%22%3A%7B%22line%22%3A16%2C%22character%22%3A12%7D%7D%2C%7B%22uri%22%3A%7B%22%24mid%22%3A1%2C%22fsPath%22%3A%22%2FUsers%2Ftrenthouliston%2FCode%2FNUClear%2Fdocs%2Fstartup.rst%22%2C%22external%22%3A%22file%3A%2F%2F%2FUsers%2Ftrenthouliston%2FCode%2FNUClear%2Fdocs%2Fstartup.rst%22%2C%22path%22%3A%22%2FUsers%2Ftrenthouliston%2FCode%2FNUClear%2Fdocs%2Fstartup.rst%22%2C%22scheme%22%3A%22file%22%7D%2C%22pos%22%3A%7B%22line%22%3A104%2C%22character%22%3A10%7D%7D%2C%7B%22uri%22%3A%7B%22%24mid%22%3A1%2C%22fsPath%22%3A%22%2FUsers%2Ftrenthouliston%2FCode%2FNUClear%2Fdocs2%2Fsystem%2FStart%20Up.md%22%2C%22external%22%3A%22file%3A%2F%2F%2FUsers%2Ftrenthouliston%2FCode%2FNUClear%2Fdocs2%2Fsystem%2FStart%2520Up.md%22%2C%22path%22%3A%22%2FUsers%2Ftrenthouliston%2FCode%2FNUClear%2Fdocs2%2Fsystem%2FStart%20Up.md%22%2C%22scheme%22%3A%22file%22%7D%2C%22pos%22%3A%7B%22line%22%3A5%2C%22character%22%3A5%7D%7D%2C%7B%22uri%22%3A%7B%22%24mid%22%3A1%2C%22fsPath%22%3A%22%2FUsers%2Ftrenthouliston%2FCode%2FNUClear%2Fsrc%2FEnvironment.hpp%22%2C%22external%22%3A%22file%3A%2F%2F%2FUsers%2Ftrenthouliston%2FCode%2FNUClear%2Fsrc%2FEnvironment.hpp%22%2C%22path%22%3A%22%2FUsers%2Ftrenthouliston%2FCode%2FNUClear%2Fsrc%2FEnvironment.hpp%22%2C%22scheme%22%3A%22file%22%7D%2C%22pos%22%3A%7B%22line%22%3A33%2C%22character%22%3A6%7D%7D%2C%7B%22uri%22%3A%7B%22%24mid%22%3A1%2C%22fsPath%22%3A%22%2FUsers%2Ftrenthouliston%2FCode%2FNUClear%2Fsrc%2FPowerPlant.cpp%22%2C%22external%22%3A%22file%3A%2F%2F%2FUsers%2Ftrenthouliston%2FCode%2FNUClear%2Fsrc%2FPowerPlant.cpp%22%2C%22path%22%3A%22%2FUsers%2Ftrenthouliston%2FCode%2FNUClear%2Fsrc%2FPowerPlant.cpp%22%2C%22scheme%22%3A%22file%22%7D%2C%22pos%22%3A%7B%22line%22%3A46%2C%22character%22%3A0%7D%7D%2C%7B%22uri%22%3A%7B%22%24mid%22%3A1%2C%22fsPath%22%3A%22%2FUsers%2Ftrenthouliston%2FCode%2FNUClear%2Fsrc%2FPowerPlant.hpp%22%2C%22external%22%3A%22file%3A%2F%2F%2FUsers%2Ftrenthouliston%2FCode%2FNUClear%2Fsrc%2FPowerPlant.hpp%22%2C%22path%22%3A%22%2FUsers%2Ftrenthouliston%2FCode%2FNUClear%2Fsrc%2FPowerPlant.hpp%22%2C%22scheme%22%3A%22file%22%7D%2C%22pos%22%3A%7B%22line%22%3A65%2C%22character%22%3A6%7D%7D%5D%5D "Go to definition") starts up on a single thread. During this phase, startup messages are emitted inline, ensuring that all initial setup tasks are completed before moving to the multithreaded execution phase. + +```cpp +plant.start(); +``` + +### Startup Messages Emitted Inline + +During the startup, the [`PowerPlant`](command:_github.copilot.openSymbolFromReferences?%5B%22PowerPlant%22%2C%5B%7B%22uri%22%3A%7B%22%24mid%22%3A1%2C%22fsPath%22%3A%22%2FUsers%2Ftrenthouliston%2FCode%2FNUClear%2Fdocs%2Fcomponents.rst%22%2C%22external%22%3A%22file%3A%2F%2F%2FUsers%2Ftrenthouliston%2FCode%2FNUClear%2Fdocs%2Fcomponents.rst%22%2C%22path%22%3A%22%2FUsers%2Ftrenthouliston%2FCode%2FNUClear%2Fdocs%2Fcomponents.rst%22%2C%22scheme%22%3A%22file%22%7D%2C%22pos%22%3A%7B%22line%22%3A3%2C%22character%22%3A61%7D%7D%2C%7B%22uri%22%3A%7B%22%24mid%22%3A1%2C%22fsPath%22%3A%22%2FUsers%2Ftrenthouliston%2FCode%2FNUClear%2Fdocs%2Fstartup.rst%22%2C%22external%22%3A%22file%3A%2F%2F%2FUsers%2Ftrenthouliston%2FCode%2FNUClear%2Fdocs%2Fstartup.rst%22%2C%22path%22%3A%22%2FUsers%2Ftrenthouliston%2FCode%2FNUClear%2Fdocs%2Fstartup.rst%22%2C%22scheme%22%3A%22file%22%7D%2C%22pos%22%3A%7B%22line%22%3A16%2C%22character%22%3A12%7D%7D%2C%7B%22uri%22%3A%7B%22%24mid%22%3A1%2C%22fsPath%22%3A%22%2FUsers%2Ftrenthouliston%2FCode%2FNUClear%2Fdocs%2Fstartup.rst%22%2C%22external%22%3A%22file%3A%2F%2F%2FUsers%2Ftrenthouliston%2FCode%2FNUClear%2Fdocs%2Fstartup.rst%22%2C%22path%22%3A%22%2FUsers%2Ftrenthouliston%2FCode%2FNUClear%2Fdocs%2Fstartup.rst%22%2C%22scheme%22%3A%22file%22%7D%2C%22pos%22%3A%7B%22line%22%3A104%2C%22character%22%3A10%7D%7D%2C%7B%22uri%22%3A%7B%22%24mid%22%3A1%2C%22fsPath%22%3A%22%2FUsers%2Ftrenthouliston%2FCode%2FNUClear%2Fdocs2%2Fsystem%2FStart%20Up.md%22%2C%22external%22%3A%22file%3A%2F%2F%2FUsers%2Ftrenthouliston%2FCode%2FNUClear%2Fdocs2%2Fsystem%2FStart%2520Up.md%22%2C%22path%22%3A%22%2FUsers%2Ftrenthouliston%2FCode%2FNUClear%2Fdocs2%2Fsystem%2FStart%20Up.md%22%2C%22scheme%22%3A%22file%22%7D%2C%22pos%22%3A%7B%22line%22%3A5%2C%22character%22%3A5%7D%7D%2C%7B%22uri%22%3A%7B%22%24mid%22%3A1%2C%22fsPath%22%3A%22%2FUsers%2Ftrenthouliston%2FCode%2FNUClear%2Fsrc%2FEnvironment.hpp%22%2C%22external%22%3A%22file%3A%2F%2F%2FUsers%2Ftrenthouliston%2FCode%2FNUClear%2Fsrc%2FEnvironment.hpp%22%2C%22path%22%3A%22%2FUsers%2Ftrenthouliston%2FCode%2FNUClear%2Fsrc%2FEnvironment.hpp%22%2C%22scheme%22%3A%22file%22%7D%2C%22pos%22%3A%7B%22line%22%3A33%2C%22character%22%3A6%7D%7D%2C%7B%22uri%22%3A%7B%22%24mid%22%3A1%2C%22fsPath%22%3A%22%2FUsers%2Ftrenthouliston%2FCode%2FNUClear%2Fsrc%2FPowerPlant.cpp%22%2C%22external%22%3A%22file%3A%2F%2F%2FUsers%2Ftrenthouliston%2FCode%2FNUClear%2Fsrc%2FPowerPlant.cpp%22%2C%22path%22%3A%22%2FUsers%2Ftrenthouliston%2FCode%2FNUClear%2Fsrc%2FPowerPlant.cpp%22%2C%22scheme%22%3A%22file%22%7D%2C%22pos%22%3A%7B%22line%22%3A46%2C%22character%22%3A0%7D%7D%2C%7B%22uri%22%3A%7B%22%24mid%22%3A1%2C%22fsPath%22%3A%22%2FUsers%2Ftrenthouliston%2FCode%2FNUClear%2Fsrc%2FPowerPlant.hpp%22%2C%22external%22%3A%22file%3A%2F%2F%2FUsers%2Ftrenthouliston%2FCode%2FNUClear%2Fsrc%2FPowerPlant.hpp%22%2C%22path%22%3A%22%2FUsers%2Ftrenthouliston%2FCode%2FNUClear%2Fsrc%2FPowerPlant.hpp%22%2C%22scheme%22%3A%22file%22%7D%2C%22pos%22%3A%7B%22line%22%3A65%2C%22character%22%3A6%7D%7D%5D%5D "Go to definition") emits startup messages inline. This ensures that all reactors are properly initialized and ready to handle tasks. + +## Multithreaded Execution Phase + +### Normal System Operation + +In the multithreaded execution phase, the system operates normally. Tasks are executed on thread pools, allowing for efficient parallel processing. + +### Task Execution on Thread Pools + +Tasks are distributed across multiple threads using thread pools. This ensures that the system can handle multiple tasks concurrently, improving performance and responsiveness. + +## Shutdown Sequence + +### Shutdown Message Emitted at Lowest Priority + +When the system is shutting down, a shutdown message is emitted at the lowest priority. This ensures that all critical tasks are completed before the system shuts down. + +### Blocking New Non-Inline Tasks on Normal Pools + +During the shutdown sequence, new non-inline tasks are blocked on normal pools. This prevents new tasks from being added to the queue, allowing the system to focus on completing existing tasks. + +### Allowing Tasks on Persistent Pools + +Tasks on persistent pools are still allowed to run during the shutdown sequence. This ensures that long-running or critical tasks can complete before the system shuts down. + +### Waiting for Non-Persistent Pools to Finish + +The main thread waits for non-persistent pools to finish all tasks and for the threads to terminate. This ensures that all tasks are completed before the system shuts down. + +### Shutting Down Persistent Pool Threads + +Persistent pool threads are told to shut down once they exhaust all tasks. The main thread then returns from [`powerplant.start()`]( "Go to definition"), completing the shutdown process. + +```cpp +plant.shutdown(); +``` + +## Sequence Diagrams + +### Interaction Between Main Thread and Thread Pools + +#### Initialization Phase + +```mermaid +sequenceDiagram + participant Main Thread + participant PowerPlant + Main Thread->>PowerPlant: Create PowerPlant + Main Thread->>PowerPlant: Install Reactors + Main Thread->>PowerPlant: Start PowerPlant + PowerPlant->>Main Thread: Emit Startup Messages Inline +``` + +#### Execution Phase + +```mermaid +sequenceDiagram + participant Main Thread + participant Thread Pool + Main Thread->>Thread Pool: Submit Tasks + Thread Pool->>Main Thread: Execute Tasks Concurrently +``` + +#### Shutdown Phase + +```mermaid +sequenceDiagram + participant Main Thread + participant Thread Pool + Main Thread->>PowerPlant: Emit Shutdown Message + PowerPlant->>Thread Pool: Block New Non-Inline Tasks + Thread Pool->>Main Thread: Complete Existing Tasks + Main Thread->>Thread Pool: Wait for Non-Persistent Pools to Finish + Thread Pool->>Main Thread: Terminate Threads + Main Thread->>PowerPlant: Shutdown Persistent Pools + PowerPlant->>Main Thread: Return from Start +``` + +This document provides an overview of the execution stages in the NUClear system, helping new users understand how to set up, run, and shut down the system effectively. + +```mermaid +sequenceDiagram + participant PowerPlant; + participant Main Thread; + participant Default Pool; + participant Persistant Pool; + + activate Main Thread; + + loop Install Phase; + Main Thread->>PowerPlant: Install Reactor; + PowerPlant-->>Main Thread: ; + end; + + Main Thread->>PowerPlant: PowerPlant.start(); + Main Thread->>PowerPlant: emit(Startup); + loop Startup Phase; + PowerPlant-->>Main Thread: Execute Startup Inline; + end; + + PowerPlant->>+Default Pool: Start; + PowerPlant->>+Persistant Pool: ; + + loop Execution Phase; + par; + PowerPlant->Main Thread: Emit/Accept; + and; + PowerPlant->Default Pool: Emit/Accept; + and; + PowerPlant->Persistant Pool: Emit/Accept; + end + end + + Main Thread->>PowerPlant: powerplant.shutdown() + loop Shutdown Phase; + par; + PowerPlant->Main Thread: Emit; + and; + PowerPlant->Default Pool: Emit; + and; + PowerPlant->Persistant Pool: Emit/Accept; + end + end + + deactivate Main Thread; + deactivate Default Pool; + + loop Finalisation Phase; + PowerPlant->Persistant Pool: Execute; + end; + + deactivate Persistant Pool; + +``` diff --git a/docs2/system/Tracing.md b/docs2/system/Tracing.md new file mode 100644 index 00000000..a7cb99c7 --- /dev/null +++ b/docs2/system/Tracing.md @@ -0,0 +1,4 @@ +Talk about tracing +Being tracing +End tracing +What is output