Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Large Refactor Branch #111

Closed
wants to merge 114 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
114 commits
Select commit Hold shift + click to select a range
c42fcfe
First hacky attempt at measuring cpu time
TrentHouliston Nov 12, 2023
1d34461
.
TrentHouliston Nov 12, 2023
e0aa92f
How did that ever work
TrentHouliston Nov 12, 2023
092886d
.
TrentHouliston Nov 12, 2023
61decca
.
TrentHouliston Nov 12, 2023
3d1bb53
Add unit test
TrentHouliston Nov 13, 2023
b826cd7
Could swap to a single CPU clock?
TrentHouliston Nov 28, 2023
03ca204
Merge branch 'main' into houliston/cputime
TrentHouliston Apr 17, 2024
698b92b
clang-tidy
TrentHouliston Apr 18, 2024
6579625
fix windows
TrentHouliston Apr 18, 2024
e4e0bb4
Merge remote-tracking branch 'origin/main' into houliston/cputime
TrentHouliston Jul 14, 2024
f90f0ca
Add real time and document some stuff. Also swap to newer catch (will…
TrentHouliston Jul 14, 2024
5a60e6f
Merge branch 'main' into houliston/cputime
TrentHouliston Jul 14, 2024
1affc70
.
TrentHouliston Jul 14, 2024
9675712
In theory it's possible for things to be at the same time due to the …
TrentHouliston Jul 14, 2024
b3aef90
clang
TrentHouliston Jul 14, 2024
20f3007
clang
TrentHouliston Jul 14, 2024
3580b76
.
TrentHouliston Jul 15, 2024
f45d746
Changing a ton of stuff to allow events and clean up some APIs and docs
TrentHouliston Aug 1, 2024
b2cd1be
The beatings will continue until morale improves
TrentHouliston Aug 2, 2024
2f5a3e0
Add an idle lock class for the new scheduler
TrentHouliston Aug 5, 2024
3a0c6e8
Fix test
TrentHouliston Aug 5, 2024
7d67d0d
Add group lock and tests
TrentHouliston Aug 6, 2024
4051be0
Test the units separately which made the tests simpler
TrentHouliston Aug 6, 2024
2a4e43e
Minor changes
TrentHouliston Aug 6, 2024
b1e63e0
More minor changes
TrentHouliston Aug 6, 2024
0e0f336
Make MainThread descriptor a constexpr
TrentHouliston Aug 6, 2024
f83cb16
make tests not be in tests/tests
TrentHouliston Aug 6, 2024
4f5fd18
.
TrentHouliston Aug 6, 2024
3d15e17
.
TrentHouliston Aug 6, 2024
a5d12d3
Implement the new task scheduler and remove the old one
TrentHouliston Aug 6, 2024
c360cd3
Remove empty unit test for now
TrentHouliston Aug 6, 2024
343e90e
Fix idle test
TrentHouliston Aug 7, 2024
1afca14
Don't make idle tasks when shutting down
TrentHouliston Aug 7, 2024
95c4245
Change message
TrentHouliston Aug 8, 2024
f187e44
Sleep if there are no idle tasks to do
TrentHouliston Aug 8, 2024
795f5fa
Fix the logs
TrentHouliston Aug 8, 2024
d00e329
Swap over to a better group method
TrentHouliston Aug 8, 2024
d6760d3
Improve idle in the scheduler so it's more reliable
TrentHouliston Aug 8, 2024
c4609d7
Upgrade the sync order test to ensure it works with multiple threads
TrentHouliston Aug 8, 2024
3b92ac2
Improve atomic use in active tasks
TrentHouliston Aug 8, 2024
798319d
Chrono controller does not need to be atomic (it already guards with …
TrentHouliston Aug 8, 2024
62ae754
Split clock back into cpp files
TrentHouliston Aug 8, 2024
3f0a42e
Atomic memory ordering
TrentHouliston Aug 8, 2024
405e9ac
Tighten the mutexes and atomics in the scheduler
TrentHouliston Aug 8, 2024
77bf77f
Minor changes
TrentHouliston Aug 8, 2024
a8d2c44
Don't let main thread start holding the pools mutex
TrentHouliston Aug 8, 2024
deeb7b0
Includes
TrentHouliston Aug 8, 2024
b8dbd47
missing include
TrentHouliston Aug 8, 2024
bf13f41
Wrong header
TrentHouliston Aug 8, 2024
0482417
Remove the asan options
TrentHouliston Aug 8, 2024
14acfd4
mismatched includes
TrentHouliston Aug 8, 2024
eee5b28
Compile warnings
TrentHouliston Aug 8, 2024
6050bb7
missing header
TrentHouliston Aug 8, 2024
5827c5e
remove file
TrentHouliston Aug 8, 2024
f8710b6
Pull out the sleeping logic from chronocontroller and test it works
TrentHouliston Aug 9, 2024
a342c58
With tight enough timings the chrono controller never sleeps and ther…
TrentHouliston Aug 9, 2024
b11f19a
I trust the precise sleep is accurate enough without a spinlock
TrentHouliston Aug 9, 2024
5ffb755
stuff
TrentHouliston Aug 9, 2024
489e9f3
Fix formatting
TrentHouliston Aug 9, 2024
238bf4c
.
TrentHouliston Aug 9, 2024
26b917b
Fix error in newer clang-tidy
TrentHouliston Aug 9, 2024
6cd5848
New clang-tidy things
TrentHouliston Aug 9, 2024
c68e1b6
More clang tidy things
TrentHouliston Aug 9, 2024
ad1aa13
Remove redundant inlines
TrentHouliston Aug 9, 2024
8e3e7a4
Split get_hostname up into a cpp and hpp file
TrentHouliston Aug 9, 2024
cf84715
Remove redundant initialisers
TrentHouliston Aug 9, 2024
5da99c5
clang-tidy
TrentHouliston Aug 9, 2024
fdb561c
More redundant initialisers
TrentHouliston Aug 9, 2024
72372d6
More clang-tidy
TrentHouliston Aug 9, 2024
4ffeaf2
const lock guards
TrentHouliston Aug 9, 2024
76d4d06
When including <nuclear> stuff inside is fine
TrentHouliston Aug 9, 2024
95e6133
Anything in platform should come from platform
TrentHouliston Aug 9, 2024
1ffa07f
Didn't do anything?
TrentHouliston Aug 9, 2024
4e2d782
Fully qualify
TrentHouliston Aug 9, 2024
6b38694
Change in docs too
TrentHouliston Aug 9, 2024
38fb5c2
More clang tidy
TrentHouliston Aug 9, 2024
4ab0873
more clang-tidy
TrentHouliston Aug 9, 2024
f940202
clang-tidy
TrentHouliston Aug 9, 2024
df46942
More of that clang-tidy stuff
TrentHouliston Aug 9, 2024
3ed790d
compile error
TrentHouliston Aug 9, 2024
26af9a5
.
TrentHouliston Aug 9, 2024
c3de6d8
More clang tidy
TrentHouliston Aug 9, 2024
1b2e061
Tidy
TrentHouliston Aug 9, 2024
5132c07
last few clang-tidy errors
TrentHouliston Aug 9, 2024
3b0cc7f
Seems the type was important
TrentHouliston Aug 9, 2024
f34beb1
lint on wrong line
TrentHouliston Aug 9, 2024
1ac8c03
You are being far too difficult
TrentHouliston Aug 9, 2024
c959b09
Go away
TrentHouliston Aug 9, 2024
358bb52
Fix up the log test and add extra functions to log level
TrentHouliston Aug 9, 2024
dba56bc
ct
TrentHouliston Aug 9, 2024
eceebb8
Fix header
TrentHouliston Aug 9, 2024
42f459f
Trust was misplaced apparently
TrentHouliston Aug 9, 2024
1f4c0a2
tidy
TrentHouliston Aug 9, 2024
69c78b0
Merge remote-tracking branch 'origin/main' into houliston/changingstuff
TrentHouliston Aug 12, 2024
8610f34
Correct some undone changes
TrentHouliston Aug 12, 2024
583b8e9
fix settings.json
TrentHouliston Aug 12, 2024
6d98873
Remove unused class
TrentHouliston Aug 12, 2024
44569ba
Merge remote-tracking branch 'origin/main' into houliston/changingstuff
TrentHouliston Aug 12, 2024
165950c
Merge remote-tracking branch 'origin/main' into houliston/changingstuff
TrentHouliston Aug 12, 2024
54e2fd4
Merge remote-tracking branch 'origin/main' into houliston/changingstuff
TrentHouliston Aug 13, 2024
190ea5c
Fix
TrentHouliston Aug 13, 2024
6f16e98
Fixes
TrentHouliston Aug 13, 2024
49df23f
Merge remote-tracking branch 'origin/main' into houliston/changingstuff
TrentHouliston Aug 14, 2024
3e9470d
Merge branch 'main' into houliston/changingstuff
TrentHouliston Aug 14, 2024
a6cd4ac
Merge remote-tracking branch 'origin/main' into houliston/changingstuff
TrentHouliston Aug 14, 2024
92e9515
Merge remote-tracking branch 'origin/main' into houliston/changingstuff
TrentHouliston Aug 15, 2024
9f0be85
Fixes
TrentHouliston Aug 15, 2024
52ee36b
Merge branch 'main' into houliston/changingstuff
TrentHouliston Aug 15, 2024
e2eb2fa
Undo/normalise a lot of changes with main
TrentHouliston Aug 15, 2024
e6344dc
Merge remote-tracking branch 'origin/main' into houliston/changingstuff
TrentHouliston Aug 21, 2024
92d27d3
Merge remote-tracking branch 'origin/main' into houliston/changingstuff
TrentHouliston Aug 21, 2024
046ec8f
Undo some changes
TrentHouliston Aug 21, 2024
cb1b13a
.
TrentHouliston Aug 21, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 5 additions & 1 deletion .vscode/settings.json
Original file line number Diff line number Diff line change
Expand Up @@ -12,8 +12,12 @@
"editor.rulers": [120],
"cmake.configureOnOpen": true,
"C_Cpp.default.configurationProvider": "vector-of-bool.cmake-tools",
"files.associations": {
"*.ipp": "cpp"
},
"sonarlint.connectedMode.project": {
"connectionId": "fastcode",
"projectKey": "Fastcode_NUClear"
}
},
"cSpell.language": "en-GB"
}
88 changes: 30 additions & 58 deletions src/extension/ChronoController.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -22,43 +22,35 @@

#include "ChronoController.hpp"

#include <atomic>

#include "../util/precise_sleep.hpp"

namespace NUClear {
namespace extension {

/// The precision threshold to swap from sleeping on the condition variable to sleeping with nanosleep
constexpr std::chrono::milliseconds precise_threshold = std::chrono::milliseconds(50);

/**
* Duration cast the given type to nanoseconds
*
* @tparam T the type to cast
*
* @param t the value to cast
*
* @return the time value in nanoseconds
*/
template <typename T>
std::chrono::nanoseconds ns(T&& t) {
return std::chrono::duration_cast<std::chrono::nanoseconds>(std::forward<T>(t));
}

ChronoController::ChronoController(std::unique_ptr<NUClear::Environment> environment)
: Reactor(std::move(environment)) {

// Estimate the accuracy of our cv wait and precise sleep
for (int i = 0; i < 3; ++i) {
// Estimate the accuracy of our cv wait
std::mutex test;
std::unique_lock<std::mutex> lock(test);
const auto cv_s = NUClear::clock::now();
wait.wait_for(lock, std::chrono::milliseconds(1));
const auto cv_e = NUClear::clock::now();
const auto cv_a = NUClear::clock::duration(cv_e - cv_s - std::chrono::milliseconds(1));

// Estimate the accuracy of our precise sleep
const auto ns_s = NUClear::clock::now();
util::precise_sleep(std::chrono::milliseconds(1));
const auto ns_e = NUClear::clock::now();
const auto ns_a = NUClear::clock::duration(ns_e - ns_s - std::chrono::milliseconds(1));

// Use the largest time we have seen
cv_accuracy = cv_a > cv_accuracy ? cv_a : cv_accuracy;
ns_accuracy = ns_a > ns_accuracy ? ns_a : ns_accuracy;
}

on<Trigger<ChronoTask>>().then("Add Chrono task", [this](const std::shared_ptr<const ChronoTask>& task) {
// Lock the mutex while we're doing stuff
const std::lock_guard<std::mutex> lock(mutex);

// Add our new task to the heap if we are still running
if (running.load(std::memory_order_acquire)) {
if (running) {
tasks.push_back(*task);
std::push_heap(tasks.begin(), tasks.end(), std::greater<>());
}
Expand Down Expand Up @@ -91,7 +83,7 @@ namespace extension {
// When we shutdown we notify so we quit now
on<Shutdown>().then("Shutdown Chrono Controller", [this] {
const std::lock_guard<std::mutex> lock(mutex);
running.store(false, std::memory_order_release);
running = false;
wait.notify_all();
});

Expand Down Expand Up @@ -123,19 +115,20 @@ namespace extension {

on<Always, Priority::REALTIME>().then("Chrono Controller", [this] {
// Run until we are told to stop
while (running.load(std::memory_order_acquire)) {
while (running) {

// Acquire the mutex lock so we can wait on it
std::unique_lock<std::mutex> lock(mutex);

// If we have no chrono tasks wait until we are notified
if (tasks.empty()) {
wait.wait(lock, [this] { return !running.load(std::memory_order_acquire) || !tasks.empty(); });
wait.wait(lock, [this] { return !running || !tasks.empty(); });
}
else {
auto start = NUClear::clock::now();
auto target = tasks.front().time;

// Run the task if we are at or past the target time
if (target <= start) {
// Run our task and if it returns false remove it
const bool renew = tasks.front()();
Expand All @@ -152,44 +145,23 @@ namespace extension {
tasks.pop_back();
}
}
// Wait if we are not at the target time
else {
const NUClear::clock::duration time_until_task =
std::chrono::duration_cast<NUClear::clock::duration>((target - start) / clock::rtf());
// Calculate the real time to sleep given the rate at which time passes
const auto time_until_task = ns((target - start) / clock::rtf());

if (clock::rtf() == 0.0) {
// If we are paused then just wait until we are unpaused
wait.wait(lock, [&] {
return !running.load(std::memory_order_acquire) || clock::rtf() != 0.0
|| NUClear::clock::now() != start;
return !running || clock::rtf() != 0.0 || NUClear::clock::now() != start;
});
}
else if (time_until_task > cv_accuracy) { // A long time in the future
else if (time_until_task > precise_threshold) { // A long time in the future
// Wait on the cv
wait.wait_for(lock, time_until_task - cv_accuracy);

// Update the accuracy of our cv wait
const auto end = NUClear::clock::now();
const auto error = end - (target - cv_accuracy); // when ended - when wanted to end
if (error.count() > 0) { // only if we were late
cv_accuracy = error > cv_accuracy ? error : ((cv_accuracy * 99 + error) / 100);
}
}
else if (time_until_task > ns_accuracy) { // Somewhat close in time
// Wait on nanosleep
const NUClear::clock::duration sleep_time = time_until_task - ns_accuracy;
util::precise_sleep(sleep_time);

// Update the accuracy of our precise sleep
const auto end = NUClear::clock::now();
const auto error = end - (target - ns_accuracy); // when ended - when wanted to end
if (error.count() > 0) { // only if we were late
ns_accuracy = error > ns_accuracy ? error : ((ns_accuracy * 99 + error) / 100);
}
wait.wait_for(lock, time_until_task - precise_threshold);
}
else {
while (NUClear::clock::now() < tasks.front().time) {
// Spinlock until we get to the time
}
else { // Within precise sleep threshold
sleeper.sleep_for(time_until_task);
}
}
}
Expand Down
13 changes: 6 additions & 7 deletions src/extension/ChronoController.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@

#include "../Reactor.hpp"
#include "../message/TimeTravel.hpp"
#include "../util/Sleeper.hpp"

namespace NUClear {
namespace extension {
Expand All @@ -37,19 +38,17 @@ namespace extension {
explicit ChronoController(std::unique_ptr<NUClear::Environment> environment);

private:
/// The mutex used to guard the tasks and running flag
std::mutex mutex;
/// The list of tasks we need to process
std::vector<dsl::operation::ChronoTask> tasks;
/// The mutex we use to lock the task list
std::mutex mutex;
/// The condition variable we use to wait on
std::condition_variable wait;
/// If we are running or not
std::atomic<bool> running{true};
bool running = true;

/// The temporal accuracy when waiting on a condition variable
NUClear::clock::duration cv_accuracy{0};
/// The temporal accuracy when waiting on nanosleep
NUClear::clock::duration ns_accuracy{0};
/// The class which is able to perform high precision sleeps
util::Sleeper sleeper;
};

} // namespace extension
Expand Down
139 changes: 139 additions & 0 deletions src/util/Sleeper.cpp
Original file line number Diff line number Diff line change
@@ -0,0 +1,139 @@
/*
* MIT License
*
* Copyright (c) 2023 NUClear Contributors
*
* This file is part of the NUClear codebase.
* See https://github.com/Fastcode/NUClear for further info.
*
* Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated
* documentation files (the "Software"), to deal in the Software without restriction, including without limitation the
* rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to
* permit persons to whom the Software is furnished to do so, subject to the following conditions:
*
* The above copyright notice and this permission notice shall be included in all copies or substantial portions of the
* Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE
* WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
* COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR
* OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
*/

#include "Sleeper.hpp"

#include <chrono>
#include <cmath>

#if defined(_WIN32)

#include <chrono>
#include <cstdint>

#include "platform.hpp"

namespace NUClear {
namespace util {

// Windows requires a waitable timer to sleep for a precise amount of time
class SleeperImpl {
public:
SleeperImpl() : timer(::CreateWaitableTimer(nullptr, TRUE, nullptr)) {}
::HANDLE timer;
};

Sleeper::~Sleeper() {
::CloseHandle(sleeper->timer);
}

void Sleeper::idle_sleep(const std::chrono::nanoseconds& ns) {
::LARGE_INTEGER ft;
// TODO if ns is negative make it 0 as otherwise it'll become absolute time
// Negative for relative time, positive for absolute time
// Measures in 100ns increments so divide by 100
ft.QuadPart = -static_cast<int64_t>(ns.count() / 100);

::SetWaitableTimer(impl->timer, &ft, 0, nullptr, nullptr, 0);
::WaitForSingleObject(impl->timer, INFINITE);
}

} // namespace util
} // namespace NUClear

#else

#include <cerrno>
#include <cstdint>
#include <ctime>

namespace NUClear {
namespace util {

// No specific implementation for precise sleep on linux
class SleeperImpl {};

// Sleep using nanosleep on linux
// This can't be static because the windows version can't be static
// NOLINTNEXTLINE(readability-convert-member-functions-to-static)
void Sleeper::idle_sleep(const std::chrono::nanoseconds& ns) {
if (ns <= std::chrono::nanoseconds(0)) {
return;
}
timespec ts{};
ts.tv_sec = std::chrono::duration_cast<std::chrono::seconds>(ns).count();
ts.tv_nsec = (ns - std::chrono::seconds(ts.tv_sec)).count();

while (::nanosleep(&ts, &ts) == -1 && errno == EINTR) {
}
}

} // namespace util
} // namespace NUClear

#endif

namespace NUClear {
namespace util {

Sleeper::Sleeper() : sleeper(std::make_unique<SleeperImpl>()) {}

// This must be in the .cpp file as we need the full definition of SleeperImpl
Sleeper::~Sleeper() = default;
Sleeper::Sleeper(Sleeper&&) noexcept = default;
Sleeper& Sleeper::operator=(Sleeper&&) noexcept = default;

void NUClear::util::Sleeper::sleep_for(const std::chrono::nanoseconds& duration) {
sleep_until(std::chrono::steady_clock::now() + duration);
}

void NUClear::util::Sleeper::sleep_until(const std::chrono::steady_clock::time_point& target) {
using namespace std::chrono; // NOLINT(google-build-using-namespace) fine in function scope

for (auto start = std::chrono::steady_clock::now(); start < target; start = std::chrono::steady_clock::now()) {
// If we can accurately sleep for the target amount of time then do so
if (target - start >= sleep_accuracy) {
// Sleep as accurately as we think we can
auto target_sleep_time = target - start - sleep_accuracy;
idle_sleep(target_sleep_time);
auto end = std::chrono::steady_clock::now();

// Update the idle sleep accuracy estimate using Welford's method
const auto actual_sleep_time = end - start;

const double sleep_error =
duration_cast<duration<double, std::nano>>(actual_sleep_time - target_sleep_time).count();
const double delta = sleep_error - mean;

count = count + 1;
mean = mean + (delta / count);
const double delta2 = sleep_error - mean;
m2 = m2 + delta * delta2;

// Sleep accuracy with 3 standard deviations of the mean for a 99.7% confidence interval
sleep_accuracy = nanoseconds(std::lround(std::sqrt(m2 / count) * 3.0));
}
}
}

} // namespace util
} // namespace NUClear
Loading
Loading