Skip to content

Commit

Permalink
Improve CPU usage in tests with IO threads (#823)
Browse files Browse the repository at this point in the history
Currently, when running tests with IO threads, we set the
`events-per-io-thread` config to 0. This activated IO threads 100% of
the time, regardless of the number of IO events.

This is causing issues with tests running multiple server instances, as
it drained machine CPU resources. As a result, tests could have very
long runtimes, especially on limited instances.
For example, in
https://github.com/valkey-io/valkey/actions/runs/10066315827/job/27827426986?pr=804,
the `Cluster consistency during live resharding` test ran for 1 hour and
41 minutes.

This PR addresses the issue by:
1. Deactivating IO threads when there are no IO events
2. Continuing to offload all IO events to IO threads

Tested on 16 cores instance, after implementing these changes, the
runtime for the `Cluster consistency during live resharding` test
dropped from 7 minutes an 14 seconds to 3 minutes and 28 seconds.

Signed-off-by: Uri Yagelnik <[email protected]>
  • Loading branch information
uriyage authored Jul 24, 2024
1 parent f00c8f6 commit 3cca268
Showing 1 changed file with 3 additions and 3 deletions.
6 changes: 3 additions & 3 deletions src/io_threads.c
Original file line number Diff line number Diff line change
Expand Up @@ -164,9 +164,9 @@ void waitForClientIO(client *c) {
void adjustIOThreadsByEventLoad(int numevents, int increase_only) {
if (server.io_threads_num == 1) return; /* All I/O is being done by the main thread. */
debugServerAssertWithInfo(NULL, NULL, server.io_threads_num > 1);

int target_threads =
server.events_per_io_thread == 0 ? server.io_threads_num : numevents / server.events_per_io_thread;
/* When events_per_io_thread is set to 0, we offload all events to the IO threads.
* This is used mainly for testing purposes. */
int target_threads = server.events_per_io_thread == 0 ? (numevents + 1) : numevents / server.events_per_io_thread;

target_threads = max(1, min(target_threads, server.io_threads_num));

Expand Down

0 comments on commit 3cca268

Please sign in to comment.