Feature Request: Concurrency Limits for ReactPHP Tasks

[hasanparasteh]

The current behavior of ReactPHP's periodic timers allows multiple instances of a task to run concurrently, potentially leading to overlapping executions. To address this issue and provide better control over task concurrency, a feature inspired by the go-co-op/gocron package from Golang can be introduced.

Proposal:

Introduce a mechanism for limiting concurrency in ReactPHP periodic timers. This mechanism should ensure that only one instance of a task is running at any given time, preventing overlap.

Key Components:

• Singleton Task Instances: Implement a mechanism to create singleton instances for each running task. This ensures that only one instance of a task runs at a time.

• Concurrency Limits: Allow users to specify concurrency limits for tasks. Users can set the maximum number of concurrent instances of a task allowed to run.

• Task Queue: Maintain a queue for pending tasks. If the maximum concurrency limit is reached, subsequent tasks are queued until a slot becomes available.

Example:

$loop = Loop::get()

$concurrencyLimit = 1; // Set concurrency limit to 1

$queue = new SplQueue(); // Initialize task queue

$loop->addPeriodicTimer(1.0, function () use ($queue, $concurrencyLimit) {
    if ($queue->count() >= $concurrencyLimit) {
        return; // Concurrency limit reached, skip execution
    }

    $queue->enqueue(1); // Enqueue task

    $task = function () use ($queue) {
        echo "Hello There\n";
        await(React\Promise\Timer\sleep(1.5));
        $queue->dequeue(); // Remove task from queue after execution
    };

    $task(); // Execute task
});

$loop->run();

Conclusion:

By implementing concurrency limits inspired by the go-co-op/gocron package, ReactPHP can offer better control over task execution, preventing overlap and ensuring efficient resource utilization.

NOTE: maybe reactphp needs a job scheduling package and it couldn't implement inside event-loop package itself. i was just exploring and hoping to spread my ideas for reactphp. if it's good enough I would implement that just like go-co-op/gocron library in another repo

[WyriHaximus]

Hey @hasanparasteh, while I understand where the question comes from I'm not sure if we'll ever add it as a core package. There are a couple of reasons for that.

First off, I've got you covered for literal cron scheduling: https://github.com/wyrihaximus/reactphp-cron
Secondly, also got mutex locking packages: https://github.com/WyriHaximus/reactphp-mutex

TL;DR For the rest @clue create https://github.com/clue/reactphp-mq for this.

The reason why I personally don't think we'll add this. Aside from having two packages providing the features you've suggested, and can be combined into our preferred use. Is that these mechanisms are very tied to how you want to use them. For the ReactPHP ext-parallel limited pool packages I'm literally keeping a counter to achieve concurrency limits: https://github.com/reactphp-parallel/limited-pool/blob/2.x/src/Limited.php#L50-L71 Other ways are using a https://github.com/jakubkulhan/bunny consumer with a pre-fetch count will also be a concurrency limit. On top of that, if you use ack/nack after processing the message, you also have a single task instance.

If you think the packages list doesn't meet your exact specifications, know that we value a small set of core packages with a large ecosystem of 3rd party packages around it.

[SimonFrings]

@hasanparasteh Thanks for bringing in this proposal, I really appreciate the amount of work you put into describing your idea ❤️

While this is a good idea, I agree with @WyriHaximus. You can use clue/reactphp-mq , which will queue your operations in-memory, or you can use clue/reactphp-flux which works similar to clue/reactphp-mq without keeping you operations inside your memory.

I also gave a similar answer in #520, also describing that in the future we want to combine clue/reactphp-mq and clue/reactphp-flux and merge them with our reactphp/async component.