Skip to content

Commit

Permalink
Document standardization
Browse files Browse the repository at this point in the history
  • Loading branch information
luizcmarin committed May 6, 2024
1 parent b47a41f commit de8c49b
Show file tree
Hide file tree
Showing 20 changed files with 446 additions and 49 deletions.
2 changes: 1 addition & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Yii Queue Change Log
# Yii Queue Extension Change Log

## 1.0.0 under development

Expand Down
60 changes: 17 additions & 43 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,10 +6,6 @@
<br>
</p>

An extension for running tasks asynchronously via queues.

Documentation is at [docs/guide/README.md](docs/guide/README.md).

[![Latest Stable Version](https://poser.pugx.org/yiisoft/queue/v/stable.svg)](https://packagist.org/packages/yiisoft/queue)
[![Total Downloads](https://poser.pugx.org/yiisoft/queue/downloads.svg)](https://packagist.org/packages/yiisoft/queue)
[![Build status](https://github.com/yiisoft/queue/workflows/build/badge.svg)](https://github.com/yiisoft/queue/actions)
Expand All @@ -19,23 +15,19 @@ Documentation is at [docs/guide/README.md](docs/guide/README.md).
[![static analysis](https://github.com/yiisoft/queue/workflows/static%20analysis/badge.svg)](https://github.com/yiisoft/queue/actions?query=workflow%3A%22static+analysis%22)
[![type-coverage](https://shepherd.dev/github/yiisoft/queue/coverage.svg)](https://shepherd.dev/github/yiisoft/queue)

## Installation
An extension for running tasks asynchronously via queues.

The preferred way to install this extension is through [composer](http://getcomposer.org/download/).
## Requirements

Either run
- PHP 8.0 or higher.

```shell
composer require yiisoft/queue
```
## Installation

or add
The package could be installed with [Composer](https://getcomposer.org):

```shell
composer require yiisoft/queue
```
"yiisoft/queue": "~3.0"
```

to the `require` section of your `composer.json` file.

## Ready for yiisoft/config

Expand All @@ -53,7 +45,7 @@ change to start working with the queue:
If you have experience with `yiisoft/yii2-queue`, you will find out that this package is similar.
Though, there are some key differences which are described in the "[migrating from yii2-queue](docs/guide/migrating-from-yii2-queue.md)" article.

## Basic Usage
## General usage

Each queue task consists of two parts:

Expand Down Expand Up @@ -283,31 +275,20 @@ You can declare error handling middleware pipeline in the `FailureMiddlewareDisp

See [error handling docs](docs/guide/error-handling.md) for details.

## Extra

### Unit testing

The package is tested with [PHPUnit](https://phpunit.de/). To run tests:

```shell
./vendor/bin/phpunit
```

### Mutation testing
## Documentation

The package tests are checked with [Infection](https://infection.github.io/) mutation framework. To run it:
- Guide: [English](docs/guide/en/README.md), [Português - Brasil](docs/guide/pt-BR/README.md)
- [Internals](docs/internals.md)

```shell
./vendor/bin/infection
```
If you need help or have a question, the [Yii Forum](https://forum.yiiframework.com/c/yii-3-0/63) is a good place for that.
You may also check out other [Yii Community Resources](https://www.yiiframework.com/community).

### Static analysis
## License

The code is statically analyzed with [Psalm](https://psalm.dev/). To run static analysis:
The Yii Queue Extension is free software. It is released under the terms of the BSD License.
Please see [`LICENSE`](./LICENSE.md) for more information.

```shell
./vendor/bin/psalm
```
Maintained by [Yii Software](https://www.yiiframework.com/).

### Support the project

Expand All @@ -320,10 +301,3 @@ The code is statically analyzed with [Psalm](https://psalm.dev/). To run static
[![Telegram](https://img.shields.io/badge/telegram-join-1DA1F2?style=flat&logo=telegram)](https://t.me/yii3en)
[![Facebook](https://img.shields.io/badge/facebook-join-1DA1F2?style=flat&logo=facebook&logoColor=ffffff)](https://www.facebook.com/groups/yiitalk)
[![Slack](https://img.shields.io/badge/slack-join-1DA1F2?style=flat&logo=slack)](https://yiiframework.com/go/slack)

## License

The Yii Queue Extension is free software. It is released under the terms of the BSD License.
Please see [`LICENSE`](./LICENSE.md) for more information.

Maintained by [Yii Software](https://www.yiiframework.com/).
7 changes: 2 additions & 5 deletions UPGRADE.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,3 @@
Upgrading Instructions
======================

This file contains the upgrade notes. These notes highlight changes that could break your
application when you upgrade the package from one version to another.
# Yii Queue Extension Upgrading Instructions

## Changes summary
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
11 changes: 11 additions & 0 deletions docs/guide/pt-BR/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
# Yii Queue extension

An extension for running tasks asynchronously via queues.

## Guides and concept explanations

* [Usage basics](usage.md)
* [Migrating from `yii2-queue`](migrating-from-yii2-queue.md)
* [Errors and retryable jobs](error-handling.md)
* [Workers](worker.md)
* [Adapter list](adapter-list.md)
5 changes: 5 additions & 0 deletions docs/guide/pt-BR/adapter-list.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
Queue Adapters
-------------

* [Synchronous](adapter-sync.md) - adapter for development and test environments
* [AMQP](https://github.com/yiisoft/queue-amqp) - adapter over AMQP protocol via [amqplib](https://github.com/php-amqplib/php-amqplib)
21 changes: 21 additions & 0 deletions docs/guide/pt-BR/adapter-sync.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
Synchronous Adapter
==================

Run tasks synchronously in the same process. It could be used when developing and debugging an application.

Configuration example:

```php
$logger = $DIContainer->get(\Psr\Log\LoggerInterface::class);

$worker = $DIContainer->get(\Yiisoft\Queue\Worker\WorkerInterface::class);
$loop = $DIContainer->get(\Yiisoft\Queue\Cli\LoopInterface::class);
$adapter = new Yiisoft\Queue\Adapter\SynchronousAdapter($loop, $worker);

$queue = new Yiisoft\Queue\Queue(
$adapter,
$worker,
$loop,
$logger
);
```
90 changes: 90 additions & 0 deletions docs/guide/pt-BR/error-handling.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,90 @@
# Error handling on message processing

Often when some message handling is failing, we want to retry its execution a couple more times or redirect it to another queue channel. This can be done in `yiisoft/queue` with _Failure Handling Middleware Pipeline_. It is triggered each time message processing via Consume Middleware Pipeline is interrupted with any `Throwable`.

## Configuration

Here below is configuration via `yiisoft/config`. If you don't use it - you should add middleware definition list (in the `middlewares-fail` key here) to the `FailureMiddlewareDispatcher` by your own.

Configuration should be passed to the `yiisoft/queue.fail-strategy-pipelines` key of the `params` config to work with the `yiisoft/config`. You can define different failure handling pipelines for each queue channel. Let's see and describe an example:

```php
'yiisoft/queue' => [
'middlewares-fail' => [
FailureMiddlewareDispatcher::DEFAULT_PIPELINE => [
[
'class' => SendAgainMiddleware::class,
'__construct()' => ['id' => 'default-first-resend', 'queue' => null],
],
static fn (QueueFactoryInterface $factory) => new SendAgainMiddleware(
id: 'default-second-resend',
queue: $factory->get('failed-messages'),
),
],

'failed-messages' => [
[
'class' => ExponentialDelayMiddleware::class,
'__construct()' => [
'id' => 'failed-messages',
'maxAttempts' => 30,
'delayInitial' => 5,
'delayMaximum' => 60,
'exponent' => 1.5,
'queue' => null,
],
],
],
],
]
```

Keys here except `FailureMiddlewareDispatcher::DEFAULT_PIPELINE` are queue channel names, and values are lists of `FailureMiddlewareInterface` definitions. `FailureMiddlewareDispatcher::DEFAULT_PIPELINE` defines a default pipeline to apply to channels without explicitly defined failure strategy pipeline. Each middleware definition must be one of:
- A ready-to-use `MiddlewareFailureInterface` object like `new FooMiddleware()`.
- A valid definition for the [yiisoft/definitions](https://github.com/yiisoft/definitions). It must describe an object, implementing the `MiddlewareFailureInterface`.
- A callable: `fn() => // do stuff`, `$object->foo(...)`, etc. It will be executed through the `yiisoft/injector`, so all the dependencies of your callable will be resolved. You can also define a "callable-looking" array, where object will be instantiated with a DI container: `[FooMiddleware::class, 'handle']`.
- A string for your DI container to resolve the middleware, e.g. `FooMiddleware::class`.

In the example above failures will be handled this way (look the concrete middleware description below):

1. For the first time message will be resent to the same queue channel immediately.
2. If it fails again, it will be resent to the queue channel named `failed-messages`.
3. From now on it will be resent to the same queue channel (`failed-messages`) up to 30 times with a delay from 5 to 60 seconds, increased 1.5 times each time the message fails again.
4. If the message handler throw an exception one more (33rd) time, the exception will not be caught.

Failures of messages, which are initially sent to the `failed-messages` channel, will only be handled by the 3rd and the 4th points of this list.

## Default failure handling strategies

Let's see the built-in defaults.

### SendAgainMiddleware

This strategy simply resends the given message to a queue. Let's see the constructor parameters through which it's configured:

- `id` - A unique string. Allows to use this strategy more than once for the same message, just like in example above.
- `maxAttempts` - Maximum attempts count for this strategy with the given $id before it will give up.
- `queue` - The strategy will send the message to the given queue when it's not `null`. That means you can use this strategy to push a message not to the same queue channel it came from. When the `queue` parameter is set to `null`, a message will be sent to the same channel it came from.

### ExponentialDelayMiddleware

This strategy does the same thing as the `SendAgainMiddleware` with a single difference: it resends a message with an exponentially increasing delay. The delay **must** be implemented by the used `AdapterInterface` implementation.

It's configured via constructor parameters, too. Here they are:

- `id` - A unique string allows to use this strategy more than once for the same message, just like in example above.
- `maxAttempts` - Maximum attempts count for this strategy with the given $id before it will give up.
- `delayInitial` - The initial delay that will be applied to a message for the first time. It must be a positive float.
- `delayMaximum` - The maximum delay which can be applied to a single message. Must be above the `delayInitial`.
- `exponent` - Message handling delay will be muliplied by exponent each time it fails.
- `queue` - The strategy will send the message to the given queue when it's not `null`. That means you can use this strategy to push a message not to the same queue channel it came from. When the `queue` parameter is set to `null`, a message will be sent to the same channel it came from.

## How to create a custom Failure Middleware?

All you need is to implement the `MiddlewareFailureInterface` and add your implementation definition to the [configuration](#configuration).
This interface has the only method `handle`. And the method has these parameters:
- `ConsumeRequest $request` - a request for a message handling. It consists of a message and a queue the message came from.
- `Throwable $exception` - an exception thrown on the `request` handling
- `MessageFailureHandlerInterface $handler` - failure strategy pipeline continuation. Your Middleware should call `$pipeline->handle()` when it shouldn't interrupt failure pipeline execution.

> Note: your strategy have to check by its own if it should be applied. Look into [`SendAgainMiddleware::suites()`](../../src/Middleware/Implementation/FailureMiddleware/Middleware/SendAgainMiddleware.php#L52) for an example.
Empty file added docs/guide/pt-BR/loops.md
Empty file.
32 changes: 32 additions & 0 deletions docs/guide/pt-BR/migrating-from-yii2-queue.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
# Migrating from yii2-queue

This package is similar to [yiisoft/yii2-queue] but with improved design and code style. The new package less coupled
and more structured than the old one allowing better maintenance.

## Adapters

- Individual adapters are now separate packages. This means each adapter must be `require`d with composer in order to
be available in your application.
- Adapter may be any class which implements `AdapterInterface`. This means you can replace one adapter with another without
changing any code in your app. For example, you can use `db` adapter in development while using `amqp` in production,
then you may seamlessly switch to `redis` if needed. This also means you can write your own adapter implementation
if necessary.

## Jobs (Messages and Handlers)

There was a concept in [yiisoft/yii2-queue] called `Job`: you had to push it to the queue, and it was executed after
being consumed. In the new package it is divided into two different concepts: a message and a handler.

- A `Message` is a class implementing `MessageInterface`. It contains 2 types of data:
- Name. Worker uses it to find the right handler for a message.
- Data. Any serializable data which should be used by the message handler.

All the message data is fully serializable (that means message `data` must be serializable too). It allows you to
freely choose where and how to send and process jobs. Both can be implemented in a single application, or
separated into multiple applications, or you can do sending/processing only leaving part of the job to another
system including non-PHP ones. It is fairly popular to process heavy jobs with Go.

- A `Handler` is called by a `Worker` when a message comes. Default `Worker` finds a corresponding message handler
by the message name. [See more](worker.md#handler-format).

[yiisoft/yii2-queue]: (https://github.com/yiisoft/yii2-queue)
86 changes: 86 additions & 0 deletions docs/guide/pt-BR/usage.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,86 @@
# Usage basics

## Configuration

You can configure it with a DI container in the following way:

```php
$logger = $DIContainer->get(\Psr\Log\LoggerInterface::class);

$worker = $DIContainer->get(\Yiisoft\Queue\Worker\WorkerInterface::class);
$loop = $DIContainer->get(\Yiisoft\Queue\Cli\LoopInterface::class);
$adapter = $DIContainer->get(\Yiisoft\Queue\Adapter\AdapterInterface::class);

$queue = new Queue(
$adapter,
$worker,
$loop,
$logger
);
```

See also the documentation for concrete adapters ([synchronous adapter](adapter-sync.md),
[AMQP adapter](https://github.com/yiisoft/queue-amqp)) and [workers](worker.md)


## Usage

Each job sent to the queue should be defined as a separate class.
For example, if you need to download and save a file the class may look like the following:

```php
$data = [
'url' => $url,
'destinationFile' => $filename,
];
$message = new \Yiisoft\Queue\Message\Message('file-download', $data);
```

Here's how to send a task to the queue:

```php
$queue->push($message);
```

To push a job into the queue that should run after 5 minutes:

```php
// TODO
```

**Important:** Not every adapter (such as synchronous adapter) supports delayed execution.


## Queue handling

The exact way how a job is executed depends on the adapter used. Most adapters can be run using
console commands, which the component registers in your application. For more details check the respective
adapter documentation.


## Job status

```php
// Push a job into the queue and get a message ID.
$id = $queue->push(new SomeJob());

// Get job status.
$status = $queue->status($id);

// Check whether the job is waiting for execution.
$status->isWaiting();

// Check whether a worker got the job from the queue and executes it.
$status->isReserved($id);

// Check whether a worker has executed the job.
$status->isDone($id);
```

## Limitations

When using queues it is important to remember that tasks are put into and obtained from the queue in separate
processes. Therefore, avoid external dependencies when executing a task if you're not sure if they are available in
the environment where the worker does its job.

All the data to process the task should be provided with your payload `getData()` method.
Loading

0 comments on commit de8c49b

Please sign in to comment.