[Host.RabbitMq] Multiple consumers with varying concurrency

[robert94p]

how to process messages in parallel when using RabbitMq?

[zarusz]

hello @robert94p ,

By default, the underlying RabbitMq net driver uses a concurrency of 1.
Following this it can be adjusted on the ConnectionFactory object.

For example, try to set this via SMB like so:

services.AddSlimMessageBus((mbb) =>
{
    mbb.WithProviderRabbitMQ(cfg =>
    {
        cfg.ConnectionFactory.ConsumerDispatchConcurrency = 2; // default is 1
        // ...
    }
}

Please let me know if this worked for you.

[robert94p]

registered several buses for one consumer. While it works

[zarusz]

Did you try the provided suggestion (with ConsumerDispatchConcurrency setting)?
Sounds like you've taken another route, can you explain what you mean several buses for one consumer?
Thanks

[zarusz]

As part of #207 I have updated the docs to include information on how to increase concurrency.

If you run into any problems, please re-open the issue and let me know.

Thanks!

[zarusz]

Moving the following from @robert94p #310 (comment):

Apologies, I wrote this under the wrong issue; it should have been under https://github.com/zarusz/SlimMessageBus/issues/205

I need 2 or more consumers. I have one producer, one exchange, and three queues for three different consumers. When publishing a message, the target queue is determined via the routingKey.

One of the consumers performs a resource-intensive operation, so it needs to be limited to processing only 2 messages in parallel. The other consumers can handle more messages simultaneously.

I would like to configure the number of messages that can be processed concurrently per consumer, similar to how it works in MemoryMessageBus. Currently, messages are processed based on the ConsumerDispatchConcurrency setting, which applies globally to all consumers.

[zarusz]

@robert94p I believe this is a good example of what you are trying to achieve, correct?
https://www.rabbitmq.com/tutorials/tutorial-four-dotnet

Here is the preview release to try out:
https://www.nuget.org/packages/SlimMessageBus.Host.RabbitMQ/2.6.2-rc10

Here is a pseudo sample:

services.AddSlimMessageBus(mbb => 
{
  mbb.Produce<PingMessage>(x => x
        .Exchange("exchange", exchangeType: ExchangeType.Direct)
        .RoutingKeyProvider((m, p) => m.Label));

  // messages with blue routing key will get 1 concurrency
  mbb.Consume<PingMessage>(x => x
      .Queue("queue", autoDelete: false)
      .ExchangeBinding("orders", routingKey: "blue")
      .Instances(1));

  // messages with red routing key will get 10 concurrency
  mbb.Consume<PingMessage>(x => x
      .Queue("queue", autoDelete: false)
      .ExchangeBinding("orders", routingKey: "red") 
      .Instances(10));  
});

Let me know if that works for you.

[zarusz]

I just pushed a refined version:
https://www.nuget.org/packages/SlimMessageBus.Host.RabbitMQ/2.6.2-rc12

With that, the overall concurrency should be improved.