Load Balancing with Queues
The property of queues, that requires that any given message is delivered only to a single consumer, provides a reliable mechanism to load balance workload between multiple consumers of the queue.
In addition to a simple round-robin load-balancing between consumers (which is the default for queues), ActiveMQ offers more advanced features such as:
- Exclusive consumers, where a given consumer receives all messages until it fails (or disconnects) and then the broker designates another consumer as the new exclusive consumer.
- Batch message prefetching per consumer, where a consumer will receive a set of messages before the queue forwards the next batch of messages to another consumer.
Queue-based load-balancing paired with a master-slave broker setup and a persistent message store results in a fully redundant message processing node.
Load Balancing with Topics?
Unlike queues, topics deliver a copy of the message to each consumer (topic dispatch policies allow violating this rule). So queues are not suitable for load balancing.
Nevertheless, topics are popular, because they are used as a publish-subscribe abstraction, where the publisher disseminates events to the topic, and any number of subscribers receive the event.
Since load balancing is not on option with topics, when a topic subscriber fails, the subsystem that relied on the subscriber is cut-off from any messages published from the topic.
Virtual topics provide an elegant solution to this problem. With a virtual topic, the publisher publishes to a topic, however subscribers can choose to either consume from the topic or from a queue linked to the topic. Thus incoming messages are distributed amongst the queue subscribers.
In addition consuming from a queue attached to a virtual topic allows enabling multiple concurrent consumers which would not have been possible, had the consumers been attached directly to the topic.