Pulsar load balance
Load balance across Pulsar brokers​
Pulsar is an horizontally scalable messaging system, so the traffic in a logical cluster must be balanced across all the available Pulsar brokers as evenly as possible, which is a core requirement.
You can use multiple settings and tools to control the traffic distribution which require a bit of context to understand how the traffic is managed in Pulsar. Though, in most cases, the core requirement mentioned above is true out of the box and you should not worry about it.
Pulsar load manager architecture​
The following part introduces the basic architecture of the Pulsar load manager.
Assign topics to brokers dynamically​
Topics are dynamically assigned to brokers based on the load conditions of all brokers in the cluster.
When a client starts using new topics that are not assigned to any broker, a process is triggered to choose the best suited broker to acquire ownership of these topics according to the load conditions.
In case of partitioned topics, different partitions are assigned to different brokers. Here "topic" means either a non-partitioned topic or one partition of a topic.
The assignment is "dynamic" because the assignment changes quickly. For example, if the broker owning the topic crashes, the topic is reassigned immediately to another broker. Another scenario is that the broker owning the topic becomes overloaded. In this case, the topic is reassigned to a less loaded broker.
The stateless nature of brokers makes the dynamic assignment possible, so you can quickly expand or shrink the cluster based on usage.