Skip to main content
Version: Next

Pulsar isolation

In an organization, a Pulsar instance provides services to multiple teams. When organizing the resources across multiple teams, you want to make a suitable isolation plan to avoid the resource competition between different teams and applications and provide high-quality messaging service. In this case, you need to take resource isolation into consideration and weigh your intended actions against expected and unexpected consequences.

The multi-layer and segment-centric architecture and hierarchical resource management of Pulsar provide a solid foundation for isolation, which allows you to isolate resources in your desired manner, prevent resource competition, and attain stability.

Isolation methods

To enforce resource isolation within Pulsar, you can use the following methods of isolating broker and bookie resources:

Deployments to achieve isolation within Pulsar

Separate Pulsar clusters

The following illustration demonstrates the deployment of separate Pulsar clusters to achieve the highest-level isolation.

Deployment of separate Pulsar clusters

Here are some key points for understanding how it works:

  • Separate Pulsar clusters use a shared configuration store.
  • Each cluster exposes its service through a DNS entry point and makes sure a client can access the cluster through the DNS entry point. Clients can use one or multiple Pulsar URLs that the Pulsar cluster exposes as the service URL.
  • Each Pulsar cluster has one or multiple brokers and bookies.
  • Each Pulsar cluster has one metadata store, which can be separated into Pulsar metadata store and BookKeeper metadata store.
note

When using this approach, if you want to achieve namespace isolation, you need to specify a cluster for a namespace. The cluster must be in the allowed cluster list of the tenant. Topics under the namespace are assigned to this cluster.

Shared BookKeeper cluster

The following illustration demonstrates the deployment of shared BookKeeper clusters to achieve isolation.

Deployment of shared BookKeeper cluster

Here are some key points for understanding how it works:

  • Separate Pulsar clusters share a BookKeeper cluster and a configuration store.
  • Each cluster exposes its service through a DNS entry point and makes sure a client can access the cluster through the DNS entry point. Clients can use one or multiple Pulsar URLs that the Pulsar cluster exposes as the service URL.
  • Each Pulsar cluster has one or multiple brokers.
  • Each Pulsar cluster has one metadata store.

As illustrated below, storage isolation is achieved by setting bookie affinity groups. All bookie groups use a shared BookKeeper cluster and a metadata store, and each bookie isolation group has one or several bookies. You can specify one or multiple primary/secondary groups for a namespace. Topics under the namespace are created on the bookies in the primary group firstly and then created on the bookies in the secondary group.

Storage isolation achieved by bookie affinity groups

Single Pulsar cluster

The following illustration demonstrates how to achieve isolation inside a single Pulsar cluster.

Deployment of a single Pulsar cluster

Here are some key points for understanding how it works:

  • Each cluster exposes its service through a DNS entry point and makes sure a client can access the cluster through the DNS entry point. Clients can use one or multiple Pulsar URLs that the Pulsar cluster exposes as the service URL.
  • Broker isolation is achieved by setting namespace isolation policy.