Managing Clusters
Important
This page only shows some frequently used operations.
For the latest and complete information about
Pulsar admin
, including commands, flags, descriptions, and more, see Pulsar admin docFor the latest and complete information about
REST API
, including parameters, responses, samples, and more, see REST API doc.For the latest and complete information about
Java admin API
, including classes, methods, descriptions, and more, see Java admin API doc.
Pulsar clusters consist of one or more Pulsar brokers, one or more BookKeeper servers (aka bookies), and a ZooKeeper cluster that provides configuration and coordination management.
Clusters can be managed via:
- The
clusters
command of thepulsar-admin
tool - The
/admin/v2/clusters
endpoint of the admin REST API - The
clusters
method of thePulsarAdmin
object in the Java API
Clusters resources​
Provision​
New clusters can be provisioned using the admin interface.
Please note that this operation requires superuser privileges.
- pulsar-admin
- REST API
- JAVA
You can provision a new cluster using the create
subcommand. Here's an example:
$ pulsar-admin clusters create cluster-1 \
--url http://my-cluster.org.com:8080 \
--broker-url pulsar://my-cluster.org.com:6650
ClusterData clusterData = new ClusterData(
serviceUrl,
serviceUrlTls,
brokerServiceUrl,
brokerServiceUrlTls
);
admin.clusters().createCluster(clusterName, clusterData);
Initialize cluster metadata​
When provision a new cluster, you need to initialize that cluster's metadata. When initializing cluster metadata, you need to specify all of the following:
- The name of the cluster
- The local metadata store connection string for the cluster
- The configuration store connection string for the entire instance
- The web service URL for the cluster
- A broker service URL enabling interaction with the brokers in the cluster
You must initialize cluster metadata before starting up any brokers that will belong to the cluster.
No cluster metadata initialization through the REST API or the Java admin API
Unlike most other admin functions in Pulsar, cluster metadata initialization cannot be performed via the admin REST API or the admin Java client, as metadata initialization involves communicating with ZooKeeper directly. Instead, you can use the
pulsar
CLI tool, in particular theinitialize-cluster-metadata
command.
Here's an example cluster metadata initialization command:
bin/pulsar initialize-cluster-metadata \
--cluster us-west \
--metadata-store zk:zk1.us-west.example.com:2181,zk2.us-west.example.com:2181/my-chroot-path \
--configuration-metadata-store zk:zk1.us-west.example.com:2181,zk2.us-west.example.com:2181/my-chroot-path \
--web-service-url http://pulsar.us-west.example.com:8080/ \
--web-service-url-tls https://pulsar.us-west.example.com:8443/ \
--broker-service-url pulsar://pulsar.us-west.example.com:6650/ \
--broker-service-url-tls pulsar+ssl://pulsar.us-west.example.com:6651/
You'll need to use --*-tls
flags only if you're using TLS authentication in your instance.
Get configuration​
You can fetch the configuration for an existing cluster at any time.
- pulsar-admin
- REST API
- JAVA
Use the get
subcommand and specify the name of the cluster. Here's an example:
$ pulsar-admin clusters get cluster-1
{
"serviceUrl": "http://my-cluster.org.com:8080/",
"serviceUrlTls": null,
"brokerServiceUrl": "pulsar://my-cluster.org.com:6650/",
"brokerServiceUrlTls": null
"peerClusterNames": null
}
admin.clusters().getCluster(clusterName);
Update​
You can update the configuration for an existing cluster at any time.
- pulsar-admin
- REST API
- JAVA
Use the update
subcommand and specify new configuration values using flags.
$ pulsar-admin clusters update cluster-1 \
--url http://my-cluster.org.com:4081 \
--broker-url pulsar://my-cluster.org.com:3350
ClusterData clusterData = new ClusterData(
serviceUrl,
serviceUrlTls,
brokerServiceUrl,
brokerServiceUrlTls
);
admin.clusters().updateCluster(clusterName, clusterData);
Delete​
Clusters can be deleted from a Pulsar instance.
- pulsar-admin
- REST API
- JAVA
Use the delete
subcommand and specify the name of the cluster.
$ pulsar-admin clusters delete cluster-1
admin.clusters().deleteCluster(clusterName);
List​
You can fetch a list of all clusters in a Pulsar instance.
- pulsar-admin
- REST API
- JAVA
Use the list
subcommand.
$ pulsar-admin clusters list
cluster-1
cluster-2
admin.clusters().getClusters();
Update peer-cluster data​
Peer clusters can be configured for a given cluster in a Pulsar instance.
- pulsar-admin
- REST API
- JAVA
Use the update-peer-clusters
subcommand and specify the list of peer-cluster names.
$ pulsar-admin update-peer-clusters cluster-1 --peer-clusters cluster-2
admin.clusters().updatePeerClusterNames(clusterName, peerClusterList);