There are different ways to monitor a Pulsar cluster, exposing both metrics relative to the usage of topics and the overall health of the individual components of the cluster.
Pulsar broker metrics can be collected from brokers and exported in JSON format. There are two main types of metrics:
Destination dumps, which containing stats for each individual topic. They can be fetched using
bin/pulsar-admin broker-stats destinations
Broker metrics, containing broker info and topics stats aggregated at namespace level:
bin/pulsar-admin broker-stats monitoring-metrics
All the message rates are updated every 1min.
The aggregated broker metrics are also exposed in the Prometheus format at:
The local Zookeeper/configuration store server and clients that are shipped with Pulsar have been instrumented to expose detailed stats through Prometheus as well.
The default port of local ZooKeeper is
8000 and that of configuration store is
These can be changed by specifying system property
For BookKeeper you can configure the stats frameworks by changing the
By default, the default BookKeeper configuration included with Pulsar distribution will enable the Prometheus exporter.
For bookies, the default port is
8000 (instead of
8080) and that can be configured by changing
You can configure Prometheus to collect and store the metrics data by following the Prometheus Getting started guide.
When running on bare metal, you can provide the list of nodes that needs to be probed. When deploying in a Kubernetes cluster, the monitoring is automatically setup with the provided instructions.
When collecting time series statistics, the major problem is to make sure the number of dimensions attached to the data does not explode.
For that reason we only collect time series of metrics aggregated at the namespace level.
Pulsar per-topic dashboard
The per-topic dashboard instructions are available at Dashboard.
You can use grafana to easily create dashboard driven by the data stored in Prometheus.
There is a
pulsar-grafana Docker image that is ready to use with the principal dashboards already
in place. This is enabled by default when deploying Pulsar on Kubernetes.
To use the dashboard manually:
docker run -p3000:3000 \ -e PROMETHEUS_URL=http://$PROMETHEUS_HOST:9090/ \ apachepulsar/pulsar-grafana:latest