The Pulsar Perf is a built-in performance test tool for Apache Pulsar. You can use the Pulsar Perf to test message writing or reading performance. For detailed information about performance tuning, see here.
From the above test data, you can get the throughput statistics and the write latency statistics. The aggregated statistics is printed when the Pulsar Perf is stopped. You can press Ctrl+C to stop the Pulsar Perf. After the Pulsar Perf is stopped, the HdrHistogram formatted test result appears under your directory. The document looks like perf-producer-1589370810837.hgrm. You can also check the test result through HdrHistogram Plotter. For details about how to check the test result through HdrHistogram Plotter, see HdrHistogram Plotter.
This example shows how the Pulsar Perf produces messages with the transaction option.
Input
bin/pulsar-perf produce my-topic -r10-m100-txn
Output
2021-10-11T13:36:15,595+0800 INFO [Thread-3] o.a.p.t.PerformanceProducer@499 - --- Transaction :2 transaction end successfully ---0 transaction end failed --- 0.200 Txn/s 2021-10-11T13:36:15,614+0800 INFO [Thread-3] o.a.p.t.PerformanceProducer@503 - Throughput produced: 100 msg --- 0.0 msg/s --- 0.1 Mbit/s --- failure 0.0 msg/s --- Latency: mean: 3.067 ms - med: 3.104 - 95pct: 3.747 - 99pct: 4.619 - 99.9pct: 6.760 - 99.99pct: 6.760 - Max: 6.760 2021-10-11T13:36:15,710+0800 INFO [pulsar-perf-producer-exec-46-1] o.a.p.t.PerformanceProducer@834 - Aggregated latency stats --- Latency: mean: 3.067 ms - med: 3.104 - 95pct: 3.747 - 99pct: 4.619 - 99.9pct: 6.760 - 99.99pct: 6.760 - 99.999pct: 6.760 - Max: 6.760 2021-10-11T13:36:29,976+0800 INFO [Thread-4] o.a.p.t.PerformanceProducer@815 - --- Transaction :2 transaction end successfully --- 0 transaction end failed --- 2 transaction open successfully --- 0 transaction open failed --- 12.237 Txn/s 2021-10-11T13:36:29,976+0800 INFO [Thread-4] o.a.p.t.PerformanceProducer@824 - Aggregated throughput stats --- 102 records sent --- 4.168 msg/s --- 0.033 Mbit/s
For the latest and complete information about pulsar-perf, including commands, flags, descriptions, and more, see pulsar-perf.
This example shows how the Pulsar Perf consumes messages with default options.
Input
note
If you have not created a topic (in this example, it is my-topic) before, the broker creates a new topic without partitions and messages, then the consumer can not receive any messages. Consequently, before using pulsar-perf consume, make sure your topic has enough messages to consume.
bin/pulsar-perf consume my-topic
After the command is executed, the test data is continuously output on the Console.
From the output test data, you can get the throughput statistics and the end-to-end latency statistics. The aggregated statistics are printed after the Pulsar Perf is stopped. You can press Ctrl+C to stop the Pulsar Perf.
This example shows how the Pulsar Perf consumes messages with the transaction option.
If you have not created a topic (in this example, it is my-topic) before, the broker creates a new topic without partitions and messages, then the consumer can not receive any messages. Consequently, before using pulsar-perf consume, make sure your topic has enough messages to consume.
Output
2021-10-11T13:43:36,052+0800 INFO [Thread-3] o.a.p.t.PerformanceConsumer@538 - --- Transaction: 6 transaction end successfully --- 0 transaction end failed --- 0.199 Txn/s --- AckRate: 9.952 msg/s 2021-10-11T13:43:36,065+0800 INFO [Thread-3] o.a.p.t.PerformanceConsumer@545 - Throughput received: 306 msg --- 9.952 msg/s -- 0.000 Mbit/s --- Latency: mean: 26177.380 ms - med: 26128 - 95pct: 30531 - 99pct: 30923 - 99.9pct: 31021 - 99.99pct: 31021 - Max: 31021 2021-10-11T13:43:59,854+0800 INFO [Thread-5] o.a.p.t.PerformanceConsumer@579 - -- Transaction: 10 transaction end successfully --- 0 transaction end failed --- 10 transaction open successfully --- 0 transaction open failed --- 0.185 Txn/s 2021-10-11T13:43:59,854+0800 INFO [Thread-5] o.a.p.t.PerformanceConsumer@588 - Aggregated throughput stats --- 505 records received --- 9.345 msg/s --- 0.000 Mbit/s--- AckRate: 9.27065308842743 msg/s --- ack failed 4 msg 2021-10-11T13:43:59,882+0800 INFO [Thread-5] o.a.p.t.PerformanceConsumer@601 - Aggregated latency stats --- Latency: mean: 50593.000 ms - med: 50593 - 95pct: 50593 - 99pct: 50593 - 99.9pct: 50593 - 99.99pct: 50593 - 99.999pct: 50593 - Max: 50593
If you have not created a topic (in this example, it is myConsumerTopic) before, the broker creates a new topic without partitions and messages, then the consumer can not receive any messages. Consequently, before using pulsar-perf transaction, make sure your topic has enough messages to consume.
If you have not created a topic (in this example, it is myConsumerTopic) before, the broker creates a new topic without partitions and messages, then the consumer can not receive any messages. Consequently, before using pulsar-perf transaction --txn-disEnable, make sure your topic has enough messages to consume.
By default, the Pulsar Perf uses conf/client.conf as the default configuration and uses conf/log4j2.yaml as the default Log4j configuration. If you want to connect to other Pulsar clusters, you can update the brokerServiceUrl in the client configuration.
You can use the following commands to change the configuration file and the Log4j configuration file.