Skip to main content

Setting up a local standalone cluster

For the purposes of local development and testing, you can run Pulsar in standalone mode on your own machine. Standalone mode includes a Pulsar broker as well as the necessary ZooKeeper and BookKeeper components running inside of a single Java Virtual Machine (JVM) process.

Pulsar in production?​

If you're looking to run a full production Pulsar installation, see the Deploying a Pulsar instance guide.

Run Pulsar Standalone Manually​

System requirements​

Currently, Pulsar is available for 64-bit macOS, Linux, and Windows. To use Pulsar, you need to install 64-bit JRE/JDK 8 or later versions.

Installing Pulsar​

To get started running Pulsar, download a binary tarball release in one of the following ways:

Once the tarball is downloaded, untar it and cd into the resulting directory:


$ tar xvfz apache-pulsar-2.2.1-bin.tar.gz
$ cd apache-pulsar-2.2.1

What your package contains​

The Pulsar binary package initially contains the following directories:

DirectoryContains
binPulsar's command-line tools, such as pulsar and pulsar-admin
confConfiguration files for Pulsar, including for broker configuration, ZooKeeper configuration, and more
examplesA Java JAR file containing example Pulsar Functions
libThe JAR files used by Pulsar
licensesLicense files, in .txt form, for various components of the Pulsar codebase

These directories will be created once you begin running Pulsar:

DirectoryContains
dataThe data storage directory used by ZooKeeper and BookKeeper
instancesArtifacts created for Pulsar Functions
logsLogs created by the installation

Installing Builtin Connectors (optional)​

Since release 2.1.0-incubating, Pulsar releases a separate binary distribution, containing all the builtin connectors. If you would like to enable those builtin connectors, you can download the connectors tarball release in one of the following ways:

Once the tarball is downloaded, in the pulsar directory, untar the io-connectors package and copy the connectors as connectors in the pulsar directory:


$ tar xvfz /path/to/apache-pulsar-io-connectors-2.2.1-bin.tar.gz

// you will find a directory named `apache-pulsar-io-connectors-2.2.1` in the pulsar directory
// then copy the connectors

$ cd apache-pulsar-io-connectors-2.2.1/connectors connectors

$ ls connectors
pulsar-io-aerospike-2.2.1.nar
pulsar-io-cassandra-2.2.1.nar
pulsar-io-kafka-2.2.1.nar
pulsar-io-kinesis-2.2.1.nar
pulsar-io-rabbitmq-2.2.1.nar
pulsar-io-twitter-2.2.1.nar
...

NOTES​

If you are running Pulsar in a bare metal cluster, you need to make sure connectors tarball is unzipped in every broker's pulsar directory (or in every function-worker's pulsar directory if you are running a separate worker cluster for Pulsar functions).

If you are running Pulsar in Docker or deploying Pulsar using a docker image (e.g. K8S or DC/OS), you can use apachepulsar/pulsar-all image instead of apachepulsar/pulsar image. apachepulsar/pulsar-all image has already bundled all builtin connectors.

Installing Tiered Storage Offloaders (optional)​

Since release 2.2.0, Pulsar releases a separate binary distribution, containing the tiered storage offloaders. If you would like to enable tiered storage feature, you can follow the instructions as below; otherwise you can skip this section for now.

To get started using tiered storage offloaders, you'll need to download the offloaders tarball release on every broker node in one of the following ways:

Once the tarball is downloaded, in the pulsar directory, untar the offloaders package and copy the offloaders as offloaders in the pulsar directory:


$ tar xvfz apache-pulsar-offloaders-2.2.1-bin.tar.gz

// you will find a directory named `apache-pulsar-offloaders-2.2.1` in the pulsar directory
// then copy the offloaders

$ mv apache-pulsar-offloaders-2.2.1/offloaders offloaders

$ ls offloaders
tiered-storage-jcloud-2.2.1.nar

For more details of how to configure tiered storage feature, you could reference this Tiered storage cookbook

NOTES​

If you are running Pulsar in a bare metal cluster, you need to make sure offloaders tarball is unzipped in every broker's pulsar directory

If you are running Pulsar in Docker or deploying Pulsar using a docker image (e.g. K8S or DC/OS), you can use apachepulsar/pulsar-all image instead of apachepulsar/pulsar image. apachepulsar/pulsar-all image has already bundled tiered storage offloaders.

Starting the cluster​

Once you have an up-to-date local copy of the release, you can start up a local cluster using the pulsar command, which is stored in the bin directory, and specifying that you want to start up Pulsar in standalone mode:


$ bin/pulsar standalone

If Pulsar has been successfully started, you should see INFO-level log messages like this:


2017-06-01 14:46:29,192 - INFO  - [main:WebSocketService@95] - Configuration Store cache started
2017-06-01 14:46:29,192 - INFO  - [main:AuthenticationService@61] - Authentication is disabled
2017-06-01 14:46:29,192 - INFO  - [main:WebSocketService@108] - Pulsar WebSocket Service started

Automatically created namespace​

When you start a local standalone cluster, Pulsar will automatically create a public/default namespace that you can use for development purposes. All Pulsar topics are managed within namespaces. For more info, see Topics.

Run Pulsar Standalone in Docker​

Alternatively, you can run pulsar standalone locally in docker.


docker run -it -p 80:80 -p 8080:8080 -p 6650:6650 apachepulsar/pulsar-standalone

The command forwards following port to localhost:

  • 80: the port for pulsar dashboard
  • 8080: the http service url for pulsar service
  • 6650: the binary protocol service url for pulsar service

After the docker container is running, you can access the dashboard under http://localhost .

Testing your cluster setup​

Pulsar provides a CLI tool called pulsar-client that enables you to do things like send messages to a Pulsar topic in a running cluster. This command will send a simple message saying hello-pulsar to the my-topic topic:


$ bin/pulsar-client produce my-topic --messages "hello-pulsar"

If the message has been successfully published to the topic, you should see a confirmation like this in the pulsar-client logs:


13:09:39.356 [main] INFO  org.apache.pulsar.client.cli.PulsarClientTool - 1 messages successfully produced

No need to explicitly create new topics​

You may have noticed that we did not explicitly create the my-topic topic to which we sent the hello-pulsar message. If you attempt to write a message to a topic that does not yet exist, Pulsar will automatically create that topic for you.

Using Pulsar clients locally​

Pulsar currently offers client libraries for Java, Go, Python and C++. If you're running a local standalone cluster, you can use one of these root URLs for interacting with your cluster:

  • http://localhost:8080
  • pulsar://localhost:6650

Here's an example producer for a Pulsar topic using the Java client:


String localClusterUrl = "pulsar://localhost:6650";

PulsarClient client = PulsarClient.builder().serviceUrl(localClusterUrl).build();
Producer<byte[]> producer = client.newProducer().topic("my-topic").create();

Here's an example Python producer:


import pulsar

client = pulsar.Client('pulsar://localhost:6650')
producer = client.create_producer('my-topic')

Finally, here's an example C++ producer:


Client client("pulsar://localhost:6650");
Producer producer;
Result result = client.createProducer("my-topic", producer);
if (result != ResultOk) {
    LOG_ERROR("Error creating producer: " << result);
    return -1;
}

Was this helpful?
💡 Suggest changes🛟 Get support