Skip to main content

Non-persistent messaging

Non-persistent topics are Pulsar topics in which message data is never persistently stored and kept only in memory. This cookbook provides:

Overview​

By default, Pulsar persistently stores all unacknowledged messages on multiple BookKeeper bookies (storage nodes). Data for messages on persistent topics can thus survive broker restarts and subscriber failover.

Pulsar also, however, supports non-persistent topics, which are topics on which messages are never persisted to disk and live only in memory. When using non-persistent delivery, killing a Pulsar broker or disconnecting a subscriber to a topic means that all in-transit messages are lost on that (non-persistent) topic, meaning that clients may see message loss.

Non-persistent topics have names of this form (note the non-persistent in the name):


non-persistent://tenant/namespace/topic

For more high-level information about non-persistent topics, see the Concepts and Architecture documentation.

Using​

In order to use non-persistent topics, they must be enabled in your Pulsar broker configuration.

In order to use non-persistent topics, you only need to differentiate them by name when interacting with them. This pulsar-client produce command, for example, would produce one message on a non-persistent topic in a standalone cluster:


$ bin/pulsar-client produce non-persistent://public/default/example-np-topic \
  --num-produce 1 \
  --messages "This message will be stored only in memory"

For a more thorough guide to non-persistent topics from an administrative perspective, see the Non-persistent topics guide.

Enabling​

In order to enable non-persistent topics in a Pulsar broker, the enableNonPersistentTopics must be set to true. This is the default, and so you won't need to take any action to enable non-persistent messaging.

Configuration for standalone mode​

If you're running Pulsar in standalone mode, the same configurable parameters are available but in the standalone.conf configuration file.

If you'd like to enable only non-persistent topics in a broker, you can set the enablePersistentTopics parameter to false and the enableNonPersistentTopics parameter to true.

Managing with cli​

Non-persistent topics can be managed using the pulsar-admin non-persistent command-line interface. With that interface you can perform actions like create a partitioned non-persistent topic, get stats for a non-persistent topic, list non-persistent topics under a namespace, and more.

Using with Pulsar clients​

You shouldn't need to make any changes to your Pulsar clients to use non-persistent messaging beyond making sure that you use proper topic names with non-persistent as the topic type.

Was this helpful?
💡 Suggest changes🛟 Get support