Skip to main content

Non-persistent messaging

Non-persistent topics are Pulsar topics in which message data is never persistently stored and kept only in memory. This cookbook provides:

Overview​

By default, Pulsar persistently stores all unacknowledged messages on multiple bookies (storage nodes). Data for messages on persistent topics can thus survive broker restarts and subscriber failover.

Pulsar also, however, supports non-persistent topics, which are topics on which messages are never persisted to disk and live only in memory. When using non-persistent delivery, killing a Pulsar broker or disconnecting a subscriber to a topic means that all in-transit messages are lost on that (non-persistent) topic, meaning that clients may see message loss.

Non-persistent topics have names of this form (note the non-persistent in the name):

non-persistent://tenant/namespace/topic

For more high-level information about non-persistent topics, see the Concepts and Architecture documentation.

Use​

To use non-persistent topics, you need to enable them in your Pulsar broker configuration and differentiate them by name when interacting with them. This pulsar-client produce command, for example, would produce one message on a non-persistent topic in a standalone cluster:

bin/pulsar-client produce non-persistent://public/default/example-np-topic \
  --num-produce 1 \
  --messages "This message will be stored only in memory"

For a more thorough guide to non-persistent topics from an administrative perspective, see the Non-persistent topics guide.

Enable​

To enable non-persistent topics in a Pulsar broker, the enableNonPersistentTopics must be set to true. This is the default, so you won't need to take any action to enable non-persistent messaging.

Configuration for standalone mode​

If you're running Pulsar in standalone mode, the same configurable parameters are available but in the standalone.conf configuration file.

If you'd like to enable only non-persistent topics in a broker, you can set the enablePersistentTopics parameter to false and the enableNonPersistentTopics parameter to true.

Manage with cli​

Non-persistent topics can be managed using the pulsar-admin non-persistent command-line interface. With that interface, you can perform actions like create a partitioned non-persistent topic, get stats for a non-persistent topic, list non-persistent topics under a namespace, and more.

Use with Pulsar clients​

You shouldn't need to make any changes to your Pulsar clients to use non-persistent messaging beyond making sure that you use proper topic names with non-persistent as the topic type.

Was this helpful?
💡 Suggest changes🛟 Get support