Skip to main content

Managing Namespaces

Pulsar namespaces are logical groupings of topics.

Namespaces can be managed via:

Namespaces resources

Create namespaces

You can create new namespaces under a given tenant.

Use the create subcommand and specify the namespace by name:


$ pulsar-admin namespaces create test-tenant/test-namespace

Get policies

You can fetch the current policies associated with a namespace at any time.

Use the policies subcommand and specify the namespace:


$ pulsar-admin namespaces policies test-tenant/test-namespace
{
"auth_policies": {
"namespace_auth": {},
"destination_auth": {}
},
"replication_clusters": [],
"bundles_activated": true,
"bundles": {
"boundaries": [
"0x00000000",
"0xffffffff"
],
"numBundles": 1
},
"backlog_quota_map": {},
"persistence": null,
"latency_stats_sample_rate": {},
"message_ttl_in_seconds": 0,
"retention_policies": null,
"deleted": false
}

List namespaces

You can list all namespaces within a given Pulsar tenant.

Use the list subcommand and specify the tenant:


$ pulsar-admin namespaces list test-tenant
test-tenant/ns1
test-tenant/ns2

Delete namespaces

You can delete existing namespaces from a tenant.

Use the delete subcommand and specify the namespace:


$ pulsar-admin namespaces delete test-tenant/ns1

Configure replication clusters

Set replication cluster

It sets replication clusters for a namespace, so Pulsar can internally replicate publish message from one colo to another colo.


$ pulsar-admin namespaces set-clusters test-tenant/ns1 \
--clusters cl1

Get replication cluster

It gives a list of replication clusters for a given namespace.


$ pulsar-admin namespaces get-clusters test-tenant/cl1/ns1


cl2

Configure backlog quota policies

Set backlog quota policies

Backlog quota helps the broker to restrict bandwidth/storage of a namespace once it reaches a certain threshold limit. Admin can set the limit and take corresponding action after the limit is reached.

  1. producer_request_hold: broker will hold and not persist produce request payload
  2. producer_exception: broker disconnects with the client by giving an exception.
  3. consumer_backlog_eviction: broker will start discarding backlog messages Backlog quota restriction can be taken care by defining restriction of backlog-quota-type: destination_storage

$ pulsar-admin namespaces set-backlog-quota --limit 10 --policy producer_request_hold test-tenant/ns1


N/A

Get backlog quota policies

It shows a configured backlog quota for a given namespace.


$ pulsar-admin namespaces get-backlog-quotas test-tenant/ns1


{
"destination_storage": {
"limit": 10,
"policy": "producer_request_hold"
}
}

Remove backlog quota policies

It removes backlog quota policies for a given namespace


$ pulsar-admin namespaces remove-backlog-quota test-tenant/ns1


N/A

Configure persistence policies

Set persistence policies

Persistence policies allow to configure persistency-level for all topic messages under a given namespace.

  • Bookkeeper-ack-quorum: Number of acks (guaranteed copies) to wait for each entry, default: 0
  • Bookkeeper-ensemble: Number of bookies to use for a topic, default: 0
  • Bookkeeper-write-quorum: How many writes to make of each entry, default: 0
  • Ml-mark-delete-max-rate: Throttling rate of mark-delete operation (0 means no throttle), default: 0.0

$ pulsar-admin namespaces set-persistence --bookkeeper-ack-quorum 2 --bookkeeper-ensemble 3 --bookkeeper-write-quorum 2 --ml-mark-delete-max-rate 0 test-tenant/ns1


N/A

Get persistence policies

It shows the configured persistence policies of a given namespace.


$ pulsar-admin namespaces get-persistence test-tenant/ns1


{
"bookkeeperEnsemble": 3,
"bookkeeperWriteQuorum": 2,
"bookkeeperAckQuorum": 2,
"managedLedgerMaxMarkDeleteRate": 0
}

Configure namespace bundles

Unload namespace bundles

The namespace bundle is a virtual group of topics which belong to the same namespace. If the broker gets overloaded with the number of bundles, this command can help unload a bundle from that broker, so it can be served by some other less-loaded brokers. The namespace bundle ID ranges from 0x00000000 to 0xffffffff.


$ pulsar-admin namespaces unload --bundle 0x00000000_0xffffffff test-tenant/ns1


N/A

Split namespace bundles

Each namespace bundle can contain multiple topics and each bundle can be served by only one broker. If a single bundle is creating an excessive load on a broker, an admin splits the bundle using this command permitting one or more of the new bundles to be unloaded thus spreading the load across the brokers.


$ pulsar-admin namespaces split-bundle --bundle 0x00000000_0xffffffff test-tenant/ns1


N/A

Configure message TTL

Set message-ttl

It configures message’s time to live (in seconds) duration.


$ pulsar-admin namespaces set-message-ttl --messageTTL 100 test-tenant/ns1


N/A

Get message-ttl

It gives a message ttl of configured namespace.


$ pulsar-admin namespaces get-message-ttl test-tenant/ns1


100

Remove message-ttl

Remove a message TTL of the configured namespace.


$ pulsar-admin namespaces remove-message-ttl test-tenant/ns1


100

Clear backlog

Clear namespace backlog

It clears all message backlog for all the topics that belong to a specific namespace. You can also clear backlog for a specific subscription as well.


$ pulsar-admin namespaces clear-backlog --sub my-subscription test-tenant/ns1


N/A

Clear bundle backlog

It clears all message backlog for all the topics that belong to a specific NamespaceBundle. You can also clear backlog for a specific subscription as well.


$ pulsar-admin namespaces clear-backlog --bundle 0x00000000_0xffffffff --sub my-subscription test-tenant/ns1


N/A

Configure retention

Set retention

Each namespace contains multiple topics and the retention size (storage size) of each topic should not exceed a specific threshold or it should be stored for a certain period. This command helps configure the retention size and time of topics in a given namespace.


$ pulsar-admin set-retention --size 10 --time 100 test-tenant/ns1


N/A

Get retention

It shows retention information of a given namespace.


$ pulsar-admin namespaces get-retention test-tenant/ns1


{
"retentionTimeInMinutes": 10,
"retentionSizeInMB": 100
}

Configure dispatch throttling for topics

Set dispatch throttling for topics

It sets message dispatch rate for all the topics under a given namespace. The dispatch rate can be restricted by the number of messages per X seconds (msg-dispatch-rate) or by the number of message-bytes per X second (byte-dispatch-rate). dispatch rate is in second and it can be configured with dispatch-rate-period. Default value of msg-dispatch-rate and byte-dispatch-rate is -1 which disables the throttling.

note
  • If neither clusterDispatchRate nor topicDispatchRate is configured, dispatch throttling is disabled. >
  • If topicDispatchRate is not configured, clusterDispatchRate takes effect. >
  • If topicDispatchRate is configured, topicDispatchRate takes effect.

$ pulsar-admin namespaces set-dispatch-rate test-tenant/ns1 \
--msg-dispatch-rate 1000 \
--byte-dispatch-rate 1048576 \
--dispatch-rate-period 1

Get configured message-rate for topics

It shows configured message-rate for the namespace (topics under this namespace can dispatch this many messages per second)


$ pulsar-admin namespaces get-dispatch-rate test-tenant/ns1


{
"dispatchThrottlingRatePerTopicInMsg" : 1000,
"dispatchThrottlingRatePerTopicInByte" : 1048576,
"ratePeriodInSecond" : 1
}

Configure dispatch throttling for subscription

Set dispatch throttling for subscription

It sets message dispatch rate for all the subscription of topics under a given namespace. The dispatch rate can be restricted by the number of messages per X seconds (msg-dispatch-rate) or by the number of message-bytes per X second (byte-dispatch-rate). dispatch rate is in second and it can be configured with dispatch-rate-period. Default value of msg-dispatch-rate and byte-dispatch-rate is -1 which disables the throttling.


$ pulsar-admin namespaces set-subscription-dispatch-rate test-tenant/ns1 \
--msg-dispatch-rate 1000 \
--byte-dispatch-rate 1048576 \
--dispatch-rate-period 1

Get configured message-rate for subscription

It shows configured message-rate for the namespace (topics under this namespace can dispatch this many messages per second)


$ pulsar-admin namespaces get-subscription-dispatch-rate test-tenant/ns1


{
"dispatchThrottlingRatePerTopicInMsg" : 1000,
"dispatchThrottlingRatePerTopicInByte" : 1048576,
"ratePeriodInSecond" : 1
}

Configure dispatch throttling for replicator

Set dispatch throttling for replicator

It sets message dispatch rate for all the replicator between replication clusters under a given namespace. The dispatch rate can be restricted by the number of messages per X seconds (msg-dispatch-rate) or by the number of message-bytes per X second (byte-dispatch-rate). dispatch rate is in second and it can be configured with dispatch-rate-period. Default value of msg-dispatch-rate and byte-dispatch-rate is -1 which disables the throttling.


$ pulsar-admin namespaces set-replicator-dispatch-rate test-tenant/ns1 \
--msg-dispatch-rate 1000 \
--byte-dispatch-rate 1048576 \
--dispatch-rate-period 1

Get configured message-rate for replicator

It shows configured message-rate for the namespace (topics under this namespace can dispatch this many messages per second)


$ pulsar-admin namespaces get-replicator-dispatch-rate test-tenant/ns1


{
"dispatchThrottlingRatePerTopicInMsg" : 1000,
"dispatchThrottlingRatePerTopicInByte" : 1048576,
"ratePeriodInSecond" : 1
}

Configure deduplication snapshot interval

Get deduplication snapshot interval

It shows configured deduplicationSnapshotInterval for a namespace (Each topic under the namespace will take a deduplication snapshot according to this interval)


$ pulsar-admin namespaces get-deduplication-snapshot-interval test-tenant/ns1

Set deduplication snapshot interval

Set configured deduplicationSnapshotInterval for a namespace. Each topic under the namespace will take a deduplication snapshot according to this interval. brokerDeduplicationEnabled must be set to true for this property to take effect.


$ pulsar-admin namespaces set-deduplication-snapshot-interval test-tenant/ns1 --interval 1000

Remove deduplication snapshot interval

Remove configured deduplicationSnapshotInterval of a namespace (Each topic under the namespace will take a deduplication snapshot according to this interval)


$ pulsar-admin namespaces remove-deduplication-snapshot-interval test-tenant/ns1

Namespace isolation

You can use the Pulsar isolation policy to allocate resources (broker and bookie) for a namespace.

Unload namespaces from a broker

You can unload a namespace, or a namespace bundle, from the Pulsar broker that is currently responsible for it.

pulsar-admin

Use the unload subcommand of the namespaces command.


$ pulsar-admin namespaces unload my-tenant/my-ns