Skip to main content
Version: 2.10.x

Managing Namespaces

Important

This page only shows some frequently used operations.

  • For the latest and complete information about Pulsar admin, including commands, flags, descriptions, and more information, see Pulsar admin doc.

  • For the latest and complete information about REST API, including parameters, responses, samples, and more, see REST API doc.

  • For the latest and complete information about Java admin API, including classes, methods, descriptions, and more, see Java admin API doc.

Pulsar namespaces are logical groupings of topics.

Namespaces can be managed via:

  • The namespaces command of the pulsar-admin tool
  • The /admin/v2/namespaces endpoint of the admin REST API
  • The namespaces method of the PulsarAdmin object in the Java API

Namespaces resourcesโ€‹

Create namespacesโ€‹

You can create new namespaces under a given tenant.

Use the create subcommand and specify the namespace by name:


$ pulsar-admin namespaces create test-tenant/test-namespace

Get policiesโ€‹

You can fetch the current policies associated with a namespace at any time.

Use the policies subcommand and specify the namespace:


$ pulsar-admin namespaces policies test-tenant/test-namespace
{
"auth_policies": {
"namespace_auth": {},
"destination_auth": {}
},
"replication_clusters": [],
"bundles_activated": true,
"bundles": {
"boundaries": [
"0x00000000",
"0xffffffff"
],
"numBundles": 1
},
"backlog_quota_map": {},
"persistence": null,
"latency_stats_sample_rate": {},
"message_ttl_in_seconds": 0,
"retention_policies": null,
"deleted": false
}

List namespacesโ€‹

You can list all namespaces within a given Pulsar tenant.

Use the list subcommand and specify the tenant:


$ pulsar-admin namespaces list test-tenant
test-tenant/ns1
test-tenant/ns2

Delete namespacesโ€‹

You can delete existing namespaces from a tenant.

Use the delete subcommand and specify the namespace:


$ pulsar-admin namespaces delete test-tenant/ns1

Configure replication clustersโ€‹

Set replication clusterโ€‹

You can set replication clusters for a namespace to enable Pulsar to internally replicate the published messages from one colocation facility to another.


$ pulsar-admin namespaces set-clusters test-tenant/ns1 \
--clusters cl1

Get replication clusterโ€‹

You can get the list of replication clusters for a given namespace.


$ pulsar-admin namespaces get-clusters test-tenant/cl1/ns1


cl2

Configure backlog quota policiesโ€‹

Set backlog quota policiesโ€‹

Backlog quota helps the broker to restrict bandwidth/storage of a namespace once it reaches a certain threshold limit. Admin can set the limit and take corresponding action after the limit is reached.

  1. producer_request_hold: broker holds but does not persist produce request payload

  2. producer_exception: broker disconnects with the client by giving an exception

  3. consumer_backlog_eviction: broker starts discarding backlog messages

Backlog quota restriction can be taken care of by defining the restriction of backlog-quota-type: destination_storage.


$ pulsar-admin namespaces set-backlog-quota --limit 10G --limitTime 36000 --policy producer_request_hold test-tenant/ns1

Get backlog quota policiesโ€‹

You can get a configured backlog quota for a given namespace.


$ pulsar-admin namespaces get-backlog-quotas test-tenant/ns1


{
"destination_storage": {
"limit": 10,
"policy": "producer_request_hold"
}
}

Remove backlog quota policiesโ€‹

You can remove backlog quota policies for a given namespace.


$ pulsar-admin namespaces remove-backlog-quota test-tenant/ns1

Configure persistence policiesโ€‹

Set persistence policiesโ€‹

Persistence policies allow users to configure persistency-level for all topic messages under a given namespace.

  • Bookkeeper-ack-quorum: Number of acks (guaranteed copies) to wait for each entry, default: 0

  • Bookkeeper-ensemble: Number of bookies to use for a topic, default: 0

  • Bookkeeper-write-quorum: How many writes to make of each entry, default: 0

  • Ml-mark-delete-max-rate: Throttling rate of mark-delete operation (0 means no throttle), default: 0.0


$ pulsar-admin namespaces set-persistence --bookkeeper-ack-quorum 2 --bookkeeper-ensemble 3 --bookkeeper-write-quorum 2 --ml-mark-delete-max-rate 0 test-tenant/ns1

Get persistence policiesโ€‹

You can get the configured persistence policies of a given namespace.


$ pulsar-admin namespaces get-persistence test-tenant/ns1


{
"bookkeeperEnsemble": 3,
"bookkeeperWriteQuorum": 2,
"bookkeeperAckQuorum": 2,
"managedLedgerMaxMarkDeleteRate": 0
}

Configure namespace bundlesโ€‹

Unload namespace bundlesโ€‹

The namespace bundle is a virtual group of topics which belong to the same namespace. If the broker gets overloaded with the number of bundles, this command can help unload a bundle from that broker, so it can be served by some other less-loaded brokers. The namespace bundle ID ranges from 0x00000000 to 0xffffffff.


$ pulsar-admin namespaces unload --bundle 0x00000000_0xffffffff test-tenant/ns1

Split namespace bundlesโ€‹

One namespace bundle can contain multiple topics but can be served by only one broker. If a single bundle is creating an excessive load on a broker, an admin can split the bundle using the command below, permitting one or more of the new bundles to be unloaded, thus balancing the load across the brokers.


$ pulsar-admin namespaces split-bundle --bundle 0x00000000_0xffffffff test-tenant/ns1

Configure message TTLโ€‹

Set message-ttlโ€‹

You can configure the time to live (in seconds) duration for messages. In the example below, the message-ttl is set as 100s.


$ pulsar-admin namespaces set-message-ttl --messageTTL 100 test-tenant/ns1

Get message-ttlโ€‹

When the message-ttl for a namespace is set, you can use the command below to get the configured value. This example comtinues the example of the command set message-ttl, so the returned value is 100(s).


$ pulsar-admin namespaces get-message-ttl test-tenant/ns1


100

Remove message-ttlโ€‹

Remove a message TTL of the configured namespace.


$ pulsar-admin namespaces remove-message-ttl test-tenant/ns1

Clear backlogโ€‹

Clear namespace backlogโ€‹

It clears all message backlogs for all the topics that belong to a specific namespace. You can also clear backlogs for a specific subscription as well.


$ pulsar-admin namespaces clear-backlog --sub my-subscription test-tenant/ns1

Clear bundle backlogโ€‹

It clears all message backlogs for all the topics that belong to a specific NamespaceBundle. You can also clear backlogs for a specific subscription as well.


$ pulsar-admin namespaces clear-backlog --bundle 0x00000000_0xffffffff --sub my-subscription test-tenant/ns1

Configure retentionโ€‹

Set retentionโ€‹

Each namespace contains multiple topics and the retention size (storage size) of each topic should not exceed a specific threshold or it should be stored for a certain period. This command helps configure the retention size and time of topics in a given namespace.


$ pulsar-admin namespaces set-retention --size 100 --time 10 test-tenant/ns1

Get retentionโ€‹

It shows the retention information of a given namespace.


$ pulsar-admin namespaces get-retention test-tenant/ns1


{
"retentionTimeInMinutes": 10,
"retentionSizeInMB": 100
}

Configure dispatch throttling for topicsโ€‹

Set dispatch throttling for topicsโ€‹

It sets message dispatch rate for all the topics under a given namespace. The dispatch rate can be restricted by the number of messages per X seconds (msg-dispatch-rate) or by the number of message-bytes per X second (byte-dispatch-rate). dispatch rate is in second and it can be configured with dispatch-rate-period. Default value of msg-dispatch-rate and byte-dispatch-rate is -1 which disables the throttling.

note
  • If neither clusterDispatchRate nor topicDispatchRate is configured, dispatch throttling is disabled.
  • If topicDispatchRate is not configured, clusterDispatchRate takes effect.
  • If topicDispatchRate is configured, topicDispatchRate takes effect.

$ pulsar-admin namespaces set-dispatch-rate test-tenant/ns1 \
--msg-dispatch-rate 1000 \
--byte-dispatch-rate 1048576 \
--dispatch-rate-period 1

Get configured message-rate for topicsโ€‹

It shows the configured message-rate for the namespace (topics under this namespace can dispatch this many messages per second)


$ pulsar-admin namespaces get-dispatch-rate test-tenant/ns1


{
"dispatchThrottlingRatePerTopicInMsg" : 1000,
"dispatchThrottlingRatePerTopicInByte" : 1048576,
"ratePeriodInSecond" : 1
}

Configure dispatch throttling for subscriptionโ€‹

Set dispatch throttling for subscriptionโ€‹

It sets message dispatch rate for all the subscription of topics under a given namespace. The dispatch rate can be restricted by the number of messages per X seconds (msg-dispatch-rate) or by the number of message-bytes per X second (byte-dispatch-rate). dispatch rate is in second and it can be configured with dispatch-rate-period. Default value of msg-dispatch-rate and byte-dispatch-rate is -1 which disables the throttling.


$ pulsar-admin namespaces set-subscription-dispatch-rate test-tenant/ns1 \
--msg-dispatch-rate 1000 \
--byte-dispatch-rate 1048576 \
--dispatch-rate-period 1

Get configured message-rate for subscriptionโ€‹

It shows the configured message-rate for the namespace (topics under this namespace can dispatch this many messages per second)


$ pulsar-admin namespaces get-subscription-dispatch-rate test-tenant/ns1


{
"dispatchThrottlingRatePerTopicInMsg" : 1000,
"dispatchThrottlingRatePerTopicInByte" : 1048576,
"ratePeriodInSecond" : 1
}

Configure dispatch throttling for replicatorโ€‹

Set dispatch throttling for replicatorโ€‹

It sets message dispatch rate for all the replicator between replication clusters under a given namespace. The dispatch rate can be restricted by the number of messages per X seconds (msg-dispatch-rate) or by the number of message-bytes per X second (byte-dispatch-rate). dispatch rate is in second and it can be configured with dispatch-rate-period. Default value of msg-dispatch-rate and byte-dispatch-rate is -1 which disables the throttling.


$ pulsar-admin namespaces set-replicator-dispatch-rate test-tenant/ns1 \
--msg-dispatch-rate 1000 \
--byte-dispatch-rate 1048576 \
--dispatch-rate-period 1

Get configured message-rate for replicatorโ€‹

It shows the configured message-rate for the namespace (topics under this namespace can dispatch this many messages per second)


$ pulsar-admin namespaces get-replicator-dispatch-rate test-tenant/ns1


{
"dispatchThrottlingRatePerTopicInMsg" : 1000,
"dispatchThrottlingRatePerTopicInByte" : 1048576,
"ratePeriodInSecond" : 1
}

Configure deduplication snapshot intervalโ€‹

Get deduplication snapshot intervalโ€‹

It shows the configured deduplicationSnapshotInterval for a namespace (Each topic under the namespace will take a deduplication snapshot according to this interval)


$ pulsar-admin namespaces get-deduplication-snapshot-interval test-tenant/ns1

Set deduplication snapshot intervalโ€‹

Set configured deduplicationSnapshotInterval for a namespace. Each topic under the namespace will take a deduplication snapshot according to this interval. brokerDeduplicationEnabled must be set to true for this property to take effect.


$ pulsar-admin namespaces set-deduplication-snapshot-interval test-tenant/ns1 --interval 1000

Remove deduplication snapshot intervalโ€‹

Remove configured deduplicationSnapshotInterval of a namespace (Each topic under the namespace will take a deduplication snapshot according to this interval)


$ pulsar-admin namespaces remove-deduplication-snapshot-interval test-tenant/ns1

Namespace isolationโ€‹

You can use the Pulsar isolation policy to allocate resources (broker and bookie) for a namespace.

Unload namespaces from a brokerโ€‹

You can unload a namespace, or a namespace bundle, from the Pulsar broker that is currently responsible for it.

pulsar-adminโ€‹

Use the unload subcommand of the namespaces command.


$ pulsar-admin namespaces unload my-tenant/my-ns