Managing Namespaces
Pulsar namespaces are logical groupings of topics.
Namespaces can be managed via:
- The
namespaces
command of thepulsar-admin
tool - The
/admin/v2/namespaces
endpoint of the admin REST API - The
namespaces
method of the PulsarAdmin object in the Java API
Namespaces resources​
Create namespaces​
You can create new namespaces under a given tenant.
- pulsar-admin
- REST API
- Java
Use the create
subcommand and specify the namespace by name:
$ pulsar-admin namespaces create test-tenant/test-namespace
admin.namespaces().createNamespace(namespace);
Get policies​
You can fetch the current policies associated with a namespace at any time.
- pulsar-admin
- REST API
- Java
Use the policies
subcommand and specify the namespace:
$ pulsar-admin namespaces policies test-tenant/test-namespace
{
"auth_policies": {
"namespace_auth": {},
"destination_auth": {}
},
"replication_clusters": [],
"bundles_activated": true,
"bundles": {
"boundaries": [
"0x00000000",
"0xffffffff"
],
"numBundles": 1
},
"backlog_quota_map": {},
"persistence": null,
"latency_stats_sample_rate": {},
"message_ttl_in_seconds": 0,
"retention_policies": null,
"deleted": false
}
admin.namespaces().getPolicies(namespace);
List namespaces​
You can list all namespaces within a given Pulsar tenant.
- pulsar-admin
- REST API
- Java
Use the list
subcommand and specify the tenant:
$ pulsar-admin namespaces list test-tenant
test-tenant/ns1
test-tenant/ns2
admin.namespaces().getNamespaces(tenant);
Delete namespaces​
You can delete existing namespaces from a tenant.
- pulsar-admin
- REST API
- Java
Use the delete
subcommand and specify the namespace:
$ pulsar-admin namespaces delete test-tenant/ns1
admin.namespaces().deleteNamespace(namespace);
Configure replication clusters​
Set replication cluster​
It sets replication clusters for a namespace, so Pulsar can internally replicate publish message from one colo to another colo.
- pulsar-admin
- REST API
- Java
$ pulsar-admin namespaces set-clusters test-tenant/ns1 \
--clusters cl1
admin.namespaces().setNamespaceReplicationClusters(namespace, clusters);
Get replication cluster​
It gives a list of replication clusters for a given namespace.
- pulsar-admin
- REST API
- Java
$ pulsar-admin namespaces get-clusters test-tenant/cl1/ns1
cl2
admin.namespaces().getNamespaceReplicationClusters(namespace)
Configure backlog quota policies​
Set backlog quota policies​
Backlog quota helps the broker to restrict bandwidth/storage of a namespace once it reaches a certain threshold limit. Admin can set the limit and take corresponding action after the limit is reached.
-
producer_request_hold: broker will hold and not persist produce request payload
-
producer_exception: broker disconnects with the client by giving an exception.
-
consumer_backlog_eviction: broker will start discarding backlog messages
Backlog quota restriction can be taken care by defining restriction of backlog-quota-type: destination_storage
- pulsar-admin
- REST API
- Java
$ pulsar-admin namespaces set-backlog-quota --limit 10 --policy producer_request_hold test-tenant/ns1
N/A
admin.namespaces().setBacklogQuota(namespace, new BacklogQuota(limit, policy))
Get backlog quota policies​
It shows a configured backlog quota for a given namespace.
- pulsar-admin
- REST API
- Java
$ pulsar-admin namespaces get-backlog-quotas test-tenant/ns1
{
"destination_storage": {
"limit": 10,
"policy": "producer_request_hold"
}
}
admin.namespaces().getBacklogQuotaMap(namespace);
Remove backlog quota policies​
It removes backlog quota policies for a given namespace
- pulsar-admin
- REST API
- Java
$ pulsar-admin namespaces remove-backlog-quota test-tenant/ns1
N/A
admin.namespaces().removeBacklogQuota(namespace, backlogQuotaType)
Configure persistence policies​
Set persistence policies​
Persistence policies allow to configure persistency-level for all topic messages under a given namespace.
-
Bookkeeper-ack-quorum: Number of acks (guaranteed copies) to wait for each entry, default: 0
-
Bookkeeper-ensemble: Number of bookies to use for a topic, default: 0
-
Bookkeeper-write-quorum: How many writes to make of each entry, default: 0
-
Ml-mark-delete-max-rate: Throttling rate of mark-delete operation (0 means no throttle), default: 0.0
- pulsar-admin
- REST API
- Java
$ pulsar-admin namespaces set-persistence --bookkeeper-ack-quorum 2 --bookkeeper-ensemble 3 --bookkeeper-write-quorum 2 --ml-mark-delete-max-rate 0 test-tenant/ns1
N/A
admin.namespaces().setPersistence(namespace,new PersistencePolicies(bookkeeperEnsemble, bookkeeperWriteQuorum,bookkeeperAckQuorum,managedLedgerMaxMarkDeleteRate))
Get persistence policies​
It shows the configured persistence policies of a given namespace.
- pulsar-admin
- REST API
- Java
$ pulsar-admin namespaces get-persistence test-tenant/ns1
{
"bookkeeperEnsemble": 3,
"bookkeeperWriteQuorum": 2,
"bookkeeperAckQuorum": 2,
"managedLedgerMaxMarkDeleteRate": 0
}
admin.namespaces().getPersistence(namespace)
Configure namespace bundles​
Unload namespace bundles​
The namespace bundle is a virtual group of topics which belong to the same namespace. If the broker gets overloaded with the number of bundles, this command can help unload a bundle from that broker, so it can be served by some other less-loaded brokers. The namespace bundle ID ranges from 0x00000000 to 0xffffffff.
- pulsar-admin
- REST API
- Java
$ pulsar-admin namespaces unload --bundle 0x00000000_0xffffffff test-tenant/ns1
N/A
admin.namespaces().unloadNamespaceBundle(namespace, bundle)
Split namespace bundles​
Each namespace bundle can contain multiple topics and each bundle can be served by only one broker. If a single bundle is creating an excessive load on a broker, an admin splits the bundle using this command permitting one or more of the new bundles to be unloaded thus spreading the load across the brokers.
- pulsar-admin
- REST API
- Java
$ pulsar-admin namespaces split-bundle --bundle 0x00000000_0xffffffff test-tenant/ns1
N/A
admin.namespaces().splitNamespaceBundle(namespace, bundle, unloadSplitBundles, splitAlgorithmName)
Configure message TTL​
Set message-ttl​
It configures message’s time to live (in seconds) duration.
- pulsar-admin
- REST API
- Java
$ pulsar-admin namespaces set-message-ttl --messageTTL 100 test-tenant/ns1
N/A
admin.namespaces().setNamespaceMessageTTL(namespace, messageTTL)
Get message-ttl​
It gives a message ttl of configured namespace.
- pulsar-admin
- REST API
- Java
$ pulsar-admin namespaces get-message-ttl test-tenant/ns1
100
admin.namespaces().getNamespaceMessageTTL(namespace)
Remove message-ttl​
Remove a message TTL of the configured namespace.
- pulsar-admin
- REST API
- Java
$ pulsar-admin namespaces remove-message-ttl test-tenant/ns1
100
admin.namespaces().removeNamespaceMessageTTL(namespace)
Clear backlog​
Clear namespace backlog​
It clears all message backlog for all the topics that belong to a specific namespace. You can also clear backlog for a specific subscription as well.
- pulsar-admin
- REST API
- Java
$ pulsar-admin namespaces clear-backlog --sub my-subscription test-tenant/ns1
N/A
admin.namespaces().clearNamespaceBacklogForSubscription(namespace, subscription)
Clear bundle backlog​
It clears all message backlog for all the topics that belong to a specific NamespaceBundle. You can also clear backlog for a specific subscription as well.
- pulsar-admin
- REST API
- Java
$ pulsar-admin namespaces clear-backlog --bundle 0x00000000_0xffffffff --sub my-subscription test-tenant/ns1
N/A
admin.namespaces().clearNamespaceBundleBacklogForSubscription(namespace, bundle, subscription)
Configure retention​
Set retention​
Each namespace contains multiple topics and the retention size (storage size) of each topic should not exceed a specific threshold or it should be stored for a certain period. This command helps configure the retention size and time of topics in a given namespace.
- pulsar-admin
- REST API
- Java
$ pulsar-admin set-retention --size 10 --time 100 test-tenant/ns1
N/A
admin.namespaces().setRetention(namespace, new RetentionPolicies(retentionTimeInMin, retentionSizeInMB))
Get retention​
It shows retention information of a given namespace.
- pulsar-admin
- REST API
- Java
$ pulsar-admin namespaces get-retention test-tenant/ns1
{
"retentionTimeInMinutes": 10,
"retentionSizeInMB": 100
}
admin.namespaces().getRetention(namespace)
Configure dispatch throttling for topics​
Set dispatch throttling for topics​
It sets message dispatch rate for all the topics under a given namespace.
The dispatch rate can be restricted by the number of messages per X seconds (msg-dispatch-rate
) or by the number of message-bytes per X second (byte-dispatch-rate
).
dispatch rate is in second and it can be configured with dispatch-rate-period
. Default value of msg-dispatch-rate
and byte-dispatch-rate
is -1 which
disables the throttling.
-
If neither
clusterDispatchRate
nortopicDispatchRate
is configured, dispatch throttling is disabled. -
If
topicDispatchRate
is not configured,clusterDispatchRate
takes effect. -
If
topicDispatchRate
is configured,topicDispatchRate
takes effect.
- pulsar-admin
- REST API
- Java
$ pulsar-admin namespaces set-dispatch-rate test-tenant/ns1 \
--msg-dispatch-rate 1000 \
--byte-dispatch-rate 1048576 \
--dispatch-rate-period 1
admin.namespaces().setDispatchRate(namespace, new DispatchRate(1000, 1048576, 1))
Get configured message-rate for topics​
It shows configured message-rate for the namespace (topics under this namespace can dispatch this many messages per second)
- pulsar-admin
- REST API
- Java
$ pulsar-admin namespaces get-dispatch-rate test-tenant/ns1
{
"dispatchThrottlingRatePerTopicInMsg" : 1000,
"dispatchThrottlingRatePerTopicInByte" : 1048576,
"ratePeriodInSecond" : 1
}
admin.namespaces().getDispatchRate(namespace)
Configure dispatch throttling for subscription​
Set dispatch throttling for subscription​
It sets message dispatch rate for all the subscription of topics under a given namespace.
The dispatch rate can be restricted by the number of messages per X seconds (msg-dispatch-rate
) or by the number of message-bytes per X second (byte-dispatch-rate
).
dispatch rate is in second and it can be configured with dispatch-rate-period
. Default value of msg-dispatch-rate
and byte-dispatch-rate
is -1 which
disables the throttling.
- pulsar-admin
- REST API
- Java
$ pulsar-admin namespaces set-subscription-dispatch-rate test-tenant/ns1 \
--msg-dispatch-rate 1000 \
--byte-dispatch-rate 1048576 \
--dispatch-rate-period 1
admin.namespaces().setSubscriptionDispatchRate(namespace, new DispatchRate(1000, 1048576, 1))
Get configured message-rate for subscription​
It shows configured message-rate for the namespace (topics under this namespace can dispatch this many messages per second)
- pulsar-admin
- REST API
- Java
$ pulsar-admin namespaces get-subscription-dispatch-rate test-tenant/ns1
{
"dispatchThrottlingRatePerTopicInMsg" : 1000,
"dispatchThrottlingRatePerTopicInByte" : 1048576,
"ratePeriodInSecond" : 1
}
admin.namespaces().getSubscriptionDispatchRate(namespace)
Configure dispatch throttling for replicator​
Set dispatch throttling for replicator​
It sets message dispatch rate for all the replicator between replication clusters under a given namespace.
The dispatch rate can be restricted by the number of messages per X seconds (msg-dispatch-rate
) or by the number of message-bytes per X second (byte-dispatch-rate
).
dispatch rate is in second and it can be configured with dispatch-rate-period
. Default value of msg-dispatch-rate
and byte-dispatch-rate
is -1 which
disables the throttling.
- pulsar-admin
- REST API
- Java
$ pulsar-admin namespaces set-replicator-dispatch-rate test-tenant/ns1 \
--msg-dispatch-rate 1000 \
--byte-dispatch-rate 1048576 \
--dispatch-rate-period 1
admin.namespaces().setReplicatorDispatchRate(namespace, new DispatchRate(1000, 1048576, 1))
Get configured message-rate for replicator​
It shows configured message-rate for the namespace (topics under this namespace can dispatch this many messages per second)
- pulsar-admin
- REST API
- Java
$ pulsar-admin namespaces get-replicator-dispatch-rate test-tenant/ns1
{
"dispatchThrottlingRatePerTopicInMsg" : 1000,
"dispatchThrottlingRatePerTopicInByte" : 1048576,
"ratePeriodInSecond" : 1
}
admin.namespaces().getReplicatorDispatchRate(namespace)
Configure deduplication snapshot interval​
Get deduplication snapshot interval​
It shows configured deduplicationSnapshotInterval
for a namespace (Each topic under the namespace will take a deduplication snapshot according to this interval)
- pulsar-admin
- REST API
- Java
$ pulsar-admin namespaces get-deduplication-snapshot-interval test-tenant/ns1
admin.namespaces().getDeduplicationSnapshotInterval(namespace)
Set deduplication snapshot interval​
Set configured deduplicationSnapshotInterval
for a namespace. Each topic under the namespace will take a deduplication snapshot according to this interval.
brokerDeduplicationEnabled
must be set to true
for this property to take effect.
- pulsar-admin
- REST API
- Java
$ pulsar-admin namespaces set-deduplication-snapshot-interval test-tenant/ns1 --interval 1000
admin.namespaces().setDeduplicationSnapshotInterval(namespace, 1000)
Remove deduplication snapshot interval​
Remove configured deduplicationSnapshotInterval
of a namespace (Each topic under the namespace will take a deduplication snapshot according to this interval)
- pulsar-admin
- REST API
- Java
$ pulsar-admin namespaces remove-deduplication-snapshot-interval test-tenant/ns1
admin.namespaces().removeDeduplicationSnapshotInterval(namespace)
Namespace isolation​
You can use the Pulsar isolation policy to allocate resources (broker and bookie) for a namespace.
Unload namespaces from a broker​
You can unload a namespace, or a namespace bundle, from the Pulsar broker that is currently responsible for it.
pulsar-admin​
Use the unload
subcommand of the namespaces
command.
- pulsar-admin
- REST
- Java
$ pulsar-admin namespaces unload my-tenant/my-ns
admin.namespaces().unload(namespace)