Managing Namespaces
This page only shows some frequently used operations.
-
For the latest and complete information about
Pulsar admin
, including commands, flags, descriptions, and more information, see Pulsar admin docs. -
For the latest and complete information about
REST API
, including parameters, responses, samples, and more, see REST API doc. -
For the latest and complete information about
Java admin API
, including classes, methods, descriptions, and more, see Java admin API doc.
Pulsar namespaces are logical groupings of topics.
Namespaces can be managed via:
- The
namespaces
command of thepulsar-admin
tool - The
/admin/v2/namespaces
endpoint of the admin REST API - The
namespaces
method of thePulsarAdmin
object in the Java API
Namespaces resources
Create namespaces
You can create new namespaces under a given tenant.
- pulsar-admin
- REST API
- Java
Use the create
subcommand and specify the namespace by name:
pulsar-admin namespaces create test-tenant/test-namespace
admin.namespaces().createNamespace(namespace);
Get policies
You can fetch the current policies associated with a namespace at any time.
- pulsar-admin
- REST API
- Java
Use the policies
subcommand and specify the namespace:
pulsar-admin namespaces policies test-tenant/test-namespace
Example output:
{
"auth_policies": {
"namespace_auth": {},
"destination_auth": {}
},
"replication_clusters": [],
"bundles_activated": true,
"bundles": {
"boundaries": [
"0x00000000",
"0xffffffff"
],
"numBundles": 1
},
"backlog_quota_map": {},
"persistence": null,
"latency_stats_sample_rate": {},
"message_ttl_in_seconds": 0,
"retention_policies": null,
"deleted": false
}
admin.namespaces().getPolicies(namespace);
List namespaces
You can list all namespaces within a given Pulsar tenant.
- pulsar-admin
- REST API
- Java
Use the list
subcommand and specify the tenant:
pulsar-admin namespaces list test-tenant
Example output:
test-tenant/namespace1
test-tenant/namespace2
admin.namespaces().getNamespaces(tenant);
Delete namespaces
You can delete existing namespaces from a tenant.
- pulsar-admin
- REST API
- Java
Use the delete
subcommand and specify the namespace:
pulsar-admin namespaces delete test-tenant/namespace1
admin.namespaces().deleteNamespace(namespace);
Configure replication clusters
Set replication cluster
You can set replication clusters for a namespace to enable Pulsar to internally replicate the published messages from one colocation facility to another.
- pulsar-admin
- REST API
- Java
pulsar-admin namespaces set-clusters test-tenant/namespace1 --clusters cl1
admin.namespaces().setNamespaceReplicationClusters(namespace, clusters);
Get replication cluster
You can get the list of replication clusters for a given namespace.
- pulsar-admin
- REST API
- Java
pulsar-admin namespaces get-clusters test-tenant/cluster1/namespace1
Example output:
cluster2
admin.namespaces().getNamespaceReplicationClusters(namespace)
Configure backlog quota policies
Set backlog quota policies
Backlog quota helps the broker to restrict bandwidth/storage of a namespace once it reaches a certain threshold limit. Admin can set the limit and take the corresponding action after the limit is reached.
-
producer_request_hold: the producer holds the message and retries until client configuration
sendTimeoutMs
is exceeded -
producer_exception: the producer throws an exception when trying to send a message
-
consumer_backlog_eviction: broker starts discarding backlog messages
Backlog quota restriction can be taken care of by defining the restriction of backlog-quota-type: destination_storage.
- pulsar-admin
- REST API
- Java
pulsar-admin namespaces set-backlog-quota --limit 10G \
--limitTime 36000 \
--policy producer_request_hold \
test-tenant/namespace1
admin.namespaces().setBacklogQuota(namespace, new BacklogQuota(limit, limitTime, policy))
Get backlog quota policies
You can get a configured backlog quota for a given namespace.
- pulsar-admin
- REST API
- Java
pulsar-admin namespaces get-backlog-quotas test-tenant/namespace1
Example output:
destination_storage BacklogQuotaImpl(limit=10737418240, limitSize=10737418240, limitTime=36000, policy=producer_request_hold)
admin.namespaces().getBacklogQuotaMap(namespace);
Remove backlog quota policies
You can remove backlog quota policies for a given namespace.
- pulsar-admin
- REST API
- Java
pulsar-admin namespaces remove-backlog-quota test-tenant/namespace1
admin.namespaces().removeBacklogQuota(namespace, backlogQuotaType)
Configure persistence policies
Set persistence policies
Persistence policies allow users to configure persistency-level for all topic messages under a given namespace.
-
Bookkeeper-ack-quorum: Number of acks (guaranteed copies) to wait for each entry, default: 2
-
Bookkeeper-ensemble: Number of bookies to use for a topic, default: 2
-
Bookkeeper-write-quorum: How many writes to make of each entry, default: 2
-
Ml-mark-delete-max-rate: Throttling rate of mark-delete operation (0 means no throttle), default: 0
- pulsar-admin
- REST API
- Java
pulsar-admin namespaces set-persistence \
--bookkeeper-ack-quorum 2 --bookkeeper-ensemble 3 \
--bookkeeper-write-quorum 2 --ml-mark-delete-max-rate 0 \
test-tenant/namespace1
admin.namespaces().setPersistence(namespace,new PersistencePolicies(bookkeeperEnsemble, bookkeeperWriteQuorum,bookkeeperAckQuorum,managedLedgerMaxMarkDeleteRate))
Get persistence policies
You can get the configured persistence policies of a given namespace.
- pulsar-admin
- REST API
- Java
pulsar-admin namespaces get-persistence test-tenant/namespace1
Example output:
{
"bookkeeperEnsemble": 3,
"bookkeeperWriteQuorum": 2,
"bookkeeperAckQuorum": 2,
"managedLedgerMaxMarkDeleteRate": 0
}
admin.namespaces().getPersistence(namespace)
Configure namespace bundles
Unload namespace bundles
A namespace bundle is a virtual group of topics that belong to the same namespace. If the broker gets overloaded with the number of bundles, this command can help unload a bundle from that broker, so it can be served by some other less-loaded brokers. The namespace bundle ID ranges from 0x00000000 to 0xffffffff.
- pulsar-admin
- REST API
- Java
pulsar-admin namespaces unload --bundle 0x00000000_0xffffffff test-tenant/namespace1
pulsar-admin namespaces unload --bundle 0x00000000_0xffffffff test-tenant/namespace1
pulsar-admin namespaces unload --bundle 0x00000000_0xffffffff --destinationBroker broker1.use.org.com:8080 test-tenant/namespace1
admin.namespaces().unloadNamespaceBundle(namespace, bundle)
Split namespace bundles
One namespace bundle can contain multiple topics but can be served by only one broker. If a single bundle is creating an excessive load on a broker, an admin can split the bundle using the command below, permitting one or more of the new bundles to be unloaded, thus balancing the load across the brokers.
- pulsar-admin
- REST API
- Java
pulsar-admin namespaces split-bundle --bundle 0x00000000_0xffffffff test-tenant/namespace1
admin.namespaces().splitNamespaceBundle(namespace, bundle, unloadSplitBundles, splitAlgorithmName)
Configure message TTL
Set message-ttl
You can configure the time to live (in seconds) duration for messages. In the example below, the message-ttl is set as 100s.
- pulsar-admin
- REST API
- Java
pulsar-admin namespaces set-message-ttl --messageTTL 100 test-tenant/namespace1
admin.namespaces().setNamespaceMessageTTL(namespace, messageTTL)
Get message-ttl
When the message-ttl for a namespace is set, you can use the command below to get the configured value. This example comtinues the example of the command set message-ttl
, so the returned value is 100(s).
- pulsar-admin
- REST API
- Java
pulsar-admin namespaces get-message-ttl test-tenant/namespace1
Example output:
100
admin.namespaces().getNamespaceMessageTTL(namespace)
100
Remove message-ttl
Remove a message TTL of the configured namespace.
- pulsar-admin
- REST API
- Java
pulsar-admin namespaces remove-message-ttl test-tenant/namespace1
admin.namespaces().removeNamespaceMessageTTL(namespace)
Clear backlog
Clear namespace backlog
It clears all message backlogs for all the topics that belong to a specific namespace. You can also clear backlogs for a specific subscription as well.
- pulsar-admin
- REST API
- Java
pulsar-admin namespaces clear-backlog --sub my-subscription test-tenant/namespace1
admin.namespaces().clearNamespaceBacklogForSubscription(namespace, subscription)
Clear bundle backlog
It clears all message backlogs for all the topics that belong to a specific NamespaceBundle. You can also clear backlogs for a specific subscription as well.
- pulsar-admin
- REST API
- Java
pulsar-admin namespaces clear-backlog \
--bundle 0x00000000_0xffffffff \
--sub my-subscription \
test-tenant/namespace1
admin.namespaces().clearNamespaceBundleBacklogForSubscription(namespace, bundle, subscription)
Configure retention
Set retention
Each namespace contains multiple topics and the retention size (storage size) of each topic should not exceed a specific threshold or it should be stored for a certain period. This command helps configure the retention size and time of topics in a given namespace.
- pulsar-admin
- REST API
- Java
pulsar-admin namespaces set-retention --size 100M --time 10m test-tenant/namespace1
admin.namespaces().setRetention(namespace, new RetentionPolicies(retentionTimeInMin, retentionSizeInMB))
Get retention
It shows the retention information of a given namespace.
- pulsar-admin
- REST API
- Java
pulsar-admin namespaces get-retention test-tenant/namespace1
{
"retentionTimeInMinutes": 10,
"retentionSizeInMB": 100
}
admin.namespaces().getRetention(namespace)
Configure dispatch throttling for topics
Set dispatch throttling for topics
It sets the message dispatch rate for all the topics under a given namespace. The dispatch rate can be restricted by the number of messages per X seconds (msg-dispatch-rate
) or by the number of message-bytes per X second (byte-dispatch-rate
). The dispatch rate is in second and it can be configured with dispatch-rate-period
. The default value of msg-dispatch-rate
and byte-dispatch-rate
is -1 which
disables the throttling.
- If neither
clusterDispatchRate
nortopicDispatchRate
is configured, dispatch throttling is disabled. - If
topicDispatchRate
is not configured,clusterDispatchRate
takes effect. - If
topicDispatchRate
is configured,topicDispatchRate
takes effect.
- pulsar-admin
- REST API
- Java
pulsar-admin namespaces set-dispatch-rate test-tenant/namespace1 \
--msg-dispatch-rate 1000 \
--byte-dispatch-rate 1048576 \
--dispatch-rate-period 1
admin.namespaces().setDispatchRate(namespace, new DispatchRate(1000, 1048576, 1))
Get configured message-rate for topics
It shows the configured message-rate for the namespace (topics under this namespace can dispatch this many messages per second)
- pulsar-admin
- REST API
- Java
pulsar-admin namespaces get-dispatch-rate test-tenant/namespace1
Example output:
{
"dispatchThrottlingRateInMsg" : 1000,
"dispatchThrottlingRateInByte" : 1048576,
"relativeToPublishRate" : false,
"ratePeriodInSecond" : 1
}
admin.namespaces().getDispatchRate(namespace)
Configure dispatch throttling for subscription
Set dispatch throttling for subscription
It sets the message dispatch rate for all the subscriptions of topics under a given namespace. The dispatch rate can be restricted by the number of messages per X seconds (msg-dispatch-rate
) or by the number of message-bytes per X second (byte-dispatch-rate
). The dispatch rate is in second and it can be configured with dispatch-rate-period
. The default value of msg-dispatch-rate
and byte-dispatch-rate
is -1 which
disables the throttling.
- pulsar-admin
- REST API
- Java
pulsar-admin namespaces set-subscription-dispatch-rate test-tenant/namespace1 \
--msg-dispatch-rate 1000 \
--byte-dispatch-rate 1048576 \
--dispatch-rate-period 1
admin.namespaces().setSubscriptionDispatchRate(namespace, new DispatchRate(1000, 1048576, 1))
Get configured message-rate for subscription
It shows the configured message-rate for the namespace (topics under this namespace can dispatch this many messages per second).
- pulsar-admin
- REST API
- Java
pulsar-admin namespaces get-subscription-dispatch-rate test-tenant/namespace1
Example output:
{
"dispatchThrottlingRateInMsg" : 1000,
"dispatchThrottlingRateInByte" : 1048576,
"relativeToPublishRate" : false,
"ratePeriodInSecond" : 1
}
admin.namespaces().getSubscriptionDispatchRate(namespace)
Configure dispatch throttling for replicators
Set dispatch throttling for replicators
It sets the message dispatch rate for all the replicators between replication clusters under a given namespace. The dispatch rate can be restricted by the number of messages per X seconds (msg-dispatch-rate
) or by the number of message-bytes per X second (byte-dispatch-rate
). The dispatch rate is in second and it can be configured with dispatch-rate-period
. The default value of msg-dispatch-rate
and byte-dispatch-rate
is -1 which disables the throttling.
- pulsar-admin
- REST API
- Java
pulsar-admin namespaces set-replicator-dispatch-rate test-tenant/namespace1 \
--msg-dispatch-rate 1000 \
--byte-dispatch-rate 1048576 \
--dispatch-rate-period 1
admin.namespaces().setReplicatorDispatchRate(namespace, new DispatchRate(1000, 1048576, 1))
Get configured message-rate for replicators
It shows the configured message-rate for the namespace (topics under this namespace can dispatch this many messages per second)
- pulsar-admin
- REST API
- Java
pulsar-admin namespaces get-replicator-dispatch-rate test-tenant/namespace1
{
"dispatchThrottlingRatePerTopicInMsg" : 1000,
"dispatchThrottlingRatePerTopicInByte" : 1048576,
"ratePeriodInSecond" : 1
}
admin.namespaces().getReplicatorDispatchRate(namespace)
Configure deduplication snapshot interval
Get deduplication snapshot interval
It shows the configured deduplicationSnapshotInterval
for a namespace (Each topic under the namespace will take a deduplication snapshot according to this interval)
- pulsar-admin
- REST API
- Java
pulsar-admin namespaces get-deduplication-snapshot-interval test-tenant/namespace1
admin.namespaces().getDeduplicationSnapshotInterval(namespace)
Set deduplication snapshot interval
Set the configured deduplicationSnapshotInterval
for a namespace. Each topic under the namespace will take a deduplication snapshot according to this interval. brokerDeduplicationEnabled
must be set to true
for this property to take effect.
- pulsar-admin
- REST API
- Java
pulsar-admin namespaces set-deduplication-snapshot-interval test-tenant/namespace1 --interval 1000
admin.namespaces().setDeduplicationSnapshotInterval(namespace, 1000)
Remove deduplication snapshot interval
Remove configured deduplicationSnapshotInterval
of a namespace (Each topic under the namespace will take a deduplication snapshot according to this interval).
- pulsar-admin
- REST API
- Java
pulsar-admin namespaces remove-deduplication-snapshot-interval test-tenant/namespace1
admin.namespaces().removeDeduplicationSnapshotInterval(namespace)
Namespace isolation
You can use the Pulsar isolation policy to allocate resources (broker and bookie) for a namespace.
Unload namespaces from a broker
You can unload a namespace, or a namespace bundle, from the Pulsar broker that is currently responsible for it.
pulsar-admin
Use the unload
subcommand of the namespaces
command.
- pulsar-admin
- REST API
- Java
pulsar-admin namespaces unload my-tenant/my-ns
admin.namespaces().unload(namespace)
Configure entry filters policy
Set entry filters policy
Entry filter helps with filtering message on server side.
- pulsar-admin
- REST API
- Java
pulsar-admin namespaces set-entry-filters \
--desc "The description of the entry filter to be used for user help." \
--entry-filters-name "The class name for the entry filter." \
--entry-filters-dir "The directory for all the entry filter implementations." \
test-tenant/namespace1
[POST /admin/v2/namespaces/:tenant/:namespace/entryFilters/setEntryFilters? version=3.1.3](https://pulsar.apache.org/admin-rest-api?version=master&apiversion=v2#operation/setEntryFilters? version=3.1.3)
admin.namespaces().setEntryFilters(namespace, new EntryFilters("desc", "classes name", "class files localtion"))
Get entry filters policies
You can get a configured entry filter for a given namespace.
- pulsar-admin
- REST API
- Java
pulsar-admin namespaces get-entry-filters test-tenant/namespace1
[GET /admin/v2/namespaces/:tenant/:namespace/entryFilters/getEntryFilters? version=3.1.3](https://pulsar.apache.org/admin-rest-api?version=master&apiversion=v2#operation/getEntryFilters? version=3.1.3)
admin.namespaces().getEntryFilters(namespace);
Remove entry filters policies
You can remove entry filters policies for a given namespace.
- pulsar-admin
- REST API
- Java
pulsar-admin namespaces remove-entry-filters test-tenant/namespace1
[DELETE /admin/v2/namespaces/:tenant/:namespace/entryFilters/removeEntryFilters? version=3.1.3](https://pulsar.apache.org/admin-rest-api?version=master&apiversion=v2#operation/removeEntryFilters? version=3.1.3)
admin.namespaces().removeEntryFilters(namespace)