Manage packages
This page only shows some frequently used operations.
-
For the latest and complete information about
Pulsar admin
, including commands, flags, descriptions, and more, see Pulsar admin doc. -
For the latest and complete information about
REST API
, including parameters, responses, samples, and more, see REST API doc. -
For the latest and complete information about
Java admin API
, including classes, methods, descriptions, and more, see Java admin API doc.
Package managers or package-management systems automatically manage packages in a consistent manner. These tools simplify the installation tasks, upgrade process, and deletion operations for users. A package is a minimal unit that a package manager deals with. In Pulsar, packages are organized at the tenant-level and namespace-level to manage Pulsar Functions and Pulsar IO connectors (i.e., source and sink).
What is a package?
A package is a set of elements that the user would like to reuse in later operations. In Pulsar, a package can be a group of functions, sources, and sinks. You can define a package according to your needs.
The package management system in Pulsar stores the data and metadata of each package (as shown in the table below) and tracks the package versions.
Metadata | Description |
---|---|
description | The description of the package. |
contact | The contact information of a package. For example, an email address of the developer team. |
create_time | The time when the package is created. |
modification_time | The time when the package is lastly modified. |
properties | A user-defined key/value map to store other information. |
How to use a package
Packages can efficiently use the same set of functions and IO connectors. For example, you can use the same function, source, and sink in multiple namespaces. The main steps are:
-
Create a package in the package manager by providing the following information: type, tenant, namespace, package name, and version.
Component Description type Specify one of the supported package types: function, sink and source. tenant Specify the tenant where you want to create the package. namespace Specify the namespace where you want to create the package. name Specify the complete name of the package, using the format <tenant>/<namespace>/<package name>
.version Specify the version of the package using the format MajorVerion.MinorVersion
in numerals.The information you provide creates a URL for a package, in the format
<type>://<tenant>/<namespace>/<package name>/<version>
. -
Upload the elements to the package, i.e., the functions, sources, and sinks that you want to use across namespaces.
-
Apply permissions to this package from various namespaces.
Now, you can use the elements you defined in the package by calling this package from within the package manager. The package manager locates it by the URL. For example,
sink://public/default/mysql-sink@1.0
function://my-tenant/my-ns/my-function@0.1
source://my-tenant/my-ns/mysql-cdc-source@2.3
Package management in Pulsar
You can use the command line tools, REST API, or the Java client to manage your package resources in Pulsar. More specifically, you can use these tools to upload, download, and delete a package, get the metadata and update the metadata of a package, get the versions of a package, and get all packages of a specific type under a namespace.
To use package management service, ensure that the package management service has been enabled in your cluster by setting the following properties in broker.conf
.
Package management service is not enabled by default.
enablePackagesManagement=true
packagesManagementStorageProvider=org.apache.pulsar.packages.management.storage.bookkeeper.BookKeeperPackagesStorageProvider
packagesReplicas=1
packagesManagementLedgerRootPath=/ledgers
Upload a package
You can use the following commands to upload a package.
- pulsar-admin
- REST API
- Java
bin/pulsar-admin packages upload function://public/default/example@v0.1 --path package-file --description package-description
Upload a package to the package management service synchronously.
void upload(PackageMetadata metadata, String packageName, String path) throws PulsarAdminException;
Upload a package to the package management service asynchronously.
CompletableFuture<Void> uploadAsync(PackageMetadata metadata, String packageName, String path);
Download a package
You can use the following commands to download a package.
- pulsar-admin
- REST API
- Java
bin/pulsar-admin packages download function://public/default/example@v0.1 --path package-file
Download a package from the package management service synchronously.
void download(String packageName, String path) throws PulsarAdminException;
Download a package from the package management service asynchronously.
CompletableFuture<Void> downloadAsync(String packageName, String path);
Delete a package
You can use the following commands to delete a package.
- pulsar-admin
- REST API
- Java
The following command deletes a package of version 0.1.
bin/pulsar-admin packages delete functions://public/default/example@v0.1
Delete a specified package synchronously.
void delete(String packageName) throws PulsarAdminException;
Delete a specified package asynchronously.
CompletableFuture<Void> deleteAsync(String packageName);
Get the metadata of a package
You can use the following commands to get the metadata of a package.
- pulsar-admin
- REST API
- Java
bin/pulsar-admin packages get-metadata function://public/default/test@v1
Get the metadata of a package synchronously.
PackageMetadata getMetadata(String packageName) throws PulsarAdminException;
Get the metadata of a package asynchronously.
CompletableFuture<PackageMetadata> getMetadataAsync(String packageName);
Update the metadata of a package
You can use the following commands to update the metadata of a package.
- pulsar-admin
- REST API
- Java
bin/pulsar-admin packages update-metadata function://public/default/example@v0.1 --description update-description
Update the metadata of a package synchronously.
void updateMetadata(String packageName, PackageMetadata metadata) throws PulsarAdminException;
Update the metadata of a package asynchronously.
CompletableFuture<Void> updateMetadataAsync(String packageName, PackageMetadata metadata);
List all versions of a package
You can use the following commands to list all versions of a package.
- pulsar-admin
- REST API
- Java
bin/pulsar-admin packages list-versions type://tenant/namespace/packageName
List all versions of a package synchronously.
List<String> listPackageVersions(String packageName) throws PulsarAdminException;
List all versions of a package asynchronously.
CompletableFuture<List<String>> listPackageVersionsAsync(String packageName);
List all packages of a specific type under a namespace
You can use the following commands to list all packages of a specific type under a namespace.
- pulsar-admin
- REST API
- Java
bin/pulsar-admin packages list --type function public/default
List all packages of a specific type under a namespace synchronously.
List<String> listPackages(String type, String namespace) throws PulsarAdminException;
List all packages of a specific type under a namespace asynchronously.
CompletableFuture<List<String>> listPackagesAsync(String type, String namespace);