Skip to main content
Version: Next

Manage Schemas

tip

This page only shows some frequently used operations.

  • For the latest and complete information about Pulsar admin, including commands, flags, descriptions, and more, see Pulsar admin docs.

  • For the latest and complete information about REST API, including parameters, responses, samples, and more, see REST API doc.

  • For the latest and complete information about Java admin API, including classes, methods, descriptions, and more, see Java admin API doc.

Manage schema

Upload a schema

To upload (register) a new schema for a topic, you can use one of the following methods.

Use the upload subcommand.

pulsar-admin schemas upload --filename <schema-definition-file> <topic-name>

The schema-definition-file is in JSON format.

{
"type": "<schema-type>",
"schema": "<an-utf8-encoded-string-of-schema-definition-data>",
"properties": {} // the properties associated with the schema
}

The payload includes the following fields:

FieldDescription
type
  • Allowed values for primitive-type schemas are listed on the following page: Primitive types
  • Allowed values for struct-type schemas are AVRO, PROTOBUF, PROTOBUF_NATIVE and JSON.
  • schemaThe schema definition data, which is encoded in UTF 8 charset.
  • If the schema type is AVRO, PROTOBUF or JSON schema, this field should be an Avro schema definition in JSON format.
  • If the schema type is PROTOBUF_NATIVE schema, this field should contain a Protobuf descriptor.
  • Otherwise, this field should be blank.
  • propertiesThe additional properties associated with the schema.

    The following is an example for a JSON schema.

    Example

    {
    "type": "JSON",
    "schema": "{\"type\":\"record\",\"name\":\"User\",\"namespace\":\"com.foo\",\"fields\":[{\"name\":\"file1\",\"type\":[\"null\",\"string\"],\"default\":null},{\"name\":\"file2\",\"type\":[\"null\",\"string\"],\"default\":null},{\"name\":\"file3\",\"type\":[\"string\",\"null\"],\"default\":\"dfdf\"}]}",
    "properties": {}
    }

    Get the latest schema

    To get the latest schema for a topic, you can use one of the following methods.

    Use the get subcommand.

    pulsar-admin schemas get <topic-name>

    Example output:

    {
    "version": 0,
    "type": "String",
    "timestamp": 0,
    "data": "string",
    "properties": {
    "property1": "string",
    "property2": "string"
    }
    }

    Get a specific schema

    To get a specific version of a schema, you can use one of the following methods.

    Use the get subcommand.

    pulsar-admin schemas get <topic-name> --version <version>

    Extract a schema

    To extract (provide) a schema via a topic, use the following method.

    Use the extract subcommand.

    pulsar-admin schemas extract --classname <class-name> --jar <absolute-jar-path> --type <type-name>

    Delete a schema

    note

    In any case, the delete action deletes all versions of a schema registered for a topic.

    To delete a schema for a topic, you can use one of the following methods.

    Use the delete subcommand.

    pulsar-admin schemas delete <topic-name>

    Manage schema AutoUpdate

    Enable schema AutoUpdate

    To enable/enforce schema auto-update at the namespace level, you can use one of the following methods.

    Use the set-is-allow-auto-update-schema subcommand.

    bin/pulsar-admin namespaces set-is-allow-auto-update-schema --enable tenant/namespace

    Disable schema AutoUpdate

    note

    When schema auto-update is disabled, you can only register a new schema.

    To disable schema auto-update at the namespace level, you can use one of the following commands.

    Use the set-is-allow-auto-update-schema subcommand.

    bin/pulsar-admin namespaces set-is-allow-auto-update-schema --disable tenant/namespace

    Manage schema validation enforcement

    Enable schema validation enforcement

    To enforce schema validation enforcement at the cluster level, you can configure isSchemaValidationEnforced to true in the conf/broker.conf file.

    To enable schema validation enforcement at the namespace level, you can use one of the following commands.

    Use the set-schema-validation-enforce subcommand.

    bin/pulsar-admin namespaces set-schema-validation-enforce --enable tenant/namespace

    Disable schema validation enforcement

    To disable schema validation enforcement at the namespace level, you can use one of the following commands.

    Use the set-schema-validation-enforce subcommand.

    bin/pulsar-admin namespaces set-schema-validation-enforce --disable tenant/namespace

    Manage schema compatibility strategy

    The schema compatibility check strategy configured at different levels has priority: topic level > namespace level > cluster level. In other words:

    • If you set the strategy at both topic and namespace levels, the topic-level strategy is used.
    • If you set the strategy at both namespace and cluster levels, the namespace-level strategy is used.

    Set schema compatibility strategy

    Set topic-level schema compatibility strategy

    To set a schema compatibility check strategy at the topic level, you can use one of the following methods.

    Use the pulsar-admin topicPolicies set-schema-compatibility-strategy command.

    pulsar-admin topicPolicies set-schema-compatibility-strategy <strategy> <topicName>

    Set namespace-level schema compatibility strategy

    To set schema compatibility check strategy at the namespace level, you can use one of the following methods.

    Use the pulsar-admin namespaces set-schema-compatibility-strategy command.

    pulsar-admin namespaces set-schema-compatibility-strategy options

    Set cluster-level schema compatibility strategy

    To set schema compatibility check strategy at the cluster level, set schemaCompatibilityStrategy in the conf/broker.conf file.

    The following is an example:

    schemaCompatibilityStrategy=ALWAYS_INCOMPATIBLE

    Get schema compatibility strategy

    Get topic-level schema compatibility strategy

    To get the topic-level schema compatibility check strategy, you can use one of the following methods.

    Use the pulsar-admin topicPolicies get-schema-compatibility-strategy command.

    pulsar-admin topicPolicies get-schema-compatibility-strategy <topicName>

    Get namespace-level schema compatibility strategy

    You can get schema compatibility check strategy at namespace level using one of the following methods.

    Use the pulsar-admin namespaces get-schema-compatibility-strategy command.

    pulsar-admin namespaces get-schema-compatibility-strategy options