Manage schema
This guide demonstrates the ways to manage schemas:
-
Automatically
-
Manually
Schema AutoUpdateβ
If a schema passes the schema compatibility check, Pulsar producer automatically updates this schema to the topic it produces by default.
AutoUpdate for producerβ
For a producer, the AutoUpdate
happens in the following cases:
-
If a topic doesnβt have a schema, Pulsar registers a schema automatically.
-
If a topic has a schema:
-
If a producer doesnβt carry a schema:
-
If
isSchemaValidationEnforced
orschemaValidationEnforced
is disabled in the namespace to which the topic belongs, the producer is allowed to connect to the topic and produce data. -
If
isSchemaValidationEnforced
orschemaValidationEnforced
is enabled in the namespace to which the topic belongs, the producer is rejected and disconnected. -
If a producer carries a schema:
A broker performs the compatibility check based on the configured compatibility check strategy of the namespace to which the topic belongs.
-
If the schema is registered, a producer is connected to a broker.
-
If the schema is not registered:
-
If
isAllowAutoUpdateSchema
sets to false, the producer is rejected to connect to a broker. -
If
isAllowAutoUpdateSchema
sets to true:-
If the schema passes the compatibility check, then the broker registers a new schema automatically for the topic and the producer is connected.
-
If the schema does not pass the compatibility check, then the broker does not register a schema and the producer is rejected to connect to a broker.
-
-
-
AutoUpdate for consumerβ
For a consumer, the AutoUpdate
happens in the following cases:
-
If a consumer connects to a topic without a schema (which means the consumer receiving raw bytes), the consumer can connect to the topic successfully without doing any compatibility check.
-
If a consumer connects to a topic with a schema.
-
If a topic does not have all of them (a schema/data/a local consumer and a local producer):
-
If
isAllowAutoUpdateSchema
sets to true, then the consumer registers a schema and it is connected to a broker. -
If
isAllowAutoUpdateSchema
sets to false, then the consumer is rejected to connect to a broker.
-
-
If a topic has one of them (a schema/data/a local consumer and a local producer), then the schema compatibility check is performed.
-
If the schema passes the compatibility check, then the consumer is connected to the broker.
-
If the schema does not pass the compatibility check, then the consumer is rejected to connect to the broker.
-
-
Manage AutoUpdate strategyβ
You can use the pulsar-admin
command to manage the AutoUpdate
strategy as below:
Enable AutoUpdateβ
To enable AutoUpdate
on a namespace, you can use the pulsar-admin
command.
bin/pulsar-admin namespaces set-is-allow-auto-update-schema --enable tenant/namespace
Disable AutoUpdateβ
To disable AutoUpdate
on a namespace, you can use the pulsar-admin
command.
bin/pulsar-admin namespaces set-is-allow-auto-update-schema --disable tenant/namespace
Once the AutoUpdate
is disabled, you can only register a new schema using the pulsar-admin
command.
Adjust compatibilityβ
To adjust the schema compatibility level on a namespace, you can use the pulsar-admin
command.
bin/pulsar-admin namespaces set-schema-compatibility-strategy --compatibility <compatibility-level> tenant/namespace
Schema validationβ
By default, schemaValidationEnforced
is disabled for producers:
-
This means a producer without a schema can produce any kind of messages to a topic with schemas, which may result in producing trash data to the topic.
-
This allows non-java language clients that donβt support schema can produce messages to a topic with schemas.
However, if you want a stronger guarantee on the topics with schemas, you can enable schemaValidationEnforced
across the whole cluster or on a per-namespace basis.