Skip to main content
Version: Next

Azure Data Explorer sink connector


You can download all the Pulsar connectors on download page.

The Azure Data Explorer(ADX) sink connector pulls messages from Pulsar topics and sink the messages to ADX clusters.


The configuration of the ADX sink connector has the following properties.


clusterUrlStringtrue(empty string)The ADX cluster URL.
databaseStringtrue(empty string)The database name to which data need to be ingested
tableStringtrue(empty string)Table name to which pulsar data need to be ingested
appIdStringfalse(empty string)The AAD app Id for authentication
appKeyStringfalse(empty string)The AAD app secret for authentication
tenantIdStringfalse(empty string)The tenant Id for authentication
managedIdentityIdStringfalse(empty string)The Managed Identity credential for authentication. Set this with clientId in case of user assigned MI and 'system' in case of system assigned managed identity
mappingRefNameStringfalse(empty string)The mapping reference for ingestion
mappingRefTypeStringfalse(empty string)The type of mapping reference provided. Eg. CSV, JSON etc.
flushImmediatelyBooleanfalsefalseDenotes if flush should happen immediately without aggregation. Not recommended to enable flushImmediately for production workloads
batchSizeIntfalse100For batching, this defines the number of records to hold for batching, to sink data to ADX
batchTimeMsLongfalse10000For batching, this defines the time(in ms) to hold records before sink to ADX
maxRetryAttemptsIntfalse1Max retry attempts, In case of transient ingestion errors
retryBackOffTimeLongfalse10Period of time in milliseconds to backoff before retry for transient errors


Before using the ADX sink connector, you need to create a configuration file through one of the following methods.

  • JSON

    "configs": {
    "table": "tableName",
    "appId": "xxxx-xxxx-xxxx-xxxx",
    "appKey": "xxxx-xxxx-xxxx-xxxx",
    "tenantId": "xxxx-xxxx-xxxx-xxxx",
    "mappingRefName": "mapping ref name",
  • YAML

    database: somedb,
    table: tableName,
    appId: xxxx-xxxx-xxxx-xxxx,
    appKey: xxxx-xxxx-xxxx-xxxx,
    tenantId: xxxx-xxxx-xxxx-xxxx,
    mappingRefName: mapping ref name,
    mappingRefType: CSV,
    flushImmediately: true,
    batchSize: 100,
    batchTimeMs: 10000,