Skip to main content
Version: 3.3.x

MongoDB source connector

note

You can download all the Pulsar connectors on download page.

The MongoDB source connector pulls documents from MongoDB and persists the messages to Pulsar topics.

This guide explains how to configure and use the MongoDB source connector.

Configuration

The configuration of the MongoDB source connector has the following properties.

Property

NameTypeRequiredDefaultDescription
mongoUriStringtrue" " (empty string)The MongoDB URI to which the connector connects.

For more information, see connection string URI format.
databaseStringfalse" " (empty string)The name of the watched database.

If this field is not set, the source connector will watch the entire MongoDB for all changes.
collectionStringfalse" " (empty string)The name of the watched collection.

If this field is not set, the source connector will watch the database for all changes.
syncTypeStringfalse"INCR_SYNC"The synchronization type between MongoDB and Pulsar: full synchronization or incremental synchronization.

Valid values are full_sync, FULL_SYNC, incr_sync and INCR_SYNC.
batchSizeintfalse100The batch size of pulling documents from collections.
batchTimeMslongfalse1000The batch operation interval in milliseconds.

Example

Before using the Mongo source connector, you need to create a configuration file through one of the following methods.

  • JSON

    {
    "configs": {
    "mongoUri": "mongodb://localhost:27017",
    "database": "pulsar",
    "collection": "messages",
    "syncType": "full_sync",
    "batchSize": "2",
    "batchTimeMs": "500"
    }
    }
  • YAML

    configs:
    mongoUri: "mongodb://localhost:27017"
    database: "pulsar"
    collection: "messages"
    syncType: "full_sync",
    batchSize: 2
    batchTimeMs: 500