mongodb

Executes a query and creates a message for each document received.

Introduced in version 3.64.0.

Common

# Common config fields, showing default values
input:
  label: ""
  mongodb:
    url: mongodb://localhost:27017 # No default (required)
    database: "" # No default (required)
    username: ""
    password: ""
    collection: "" # No default (required)
    query: |2 # No default (required)
        root.from = {"$lte": timestamp_unix()}
        root.to = {"$gte": timestamp_unix()}
    auto_replay_nacks: true
    batch_size: 1000 # No default (optional)
    sort: {} # No default (optional)
    limit: 0 # No default (optional)

Advanced

# Advanced config fields, showing default values
input:
  label: ""
  mongodb:
    url: mongodb://localhost:27017 # No default (required)
    database: "" # No default (required)
    username: ""
    password: ""
    app_name: benthos
    collection: "" # No default (required)
    operation: find
    json_marshal_mode: canonical
    query: |2 # No default (required)
        root.from = {"$lte": timestamp_unix()}
        root.to = {"$gte": timestamp_unix()}
    auto_replay_nacks: true
    batch_size: 1000 # No default (optional)
    sort: {} # No default (optional)
    limit: 0 # No default (optional)

Once the documents from the query are exhausted, this input shuts down, allowing the pipeline to gracefully terminate (or the next input in a sequence to execute).

Fields

url

The URL of the target MongoDB server.

Type: string

# Examples

url: mongodb://localhost:27017

database

The name of the target MongoDB database.

Type: string

username

The username to connect to the database.

Type: string

Default: ""

password

The password to connect to the database.

Sensitive Information

This field contains sensitive information that usually shouldn’t be added to a config directly, read our secrets page for more info.

Type: string

Default: ""

app_name

The client application name.

Type: string

Default: "benthos"

collection

The collection to select from.

Type: string

operation

The mongodb operation to perform.

Type: string

Default: "find" Requires version 4.2.0 or newer

Options: find , aggregate .

json_marshal_mode

The json_marshal_mode setting is optional and controls the format of the output message.

Type: string

Default: "canonical" Requires version 4.7.0 or newer

Option	Summary
canonical	A string format that emphasizes type preservation at the expense of readability and interoperability. That is, conversion from canonical to BSON will generally preserve type information except in certain specific cases.
relaxed	A string format that emphasizes readability and interoperability at the expense of type preservation.That is, conversion from relaxed format to BSON can lose type information.

query

Bloblang expression describing MongoDB query.

Type: string

# Examples

query: |2
    root.from = {"$lte": timestamp_unix()}
    root.to = {"$gte": timestamp_unix()}

auto_replay_nacks

Whether messages that are rejected (nacked) at the output level should be automatically replayed indefinitely, eventually resulting in back pressure if the cause of the rejections is persistent. If set to false these messages will instead be deleted. Disabling auto replays can greatly improve memory efficiency of high throughput streams as the original shape of the data can be discarded immediately upon consumption and mutation.

Type: bool

Default: true

batch_size

A explicit number of documents to batch up before flushing them for processing. Must be greater than 0. Operations: find, aggregate

Type: int

Requires version 4.26.0 or newer

# Examples

batch_size: 1000

sort

An object specifying fields to sort by, and the respective sort order (1 ascending, -1 descending). Note: The driver currently appears to support only one sorting key. Operations: find

Type: object

Requires version 4.26.0 or newer

# Examples

sort:
  name: 1

sort:
  age: -1

limit

An explicit maximum number of documents to return. Operations: find

Type: int

Requires version 4.26.0 or newer