Skip to content
Wombat

cyborgdb

Inserts items into a CyborgDB encrypted vector index.

# Common config fields, showing default values
output:
  label: ""
  cyborgdb:
    max_in_flight: 64
    batching:
      count: 0
      byte_size: 0
      period: ""
      check: ""
    host: api.cyborg.com # No default (required)
    api_key: "" # No default (required)
    index_name: redpanda-vectors
    index_key: '!!!SECRET_SCRUBBED!!!' # No default (required)
    operation: upsert
    id: "" # No default (required)
    vector_mapping: root = this.embeddings_vector # No default (optional)
    metadata_mapping: root = @ # No default (optional)

This output allows you to write vectors to a CyborgDB encrypted index. CyborgDB provides end-to-end encrypted vector storage with automatic dimension detection and index optimization.

All vector data is encrypted client-side before being sent to the server, ensuring complete data privacy. The encryption key never leaves your infrastructure.

Fields

max_in_flight

The maximum number of messages to have in flight at a given time. Increase this to improve throughput.

Type: int

Default: 64

batching

Allows you to configure a batching policy.

Type: object

# Examples


batching:
  byte_size: 5000
  count: 0
  period: 1s


batching:
  count: 10
  period: 1s


batching:
  check: this.contains("END BATCH")
  count: 0
  period: 1m

batching.count

A number of messages at which the batch should be flushed. If 0 disables count based batching.

Type: int

Default: 0

batching.byte_size

An amount of bytes at which the batch should be flushed. If 0 disables size based batching.

Type: int

Default: 0

batching.period

A period in which an incomplete batch should be flushed regardless of its size.

Type: string

Default: ""

# Examples


period: 1s


period: 1m


period: 500ms

batching.check

A Bloblang query that should return a boolean value indicating whether a message should end a batch.

Type: string

Default: ""

# Examples


check: this.type == "end_of_transaction"

batching.processors

A list of processors to apply to a batch as it is flushed. This allows you to aggregate and archive the batch however you see fit. Please note that all resulting messages are flushed as a single batch, therefore splitting the batch into smaller batches using these processors is a no-op.

Type: array

# Examples


processors:
  - archive:
      format: concatenate


processors:
  - archive:
      format: lines


processors:
  - archive:
      format: json_array

host

The host for the CyborgDB instance.

Type: string

# Examples


host: api.cyborg.com


host: localhost:8000

api_key

The CyborgDB API key for authentication.

Type: string

index_name

The name of the index to write to.

Type: string

Default: "redpanda-vectors"

index_key

The base64-encoded encryption key for the index. Must be exactly 32 bytes when decoded.

Type: string

# Examples


index_key: your-base64-encoded-32-byte-key

create_if_missing

If true, create the index if it doesn’t exist. CyborgDB will auto-detect dimension and optimize the index.

Type: bool

Default: false

operation

The operation to perform against the CyborgDB index.

Type: string

Default: "upsert"

Options: upsert , delete .

id

The ID for the vector entry in CyborgDB. This field supports interpolation functions.

Type: string

vector_mapping

The mapping to extract out the vector from the document. The result must be a floating point array. Required for upsert operations.

Type: string

# Examples


vector_mapping: root = this.embeddings_vector


vector_mapping: root = [1.2, 0.5, 0.76]

metadata_mapping

An optional mapping of message to metadata for the vector entry.

Type: string

# Examples


metadata_mapping: root = @


metadata_mapping: root = metadata()


metadata_mapping: 'root = {"summary": this.summary, "category": this.category}'