http_server

Sets up an HTTP server that will send messages over HTTP(S) GET requests. HTTP 2.0 is supported when using TLS, which is enabled when key and cert files are specified.

Common

# Common config fields, showing default values
output:
  label: ""
  http_server:
    address: ""
    path: /get
    stream_path: /get/stream
    ws_path: /get/ws
    allowed_verbs:
      - GET

Advanced

# Advanced config fields, showing default values
output:
  label: ""
  http_server:
    address: ""
    path: /get
    stream_path: /get/stream
    ws_path: /get/ws
    allowed_verbs:
      - GET
    timeout: 5s
    cert_file: ""
    key_file: ""
    cors:
      enabled: false
      allowed_origins: []

Sets up an HTTP server that will send messages over HTTP(S) GET requests. If the address config field is left blank the service-wide HTTP server will be used.

Three endpoints will be registered at the paths specified by the fields path, stream_path and ws_path. Which allow you to consume a single message batch, a continuous stream of line delimited messages, or a websocket of messages for each request respectively.

When messages are batched the path endpoint encodes the batch according to RFC1341. This behavior can be overridden by archiving your batches.

Please note, messages are considered delivered as soon as the data is written to the client. There is no concept of at least once delivery on this output.

[CAUTION] .Endpoint caveats

Components within a Redpanda Connect config will register their respective endpoints in a non-deterministic order. This means that establishing precedence of endpoints that are registered via multiple http_server inputs or outputs (either within brokers or from cohabiting streams) is not possible in a predictable way.

This ambiguity makes it difficult to ensure that paths which are both a subset of a path registered by a separate component, and end in a slash (/) and will therefore match against all extensions of that path, do not prevent the more specific path from matching against requests.

It is therefore recommended that you ensure paths of separate components do not collide unless they are explicitly non-competing.

For example, if you were to deploy two separate http_server inputs, one with a path /foo/ and the other with a path /foo/bar, it would not be possible to ensure that the path /foo/ does not swallow requests made to /foo/bar.

Fields

address

An alternative address to host from. If left empty the service wide address is used.

Type: string

Default: ""

path

The path from which discrete messages can be consumed.

Type: string

Default: "/get"

stream_path

The path from which a continuous stream of messages can be consumed.

Type: string

Default: "/get/stream"

ws_path

The path from which websocket connections can be established.

Type: string

Default: "/get/ws"

allowed_verbs

An array of verbs that are allowed for the path and stream_path HTTP endpoint.

Type: array

Default: ["GET"]

timeout

The maximum time to wait before a blocking, inactive connection is dropped (only applies to the path endpoint).

Type: string

Default: "5s"

cert_file

Enable TLS by specifying a certificate and key file. Only valid with a custom address.

Type: string

Default: ""

key_file

Enable TLS by specifying a certificate and key file. Only valid with a custom address.

Type: string

Default: ""

cors

Adds Cross-Origin Resource Sharing headers. Only valid with a custom address.

Type: object

Requires version 3.63.0 or newer

cors.enabled

Whether to allow CORS requests.

Type: bool

Default: false

cors.allowed_origins

An explicit list of origins that are allowed for CORS requests.

Type: array

Default: []