Skip to content
Wombat

avro

Consume a stream of Avro OCF datum.

# Common config fields, showing default values
avro: {}

Avro JSON format

This scanner yields documents formatted as Avro JSON when decoding with Avro schemas. In this format the value of a union is encoded in JSON as follows:

  • if its type is null, then it is encoded as a JSON null;
  • otherwise it is encoded as a JSON object with one name/value pair whose name is the type’s name and whose value is the recursively encoded value. For Avro’s named types (record, fixed or enum) the user-specified name is used, for other types the type name is used.

For example, the union schema ["null","string","Foo"], where Foo is a record name, would encode:

  • null as null;
  • the string "a" as {"string": "a"}; and
  • a Foo instance as {"Foo": {...}}, where {...} indicates the JSON encoding of a Foo instance.

However, it is possible to instead create documents in standard/raw JSON format by setting the field avro_raw_json to true.

This scanner also emits the canonical Avro schema as @avro_schema metadata, along with the schema’s fingerprint available via @avro_schema_fingerprint.

Fields

raw_json

Whether messages should be decoded into normal JSON (“json that meets the expectations of regular internet json”) rather than Avro JSON. If true the schema returned from the subject should be decoded as standard json instead of as avro json. There is a comment in goavro, the underlining library used for avro serialization, that explains in more detail the difference between the standard json and avro json.

Type: bool

Default: false