User-defined protocols (protocols.d)

Proprietary or in-house protocols that are not built in can be added as YAML files, without writing Rust. They become the same definition IR as the built-ins (wpg_core::ProtocolDef), so building (including Auto lengths and checksums), parsing (wpg import, the tree of the capture page), generators and the GUI builder and preview all work with them.

Loading them

The *.yaml / *.yml files directly in each directory are loaded in file name order (subdirectories are not read). On an invalid definition the command stops with exit status 2 and prints the file and the reason. wpg config validate checks protocols.dirs too.

wpg render examples/acme-telemetry.wpg.yaml -o acme.pcapng --protocols-dir examples/protocols.d
wpg import acme.pcapng -o acme.wpg.yaml --protocols-dir examples/protocols.d

Reloading a running server

wpg serve reads the directories at startup. After adding, changing or removing files, reload them without a restart:

• Web UI: Settings → User-defined protocols → Reload.
• API: POST /api/protocols/reload (admin session and X-CSRF-Token, like other changes). The answer lists the directories and the files now in use; connected browsers get a protocols_changed event and fetch the definitions again.

The same directories are read again (a changed [protocols] dirs in wpg.toml needs a restart). If a file does not load, the answer is 422 invalid_protocols with details.file / details.reason, and the protocols in use stay as they were. Jobs that are already running keep the definitions they started with.

Every distinct set of definitions that is loaded stays in memory until the server exits (running jobs may still use it); loading a set that was loaded before reuses it. This is small, but restart now and then if you reload very often during development.

Tools: wpg protocols

check reads each file, then merges them with the built-in protocols and the other files of --protocols-dir (so files may refer to each other). Errors name the file and the place in it, for example:

my.yaml: protocols[0].fields[2]: missing field `bits` at line 12 column 9
acme.yaml: bind[0]: `udp.dst_port` value 4789 is already bound to `vxlan`

To start from an existing protocol:

wpg protocols show udp > protocols.d/my-proto.yaml
# change `id` (and `name`), remove the `description_key` lines, edit the fields
wpg protocols check protocols.d/my-proto.yaml

File format

A file has protocols (new protocol definitions) and bind (entries added to the next-layer tables of existing protocols); either may be omitted. The repository has an example in examples/protocols.d/acme-telemetry.yaml.

protocols:
  - id: acme_tlm            # unique id (the name used as `protocol:` in templates)
    name: ACME Telemetry    # display name
    layer: l7               # palette class: l2 / l3 / l4 / l7 / payload
    fields:                 # in wire order
      - name: version
        type: { kind: uint, bits: 4 }
        default: { uint: 1 }
      - name: msg_type
        type:
          kind: enum
          bits: 4
          variants:
            - { value: 1, name: report }
            - { value: 2, name: alarm }
        default: { uint: 1 }
      - name: length
        type: { kind: uint, bits: 16 }
        auto: { kind: length, of: total }   # octets of the header and what follows
      - name: readings
        type: { kind: bytes, len: { kind: to_end } }

bind:
  - under: udp          # lower protocol
    value: 47000        # value of its selector field (UDP: dst_port, and src_port when parsing)
    protocol: acme_tlm

Protocol definitions

Each definition has the shape of the JSON returned by GET /api/protocols/{id}; wpg protocols show <id> prints an existing protocol in this format as a starting point.

• Field types (type.kind): uint / int (bits), enum (bits, variants), flags (bits, flags), bytes (len), mac, ipv4, ipv6, nsap, cstring, tlv, list, padding, raw.
• Defaults (default) are single-key maps naming the value type: { uint: 1 }, { bytes: [0, 1] }.
• Auto computations (auto): lengths (kind: length, of: header | payload | total), checksums (kind: checksum, algorithm: internet | internet_pseudo_header | fletcher | crc32 | crc32c ...), next header (kind: next_protocol) and more.
• Next-layer selection (next): to carry other protocols on top of this one, give field and a table (value → protocol).

Definitions go through the same checks as the built-ins when loaded (bit widths, octet alignment, Auto targets …), and unknown keys are errors.

Bindings (bind)

To bind by EtherType, add one entry per lower layer you need, e.g. ethernet2 and dot1q (after a VLAN tag).

Limitations

• Built-in protocol ids can not be reused and built-in definitions can not be changed (a second file reusing an id is an error as well).
• A value the lower protocol already maps to another protocol (UDP 4789 = VXLAN, for example) can not be bound.
• UDP / TCP ports are not Auto: set dst_port in templates (bind is used for parsing and for the GUI’s suggestions).
• Field descriptions (the Fluent description_key) are for built-ins; leave them out in user-defined protocols.
• The protocol reference (wpg docs gen) covers the built-in protocols only.

Generated reference

Field reference of the protocols on this page (generated from the definitions):

ethernet2 · dot1q · ipv4 · tcp · udp · ipv6 · raw