Skip to content
STAGING SERVER
DEVELOPMENT SERVER

Basler Vision Connector Messaging: Recipe Payload Format#

The Recipe Payload Format is a more generic format compared to the standard vision connector image format. It's a proprietary protocol that is only supported by the Basler Vision Connector. The standard format only supports image data while the Recipe Payload Format supports the following data types:

  • Integer (64-bit, signed and unsigned)
  • Boolean
  • String
  • Float
  • Transformation data (a 2D matrix)
  • Image (only supported via ZeroMQ, but not MQTT)
  • Region (only supported via ZeroMQ, but not MQTT)
  • Composite types (e.g., rectangle, circle, ellipse, point) that consist of multiple other types
  • Arrays of all types mentioned above

Image and region data can only be transported via ZeroMQ, but not via MQTT, because these data types are very bulky and would therefore cause too much traffic for an MQTT broker. The Recipe Payload Format is organized in a hierarchical way.

For ZeroMQ, the messages will consist of multiple (at least 2) message parts:

ZeroMQ Message Envelope#

Frame Field Description Type
1 topic Device/recipe identification. Binary string (UTF-8) to identify the data source in the following syntax:
- Devices: device/{DeviceID}/cstm_images
- Recipes: cstm_recipe/{RecipeID}/data
2 metadata Metadata Binary metadata JSON (UTF-8)
3 … n binary data Additional binary data Binary (e.g., image or region data)

Additional binary data parts are only present, if data types that are transmitted require any additional binary data and if the additional data isn't embedded directly, but uses the data URI scheme instead. Which data is embedded in which additional data part is described in the Metadata JSON section below.

Metadata JSON#

Field Optional Type Default Description
Version No String 1 Metadata format version
SequentialNumber No Integer – Sequential number for the data source. Starting at 1, incremented by 1 for each message sent.
Timestamp No String – Date and time string UTF-8 ISO8601 of the host PC
Data No Metadata JSON entries – JSON sub object that contains the actual data entries. The key of each entry is an identifier and the value of the entry is described below.

Metadata JSON Entries#

Field Optional Type Default Description
Type No String – Data type of the entry
Data No JSON payload data of the entry – JSON sub-object containing the actual payload of the entry. The payload depends on the data type and is described below for each data type.

The payload of an entry can either carry the actual value for the data type or contain an error.

Metadata JSON Example#

{
  "Version": "1",
  "SequentialNumber": 123,
  "HostTimestamp": "2024-07-08T11:57:22.743739+0200",
  "Data": {
    "SomeValidEntry": {
      "Type": "Int64",
      "Data": {
        "Value": 4711
      }
    },
    "SomeErrorEntry": {
      "Type": "Boolean",
      "Data": {
        "Error": "Some error description."
      }
    },
    ...
  }
}

Metadata JSON Entry Types#

Integer Data Type#

64-bit signed and unsigned integer values can be represented.

Data type:

  • Signed integer: Int64
  • Unsigned integer: UInt64

Valid JSON entry:

Field Optional Type Default Description
Value No Integer – The signed or unsigned (depending on the data type) integer value

JSON entry carrying an error:

Field Optional Type Default Description
Error No String – An error description with the reason for the error

Examples:

{
  "Type": "Int64",
  "Data": {
    "Value": -1701
  }
}

{
  "Type": "UInt64",
  "Data": {
    "Value": 4711
  }
}

{
  "Type": "Int64",
  "Data": {
    "Error": "Some error message."
  }
}

{
  "Type": "UInt64",
  "Data": {
    "Error": "Some error message."
  }
}

Boolean Data Type#

Data type: Boolean

Valid JSON entry:

Field Optional Type Default Description
Value No Boolean – The boolean value

JSON entry carrying an error:

Field Optional Type Default Description
Error No String – An error description with the reason for the error

Examples:

{
  "Type": "Boolean",
  "Data": {
    "Value": true
  }
}

{
  "Type": "Boolean",
  "Data": {
    "Error": "Some error message."
  }
}

String Data Type#

Strings are always encoded using UTF-8.

Data type: String

Valid JSON entry:

Field Optional Type Default Description
Value No String – The UTF-8 encoded string value

JSON entry carrying an error:

Field Optional Type Default Description
Error No String – An error description with the reason for the error

Examples:

{
  "Type": "String",
  "Data": {
    "Value": "Example string"
  }
}

{
  "Type": "String",
  "Data": {
    "Error": "Some error message."
  }
}

Float Data Type#

Internally float data types are represented as 64-bit floating point variables.

Data type: Float

Valid JSON entry:

Field Optional Type Default Description
Value No Float – The float value

JSON entry carrying an error:

Field Optional Type Default Description
Error No String – An error description with the reason for the error

Examples:

{
  "Type": "Float",
  "Data": {
    "Value": 3.14
  }
}

{
  "Type": "Float",
  "Data": {
    "Error": "Some error message."
  }
}

Data Type: Transformation Data#

Transformation data represents a 2D mathematical matrix of 64-bit floating point values usually used for point/vector transformations.

Data type: TransformationData

Valid JSON entry:

Field Optional Type Default Description
Entries No 2D array of float values – The floating point values of the matrix organized in an array of arrays

JSON entry carrying an error:

Field Optional Type Default Description
Error No String – An error description with the reason for the error

Examples:

{
  "Type": "TransformationData",
  "Data": {
    "Entries": [
      [
        1.2,
        4.1
      ],
      [
        2.4,
        3.14
      ],
      [
        0,
        77.21
      ]
    ]
  }
}

{
  "Type": "TransformationData",
  "Data": {
    "Error": "Some error message."
  }
}

Image Data Type#

Images can occur in ZeroMQ and MQTT messages, but they only reference actual image data in ZeroMQ, because image data is very bulky and would therefore cause too much traffic for an MQTT broker. In MQTT messages, images are only informative. If the actual image data is required, use ZeroMQ.

Data type: Image

Valid JSON entry:

Field Optional Type Default Description
FormatNamespace Yes String GenICam Image format namespace. For GenICam pixel formats, use GenICam and Compressed for compressed formats.b
Format No String – Image format abc
Width No / Yes (encoded formats) Integer – Image width
Height No / Yes (encoded formats) Integer – Image height
LinePadding Yes Integer 0 Number of bytes added on the end of the byte stream to reach a multiple of 4 bytes. For example, an image with 61 pixels, mono8, has a line padding of 3, filling the stream to reach 64 bytes.
ImageData No / Yes (for MQTT) String – URI with the image data or a reference to the image data.d This field is mandatory for ZeroMQ and isn't available for MQTT.

  1. For cameras using a GenICam standardized pixel format, it's recommended to use the GenICam naming convention as described in section 4.35 GenICam_PFNC_2_4.pdf.

  2. For cameras using the Compressed image format namespace, the only supported format is JPEG.

  3. The Basler Vision Connector only supports GenICam, no compressed formats.

  4. A proprietary message part URI with MIME-Type is used here. It references raw data in another message part. This is only possible, if the underlying protocol supports multipart messages.
    Syntax: msgpart:{Message-Part-Index};{MIME-Type}
    Examples:

    • For GenICam image data: msgpart:2;application/octet-stream
    • For JPEG: msgpart:4;image/jpeg
    • For PNG: msgpart:3;image/png
    • For BMP: msgpart:5;image/bmp

JSON entry carrying an error:

Field Optional Type Default Description
Error No String – An error description with the reason for the error

Examples:

{
  "Type": "Image",
  "Data": {
    "FormatNamespace": "Compressed",
    "Format": "PNG",
    "Width": 80,
    "Height": 60,
    "ImageData": "msgpart:2;image/png"
  }
}

{
  "Type": "Image",
  "Data": {
    "FormatNamespace": "GenICam",
    "Format": "Mono8",
    "Width": 640,
    "Height": 480,
    "LinePadding": 0,
    "ImageData": "msgpart:3;application/octet-stream"
  }
}

{
  "Type": "Image",
  "Data": {
    "Error": "Some error message."
  }
}

Region Data Type#

Regions can occur in ZeroMQ and MQTT messages, but they only reference actual region data in ZeroMQ, because region data is very bulky and would therefore cause too much traffic for an MQTT broker. In MQTT messages, they're only informative. If the actual region data is required, use ZeroMQ.

Data type: Region

Valid JSON entry:

Field Optional Type Default Description
Format No String – Region format. Only RLE32 is supported.
BoundingBox Yes Bounding box object – JSON object with bounding box values (see below). If no bounding box is available for this region, this field doesn't exist.
ReferenceSize Yes Reference size object – JSON object with reference size values (see below). If no reference size is available for this region, this field doesn't exist.
RegionData ZeroMQ: No
MQTT: Yes		 String – URI with the region data or a reference to the region datad. This field is mandatory for ZeroMQ and isn't available for MQTT.

Bounding Box Object:

The bounding box of a region indicates the horizontal and vertical extent of the pixels contained in the region.

Field Optional Type Default Description
TopLeftX No Integer – X component of the top left pixel of the bounding box
TopLeftY No Integer – Y component of the top left pixel of the bounding box
Width No Integer – Horizontal size of the bounding box
Height No Integer – Vertical size of the bounding box

Reference Size Object:

The reference size of a region provides information about the size of the original image that was used when the region was created during image processing.

Field Optional Type Default Description
Width No Integer – Horizontal reference size
Height No Integer – Vertical reference size

JSON entry carrying an error:

Field Optional Type Default Description
Error No String – An error description with the reason for the error

Examples:

{
  "Type": "Region",
  "Data": {
    "Format": "RLE32",
    "BoundingBox": {
      "TopLeftX": 0,
      "TopLeftY": 0,
      "Width": 511,
      "Height": 256
    },
    "ReferenceSize": {
      "Width": 1024,
      "Height": 768
    },
    "RegionData": "msgpart:6;application/octet-stream"
  }
}

{
  "Type": "Region",
  "Data": {
    "Error": "Some error message."
  }
}

Composite Data Types#

Composite data types consist of one or multiple other types. With these data types you can create a hierarchical structure of data types.

It's valid that some sub data types carry errors, while the parent composite data type doesn't carry an error. In this case, only parts of the composite data type carry values. The sub data types are always added to the JSON regardless whether the parent composite data type carries an error.

Currently, the following composite data types exist:

  • 2D point (data type: PointF2D). Consists of:
  • X (data type: Float)
  • Y (data type: Float)
  • 2D line (data type: LineF2D). Consists of:
  • PointA (data type: PointF2D)
  • PointB (data type: PointF2D)
  • Rectangle (data type: RectangleF). Consists of:
  • Center (data type: PointF2D)
  • Width (data type: Float)
  • Height (data type: Float)
  • Rotation (data type: Float)
  • Circle (data type: CircleF). Consists of:
  • Center (data type: PointF2D)
  • Radius (data type: Float)
  • Ellipse (data type: EllipseF). Consists of:
  • Center (data type: PointF2D)
  • Radius1 (data type: Float)
  • Radius2 (data type: Float)
  • Rotation (data type: Float)

Examples for 2D point:

{
  "Type": "PointF2D",
  "Data": {
    "X": {
      "Type": "Float",
      "Data": {
        "Value": 2.4
      }
    },
    "Y": {
      "Type": "Float",
      "Data": {
        "Value": -3.14
      }
    }
  }
}

{
  "Type": "PointF2D",
  "Data": {
    "Error": "Some error message.",
    "X": {
      "Type": "Float",
      "Data": {
        "Error": "Some error message."
      }
    },
    "Y": {
      "Type": "Float",
      "Data": {
        "Error": "Some error message."
      }
    }
  }
}

Examples for 2D line:

{
  "Type": "LineF2D",
  "Data": {
    "PointA": {
      "Type": "PointF2D",
      "Data": {
        "X": {
          "Type": "Float",
          "Data": {
            "Value": 2.4
          }
        },
        "Y": {
          "Type": "Float",
          "Data": {
            "Value": -3.14
          }
        }
      }
    },
    "PointB": {
      "Type": "PointF2D",
      "Data": {
        "X": {
          "Type": "Float",
          "Data": {
            "Value": 5.2
          }
        },
        "Y": {
          "Type": "Float",
          "Data": {
            "Value": -6.6
          }
        }
      }
    }
  }
}

{
  "Type": "LineF2D",
  "Data": {
    "Error": "Some error message.",
    "PointA": {
      "Type": "PointF2D",
      "Data": {
        "Error": "Some error message.",
        "X": {
          "Type": "Float",
          "Data": {
            "Error": "Some error message."
          }
        },
        "Y": {
          "Type": "Float",
          "Data": {
            "Error": "Some error message."
          }
        }
      }
    },
    "PointB": {
      "Type": "PointF2D",
      "Data": {
        "Error": "Some error message.",
        "X": {
          "Type": "Float",
          "Data": {
            "Error": "Some error message."
          }
        },
        "Y": {
          "Type": "Float",
          "Data": {
            "Error": "Some error message."
          }
        }
      }
    }
  }
}

Examples for rectangle:

{
  "Type": "RectangleF",
  "Data": {
    "Center": {
      "Type": "PointF2D",
      "Data": {
        "X": {
          "Type": "Float",
          "Data": {
            "Value": -3.4
          }
        },
        "Y": {
          "Type": "Float",
          "Data": {
            "Value": 11.3
          }
        }
      }
    },
    "Height": {
      "Type": "Float",
      "Data": {
        "Value": 23.2
      }
    },
    "Rotation": {
      "Type": "Float",
      "Data": {
        "Value": -17
      }
    },
    "Width": {
      "Type": "Float",
      "Data": {
        "Value": -46.2
      }
    }
  }
}

{
  "Type": "RectangleF",
  "Data": {
    "Error": "Some error message.",
    "Center": {
      "Type": "PointF2D",
      "Data": {
        "Error": "Some error message.",
        "X": {
          "Type": "Float",
          "Data": {
            "Error": "Some error message."
          }
        },
        "Y": {
          "Type": "Float",
          "Data": {
            "Error": "Some error message."
          }
        }
      }
    },
    "Height": {
      "Type": "Float",
      "Data": {
        "Error": "Some error message."
      }
    },
    "Rotation": {
      "Type": "Float",
      "Data": {
        "Error": "Some error message."
      }
    },
    "Width": {
      "Type": "Float",
      "Data": {
        "Error": "Some error message."
      }
    }
  }
}

Examples for circle:

{
  "Type": "CircleF",
  "Data": {
    "Center": {
      "Type": "PointF2D",
      "Data": {
        "X": {
          "Type": "Float",
          "Data": {
            "Value": -2.1
          }
        },
        "Y": {
          "Type": "Float",
          "Data": {
            "Value": 34.1
          }
        }
      }
    },
    "Radius": {
      "Type": "Float",
      "Data": {
        "Value": -6.6
      }
    }
  }
}

{
  "Type": "CircleF",
  "Data": {
    "Error": "Some error message.",
    "Center": {
      "Type": "PointF2D",
      "Data": {
        "Error": "Some error message.",
        "X": {
          "Type": "Float",
          "Data": {
            "Error": "Some error message."
          }
        },
        "Y": {
          "Type": "Float",
          "Data": {
            "Error": "Some error message."
          }
        }
      }
    },
    "Radius": {
      "Type": "Float",
      "Data": {
        "Error": "Some error message."
      }
    }
  }
}

Examples for ellipse:

{
  "Type": "EllipseF",
  "Data": {
    "Center": {
      "Type": "PointF2D",
      "Data": {
        "X": {
          "Type": "Float",
          "Data": {
            "Value": 13
          }
        },
        "Y": {
          "Type": "Float",
          "Data": {
            "Value": -1.3
          }
        }
      }
    },
    "Radius1": {
      "Type": "Float",
      "Data": {
        "Value": 1.4
      }
    },
    "Radius2": {
      "Type": "Float",
      "Data": {
        "Value": -2.1
      }
    },
    "Rotation": {
      "Type": "Float",
      "Data": {
        "Value": 5.14
      }
    }
  }
}

{
  "Type": "EllipseF",
  "Data": {
    "Error": "Some error message.",
    "Center": {
      "Type": "PointF2D",
      "Data": {
        "Error": "Some error message.",
        "X": {
          "Type": "Float",
          "Data": {
            "Error": "Some error message."
          }
        },
        "Y": {
          "Type": "Float",
          "Data": {
            "Error": "Some error message."
          }
        }
      }
    },
    "Radius1": {
      "Type": "Float",
      "Data": {
        "Error": "Some error message."
      }
    },
    "Radius2": {
      "Type": "Float",
      "Data": {
        "Error": "Some error message."
      }
    },
    "Rotation": {
      "Type": "Float",
      "Data": {
        "Error": "Some error message."
      }
    }
  }
}

Array Data Type#

Arrays can consist of any other type. However, the contained array elements can't be of mixed data types and multidimensional or nested array aren't supported.

It's valid that some element data types carry errors, while the parent array data type doesn't carry an error. In this case, only parts of the array data type carry values.

Data type: Array

Valid JSON entry:

Field Optional Type Default Description
ElementType No String – Data type of the elements contained in the array
Elements No Array of objects – Array of the objects described by the ElementType field

JSON entry carrying an error:

Field Optional Type Default Description
ElementType No String – Data type of the elements contained in the array
Error No String – An error description with the reason for the error

Examples:

{
  "Type": "Array",
  "Data": {
    "ElementType": "Int64",
    "Elements": [
      {
        "Value": 234
      },
      {
        "Value": 19
      }
    ]
  }
}

{
  "Type": "Array",
  "Data": {
    "ElementType": "PointF2D",
    "Elements": [
      {
        "X": {
          "Type": "Float",
          "Data": {
            "Value": 2.4
          }
        },
        "Y": {
          "Type": "Float",
          "Data": {
            "Value": -3.14
          }
        }
      },
      {
        "Error": "Some error message.",
        "X": {
          "Type": "Float",
          "Data": {
            "Error": "Some error message."
          }
        },
        "Y": {
          "Type": "Float",
          "Data": {
            "Value": 1.45
          }
        }
      }
    ]
  }
}

{
  "Type": "Array",
  "Data": {
    "ElementType": "Int64",
    "Error": "Some error message."
  }
}