Operator ImageBufferMultiRoIDyn

This operator provides support for multiple dynamic regions of interest (ROI) for each buffered image. The ROI coordinates are defined using additional input links.

For information on the latency of the operator, see Table 33, 'Individual Latencies of the Operators in Library Memory'.

All ROIs are read sequentially as individual images: ROI 0, ROI 1, ROI 2 and so on until ROI N-1. N is the maximal number of allowed ROIs. Supported are all possible rectangular regions as long as the frame dimensions are not exceeded, i.e., a single pixel, a single line, a single column, a rectangular region, or the complete frame can be defined as a ROI. Different ROIs must not be of the same size. Each ROI can be defined individually. ROIs can overlap each other. ROIs violating the maximal image dimensions are filtered out by the operator, i.e., they will be skipped. Also, all illegal ROIs are skipped, e.g., ROIs with negative height or width.

The operator buffers the image stream in the Frame Grabber RAM (DRAM). One VisualApplets resource of type RAM is required (see 'Allocation of Device Resources').

ROI coordinates are provided on separate operator input links as images containing the ROI coordinates. Each pixel in these images is treated by the operator as a valid ROI coordinate set. Thus, the size of the ROI input images define the number of ROIs. For each input image at input link I, a new ROI coordinate image set is required. The ROI input consists of 4 coordinate input links. Respectively, links for X and Y coordinates of the top left ROI corner as well as links for X and Y coordinates of the bottom right ROI corner.

Note that ROI X-coordinates are transformed to meet the x-granularity of the input link I defined by its parallelism, i.e., the operator ImageBufferMultiRoIDyn can only cut lines with the granularity of the parallelism. The coordinate XTopLeft has to be the index of the first component in the parallel word. The coordinate XBottomRight has to be the index of a last pixel in a parallel word.

Let's have a look at an example to explain the operator behavior: Suppose a parallelism of four at the input link I. Therefore XTopLeft has to be: 0, 4, 8, 12, ... XBottomRight has to be 3, 7, 11, 15, ... If the coordinates do not match with these constraints, they are rounded down (XTopLeft) or up (XBottomRight). For example, with the ROI x-coordinates 3 and 7, the first column of the ROI is column number 3 and the last column is column number 7 of the input image. The operator rounds down XTopLeft to 0; XBottomRight remains 7; Thus, the width becomes 8. The ROI height is not dependent on the parallelism and thus can be of any legal value.

[Note] Formula:

XTopLeft* = floor(XTopLeft / Parallelism) * Parallelism

XBottomRight* = ceil((XBottomRight + 1) / Parallelism) * Parallelism - 1

[Note] Examples:

With a parallelism of four at the input link I:

  • 112 to 551 becomes 112 to 551. Therefore the width becomes = 551 + 1 - 112 = 440
  • 101 to 540 becomes 100 to 543. Therefore the width becomes = 543 + 1 - 100 = 444

All 4 ROI inputs are synchronous to each other, i.e., the images on these links must be of the same size and provided at the same time time synchronously. This is always the case when they are sourced by the same M type module through an arbitrary network of O-type module. See the synchronization rules for more information ('M-type Operators with Multiple Inputs'). The ROI coordinates are asynchronous to the input link I.

The operator starts reading the ROIs as soon as the image and all ROI coordinates are fully transfered into the buffer. In practice, the ROI coordinates can be calculated and determined from the input image but there is no necessity. Users can also use for example operator CreateBlankImage for ROI coordinate generation.

When the ROI image is empty, i.e. does not contain any pixels or all ROIs are illegal, the operator will provide an empty image on its output for each illegal ROI. An empty image contains no pixels.

ImageBufferMultiRoIDyn operates on images of fixed size defined by the input link maximal image width and height parameters. This implies input frame dimension control performed by the operator.

Images that are smaller than the maximal image dimension will be buffered without modifications. However, the reading out of ROIs will be performed on a frame of maximal possible image dimension. Missing frame pixels contain random data and should be treated in further processing as undefined.

Images that exceed either the maximal frame height or the maximal frame width will be cut to the maximal frame dimension and buffered as such. Reading out of ROIs will be based again on the frame of maximal dimension.

When an overflow occurs, i.e. the buffer cannot store more images due to being full, all incoming images will be dropped until there is enough free space in the buffer to store at least one frame. Overflow can occur due to different reasons but all of them result in the reading out of being performed too slow in comparison to writing. This can either be because of defining too many ROIs or because of a blocking condition caused by the further processing pipeline or because of a too fast input image stream. In normal situations of balanced designs an overflow will never occur.

To measure the fill level of the buffer the operator provides 2 registers: FillLevel and Overflow. FillLevel shows the percentage fill level of the RAM. The Overflow parameter is set to 1 when FillLevel is close or is 100% and the next image to be stored in the buffer will exceed the RAM capacity.

The operator provides ROI images on its output as soon as the correspondent frame is completely buffered and all ROI sets for the current image were received on the ROI inputs.

The maximal amount of supported ROIs is determined by the product of the maximal image width and height settings of the XTopLeft input link. The buffering of the ROI coordinates requires FPGA-internal block RAM. Therefore, the number of ROIs is limited by the available FPGA-internal block RAM.

Parameter InfiniteSource is used to specify if the operator is directly connected to a camera or is sequenced with other memory operators. Check 'Infinite Sources / Connecting Cameras' for more information.

Operator Restrictions

  • Empty frames on input link I are not supported.

    Images with varying line lengths on port I are fully supported.

    Images with varying line lengths or empty lines on ROI ports are not supported.

Bandwidth Optimization

The theoretical bandwidth [bits/second] going through an operator that uses the Frame Grabber RAM (DRAM) is calculated in accord with the following formula:

However, the actual bandwidth is always less than the theoretical bandwidth due to the DRAM efficiency.

The maximum bandwidth going through the operator is reached if the product of Bit Width and Parallelism is equal to the internal RAM Port Width.

[Note] Platform-specific values

RAM Port Width and System Clock are platform-specific. See Appendix. Device Resources for detailed information on your individual platform.

I/O Properties

Property Value
Operator Type M
Input Links I, image data input
XTopLeft, coordinate data input
YTopLeft, coordinate data input
XBottomRight, coordinate data input
YBottomRight, coordinate data input
Output Link O, data input

Synchronous and Asynchronous Inputs

  • The 4 ROI inputs are synchronous to each other.
  • The ROI inputs are asynchronous to input I.

Supported Link Format

Link Parameter Input Link I Input Link XTopLeft Input Link YTopLeft Input Link XBottomRight Input Link YBottomRight Output Link O
Bit Width [1, 64]12 auto3 auto4 auto5 auto6 as I
Arithmetic {unsigned, signed} unsigned unsigned unsigned unsigned as I
Parallelism any7 1 1 1 1 as I
Kernel Columns 1 1 1 1 1 as I
Kernel Rows 1 1 1 1 1 as I
Img Protocol VALT_IMAGE2D VALT_IMAGE2D VALT_IMAGE2D VALT_IMAGE2D VALT_IMAGE2D as I
Color Format any VAF_GRAY VAF_GRAY VAF_GRAY VAF_GRAY as I
Color Flavor any FL_NONE FL_NONE FL_NONE FL_NONE as I
Max. Img Width any any as XTopLeft as XTopLeft as XTopLeft as I
Max. Img Height any any as XTopLeft as XTopLeft as XTopLeft as I

1

The range of the input bit width is [1, 64] for unsigned inputs. For signed inputs, the range is [2, 64]. For unsigned color inputs, the range is [3, 63] and for signed color, the range is [6, 63].

2 7

The product of the bit width and the parallelism must not exceed the native RAM data width. Check Appendix. Device Resources for more information.

3 5

The bit width is

4 6

The bit width is

Parameters

FillLevel
Type dynamic read parameter
Default 0
Range [0%, 100%]

This parameter provides the fill level of DRAM in 25% steps.

Overflow
Type dynamic read parameter
Default 0
Range [0, 1]

This parameter indicates a buffer overflow.

InfiniteSource
Type static parameter
Default Enabled
Range {ENABLED, DISABLED}

This parameter activates support for infinite source operators like Camera operators. See 'Infinite Sources / Connecting Cameras' for more information.

Examples of Use

The use of operator ImageBufferMultiRoIDyn is shown in the following examples: