Geometric Pattern Matching vTool#

The Geometric Pattern Matching vTool allows you to detect objects in images based on their shape using a taught pattern.

The Geometric Pattern Matching vTool accepts images via the Image input pin and outputs information about the matches found via its output pins.

It also outputs alignment transformation data that allows the Image Alignment vTool to align an input image with the reference image used during matching. This enables Measurements vTools to measure objects at different positions than the reference image.

If you use the ROI Creator vTool to specify a region of interest, the Geometric Pattern Matching vTool accepts the ROI data via its Roi input pin. In that case, objects are found only if the default reference point of the matching model lies within the region of interest.

A region of interest can be used to focus on objects in that specific region, to eliminate potential false candidates, or to reduce processing time.

You can use the transformation data supplied by the Calibration vTool to output the positions of the matches in world coordinates in meters (the Positions_m outpin pin becomes available). To do so, use the Calibration vTool before matching and connect the Transformation output/input pins of both vTools.

Geometric Pattern Matching Versions#

Two versions of the Geometric Pattern Matching vTool are available. The following table shows the differences between the versions. Click a version in the table heading to jump directly to the relevant section in this topic.

How It Works#

By defining a pattern and applying it to your input image, you can gather the following information about the matches:

• Matching scores
• Positions
• Orientations
• Vertical and horizontal scaling factors (only available in the Pro version of the vTool)
• Regions with visualizations of the matches

Pattern Definition#

The first step of pattern matching is the teaching of a model based on a representative image of the object to be detected by your application. In this teaching image, mark the desired pattern using one of the pens and teach it. Your application uses this model to detect identical patterns in your production environment. Patterns are formed by edges.

Using the Reference Point#

You can decide to display a reference point in the pattern you've marked. By default, the reference point is placed at the center of gravity of your pattern. For most applications, this works well.

If required, however, you can move the reference point to a different position. An example would be a robotic picking application. In such a scenario, the vision system has to determine the gripping position for the robot taking into account the gripper of the robot and the shape of the objects to pick. Here, you could place the reference point accordingly to accomodate both these aspects.

To display the reference point, you must mark a pattern in the teaching image first.

Contrast#

The Contrast setting refers to the contrast of the edges of a pattern. On the one hand, set the contrast so low that all relevant model edges are detected. On the other hand, set the contrast high enough to exclude noisy or unwanted edges. In most situations, enabling Contrast Auto provides good results. If enabled, the vTool determines a suitable contrast.

The contrast is included in the model. If you make changes here, you must re-teach the model.

Scaling#

The scaling options define the tolerance in size between the model and the objects located in the search images.

Three different scaling options are available: no scaling, isotropic scaling, or anisotropic scaling.

• No scaling: If the distance between camera to object or the object size itself is constant, choose this option.
• Isotropic scaling: Isotropic scaling can compensate for changes in the distance between camera and object or a varying object size.
• Anisotropic scaling: Anisotropic scaling means that scaling takes place in two directions: vertical and horizontal. Therefore, objects may vary to different degrees in length and width. This option is useful, for example, to detect rectangles with different lengths and widths.

The scaling settings are included in the model. If you make changes here, you must re-teach the model.

Polarity#

The pattern matching algorithm is based on edges in the template and search images. An edge is a set of neighboring edge points.

As such, edge points mark the border between background and search pattern, i.e., a transition from light to dark or dark to light pixels. The polarity of an edge point indicates the direction of this gray value transition.

Three options are available regarding the polarity of edge points in your search images:

• Identical: The polarity of all edge points in the search image has to be the same as in the template image. If the polarity of some edge points in the search image differs from that in the template image, the pattern may still be detected but with a lower score.
• Identical and inverted: Edge points with inverted polarity are detected as well. This means that identical and inverted patterns will be detected. A simple example is the detection of a circular shape where dark circles on a light background and light circles on a dard background will both be detected using the same model.
• Ignore polarity: The polarity of edge points is ignored. This means that edge points of any polarity will be detected.
  Example: You teach a pattern of a gray circle on a light background. As a result, gray circles on a light background, light circles on a dark background, and also circles on a mixed background including light and dark regions will be detected. Be aware that the matching score may go down a bit because fewer edge points in the search image fulfil the minimum edge contrast specified.

The following table illustrates the different polarity options available:

Polarity Setting	Template Image	Detected Patterns
Identical
Identical and inverted
Ignore polarity

Detection Angle#

In some applications, it may be useful to restrict the angle by which the orientation of patterns to be detected is allowed to rotate from the model. This may increase processing speed and robustness.

There are two options that define the range of the detection angle:

• Target: This is the angle by which patterns are allowed to rotate from the model while still being detected. Choose a value based on the expected average rotation of the patterns you want to detect. The default setting is 0 °, i.e., the orientation of the patterns in the search images is assumed to be the same as in the model. To allow for variations, choose an appropriate tolerance.
• Tolerance: This option allows you to specify the tolerance of the detection angle, i.e., it defines the range of angles centered around the target angle. The default setting is 180 °, i.e., 360 ° rotations are allowed.

Based on the target and tolerance settings, the range of the detection angle can be calculated like this: it starts at (target - tolerance) and ends at (target + tolerance).

Example: If you know that objects in your application may appear rotated in the range from 10 ° to 30 °, set the target angle to 20 ° and the angle tolerance to 10 °.

Restricting the angle is especially useful for objects with any kind of symmetry. For rectangles, set the tolerance to 90 °. For circles, set the tolerance to 0 °.

Execution Settings#

• Matches: If the application is such that only a limited number of objects can occur, you may want to restrict the possible number of matches to exactly this number. This will increase processing speed and robustness.
• Score: A score value is calculated for every match. This score indicates how closely the actual object in the image resembles the model of the pattern. For example, a score value of 0.9 says that 90 % of all model points could be matched to edges in the search image.
  To find the optimum Score setting, start with a moderate value and run some test images. Observe the output values of the Scores pin. In order to securely find a match, always set the Score option to a value lower than the scores output by the Scores pin. This is best practice and recommended to allow for some tolerance.
  Choose a score value between 50–80 % of the lowest score output values determined on the test image. In case the score is set too low, some unwanted matches may be detected as well. In that case, readjust the score value until only the intended patterns are found in the test images.
• Timeout: For time-critical applications, you can specify a timeout. A timeout helps in cases where the processing time varies or can't be foreseen, for example, if the image doesn't contain the pattern taught. By specifying a timeout, the detection process stops after the specified timeout and moves on to the next image.
  Basler recommends setting the timeout a bit lower than the desired time limit. This is because the vTool doesn't stop the detection process immediately. Therefore, set the timeout to 70–80 % of the desired time limit.

Using Calibration#

The resulting positions of the matches can also be output in world coordinates in meters. To achieve this, connect the Transformation output pin of the Calibration vTool to the Geometric Pattern Matching vTool's Transformation input pin.

You must use the same calibration configuration for setting up the Geometric Pattern Matching vTool as for the actual processing.

Common Use Cases#

• Counting objects: In this case, only the number of elements at the Scores output pin is relevant.
• Locating objects in image coordinates: Use the Positions_px and Angles_rad output pins for subsequent processing.
• Locating objects in world coordinates: Use the Positions_m and Angles_rad output pins to transmit the positions of objects, e.g., to a robot to grasp the objects.
• Determining the size of objects: Use the VScalingFactors and HScalingFactors output pins to measure size or object distance variations.

A typical example of a pattern matching result is shown below:

Geometric Pattern Matching Basic#

The Geometric Pattern Matching Basic vTool only detects objects that are completely visible in the image. The pixels marked during pattern definition form part of the object when teaching the pattern.

If only one pixel of an object is outside the image, the object won't be detected at all or at an incorrect position and with a lower score. Therefore, Basler recommends using the smallest pen possible.

Objects must be the exact same size and lie at the same distance to the camera as the object in the pattern taught. If you expect variations in size or distance or both, use the Geometric Pattern Matching Pro vTool as this vTool offers a Scaling option.

The Positions_px output pin returns the positions of objects in integer values. Use the Geometric Pattern Matching Pro vTool if sub-pixel accuracy is required.

Configuring the vTool#

To configure the Geometric Pattern Matching Basic vTool:

1. In the Recipe Management pane in the vTool Settings area, click Open Settings or double-click the vTool.
   The Geometric Pattern Matching Basic dialog opens.
2. Capture or open a teaching image.
   Either use the Single Shot button to grab a live image or click the Open Image button to open an existing image.
3. Use the pens to mark the desired pattern in the image.
   To correct the drawing, use the eraser or delete it completely.
4. In the Model Settings: Contrast area, adjust the Contrast to detect the relevant edges of the model.
   The model edges are visualized in the teaching image with the color selected in Edge visualization.
5. Teach the matching model by clicking the Teach button on the toolbar.
6. If required, display the reference point by clicking Show Reference Point.
   The reference point is displayed at the center of the pattern and the Set Reference Point Manually button becomes available. You can now drag the reference point to the desired position. If you click Set Reference Point Manually again, the reference point snaps back to its default position.
7. If desired, adjust the Matches setting in the Execution Settings area to limit the number of matches to a maximum.
8. If necessary, adjust the Score setting so that only the target objects are found.
9. If you want to specify a timeout, clear the No timeout check box and enter the desired timeout in the input field.

You can view the result of the pattern matching in a pin data view. Here, you can select which outputs to display.

Geometric Pattern Matching Pro#

The Geometric Pattern Matching Pro vTool offers the most flexibility for pattern detection.

Using the Scaling option, you can ensure that objects are still detected if their size differs from the pattern taught or the distance between camera and object varies. Also, objects that are not compleletely visible in the image area are detected, and positions of matches are returned with sub-pixel accuracy.

The Polarity option allows you to specify whether pattern edge points must have the same polarity as the edge points in the model or whether patterns with edge points of different polarities will be detected as well.

Using the detection angle options, you can restrict by how much the rotation angle of a pattern is allowed to deviate from the model.

Configuring the vTool#

To configure the Geometric Pattern Matching Pro vTool:

1. In the Recipe Management pane in the vTool Settings area, click Open Settings or double-click the vTool.
   The Geometric Pattern Matching Pro dialog opens.
2. Capture or open a teaching image.
   Either use the Single Shot button to grab a live image or click the Open Image button to open an existing image.
3. Use the pens to mark the desired pattern in the image.
   To correct the drawing, use the eraser or delete it completely.
4. In the Model Settings area, adjust the Contrast to detect the relevant edges of the model.
   The model edges are visualized in the teaching image with the color selected in Edge visualization.

5. In the Model Settings area, configure the following options if desired:

   • Scaling: Select the desired option and adjust the scaling range as required by your application.
   • Polarity: Select the desired option as required by your application.
   • Detection angle: Reduce or adjust the detection angle as required by your application.

6. Teach the matching model by clicking the Teach button on the toolbar.

7. If required, display the reference point by clicking Show Reference Point.
   The reference point is displayed at the center of the pattern and the Set Reference Point Manually button becomes available. You can now drag the reference point to the desired position. If you click Set Reference Point Manually again, the reference point snaps back to its default position.
8. If desired, adjust the Matches setting in the Execution Settings area to limit the number of matches to a maximum.
9. If necessary, adjust the Score setting so that only the target objects are found.
10. If you want to specify a timeout, clear the No timeout check box and enter the desired timeout in the input field.

You can view the result of the pattern matching in a pin data view. Here, you can select which outputs to display.

Inputs#

Image#

Accepts images directly from a Camera vTool or from a vTool that outputs images, e.g., the Image Format Converter vTool.

• Data type: Image
• Image format: 8-bit to 16-bit mono or color images. Color images are converted internally to mono images.

Roi#

Accepts a region of interest from the ROI Creator vTool or any other vTool that outputs regions or rectangles. Multiple rectangles or regions are merged internally to form a single region of interest.

• Data type: RectangleF, RectangleF Array, Region, Region Array

Transformation#

Accepts transformation data from the Calibration vTool.

• Data type: Transformation Data

Outputs#

Scores#

Returns the scores of the matches.

• Data type: Float Array

Positions_px#

Returns the positions of the matches in image coordinates (row/column).

• Data type: PointF Array

Positions_m#

Returns the positions of the matches in world coordinates in meters (x/y).

• Data type: PointF Array

Angles_rad#

Returns the orientations of the matches in radian.

• Data type: Float Array

VScalingFactors#

Returns the vertical scaling factors of the matches.

• Data type: Float Array

HScalingFactors#

Returns the horizontal scaling factors of the matches.

• Data type: Float Array

Regions#

Returns a region for every match found.

• Data type: Region Array

AlignmentTf#

Returns the transformation data that the Image Alignment vTool uses to align an object in the input image with the reference object used during matching.

• Data type: Transformation Data

Related vTools#

Typical Predecessors#

• Camera vTool
• Calibration vTool

Typical Successors#

• Measurements Basic vTool
• Circle Measurements Pro vTool
• Ellipse Measurements Pro vTool
• Line Measurements Pro vTool
• Rectangle Measurements Pro vTool