Depth From Focus Using Loops

Brief Description

Files: \examples\Processing\Loop\DepthFromFocus\DepthFromFocus.va
\examples\Processing\Loop\DepthFromFocus\DepthFromFocus_NoiseReduction.va

Default Platform: mE5-MA-VCL

Short Description

Calculation of a focused image, a index depth map and the image contrast from an image sequence using loops.

The VisualApplets design "DepthFromFocus.va" calculates a completely focused image from a sequence of partially defocused images. The design "DepthFromFocus_NoiseReduction.va" gives in addition the possibility to reduce noise in the focused image. For every image of the sequence the focal setting of the camera, the image plane position or the object position (in axial direction) is changed. When doing so, a part of the object observed should always be in focus, whereas all other parts of the image are defocused. In addition in this design the highest local contrast of the image sequence is calculated with the corresponding image index map. This map gives information on the image from which the current pixel in the focused output image is taken. This provides the possibility to calculate the shape of the object observed. The focused image, the image index map and the contrast are sent to PC as red, green and blue color values. The design is implemented for a grayscale camera in CameraLink Base configuration for a marathonVCL board.

Theoretical Background

To calculate a completely focused image out of a sequence of partially defocused images, the local contrast for every single image is calculated. This contrast is determined with a high-pass filter:

Equation 46.


Is the local contrast in the current image higher than the local contrast in the previous image at this position, the pixel value from the current image is used. Otherwise the pixel value from the previous image is used. This procedure is repeated for the complete sequence of images. For every pixel in the completely focused image the index of the corresponding image with the highest local contrast is saved. This information gives the user the possibility (e.g for a thin lens, if the focal length and the image distance is known) to restore the shape of the object observed.

Implementation in Visual Applets

In Fig. 349 the basic design structure is shown. For a sequence of partially defocused images from a grayscale camera a completely focused image, the local contrast and an index map of the images with the highest local contrast are sent as RGB color components via DmaToPC to PC. The calculation is performed in the HierarchicalBox DepthFromFocus.

Basic design structure

Figure 349. Basic design structure


In Fig. 350 you can see the content of the HierarchicalBox DepthfromFocus. With the transport parameters IntParamTranslator_TranslateSequenceLength, IntParamReference_ReferenceImageWidth and IntParamReference_ReferenceImageHeight you have the possibility to set the image sequence length, the image width and height in all operators necessary in the box DepthFromFocus. For this just perform a right mouse click on the box DepthFromFocus, select "properties" and set the corresponding parameters.

In the HierarchicalBox HighpassFilter the local contrast is calculated using a filter on a kernel according to Equation 46. The upper output link of this box is the current pixel value. The lower output link of this box is the current local contrast. The operators RxImageLink_Rx2_LoopContrast/TxImageLink_Tx2_LoopContrast represent the beginning and end of the loop calculating the highest local contrast from the sequence of images. As initial condition for the loop when starting the calculation on an image sequence, a blank image is inserted for synchronization and calculation reasons in the HierarchicalBox InitAndResetContrast.

Content of HierarchicalBox DepthFromFocus

Figure 350. Content of HierarchicalBox DepthFromFocus


In the HierarchicalBox CompareContrast the current local contrast is compared to the local contrast at this position in the previous image of the sequence. Fig. 351 shows the content of this box. If the current local contrast is higher than the one in the previous image, the current local contrast is forwarded to the lower output link of this HierarchicalBox. In the opposite case the value of the previous local contrast is forwarded. The upper output link of the box CompareContrast is a index 0 (current contrast is stronger) or 1 (previous contrast is stronger).

Content of HierarchicalBox CompareContrast

Figure 351. Content of HierarchicalBox CompareContrast


In the HierarchicalBox SelectDepthIndex (content see Fig. 352) in the loop RxImageLink_Rx1_LoopIndexOfBest/TxImageLink_Tx1_LoopDepthIndex the current image index (operator ModuloCount_SequenceIndexCount) is selected, if the current image has the highest local contrast. Otherwise the image index of the previous image is forwarded to the output link of this box. This procedure is repeated for the complete image sequence. As result the image indices with the highest local contrasts of the sequence are calculated. The initial condition with an insert of a blank image is performed in the box InitAndResetDepth.

Content of HierarchicalBox SelectDepthIndex

Figure 352. Content of HierarchicalBox SelectDepthIndex


The completely focused image out of the image sequence is calculated in the loop RxImageLink_Rx0_LoopPixelValue/ TxImageLink_Tx0_LoopPixelValue with the calculation performed in the box SelectPixelValue. In Fig. 353 its content is shown. The current pixel value is forwarded to the output of the IF operator if the current local contrast is higher than the local contrast in the previous image. This procedure is repeated for the complete image sequence. At the end of the sequence a completely focused image is forwarded to the input of the box LastImageOfSequenceOnly.

Content of HierarchicalBox SelectPixelValue

Figure 353. Content of HierarchicalBox SelectPixelValue


In Fig. 354 the content of this box is shown. The pixel values with the currently highest local contrasts, the corresponding image depth index and the currently highest local contrast are combined as RGB color components. Only the last RGB image with the completely focused image, the highest local contrast and the corresponding image depth index is forwarded to the output of the box and via DMA transport to PC.

Content of HierarchicalBox LastImageOfSequenceOnly

Figure 354. Content of HierarchicalBox LastImageOfSequenceOnly


In the VA design "DepthFromFocus_NoiseReduction" there is the possibility to reduce with an adaptive Median filter the local noise in the resulting focused image. You find this filter in the HierarchicalBox AdaptiveMedian. If you have a local noise at the current pixel the local Median is selected. If there is no local noise at the current position the current pixel value is selected. A local noise is determined with a lowpass filter and an adaptive threshold IS_GreaterThan_Threshold. With this method it can be prevented that contrasts of local structures are lost but noise is reduced.