pylon AI Platform#

The pylon AI Platform is a central hub for managing the lifecycle of your deep learning (DL) models.

It provides different machine vision projects for each AI vTool and also allows you to create your own projects. It supports all stages of model lifecycle management, from development to deployment, including model optimization for diverse hardware environments.

The AI Platform is available under the following URL: https://basler.ai. Simply sign up, create a Basler ID account, and get access.

How It Works#

As an introduction, the following sections provide some background information and explain key aspects of creating projects with the pylon AI Platform.

Concepts and Terminology#

Models: A model is an algorithm that can be trained using sample images. Based on the training, the model is then able to "predict" image content.

Architecture: The architecture of a model determines how a model works. Different architectures are available for different machine vision tasks.

Data sets: Data sets are a collection of data relevant to a specific problem or task. It contains all the visual properties related to the problem.

For instance, images of cats and dogs can form a data set for a classification model that distinguishes between cats and dogs. If the model needs to differentiate not only between cats and dogs but also mice, mouse image data must be added to the data set.

Typically, a data set is divided into three parts:

• Training data: Training data constitutes the largest portion of the data set used by a deep learning model for learning. The model adjusts its weights to recognize patterns within the training data.
• Validation data: Validation data is used by a DL model to assess its behavior during training. It remains independent of the training data and is not used for optimization.
• Test data: After training, the model's performance is evaluated using test data. Test data represents new, unseen examples and demonstrates how well the model generalizes to real-world data.

You can use the Recording feature of the pylon Viewer to create your own data sets.

Annotations/Labels: Annotations, also called labels, are added to the data set to inform the model about what is significant in the data. During training, the model uses this information to compare predictions with the expected/correct outcomes. For the classification task mentioned above, this information indicates whether an image depicts a dog or a cat.

To annotate images for use with pylon AI, you have to use the open-source tool CVAT. Currently, no other tools are compatible. This video demonstrates how to annotate images with bounding boxes for object detection using CVAT.

Training: In the context of deep learning, training is the process in which a neural network (a model) learns from the training data. The model makes a prediction for all training data (see "Data sets" above). The predictions are compared against the ground truth data (the annotation/label) in order to calculate the deviation. Based on the deviation, the weights of the model are adjusted in order to minimize the deviation. This process is repeated until the error has been reduced as much as possible. To verify that the model performs well, the model is used on validation data.

Inference: In the context of deep learning, inference means that a trained model is applied to new, unseen data and makes predictions about the content.

Using the pylon AI Platform#

This section takes you through the different steps of creating a machine vision project.

The pylon AI Platform is designed for intuitive use. After logging in to the pylon AI Platform, you see the Dashboard view.

This shows the 5 most recently released bundles as well as the 5 most recently run trials.

To start creating projects, click My Projects in the navigation tree on the left-hand side.

In the My Projects view, you see all projects that you, or other people who have access to the AI Platform, have created. You can choose to display projects in list view or tile view.

Project Creation#

You start by creating a project.

1. Log in to the pylon AI Platform.
2. Click My Projects in the navigation tree on the left-hand side.
3. On the My Projects page, click Create new project.
4. On the Create New Project page, specify the project type you want to create and enter a name and a short description (optional).
   A project overview page opens.
   

The tabbed navigation at the top shows you all steps that you can currently perform. Some steps only become available after you have completed the preceding steps.

In the notification area, a new message pops up for every action you perform on the AI Platform. All messages are saved until you choose to delete them.

The home button takes you back to the overview page of your project.

Data Sets#

1. Click the Add data set button or click Data Sets in the tab list above.
2. On the Add Data Set page, enter a name and a short description (optional) for the data set you want to add.
   
3. Click Add.

4. On the Upload Data page, specify the format of the data you're uploading and add the data.
   

   Info

   The data must be uploaded as a .zip file. The .zip file must contain image files and an annotation file.

   Data validation starts immediately. Once the validation has finished successfully, you can check the annotations and continue with creating data set classes.

   

5. Click Map classes.

6. In the Class Mapping dialog, click Autocreate classes to automatically create data set classes based on the annotations in your images.
   Alternatively, click Add new class in the drop-down lists under Data Set Class.
   
7. Click OK.
8. On the Upload Data page, click Upload.
   The Data Sets tab opens where you can see your data set.
9. Click the data set to view its content, add images, and edit its properties. When you are happy with the state of the data set, click Publish.
   You have to publish the data set to be able to use it in trials.
   
10. Go back to the Data Sets tab.

Models#

1. Click the Models tab.
2. On the Models page, click Add model.
3. On the Upload Model page, enter a name and a short description (optional) for the model you want to add.
   
4. Click Create.
5. On the Upload Model page, specify the configuration of the model you want to upload and add the model file.
   The upload and validation start immediately. When this has finished, you can check which classes your model contains.
   
6. Click Finish.
   The Models tab opens where you can see your model.
7. Click the model to view its properties.
   
8. Go back to Models tab.

Mapping#

Data sets and models usually contain more classes than you need for a specific task. Mapping is the process of matching data set classes to model classes to define what you're looking for with your project. For this purpose, you have to add project classes and assign the desired data set and model classes to them.

1. Click the Mapping tab.
   
2. On the Mapping tab, click Create project class to create your first project class.
   
   Alternatively, you can use the Automap feature.
   
3. Change the default name to a different name if desired.
4. Add data set classes and model classes to it.
   You can just drag and drop classes from the corresponding tabs on the right. Using the drop-down lists, you can change the data set or its version and the model version.
   
5. Repeat steps 2 to 4 until you have created all the project classes you need.
6. Exclude all data set and model classes that you don't need (see Excluding Classes for details).

Automapping#

Automapping allows you to map data set and model classes to project classes based on their names with one click. If no project class exists that matches the data set and model class names, a new project class with that name will be created. Therefore, if you have more data set classes or model classes than you want to use, make sure to exclude them before using automapping.

Automapping only works for unmapped classes. Any classes that have been mapped manually before, will remain in their classes.

If the names of data set or project classes contain the words "background" or "excluded", they will be excluded or defined as background depending on the project type.

Automapping is case-insensitive but the names of the classes have to be exactly identical.

To Use Automapping:

1. On the Data Set Classes tab on the right-hand side, exclude all data set classes that you don't need.
2. Click Automap.
   
3. In the Automapping Summary dialog, check that you want to create all the project classes listed there.
   You can deselect any classes you don't want. Deselected classes remain unmapped. Make sure to exclude them afterwards.
4. Click Map classes.
   Project classes are created and data set classes are added to them.
5. On the Model Classes tab on the right-hand side, exclude all model classes that you don't need.
6. Click Automap.
   
7. The Automapping Summary dialog shows existing project classes and which model classes will be mapped to them and any new project classes that will be created. Check that you want to create the project classes listed as new.
   You can deselect any classes you don't want. Deselected classes remain unmapped. Make sure to exclude them afterwards.
8. Click Map classes.
   Project classes that don't exist yet are created. Model classes are added to new and existing project classes.

Excluding Classes#

Model classes that you don't want to use in a project have to be excluded to complete the mapping. This is necessary to get correct statistical data from your model.

To exclude classes, select all the model classes that are not required and click Exclude. Alternatively, you can drag the classes selected to the bottom of the screen and drop them in the Excluded section.

For segmentation projects, this is slightly different. Here, all superfluous model classes have to be defined as background. To do this, select all the model classes that are not required and click Define as background. Alternatively, you can drag the classes selected to the bottom of the screen and drop them in the Background section.

Both these actions can be reversed using the corresponding buttons or by dragging and dropping.

The excluded classes won't be considered when applying your project. This means that you won't get any results for them.

Trials#

1. Click the Trials tab.
2. On the Trials page, click Add trial.
3. On the Create Trial page, enter a name and a short description (optional) for the trial you want to add.
   
4. In the Project Assets area, click Add data set to select the data set you want to use for the trial.
   
5. In the Project Assets area, click Add model to select the model you want to use for the trial.
   
6. In the Project Assets area, click Check mapping.
   You should check the mapping and make any necessary changes. After you have created the trial, you can't change the project classes anymore.
   
7. Click Create.
8. Click Add next trial run.
9. Select the hardware and model format.
   
10. Click Run trial.
    
    The trial run starts immediately. When the run has finished, the statistics are displayed on the trial page.
    
11. If you're happy with the results, you can release the bundle by clicking the Release bundle button next to the trial.
    
12. Change the name of the bundle and enter a description (optional).
    
13. Click Release bundle.
    The bundle is being generated.
14. Go back to the project overview page and click the Bundles tab.
    After a short while, your bundle appears here.

Bundles#

After they have been released, bundles are displayed on the Bundles tab. Here, you can inspect a bundle's properties and also view details of the trial run that a bundle was generated from.

The bundle contains all necessary information, e.g., hardware configuration, model, etc., that a vTool needs to perform its vision task. It is now ready to be deployed to AI vTools using the pylon AI Agent.

You can also download a bundle to a local or remote storage location. In that case, users without access to the pylon AI Platform can still use the models in the bundles.