Models

Using AnalyticOps, you can organize models within projects and store them in a common model repository. You can also train models, import models, evaluate and compare them for champion model selection, deploy models and monitor their performance.

The models module organizes and facilitates creation and management of models within the selected project. This chapter covers the following models related details:

• Types of Models

• Create a Git Model

• Define a BYOM Model

• View List of Models

• Model Monitoring Alerts

• Archive a Model

Types of Models

AnalyticOps allows you to work with two types of models:

• Git models: Using Github you add models to a given repository by following several steps, such as creating the required directory structure, editing configuration files, generating a model ID, etc.

• Bring your own models (BYOM): Bring your own models (BYOM) support lets you import models that you develop using an external application as well as open-source programming languages such as Python or R, and PMML models. The purpose is to import, govern and deploy pre-trained Python and R models.

Create a Git Model

Comment: Need to discuss what should be added in this section

Define a BYOM Model

AnalyticOps allows you to define BYOM models and import versions.

To define a BYOM model:

1. Double-click a Project from the List of Projects.

   

   The List of Models for the selected project displays in the Work area and the Models option highlights in the Navigation bar.

   

2. Click the Define BYOM Model button.
   The Define BYOM Model dialog displays.

   

3. In the Basic tab, set the properties:

   Property	Description
   Name	Specifies the BYOM model name.
   Description	Specifies the description of the BYOM model.
   Format	Lets you select the format of the BYOM model: PMML, ONNX, H2O, Python, R, Custom
   Link to BYOM	An optional property that allows linking to BYOM model workspace. It can be a GIT or DS workspace.

   

4. In the Advanced tab, set the properties:

   Property	Description
   Configuration	An optional property that allows adding further details about the BYOM model.
   Automation	An optional property that allows adding automation configuration about the BYOM model.

   

5. Click Save to save the defined BYOM model.
   

   Note: Clicking Save & Import Versions will move you directly towards the Import Versions dialog. For details, see Model Versions.

   The BYOM model adds to the List of Models.

   

Edit a BYOM Model

AnalyticOps allows you to edit an existing BYOM model. To edit an existing BYOM:

1. Select a BYOM model from the list.
   The Edit BYOM Model button enables.

   

2. Click the Edit BYOM Model button.

   The Edit BYOM Model dialog displays where you can modify the selected model.

   

3. After modifying the desired properties, click Save.
   The BYOM model saves with all the latest modifications.

View List of Models

To view models added to a project:

1. Double-click a Project from the List of Projects.
   The List of Models for the selected project displays in the Work area and the Models option highlights in the Navigation bar.

   

   The List of Models displays the following properties for each model.

   Property	Description
   Model ID	Specifies the auto-generated model ID.
   Model Name	Specifies the model name.
   Description	Specifies the description of the project.
   Type	Shows the model type as GIT and BYOM.
   Language	Specifies the programming language of the model.
   Creation Date	Lets you know the creation date of model.
   View Model Versions	Allows you to view the model versions for the selected model. For details, see Model Versions.

Model Monitoring Alerts

AnalyticOps allows you to monitor the performance of models and model versions against specific key metrics. In addition to metric gathering and tracking, the model monitoring involves alerting based on these metric values.

AnalyticOps is using Prometheus alert manager to provide the support of alerting in UI. Prometheus is a very easy-to-use yet powerful tool that allows you to make the alerting rules very sophisticated and keep the alerting rule complexity encapsulated for the end-user.

The below section describes how to enable alerts for a model and how to define new alert configurations. For more details on monitoring alert, see Monitoring Alerts.

Enable Alerts

You can enable/disable alerts for a model added to the Models list. By default, the alerts are disabled for all models.

To enable alerts for a model:

1. In the Models list, select the Enable Alert option for the model.
   The alerts for the selected model enable. A few default alerts are added to each model, however you can define new alert configurations for each model. For details, see Configure Alerts.

   

   You can disable the alerts for a model anytime by again selecting the Enable Alert option for the model.

Enabling alerts will start generating alerts if the defined rules in alert configuration become true. To view the active alerts and their details, see Monitoring Alerts.

Configure Alerts

You can define new alert configurations for a selected model. The alert configuration allows you to create rules by defining threshold values for specific metrics. If the metric values reaches to the defined threshold value, the alert will generate.

To configure an alert:

1. Select a model in the list and select Configure Alerts from the Actions menu.

   

   The Alert Configurations list displays the list of alerts defined for the model.

   

2. Click the Create Rule button.
   The Create Alert Configuration dialog displays.

   

3. In the Create Alert Configuration dialog, set the properties:

   Property	Description
   Name	Specifies the name of the alert configuration.
   Model	Specifies the read-only model ID for which the alert is defined.
   Summary	Lets you define the summary of the alert.
   Description	Lets you add the description for the alert.
   Expression	Specifies the condition expression for the alert by defining threshold values for metrics. Reset to Default allows you to reset the expression to the default expression.
   Occurs For	Lets you select the time duration for which the condition should hold true in order to generate the alert.

4. Click Save Configuration.
   The alert configuration adds to the list.

Edit an Alert Configuration

To edit an existing alert configuration:

1. Select an alert configuration from the list.
   The Edit Rule button enables.

   

2. Click the Edit Rule button.
   The Edit Alert Configuration dialog displays where you can modify the selected configuration.

   

3. After modifying the desired properties, click Save Configuration.
   The alert configuration saves with all the latest modifications.

Delete an Alert Configuration

To delete an alert configuration:

1. Select an alert configuration in the list and select Delete Rule from the Actions menu.

   

   The Delete Rule dialog displays to confirm the delete action.

2. Click Delete.

   

   The alert configuration deletes from the list and a success message displays on the top.

   

Archive a Model

The archiving feature allows you to hide a model from the list to better organize your models. You can view the archived models and unarchive them if required.

Note: The Archiving feature is provided with multiple modules including projects, models, model versions, dataset templates, datasets, and connections.

To archive a model:

1. Select a model in the list.
   The Actions button enables.

   

2. Click the Actions button.
   The Actions menu displays.

   

3. Click Archive Model.
   The model archives and hides from the current List of Models. A confirmation message displays on the top.

   

4. To view an Archived model, click the Show Archived option on the top.
   The archived model displays in the list along with an Archived label.

   

To Un-archive a model:

1. Select an archived model in the list.
   The Actions button enables.

   

2. Click the Actions button.
   The Actions menu displays.

   

3. Click Un-Archive Model.
   The model un-archives and the label Archived removes. A confirmation message displays on the top.