Machine Learning artifacts management.ipynb

Kernel: Python 3 (ipykernel)

Machine Learning artifacts management

This notebook contains steps and code to demonstrate how to manage and clean up Watson Machine Learning instance. This notebook contains steps and code to work with ibm-watson-machine-learning library available in PyPI repository. This notebook introduces commands for listing artifacts, getting artifacts details and deleting them.

Some familiarity with Python is helpful. This notebook uses Python 3.10.

Learning goals

The learning goals of this notebook are:

• List Watson Machine Learning artifacts.

• Get artifacts details.

• Delete artifacts.

Contents

This notebook contains the following parts:

1. Setup

2. Manage pipelines

3. Manage model definitions

4. Manage models

5. Manage functions

6. Manage experiments

7. Manage trainings

8. Manage deployments

9. Summary and next steps

1. Set up the environment

Before you use the sample code in this notebook, you must perform the following setup tasks:

• Contact with your Cloud Pack for Data administrator and ask him for your account credentials

Connection to WML

Authenticate the Watson Machine Learning service on IBM Cloud Pack for Data. You need to provide platform url, your username and api_key.

username = 'PASTE YOUR USERNAME HERE'
api_key = 'PASTE YOUR API_KEY HERE'
url = 'PASTE THE PLATFORM URL HERE'

wml_credentials = {
    "username": username,
    "apikey": api_key,
    "url": url,
    "instance_id": 'openshift',
    "version": '4.8'
}

Alternatively you can use username and password to authenticate WML services.

wml_credentials = {
    "username": ***,
    "password": ***,
    "url": ***,
    "instance_id": 'openshift',
    "version": '4.8'
}

Install and import the ibm-watson-machine-learning package

Note: ibm-watson-machine-learning documentation can be found here.

!pip install -U ibm-watson-machine-learning

from ibm_watson_machine_learning import APIClient

client = APIClient(wml_credentials)

Working with spaces

First of all, you need to create a space that will be used for your work. If you do not have space already created, you can use {PLATFORM_URL}/ml-runtime/spaces?context=icp4data to create one.

• Click New Deployment Space

• Create an empty space

• Go to space Settings tab

• Copy space_id and paste it below

Tip: You can also use SDK to prepare the space for your work. More information can be found here.

Action: Assign space ID below

space_id = 'PASTE YOUR SPACE ID HERE'

You can use list method to print all existing spaces.

client.spaces.list(limit=10)

To be able to interact with all resources available in Watson Machine Learning, you need to set space which you will be using.

client.set.default_space(space_id)

2. Manage pipelines

List existing pipelines. If you want to list only part of pipelines use client.pipelines.list(limit=n_pipelines).

client.pipelines.list(limit=10)

Get pipelines details. If you want to get only part of pipelines details use client.pipelines.get_details(limit=n_pipelines).

You can get each pipeline details by calling client.pipelines.get_details() and providing pipeline id from listed pipelines.

pipelines_details = client.pipelines.get_details(limit=10)
print(pipelines_details)

Delete all pipelines. You can delete one pipeline by calling client.pipelines.delete() and providing pipeline id from listed pipelines.

for pipeline in pipelines_details['resources']:
    client.pipelines.delete(pipeline['metadata']['id'])

3. Manage model definitions

List existing model definitions. If you want to list only part of model definitions use client.model_definitions.list(limit=n_model_definitions).

client.model_definitions.list(limit=10)

Get model definiton details by copying model definition uid from above cell and running client.model_definitions.get_details(model_definition_guid).

model_definition_guid = "PUT_YOUR_MODEL_DEFINITION_GUID"
model_definitions_details = client.model_definitions.get_details(model_definition_guid)
print(model_definitions_details)

Delete model definitions by calling client.model_definitions.delete(model_definition_guid).

client.model_definitions.delete(model_definition_guid)

4. Manage models

List existing models. If you want to list only part of models use client.repository.list_models(limit=n_models).

client.repository.list_models(limit=10)

Get model details by copying model uid from above cell and running client.repository.get_details(model_guid).

model_guid = "PUT_YOUR_MODEL_GUID"
model_details = client.repository.get_details(model_guid)
print(model_details)

To download selected model from repository use:

client.repository.download(model_guid, <path_to_model>)
# To obtain serialized model first decompress it
!tar xzvf <path_to_model>

client.repository.download(model_guid)

Instead of downloading model can be also loaded directly to runtime using:

model = client.repository.load(model_guid)
# Loaded model can be used to perform prediction locally
# If loaded model was a scikit-learn pipeline we can use 'predict' method
model.predict(<test_data>)

client.repository.load(model_guid)

Delete model from repository by calling client.repository.delete(model_guid).

client.repository.delete(model_guid)

5. Manage functions

List existing functions. If you want to list only part of functions use client.repository.list_functions(limit=n_functions).

client.repository.list_functions(limit=10)

Get function details by copying function uid from above cell and running client.repository.get_details(function_guid).

function_guid = "PUT_YOUR_FUNCTION_GUID"
function_details = client.repository.get_details(function_guid)
print(function_details)

Delete function from repository by calling client.repository.delete(function_guid).

client.repository.delete(function_guid)

6. Manage experiments

List existing experiments. If you want to list only part of experiments use client.pipelines.list(limit=n_experiments).

client.experiments.list(limit=10)

Get experiments details. If you want to get only part of experiments details use client.experiments.get_details(limit=n_experiments).

You can get each experiment details by calling client.experiments.get_details() and providing experiment id from listed experiments.

experiments_details = client.experiments.get_details()
print(experiments_details)

Delete all experiments. You can delete one experiment by calling client.experiments.delete() and providing experiment id from listed experiments.

for experiment in experiments_details['resources']:
    client.experiments.delete(experiment['metadata']['id'])

7. Manage trainings

List existing trainings. If you want to list only part of trainings use client.training.list(limit=n_trainings).

client.training.list(limit=10)

Get trainings details. If you want to get only part of trainings details use client.training.get_details(limit=n_trainings).

You can get each training details by calling client.training.get_details() and providing training id from listed trainings.

trainings_details = client.training.get_details(limit=10)
print(trainings_details)

Delete all trainings. You can delete one training by calling client.training.cancel() and providing training id from listed trainings.

Note The client.training.cancel() method has hard_delete parameter. Please change it to:

• True - to delete the completed or canceled training runs.

• False - to cancel the currently running training run.

Default value is False.

for training in trainings_details['resources']:
    client.training.cancel(training['metadata']['id'])

8. Manage deployments

List existing deployments. If you want to list only part of deployments use client.deployments.list(limit=n_deployments).

client.deployments.list(limit=10)

Get deployments details. If you want to get only part of deployments details use client.deployments.get_details(limit=n_deployments).

You can get each deployment details by calling client.deployments.get_details() and providing deployment id from listed deployments.

deployments_details = client.deployments.get_details()
print(deployments_details)

Delete all deployments. You can delete one deployment by calling client.deployments.delete() and providing deployment id from listed deployments.

for deployment in deployments_details['resources']:
    client.deployments.delete(deployment['metadata']['id'])

9. Summary and next steps

You successfully completed this notebook! You learned how to use ibm-watson-machine-learning client for Watson Machine Learning instance management and clean up.

Check out our Online Documentation for more samples, tutorials, documentation, how-tos, and blog posts.

Authors

Szymon Kucharczyk, Software Engineer at IBM.

Copyright © 2020-2025 IBM. This notebook and its source code are released under the terms of the MIT License.