Machine Learning artifacts management.ipynb

Kernel: Python 3

Machine Learning artifacts management

This notebook contains steps and code to demonstrate how to manage and clean up watsonx.ai Runtime instance. This notebook contains steps and code to work with ibm-watsonx-ai library available in PyPI repository. This notebook introduces commands for listing artifacts, getting artifacts details and deleting them.

Some familiarity with Python is helpful. This notebook uses Python 3.11.

Learning goals

The learning goals of this notebook are:

• List watsonx.ai Runtime artifacts.

• Get artifacts details.

• Delete artifacts.

Contents

This notebook contains the following parts:

1. Setup

2. Manage pipelines

3. Manage model definitions

4. Manage models

5. Manage functions

6. Manage experiments

7. Manage trainings

8. Manage deployments

9. Summary and next steps

1. Set up the environment

Before you use the sample code in this notebook, you must perform the following setup tasks:

• Create a watsonx.ai Runtime Service instance (a free plan is offered and information about how to create the instance can be found here).

Install and import the ibm-watsonx-ai and dependecies

Note: ibm-watsonx-ai documentation can be found here.

!pip install wget
!pip install -U ibm-watsonx-ai | tail -n 1

Connection to watsonx.ai Runtime

Authenticate the watsonx.ai Runtime service on IBM Cloud. You need to provide platform api_key and instance location.

You can use IBM Cloud CLI to retrieve platform API Key and instance location.

API Key can be generated in the following way:

ibmcloud login
ibmcloud iam api-key-create API_KEY_NAME

In result, get the value of api_key from the output.

Location of your watsonx.ai Runtime instance can be retrieved in the following way:

ibmcloud login --apikey API_KEY -a https://cloud.ibm.com
ibmcloud resource service-instance INSTANCE_NAME

In result, get the value of location from the output.

Tip: Your Cloud API key can be generated by going to the Users section of the Cloud console. From that page, click your name, scroll down to the API Keys section, and click Create an IBM Cloud API key. Give your key a name and click Create, then copy the created key and paste it below. You can also get a service specific url by going to the Endpoint URLs section of the watsonx.ai Runtime docs. You can check your instance location in your watsonx.ai Runtime Service instance details.

You can also get service specific apikey by going to the Service IDs section of the Cloud Console. From that page, click Create, then copy the created key and paste it below.

Action: Enter your api_key and location in the following cell.

api_key = 'PASTE YOUR PLATFORM API KEY HERE'
location = 'PASTE YOUR INSTANCE LOCATION HERE'

from ibm_watsonx_ai import Credentials

credentials = Credentials(
    api_key=api_key,
    url='https://' + location + '.ml.cloud.ibm.com'
)

from ibm_watsonx_ai import APIClient

client = APIClient(credentials)

Working with spaces

First of all, you need to create a space that will be used for your work with watsonx.ai Runtime. If you do not have space already created, you can use Deployment Spaces Dashboard to create one.

• Click New Deployment Space

• Create an empty space

• Select Cloud Object Storage

• Select watsonx.ai Runtime instance and press Create

• Copy space_id and paste it below

Tip: You can also use SDK to prepare the space for your work. More information here.

Action: Assign space ID below

space_id = 'PASTE YOUR SPACE ID HERE'

You can use list method to print all existing spaces.

client.spaces.list(limit=10)

To be able to interact with all resources available in watsonx.ai Runtime, you need to set space which you will be using.

client.set.default_space(space_id)

'SUCCESS'

2. Manage pipelines

List existing pipelines. If you want to list only part of pipelines use client.pipelines.list(limit=n_pipelines).

client.pipelines.list(limit=10)

Get pipelines details. If you want to get only part of pipelines details use client.pipelines.get_details(limit=n_pipelines).

You can get each pipeline details by calling client.pipelines.get_details() and providing pipeline id from listed pipelines.

pipelines_details = client.pipelines.get_details(limit=10)
print(pipelines_details)

Delete all pipelines. You can delete one pipeline by calling client.pipelines.delete() and providing pipeline id from listed pipelines.

for pipeline in pipelines_details['resources']:
    client.pipelines.delete(pipeline['metadata']['id'])

3. Manage model definitions

List existing model definitions. If you want to list only part of model definitions use client.model_definitions.list(limit=n_model_definitions).

client.model_definitions.list(limit=10)

Get model definiton details by copying model definition id from above cell and running client.model_definitions.get_details(model_definition_id).

model_definition_id = "PUT_YOUR_MODEL_DEFINITION_ID"
model_definitions_details = client.model_definitions.get_details(model_definition_id)
print(model_definitions_details)

Delete model definitions by calling client.model_definitions.delete(model_definition_id).

client.model_definitions.delete(model_definition_id)

4. Manage models

List existing models. If you want to list only part of models use client.repository.list_models(limit=n_models).

client.repository.list_models(limit=10)

Get model details by copying model id from above cell and running client.repository.get_details(model_id).

model_id = "PUT_YOUR_MODEL_ID"
model_details = client.repository.get_details(model_id)
print(model_details)

To download selected model from repository use:

client.repository.download(model_id, <path_to_model>)
# To obtain serialized model first decompress it
!tar xzvf <path_to_model>

client.repository.download(model_id)

Instead of downloading model can be also loaded directly to runtime using:

model = client.repository.load(model_id)
# Loaded model can be used to perform prediction locally
# If loaded model was a scikit-learn pipeline we can use 'predict' method
model.predict(<test_data>)

client.repository.load(model_id)

Delete model from repository by calling client.repository.delete(model_id).

client.repository.delete(model_id)

5. Manage functions

List existing functions. If you want to list only part of functions use client.repository.list_functions(limit=n_functions).

client.repository.list_functions(limit=10)

Get function details by copying function id from above cell and running client.repository.get_details(function_id).

function_id = "PUT_YOUR_FUNCTION_ID"
function_details = client.repository.get_details(function_id)
print(function_details)

Delete function from repository by calling client.repository.delete(function_id).

client.repository.delete(function_id)

6. Manage experiments

List existing experiments. If you want to list only part of experiments use client.pipelines.list(limit=n_experiments).

client.experiments.list(limit=10)

Get experiments details. If you want to get only part of experiments details use client.experiments.get_details(limit=n_experiments).

You can get each experiment details by calling client.experiments.get_details() and providing experiment id from listed experiments.

experiments_details = client.experiments.get_details()
print(experiments_details)

Delete all experiments. You can delete one experiment by calling client.experiments.delete() and providing experiment id from listed experiments.

for experiment in experiments_details['resources']:
    client.experiments.delete(experiment['metadata']['id'])

7. Manage trainings

List existing trainings. If you want to list only part of trainings use client.training.list(limit=n_trainings).

client.training.list(limit=10)

Get trainings details. If you want to get only part of trainings details use client.training.get_details(limit=n_trainings).

You can get each training details by calling client.training.get_details() and providing training id from listed trainings.

trainings_details = client.training.get_details(limit=10)
print(trainings_details)

Delete all trainings. You can delete one training by calling client.training.cancel() and providing training id from listed trainings.

Note The client.training.cancel() method has hard_delete parameter. Please change it to:

• True - to delete the completed or canceled training runs.

• False - to cancel the currently running training run.

Default value is False.

for training in trainings_details['resources']:
    client.training.cancel(training['metadata']['id'])

8. Manage deployments

List existing deployments. If you want to list only part of deployments use client.deployments.list(limit=n_deployments).

client.deployments.list(limit=10)

Get deployments details. If you want to get only part of deployments details use client.deployments.get_details(limit=n_deployments).

You can get each deployment details by calling client.deployments.get_details() and providing deployment id from listed deployments.

deployments_details = client.deployments.get_details()
print(deployments_details)

Delete all deployments. You can delete one deployment by calling client.deployments.delete() and providing deployment id from listed deployments.

for deployment in deployments_details['resources']:
    client.deployments.delete(deployment['metadata']['id'])

9. Summary and next steps

You successfully completed this notebook! You learned how to use ibm-watson-machine-learning client for watsonx.ai Runtime instance management and clean up. Check out our Online Documentation for more samples, tutorials, documentation, how-tos, and blog posts.

Authors

Szymon Kucharczyk, Software Engineer at IBM.

Mateusz Szewczyk, Software Engineer at watsonx.ai

Copyright © 2020-2025 IBM. This notebook and its source code are released under the terms of the MIT License.