GitHub Repository: ibm/watson-machine-learning-samples
Path: blob/master/cpd4.0/notebooks/python_sdk/experiments/autoai/Use AutoAI with Watson Studio project.ipynb
⁶³⁸³ views

Kernel: Python 3

Use AutoAI with Watson Studio project `ibm-watson-machine-learning`

This notebook contains the steps and code to demonstrate support of AutoAI experiments in Watson Machine Learning service inside Watson Studio's projects. It introduces commands for data retrieval, training experiments and scoring.

Some familiarity with Python is helpful. This notebook uses Python 3.9.

Learning goals

The learning goals of this notebook are:

Work with Watson Machine Learning experiments to train AutoAI model using Watson Studio project.
Online, Batch deployment and score the trained model trained.

This notebook contains the following parts:

1. Set up the environment

Before you use the sample code in this notebook, you must perform the following setup tasks:

Contact with your Cloud Pack for Data administrator and ask him for your account credentials

Connection to WML

Authenticate the Watson Machine Learning service on IBM Cloud Pack for Data. You need to provide platform url, your username and api_key.

In [ ]:

username = 'PASTE YOUR USERNAME HERE'
api_key = 'PASTE YOUR API_KEY HERE'
url = 'PASTE THE PLATFORM URL HERE'

In [2]:

wml_credentials = {
    "username": username,
    "apikey": api_key,
    "url": url,
    "instance_id": 'openshift',
    "version": '4.0'
}

Alternatively you can use username and password to authenticate WML services.

wml_credentials = {
    "username": ***,
    "password": ***,
    "url": ***,
    "instance_id": 'openshift',
    "version": '4.0'
}

Install and import the `ibm-watson-machine-learning` package

Note: ibm-watson-machine-learning documentation can be found here.

In [ ]:

!pip install -U ibm-watson-machine-learning | tail -n 1

In [2]:

from ibm_watson_machine_learning import APIClient

client = APIClient(wml_credentials)

Working with spaces

First of all, you need to create a space that will be used for your work. If you do not have space already created, you can use {PLATFORM_URL}/ml-runtime/spaces?context=icp4data to create one.

Click New Deployment Space
Create an empty space
Go to space Settings tab
Copy space_id and paste it below

Tip: You can also use SDK to prepare the space for your work. More information can be found here.

Action: Assign space ID below

In [5]:

space_id = 'PASTE YOUR SPACE ID HERE'

You can use the list method to print all existing spaces.

In [ ]:

client.spaces.list(limit=10)

Working with projects

First of all, you need to create a project that will be used for your work. If you do not have space already created follow bellow steps.

Open IBM Cloud Pak main page
Click all projects
Create an empty project
Copy project_id from url and paste it below

Action: Assign project ID below

In [5]:

project_id = 'PASTE YOUR PROJECT ID HERE'

To be able to interact with all resources available in Watson Machine Learning, you need to set the project which you will be using.

In [5]:

client.set.default_project(project_id)

Out[5]:

'SUCCESS'

2. Optimizer definition

Training data connection

Define connection information to training data CSV file. This example uses the German Credit Risk dataset.

The dataset can be downloaded from here.

In [7]:

filename = 'credit_risk_training.csv'

In [8]:

!wget -q https://raw.githubusercontent.com/IBM/watson-machine-learning-samples/master/cpd4.0/data/credit_risk/german_credit_data_biased_training.csv \
    -O credit_risk_training.csv

In [9]:

asset_details = client.data_assets.create('german_credit_data_biased_training', filename)
asset_details

Out[9]:

Creating data asset...
SUCCESS

{'metadata': {'project_id': 'c65187e1-d07b-4867-99c1-68d6e5d179c6',
  'guid': '048eae91-0379-475c-9684-d544ac063dc5',
  'href': '/v2/assets/048eae91-0379-475c-9684-d544ac063dc5?project_id=c65187e1-d07b-4867-99c1-68d6e5d179c6',
  'asset_type': 'data_asset',
  'created_at': '2021-05-05T14:52:45Z',
  'last_updated_at': '2021-05-05T14:52:45Z'},
 'entity': {'data_asset': {'mime_type': 'text/csv'}}}

In [10]:

client.data_assets.get_id(asset_details)

Out[10]:

'048eae91-0379-475c-9684-d544ac063dc5'

In [11]:

from ibm_watson_machine_learning.helpers import DataConnection, AssetLocation


credit_risk_conn = DataConnection(data_asset_id=client.data_assets.get_id(asset_details))

training_data_reference=[credit_risk_conn]

Optimizer configuration

Provide the input information for AutoAI optimizer:

name - experiment name
prediction_type - type of the problem
prediction_column - target column name
scoring - optimization metric

In [14]:

from ibm_watson_machine_learning.experiment import AutoAI

experiment = AutoAI(wml_credentials, project_id)

pipeline_optimizer = experiment.optimizer(
    name='Credit Risk Prediction - AutoAI',
    desc='Sample notebook',
    prediction_type=AutoAI.PredictionType.BINARY,
    prediction_column='Risk',
    scoring=AutoAI.Metrics.ROC_AUC_SCORE,
)

Configuration parameters can be retrieved via get_params().

In [28]:

pipeline_optimizer.get_params()

Out[28]:

{'name': 'Credit Risk Prediction - AutoAI',
 'desc': 'Sample notebook',
 'prediction_type': 'binary',
 'prediction_column': 'Risk',
 'prediction_columns': None,
 'timestamp_column_name': None,
 'scoring': 'roc_auc',
 'holdout_size': 0.1,
 'max_num_daub_ensembles': 2,
 't_shirt_size': 'm',
 'train_sample_rows_test_size': None,
 'include_only_estimators': None,
 'backtest_num': None,
 'lookback_window': None,
 'forecast_window': None,
 'backtest_gap_length': None,
 'cognito_transform_names': None,
 'data_join_graph': False,
 'csv_separator': ',',
 'excel_sheet': 0,
 'encoding': 'utf-8',
 'positive_label': None,
 'drop_duplicates': True,
 'text_processing': None,
 'word2vec_feature_number': None,
 'run_id': None}

3. Experiment run

Call the fit() method to trigger the AutoAI experiment. You can either use interactive mode (synchronous job) or background mode (asychronous job) by specifying background_model=True.

In [15]:

run_details = pipeline_optimizer.fit(
            training_data_reference=training_data_reference,
            background_mode=False)

You can use the get_run_status() method to monitor AutoAI jobs in background mode.

In [30]:

pipeline_optimizer.get_run_status()

Out[30]:

'completed'

3.1 Pipelines comparison

You can list trained pipelines and evaluation metrics information in the form of a Pandas DataFrame by calling the summary() method. You can use the DataFrame to compare all discovered pipelines and select the one you like for further testing.

In [31]:

summary = pipeline_optimizer.summary()
summary

Out[31]:

You can visualize the scoring metric calculated on a holdout data set.

In [ ]:

import pandas as pd
pd.options.plotting.backend = "plotly"

summary.holdout_roc_auc.plot()

4. Deploy and Score

In this section you will learn how to deploy and score trained model using project in a specified deployment space as a webservice and batch using WML instance.

Webservice deployment creation

In [43]:

from ibm_watson_machine_learning.deployment import WebService

service = WebService(
    source_wml_credentials=wml_credentials,
    source_project_id=project_id,
    target_wml_credentials=wml_credentials,
    target_space_id=space_id
)

service.create(
    experiment_run_id=run_details['metadata']['id'],
    model='Pipeline_1', 
    deployment_name="Credit Risk Deployment AutoAI")

Out[43]:

Preparing an AutoAI Deployment...
Published model uid: 4d648b47-e1f6-443c-b061-7cf3a43bf196
Deploying model 4d648b47-e1f6-443c-b061-7cf3a43bf196 using V4 client.

#######################################################################################

Synchronous deployment creation for uid: '4d648b47-e1f6-443c-b061-7cf3a43bf196' started

#######################################################################################

initializing.
ready

------------------------------------------------------------------------------------------------
Successfully finished deployment creation, deployment_uid='e46839f6-b6b8-406b-8b47-d2aa4974a2b6'
------------------------------------------------------------------------------------------------

Deployment object could be printed to show basic information:

In [26]:

print(service)

Out[26]:

name: Credit Risk Deployment AutoAI, id: f3649f8d-0076-4bde-a759-08f3e98a3204, scoring_url: https://cpd-wmlautoai-apr27.apps.ocp46wmlautoai2.cp.fyre.ibm.com/ml/v4/deployments/f3649f8d-0076-4bde-a759-08f3e98a3204/predictions, asset_id: a672f3a2-87c1-4413-90c9-3ea75202ab86

To show all available information about the deployment use the .get_params() method:

In [27]:

service.get_params()

Out[27]:

{'entity': {'asset': {'id': 'a672f3a2-87c1-4413-90c9-3ea75202ab86'},
  'custom': {},
  'deployed_asset_type': 'model',
  'hardware_spec': {'id': 'c076e82c-b2a7-4d20-9c0f-1f0c2fdf5a24',
   'name': 'M',
   'num_nodes': 1},
  'hybrid_pipeline_hardware_specs': [{'hardware_spec': {'name': 'S',
     'num_nodes': 1},
    'node_runtime_id': 'auto_ai.kb'}],
  'name': 'Credit Risk Deployment AutoAI',
  'online': {},
  'space_id': 'f91684fb-6a8e-4396-b77c-313bf3adb914',
  'status': {'online_url': {'url': 'https://cpd-wmlautoai-apr27.apps.ocp46wmlautoai2.cp.fyre.ibm.com/ml/v4/deployments/f3649f8d-0076-4bde-a759-08f3e98a3204/predictions'},
   'state': 'ready'}},
 'metadata': {'created_at': '2021-05-05T12:24:54.268Z',
  'id': 'f3649f8d-0076-4bde-a759-08f3e98a3204',
  'modified_at': '2021-05-05T12:24:54.268Z',
  'name': 'Credit Risk Deployment AutoAI',
  'owner': '1000330999',
  'space_id': 'f91684fb-6a8e-4396-b77c-313bf3adb914'}}

Scoring of webservice

You can make scoring request by calling score() on deployed pipeline.

In [16]:

pipeline_optimizer.get_data_connections()[0]

Out[16]:

{'type': 'data_asset', 'connection': {}, 'location': {'href': '/v2/assets/048eae91-0379-475c-9684-d544ac063dc5?project_id=c65187e1-d07b-4867-99c1-68d6e5d179c6', 'id': '048eae91-0379-475c-9684-d544ac063dc5'}}

In [17]:

train_df = pipeline_optimizer.get_data_connections()[0].read()

train_X = train_df.drop(['Risk'], axis=1)
train_y = train_df.Risk.values

In [ ]:

predictions = service.score(payload=train_X.iloc[:10])
predictions

If you want to work with the web service in an external Python application you can retrieve the service object by:

Initialize the service by service = WebService(wml_credentials)
Get deployment_id by service.list() method
Get webservice object by service.get('deployment_id') method

After that you can call service.score() method.

Deleting deployment

You can delete the existing deployment by calling the service.delete() command. To list the existing web services you can use service.list().

Batch deployment creation

A batch deployment processes input data from a inline data and return predictions in scoring details or processes from data asset and writes the output to a file.

In [44]:

batch_payload_df = train_df.drop(['Risk'], axis=1)[:5]
batch_payload_df

Out[44]:

Create batch deployment for Pipeline_2 created in AutoAI experiment with the run_id.

In [45]:

from ibm_watson_machine_learning.deployment import Batch

service_batch = Batch(
    source_wml_credentials=wml_credentials,
    source_project_id=project_id,
    target_wml_credentials=wml_credentials,
    target_space_id=space_id
)
service_batch.create(
            experiment_run_id=run_details['metadata']['id'],
            model="Pipeline_2",
            deployment_name="Credit Risk Batch Deployment AutoAI")

Out[45]:

Preparing an AutoAI Deployment...
Published model uid: 35a98cac-c3b7-448e-9a02-245400c93a82
Deploying model 35a98cac-c3b7-448e-9a02-245400c93a82 using V4 client.

#######################################################################################

Synchronous deployment creation for uid: '35a98cac-c3b7-448e-9a02-245400c93a82' started

#######################################################################################

ready.

------------------------------------------------------------------------------------------------
Successfully finished deployment creation, deployment_uid='756558ad-78f1-4d74-9e6d-555aa9cfff21'
------------------------------------------------------------------------------------------------

Score batch deployment with inline payload as pandas DataFrame.

In [46]:

scoring_params = service_batch.run_job(
            payload=batch_payload_df,
            background_mode=False)

Out[46]:

##########################################################################

Synchronous scoring for id: '478b6b5b-a80f-43f5-98f9-63e9da889720' started

##########################################################################


queued..
running
completed
Scoring job  '478b6b5b-a80f-43f5-98f9-63e9da889720' finished successfully.

In [47]:

scoring_params['entity']['scoring'].get('predictions')

Out[47]:

[{'fields': ['prediction', 'probability'],
  'values': [['No Risk', [0.7743273377418518, 0.225672647356987]],
   ['No Risk', [0.8309863209724426, 0.16901369392871857]],
   ['No Risk', [0.8892747759819031, 0.11072523891925812]],
   ['No Risk', [0.7456686496734619, 0.2543313801288605]],
   ['Risk', [0.22297507524490356, 0.7770249247550964]]]}]

Deleting deployment

You can delete the existing deployment by calling the service_batch.delete() command. To list the existing:

batch services you can use service_batch.list(),
scoring jobs you can use service_batch.list_jobs().

5. Clean up

If you want to clean up all created assets:

experiments
trainings
pipelines
model definitions
models
functions
deployments

please follow up this sample notebook.

6. Summary and next steps

You successfully completed this notebook!.

You learned how to use ibm-watson-machine-learning to run AutoAI experiments using project.

Check out our Online Documentation for more samples, tutorials, documentation, how-tos, and blog posts.

Authors

Jan Sołtysik, Intern in Watson Machine Learning

Use AutoAI with Watson Studio project `ibm-watson-machine-learning`

Learning goals

Contents

1. Set up the environment

Connection to WML

Install and import the `ibm-watson-machine-learning` package

Working with spaces

Working with projects

2. Optimizer definition

Training data connection

Optimizer configuration

3. Experiment run

3.1 Pipelines comparison

4. Deploy and Score

Webservice deployment creation

Scoring of webservice

Deleting deployment

Batch deployment creation

Score batch deployment with inline payload as pandas DataFrame.

Deleting deployment

5. Clean up

6. Summary and next steps

Authors

Product

Resources

Company

Use AutoAI with Watson Studio project ibm-watson-machine-learning

Learning goals

Contents

1. Set up the environment

Connection to WML

Install and import the ibm-watson-machine-learning package

Working with spaces

Working with projects

2. Optimizer definition

Training data connection

Optimizer configuration

3. Experiment run

3.1 Pipelines comparison

4. Deploy and Score

Webservice deployment creation

Scoring of webservice

Deleting deployment

Batch deployment creation

Score batch deployment with inline payload as pandas DataFrame.

Deleting deployment

5. Clean up

6. Summary and next steps

Authors

Use AutoAI with Watson Studio project `ibm-watson-machine-learning`

Install and import the `ibm-watson-machine-learning` package