GitHub Repository: ibm/watson-machine-learning-samples
Path: blob/master/cpd4.8/notebooks/python_sdk/deployments/custom_image/Use Custom Image, Software Specification and Runtime Definition to deploy a python function.ipynb
⁶⁴⁰⁵ views

Kernel: Python 3.10

Use Custom Image, Software Specification and Runtime Definition to deploy a python function with `ibm-watson-machine-learning`

This notebook demonstrates how to deploy in Watson Machine Learning service a python function which requires to create custom software specification and runtime definition. Familiarity with oc and python is helpful. This notebook uses Python 3.10.

Learning goals

The learning goals of this notebook are:

Creating a Custom Image
Creating custom software specification and runtime definition
Online deployment of python function
Scoring data using deployed function

This notebook contains the following parts:

1. Set up the environment

Required Role: You must be a Cloud Pak for Data Cluster Administrator

Credentials

Authenticate the Watson Machine Learning service on IBM Cloud Pack for Data. You need to provide platform url, the admin username and api_key or password

In [ ]:

username = 'PASTE YOUR USERNAME HERE' 
api_key = 'PASTE YOUR API_KEY HERE'
password = 'PASTE YOUR PASSWORD HERE'
url = 'PASTE THE PLATFORM URL HERE'

In [3]:

wml_credentials = {
    "username": username,
    "password": password,
    "url": url,
    "instance_id": 'openshift',
    "version": '4.8'
}

In [4]:

CPDHOST = url

Install and import the `ibm-watson-machine-learning` package

Note: ibm-watson-machine-learning documentation can be found here.

In [ ]:

!pip install -U ibm-watson-machine-learning

In [5]:

from ibm_watson_machine_learning import APIClient

client = APIClient(wml_credentials)

2. Create a Custom Image

The Creation of the custom image involves the following steps, not all of which can be done from a notebook. It involves:-

Downloading the runtime definition and getting the runtime image , This can be done from the notebook as shown later
Pulling, building, pushing docker images which cannot be done from the notebook. It must be done in a machine which have access to docker or podman

See https://www.ibm.com/docs/en/cloud-paks/cp-data/4.8.x?topic=runtimes-customizing-deployment-runtime-images for details.

generating auth tokens

In [6]:

import requests, json
import urllib3
urllib3.disable_warnings(urllib3.exceptions.InsecureRequestWarning)

In [7]:

headers={'Authorization': f'Bearer {client._get_icptoken()}',
         'Content-Type': 'application/json'}
VERSION = '2022-02-01'

Download the runtime definition and getting the runtime image

In this notebook we choose runtime-23.1-py3.10 See https://www.ibm.com/docs/en/cloud-paks/cp-data/4.8.x?topic=image-downloading-runtime-definition for details

In [8]:

runtime_definition_name = "runtime-23.1-py3.10"

In [9]:

import json

response = requests.get(
    f"{url}/v2/runtime_definitions?include=launch_configuration",
    headers=headers,
    verify=False
)

for r in response.json()["resources"]:
    if r["entity"]["name"] != runtime_definition_name:
        continue
    template_runtime_definition = r["entity"]
    print(f"Successfully fetched the Runtime Definition: {runtime_definition_name}")

Out[9]:

Successfully fetched the Runtime Definition: runtime-23.1-py3.10

The required runtime image will be in the key .launch_configuration.image from the runtime definition we just downloaded

In [10]:

print(f"required runtime image:\n\t{template_runtime_definition['launch_configuration']['image']}")

Out[10]:

required runtime image:
	cp.icr.io/cp/cpd/wml-dep-rt23-1-ms@sha256:096bd9906fe78159adc9118b0e8f7be656f3f13ad47f88827581481bbb2f9df5

Pulling, building, pushing docker images

These steps cannot be run from a notebook, and need to be done in a machine which have access to docker or podman. See https://www.ibm.com/docs/en/cloud-paks/cp-data/4.8.x?topic=pbci-downloading-runtime-image for more details.

Download the runtime image for customization

If you have access to the IBM Entitled registry URL, you can download the base image directly from the above, else you need to download from private container registry used at the time of installation.

Pull the base image: Here cp.stg.icr.io/cp/cpd/ is my private container registry used at the time of installation The base image:

BASE_IMAGE="cp.stg.icr.io/cp/cpd/wml-dep-rt23-1-ms@sha256:096bd9906fe78159adc9118b0e8f7be656f3f13ad47f88827581481bbb2f9df5"

registry login and pulling the image:

podman login -u <user> -p <password> cp.stg.icr.io/cp/cpd/
podman pull $BASE_IMAGE

Creating and uploading a custom image

Building the custom image

Based on the CPD release patch version. You will have two different formats for the Dockerfile here as described here https://www.ibm.com/docs/en/cloud-paks/cp-data/4.8.x?topic=image-customizing-python-deployment-images

e.g.


$ podman run --rm $BASE_IMAGE grep -q '^wsbuild:' /etc/passwd && echo "User wsbuild exists" || echo "User wsbuild does not exist"

This prints out User wsbuild exists in this case , so we have to use wsbuild, wsuser user base. We are going to install the program jq as well as the python package pendulum here. This needs to use the conda environment pointed by the variable WML_CONDA_ENV_NAME already available in the base image. microdnf can be used to install system packages

ARG  base_image_tag
FROM ${base_image_tag}

# For installing system packages, use root:root
# e.g.
USER root:root
RUN microdnf install -y jq

# For installing to conda site-packages,
# use wsbuild:wsbuild
# e.g.
USER wsbuild:wsbuild
RUN source activate $WML_CONDA_ENV_NAME && \
   pip install pendulum
USER wsuser:wscommon

If this CPD release is prior to 4.8.3 ; It would have been the case of User wsbuild does not exist , in which case we need to use a different user base wmlbaseuser, wmlfuser . Dockerfile sample for this example is as below. We are going to install the program jq as well as the python package pendulum. This needs to use the base conda environment for the python environment. yum can be used to install system packages

ARG  base_image_tag
FROM ${base_image_tag}
# For installing OS packages, use root:root
# e.g.
USER root:root
RUN yum install -y jq

# For installing to conda site-packages,
# use wmlfuser:condausers
# e.g.
USER wmlfuser:condausers
RUN source activate && \
    pip install pendulum

Build the custom image wml-demo-image:test-1:

podman build -t wml-demo-image:test-1 \
    --build-arg base_image_tag=$BASE_IMAGE \
    -f Dockerfile

Some sanity checks on image for this example. You can run the following, which should print the package version

podman run wml-demo-image:test-1 jq --version

# wsbuild case
podman run wml-demo-image:test-1 bash -c 'source activate $WML_CONDA_ENV_NAME;pip show pendulum'

# wmlbaseuser case
podman run wml-demo-image:test-1 bash -c 'source activate base;pip show pendulum'

Uploading the custom image

Push the custom image to the private container registry

# e.g
podman login -u <user> -p <password> cp.stg.icr.io/cp/cpd/
podman tag wml-demo-image:test-1 cp.stg.icr.io/cp/cpd/wml-demo-image:test-1
podman push cp.stg.icr.io/cp/cpd/wml-demo-image:test-1

Required custom image: cp.stg.icr.io/cp/cpd/wml-demo-image:test-1

3. Create a software specification for the custom image

Creation of software specification require uploading of the file to certain path in the PVC cc-home-pvc. We will use Volume Service APIs to do this.

Create a Volume service instance

namespace here is where cc-home-pvc lives oc get pvc -A | grep cc-home-pvc

In [11]:

# name for this Vol Service instance, you can give any name of choice
display_name='CCHome'

# namespace here is where cc-home-pvc lives `oc get pvc -A | grep cc-home-pvc`
namespace="PASTE THE REQUIRED NAMESPACE HERE"

In [12]:

response = requests.post(
    CPDHOST + '/zen-data/v3/service_instances',
    headers=headers,
    json={
        "addon_type": "volumes",
        "addon_version": "-",
        "create_arguments": {
            "metadata": {
                "existing_pvc_name": "cc-home-pvc"
            }
        },
        "namespace": namespace,
        "display_name": display_name
    },
    verify=False)
if response.status_code == 200:
    print("Successful",response.json())

Out[12]:

Successful {'id': '1721195823485960'}

Check Volume Instances - if it is already created

In [13]:

response = requests.get(CPDHOST + '/zen-data/v3/service_instances',
                        headers=headers,
                        verify=False)
response.status_code
vol_svc_id = None
for svc_inst in response.json()['service_instances']:
    print(svc_inst['display_name'], svc_inst['id'], svc_inst['metadata'].get('existing_pvc_name'))
    if svc_inst['display_name'] == f"{namespace}::{display_name}":
        vol_svc_id = svc_inst['id']

assert vol_svc_id is not None
print('\nvol_svc_id:', vol_svc_id)

Out[13]:

zen::CCHome 1721195823485960 cc-home-pvc

vol_svc_id: 1721195823485960

Start file server on volume

In [14]:

response = requests.post(
    CPDHOST + f'/zen-data/v1/volumes/volume_services/{namespace}::{display_name}',
    headers=headers,
    json={},
    verify=False)
print(json.dumps(response.json(), indent=2))

Out[14]:

{
  "_messageCode_": "200",
  "message": "Successfully started volume service"
}

Create the software specification

Custom Image software specification requires the following:

entity.software_specification.type have to be set to "base" to indicate custom image.
entity.software_specification.built_in have to be set boolean False to indicate custom image.
entity.software_configuration.included_packages array is informational, it can be empty list [] .
entity.software_configuration.platform.name should be set to python .

In [15]:

## The name of the Software Specification. You can give any name of your choice. e.g. here "cspec-demo". 
sw_spec_name='cspec-demo'

## The python version in the image
python_version = "3.10"


sw_spec_filename=f'{sw_spec_name}.json'
print('software spec name     :', sw_spec_name)
print('software spec filename :', sw_spec_filename)


sw_spec = {
  "metadata": {
    "name": sw_spec_name,
    "description": "Test custom image software specification"
  },
  "entity": {
    "software_specification": {
      "type": "base",
      "built_in": False,
      "package_extensions": [],
      "display_name": f"Test {sw_spec_name}",
      "software_configuration": {
        "included_packages": [
          {
            "name": "pendulum",
            "version": "3.0.0"
          }       
        ],
        "platform": {
          "name": "python",
          "version": python_version
        }
      }
    }
  }
}

Out[15]:

software spec name     : cspec-demo
software spec filename : cspec-demo.json

Upload file to the specified path on the volume

In [16]:

target_file=f'%2F_global_%2Fconfig%2Fenvironments%2Fsoftware-specifications%2F{sw_spec_filename}'
target_file

Out[16]:

'%2F_global_%2Fconfig%2Fenvironments%2Fsoftware-specifications%2Fcspec-demo.json'

In [18]:

# multipart/form-data
# Do not set the Content-type header yourself, leave that to pyrequests to generate

response = requests.put(
    CPDHOST + f'/zen-volumes/{display_name}/v1/volumes/files/{target_file}',
    headers={
        "Authorization": f"Bearer {client._get_icptoken()}",
    },
    files={
        'upFile': (sw_spec_filename, json.dumps(sw_spec), 'application/json')
    },
    verify=False)
response.status_code, response.json()

Out[18]:

(200,
 {'_messageCode_': 'Success',
  'message': 'Successfully uploaded file and created the directory structure'})

We can check if the file was successfully upload

In [19]:

response = requests.get(CPDHOST + f'/zen-volumes/{display_name}/v1/volumes/files/{target_file}',
                        headers=headers,
                        verify=False)
if response.status_code == 200:
    print("Upload completed")
else:
    print("Upload error!!", response.status_code, '\n', json.dumps(response.json(), indent=2))

Out[19]:

Upload completed

Wait for about 5 mins

Refresh intervel is handled by env-spec-sync-job cronjob

e.g.
oc get cronjobs env-spec-sync-job

NAME                SCHEDULE      SUSPEND   ACTIVE   LAST SCHEDULE   AGE
env-spec-sync-job   */5 * * * *   False     0        82s             23d

Check if the new Software Specification is activated

In [20]:

response = requests.get(
    CPDHOST + f'/v2/software_specifications?name={sw_spec_name}',
    headers=headers,
    verify=False)

if response.json()["total_results"] != 1:
    print(f"Custom software specification {sw_spec_name} is not activated yet!")
else:
    print(f"Custom software specification {sw_spec_name} is activated")

Out[20]:

Custom software specification cspec-demo is not activated yet!

Warning: Do not proceed unless the new software specification is activated!

The software specification files are processed by the cronjob, and the uploaded will be available at the endpoint "/v2/software_specifications?name={sw_spec_name}" when it gets processed. If it does not show up after 5 mins, recheck that software specification file if it is a valid json and re-upload

In [22]:

response = requests.get(
    CPDHOST + f'/v2/software_specifications?name={sw_spec_name}',
    headers=headers,
    verify=False)

if response.json()["total_results"] != 1:
    print(f"Custom software specification {sw_spec_name} is not activated yet!")
else:
    print(f"Custom software specification {sw_spec_name} is activated")

Out[22]:

Custom software specification cspec-demo is activated

4. Create the new runtime definition file for the custom image

We will create the Runtime definition file, which connects the software specification to the custom image name

In [23]:

# fully qualified name for the Internal Registry image of custom image
custom_image_fqn='cp.stg.icr.io/cp/cpd/wml-demo-image:test-1'
custom_image_fqn

Out[23]:

'cp.stg.icr.io/cp/cpd/wml-demo-image:test-1'

Create a new runtime definition for the custom image

Runtime Definition file for Custom image:

Create runtime definitions via API /v2/runtime_definitions
This runtime definition is the link between the software specification and the image name for custom image
If the custom image had been uploaded to the private container registry, then the value for custom_image_fqn would be cp.stg.icr.io/cp/cpd/wml-demo-image:test-1

We already fetched the Runtime Definition template earlier under the step "Download the runtime definition and getting the runtime image". Update the follow fields in the template

In [24]:

# here we keep the custom image runtime definition name same as the custom software spec name ; as best practise
template_runtime_definition["name"] = sw_spec_name
template_runtime_definition["display_name"] = sw_spec_name
template_runtime_definition["description"] = f"Runtime definition for custom image {sw_spec_name}"
template_runtime_definition["runtime_type"] = "wml"
template_runtime_definition["launch_configuration"]["software_specification_name"] = sw_spec_name
template_runtime_definition["launch_configuration"]["image"] = custom_image_fqn

custom_image_runtime_definition = template_runtime_definition

In [25]:

response = requests.post(
    CPDHOST + f'/v2/runtime_definitions', 
    headers=headers,
    json=custom_image_runtime_definition,
    verify=False)
if response.status_code == 201:
    print(f"Successfully created the Runtime Definition for custom image with software specification {sw_spec_name}")
else:
    print(response.status_code, response.json())

Out[25]:

Successfully created the Runtime Definition for custom image with software specification cspec-demo

In [26]:

print('The required software specification name:', sw_spec_name)
print('\t', custom_image_fqn)

Out[26]:

The required software specification name: cspec-demo
	 cp.stg.icr.io/cp/cpd/wml-demo-image:test-1

We are ready to use this software specification with a deployable asset

You can use any user with Editor role on the space to be able to use the custom software specification created by the cluster admin in the ealier steps

Working with spaces

First of all, you need to create a space that will be used for your work. If you do not have space already created, you can use {PLATFORM_URL}/ml-runtime/spaces?context=icp4data to create one.

Click New Deployment Space
Create an empty space
Go to space Settings tab
Copy space_id and paste it below

Tip: You can also use SDK to prepare the space for your work. More information can be found here.

Action: Assign space ID below

In [ ]:

space_id = 'PASTE YOUR SPACE ID HERE'

You can use list method to print all existing spaces.

In [ ]:

client.spaces.list(limit=10)

To be able to interact with all resources available in Watson Machine Learning, you need to set space which you will be using.

In [29]:

client.set.default_space(space_id)

Out[29]:

'SUCCESS'

5. Create and Deploy a WML `python function` model with the custom image

The demo python function below showcase the package pendulum which was installed for this custom image is now usable by returning the version, and does nothing else useful beyond that.

In [30]:

def score_generator():
        
    # Define scoring function
    def callModel(payload_scoring):
        import pendulum

        v4_scoring_response = {
            'predictions': [{'values': [pendulum.__version__]}]
        }
    
        return v4_scoring_response

    def score(input):

        # Score using the pre-defined model
        prediction = callModel(input);
        return prediction
    
    return score

We need use the software specification that was created earlier for the custom image sw_spec_name which is present in this variable

In [31]:

stored_function_details = client.repository.store_function(
    function=score_generator,
    meta_props={
    client.repository.FunctionMetaNames.NAME: "py-function for custom image test",
    client.repository.FunctionMetaNames.DESCRIPTION: "Test Custom Image",
    client.repository.FunctionMetaNames.SOFTWARE_SPEC_UID:
        client.software_specifications.get_id_by_name(sw_spec_name)
    }
)
function_id=client.repository.get_function_id(stored_function_details)
print(f"The required python function asset id: {function_id}")

Out[31]:

The required python function asset id: 72ce724a-044e-45df-8ad9-281935b2d413

Create an online deployment

In [32]:

meta_props = {
    client.deployments.ConfigurationMetaNames.NAME: "function dep Online custom image test",
    client.deployments.ConfigurationMetaNames.ONLINE: {},
    client.deployments.ConfigurationMetaNames.HARDWARE_SPEC : {
        "id":  client.hardware_specifications.get_id_by_name('S')}
}
deployment_details = client.deployments.create(function_id, meta_props)

Out[32]:

#######################################################################################

Synchronous deployment creation for uid: '72ce724a-044e-45df-8ad9-281935b2d413' started

#######################################################################################


initializing
Note: online_url is deprecated and will be removed in a future release. Use serving_urls instead.
.............................................................
ready


------------------------------------------------------------------------------------------------
Successfully finished deployment creation, deployment_uid='599ef2d8-2df2-4246-83e1-cda2af15ca9c'
------------------------------------------------------------------------------------------------

You can monitor the cluster for pods with `oc get pods -l WML_DEPLOYMENT_ID=`

example:

$ oc get pods -l WML_DEPLOYMENT_ID=2e4afc5b-e321-45a4-95e8-1a0086a61dfc
NAME                                                       READY   STATUS    RESTARTS   AGE
wml-2c074a33-8960-47a0-9489-1bd050bbe901-b9867df6f-9fr9g   2/2     Running   0          64s

$ oc describe pod wml-2c074a33-8960-47a0-9489-1bd050bbe901-b9867df6f-9fr9g | grep Image: | grep wml-demo-image
    Image:          cp.stg.icr.io/cp/cpd/wml-demo-image:test-1

In [33]:

deployment_id=client.deployments.get_uid(deployment_details)
deployment_id

Out[33]:

'599ef2d8-2df2-4246-83e1-cda2af15ca9c'

Predict using created deployment

In [35]:

scoring_payload = {
    'input_data': [
        {'fields': ['dummy', 'dummy_c2'], 'values': [[1, 1]]}
    ]
}
predictions = client.deployments.score(deployment_id, scoring_payload)
predictions

Out[35]:

{'predictions': [{'values': ['3.0.0']}]}

It successfully printed out the pendulum version that is present inside the custom image

6. Clean up

In [36]:

client.deployments.delete(deployment_id)

Out[36]:

'SUCCESS'

Stop File server on the Volume

In [37]:

response = requests.delete(
    CPDHOST + f'/zen-data/v1/volumes/volume_services/{namespace}::{display_name}',
    headers=headers,
    json={},
    verify=False)
print(json.dumps(response.json(), indent=2))

Out[37]:

{
  "_messageCode_": "200",
  "message": "Successfully stopped volume service"
}

Delete the Volume instance

In [38]:

response = requests.delete(CPDHOST + f'/zen-data/v3/service_instances/{vol_svc_id}',
                           headers=headers,
                           verify=False)
print(json.dumps(response.json(), indent=2))

Out[38]:

{
  "id": "1721195823485960"
}

If you want to clean up all created assets:

experiments
trainings
pipelines
model definitions
models
functions
deployments

please follow up this sample notebook.

7. Summary and next steps

You successfully completed this notebook! You learned how to use Watson Machine Learning for function deployment and scoring with custom custom_image, software_spec and runtime definition. Check out our Online Documentation for more samples, tutorials, documentation, how-tos, and blog posts.

References:

Author

Ginbiaksang Naulak Software Engineer at IBM.

In [ ]:

Use Custom Image, Software Specification and Runtime Definition to deploy a python function with `ibm-watson-machine-learning`

Learning goals

Contents

1. Set up the environment

Credentials

Install and import the `ibm-watson-machine-learning` package

2. Create a Custom Image

generating auth tokens

Download the runtime definition and getting the runtime image

Pulling, building, pushing docker images

Download the runtime image for customization

Creating and uploading a custom image

Building the custom image

Uploading the custom image

3. Create a software specification for the custom image

Create a Volume service instance

Check Volume Instances - if it is already created

Start file server on volume

Create the software specification

Upload file to the specified path on the volume

We can check if the file was successfully upload

Wait for about 5 mins

Check if the new Software Specification is activated

Warning: Do not proceed unless the new software specification is activated!

4. Create the new runtime definition file for the custom image

Create a new runtime definition for the custom image

Runtime Definition file for Custom image:

We are ready to use this software specification with a deployable asset

Working with spaces

5. Create and Deploy a WML `python function` model with the custom image

Create an online deployment

Predict using created deployment

6. Clean up

7. Summary and next steps

References:

Author

Product

Resources

Company

Use Custom Image, Software Specification and Runtime Definition to deploy a python function with ibm-watson-machine-learning

Learning goals

Contents

1. Set up the environment

Credentials

Install and import the ibm-watson-machine-learning package

2. Create a Custom Image

generating auth tokens

Download the runtime definition and getting the runtime image

Pulling, building, pushing docker images

Download the runtime image for customization

Creating and uploading a custom image

Building the custom image

Uploading the custom image

3. Create a software specification for the custom image

Create a Volume service instance

Check Volume Instances - if it is already created

Start file server on volume

Create the software specification

Upload file to the specified path on the volume

We can check if the file was successfully upload

Wait for about 5 mins

Check if the new Software Specification is activated

Warning: Do not proceed unless the new software specification is activated!

4. Create the new runtime definition file for the custom image

Create a new runtime definition for the custom image

Runtime Definition file for Custom image:

We are ready to use this software specification with a deployable asset

Working with spaces

5. Create and Deploy a WML python function model with the custom image

Create an online deployment

Predict using created deployment

6. Clean up

7. Summary and next steps

References:

Author

Use Custom Image, Software Specification and Runtime Definition to deploy a python function with `ibm-watson-machine-learning`

Install and import the `ibm-watson-machine-learning` package

5. Create and Deploy a WML `python function` model with the custom image