Book a Demo!
CoCalc Logo Icon
StoreFeaturesDocsShareSupportNewsAboutPoliciesSign UpSign In
ibm
GitHub Repository: ibm/watson-machine-learning-samples
 Path: blob/master/cpd5.2/notebooks/python_sdk/deployments/custom_library/Use custom software spec to create statsmodels function.ipynb
6411 views
Kernel: watsonx-ai-samples-py-312

Use custom software_spec to create statsmodels function describing data with ibm-watsonx-ai

This notebook demonstrates how to deploy in watsonx.ai Runtime service as Python function with statsmodel, which requires creation of custom software specification using requirements.txt file with all required libraries.

Some familiarity with Python is helpful. This notebook uses Python 3.12.

Learning goals

The learning goals of this notebook are:

  • Working with the watsonx.ai instance

  • Creating custom software specification

  • Online deployment of python function

  • Scoring data using deployed function

Contents

This notebook contains the following parts:

  1. Setup

  2. Function creation

  3. Function upload

  4. Web service creation

  5. Scoring

  6. Clean up

  7. Summary and next steps

1. Set up the environment

Before you use the sample code in this notebook, you must perform the following setup tasks:

  • Contact with your Cloud Pak for Data administrator and ask them for your account credentials

Install dependencies

Note: ibm-watsonx-ai documentation can be found here.

%pip install wget | tail -n 1
%pip install -U ibm-watsonx-ai | tail -n 1
%pip install statsmodels | tail -n 1
Successfully installed wget-3.2 Successfully installed anyio-4.9.0 certifi-2025.4.26 charset-normalizer-3.4.2 h11-0.16.0 httpcore-1.0.9 httpx-0.28.1 ibm-cos-sdk-2.14.0 ibm-cos-sdk-core-2.14.0 ibm-cos-sdk-s3transfer-2.14.0 ibm-watsonx-ai-1.3.13 idna-3.10 jmespath-1.0.1 lomond-0.3.3 numpy-2.2.5 pandas-2.2.3 pytz-2025.2 requests-2.32.2 sniffio-1.3.1 tabulate-0.9.0 typing_extensions-4.13.2 tzdata-2025.2 urllib3-2.4.0 Successfully installed patsy-1.0.1 scipy-1.15.2 statsmodels-0.14.4

Define credentials

Authenticate the watsonx.ai Runtime service on IBM Cloud Pak for Data. You need to provide the admin's username and the platform url.

username = "PASTE YOUR USERNAME HERE"
url = "PASTE THE PLATFORM URL HERE"

Use the admin's api_key to authenticate WML services:

import getpass
from ibm_watsonx_ai import Credentials

credentials = Credentials(
    username=username,
    api_key=getpass.getpass("Enter your watsonx.ai API key and hit enter: "),
    url=url,
    instance_id="openshift",
    version="5.2",
)

Alternatively you can use the admin's password:

import getpass
from ibm_watsonx_ai import Credentials

if "credentials" not in locals() or not credentials.api_key:
    credentials = Credentials(
        username=username,
        password=getpass.getpass("Enter your watsonx.ai password and hit enter: "),
        url=url,
        instance_id="openshift",
        version="5.2",
    )

Create APIClient instance

from ibm_watsonx_ai import APIClient

client = APIClient(credentials)

Working with spaces

First of all, you need to create a space that will be used for your work. If you do not have space already created, you can use {PLATFORM_URL}/ml-runtime/spaces?context=icp4data to create one.

  • Click New Deployment Space

  • Create an empty space

  • Go to space Settings tab

  • Copy space_id and paste it below

Tip: You can also use SDK to prepare the space for your work. More information can be found here.

Action: Assign space ID below

space_id = "PASTE YOUR SPACE ID HERE"

You can use list method to print all existing spaces.

client.spaces.list(limit=10)

To be able to interact with all resources available in watsonx.ai, you need to set space which you will be using.

client.set.default_space(space_id)
'SUCCESS'

2. Create function

In this section you will learn how to create deployable function with statsmodels module calculating describition of a given data.

Create deployable callable which uses statsmodels library

def deployable_callable():
    """
    Deployable python function with score
    function implemented.
    """

    try:
        from statsmodels.stats.descriptivestats import describe
    except ModuleNotFoundError as e:
        print(f"statsmodels not installed: {str(e)}")

    def score(payload):
        """
        Score method.
        """
        try:
            data = payload["input_data"][0]["values"]
            return {"predictions": [{"values": str(describe(data))}]}
        except Exception as e:
            return {"predictions": [{"values": [repr(e)]}]}

    return score

Test callable locally

import numpy as np

score_function = deployable_callable()

data = np.random.randn(10, 10)
data_description = score_function({"input_data": [{"values": data}]})

print(data_description["predictions"][0]["values"])
0 1 2 3 4 \ nobs 10.000000 10.000000 10.000000 10.000000 10.000000 missing 0.000000 0.000000 0.000000 0.000000 0.000000 mean 0.109419 -0.396759 0.230355 -0.300086 -0.404532 std_err 0.366493 0.353975 0.292359 0.310328 0.303524 upper_ci 0.827732 0.297020 0.803369 0.308147 0.190363 lower_ci -0.608894 -1.090538 -0.342659 -0.908318 -0.999428 std 1.158952 1.119368 0.924521 0.981344 0.959826 iqr 1.658504 0.502628 0.783272 1.306470 1.403269 iqr_normal 1.229451 0.372599 0.580640 0.968488 1.040245 mad 0.914476 0.643780 0.667696 0.786667 0.797205 mad_normal 1.146126 0.806858 0.836833 0.985941 0.999148 coef_var 10.591883 -2.821279 4.013468 -3.270214 -2.372679 range 3.488891 4.035260 3.012839 3.218347 3.073067 max 2.444000 0.805701 2.083121 1.060559 1.365585 min -1.044890 -3.229559 -0.929718 -2.157787 -1.707481 skew 0.863106 -1.599627 0.876143 -0.527975 0.502819 kurtosis 2.533978 5.397330 2.830348 2.352143 2.165138 jarque_bera 1.332076 6.659342 1.291370 0.639479 0.711792 jarque_bera_pval 0.513740 0.035805 0.524303 0.726338 0.700546 mode -1.044890 -3.229559 -0.929718 -2.157787 -1.707481 mode_freq 0.100000 0.100000 0.100000 0.100000 0.100000 median -0.235313 -0.331667 0.045002 -0.005845 -0.711699 1% -1.037849 -2.997851 -0.894753 -2.074665 -1.664691 5% -1.009683 -2.071020 -0.754893 -1.742177 -1.493531 10% -0.974476 -0.912480 -0.580069 -1.326566 -1.279581 25% -0.755378 -0.489836 -0.388339 -1.032802 -1.038413 50% -0.235313 -0.331667 0.045002 -0.005845 -0.711699 75% 0.903126 0.012792 0.394933 0.273668 0.364856 90% 1.385126 0.805457 1.547382 0.661974 0.605617 95% 1.914563 0.805579 1.815251 0.861267 0.985601 99% 2.338113 0.805676 2.029547 1.020701 1.289589 5 6 7 8 9 nobs 10.000000 10.000000 10.000000 10.000000 10.000000 missing 0.000000 0.000000 0.000000 0.000000 0.000000 mean 0.148414 0.137150 -0.123837 -0.223200 -0.598800 std_err 0.279302 0.271738 0.339934 0.212003 0.273558 upper_ci 0.695836 0.669747 0.542420 0.192318 -0.062636 lower_ci -0.399008 -0.395448 -0.790095 -0.638718 -1.134963 std 0.883231 0.859312 1.074964 0.670412 0.865066 iqr 0.959306 1.166451 1.045893 0.463584 1.196036 iqr_normal 0.711135 0.864691 0.775321 0.343656 0.886623 mad 0.694898 0.746995 0.816197 0.469066 0.706356 mad_normal 0.870925 0.936219 1.022952 0.587887 0.885287 coef_var 5.951126 6.265513 -8.680468 -3.003640 -1.444666 range 2.738575 2.362325 3.659138 2.252933 2.769094 max 1.015273 1.015773 1.390455 0.693056 0.683264 min -1.723302 -1.346552 -2.268684 -1.559877 -2.085830 skew -1.037692 -0.459174 -0.370394 -0.663394 -0.148528 kurtosis 2.954114 1.790141 2.809943 2.841604 2.054778 jarque_bera 1.795553 0.961300 0.243704 0.743939 0.409036 jarque_bera_pval 0.407475 0.618381 0.885280 0.689375 0.815040 mode -1.723302 -1.346552 -2.268684 -1.559877 -2.085830 mode_freq 0.100000 0.100000 0.100000 0.100000 0.100000 median 0.495079 0.284466 -0.362117 -0.139210 -0.597810 1% -1.647391 -1.308704 -2.139648 -1.512734 -2.029969 5% -1.343747 -1.157311 -1.623503 -1.324160 -1.806522 10% -0.964192 -0.968070 -0.978322 -1.088442 -1.527213 25% -0.170491 -0.307490 -0.498677 -0.368516 -1.101316 50% 0.495079 0.284466 -0.362117 -0.139210 -0.597810 75% 0.788815 0.858961 0.547216 0.095068 0.094719 90% 0.836263 0.954100 1.242465 0.530940 0.326458 95% 0.925768 0.984937 1.316460 0.611998 0.504861 99% 0.997372 1.009606 1.375656 0.676844 0.647583

3. Upload python function

In this section you will learn how to upload the Python function to watsonx.ai.

Custom software_specification

Create new software specification based on runtime-25.1-py3.12.

requirements_txt_content = "statsmodels==0.14.4"

with open("requirements.txt", "w") as file:
    file.write(requirements_txt_content)

base_sw_spec_id = client.software_specifications.get_id_by_name("runtime-25.1-py3.12")

The requirements.txt file describes details of package extension. Now you need to store new package extension using APIClient.

meta_prop_pkg_extn = {
    client.package_extensions.ConfigurationMetaNames.NAME: "statsmodels env",
    client.package_extensions.ConfigurationMetaNames.DESCRIPTION: "Environment with statsmodels",
    client.package_extensions.ConfigurationMetaNames.TYPE: "requirements_txt",
}

pkg_extn_details = client.package_extensions.store(
    meta_props=meta_prop_pkg_extn, file_path="requirements.txt"
)
pkg_extn_id = client.package_extensions.get_id(pkg_extn_details)
pkg_extn_url = client.package_extensions.get_href(pkg_extn_details)
Creating package extensions SUCCESS

Create new software specification and add created package extention to it.

meta_prop_sw_spec = {
    client.software_specifications.ConfigurationMetaNames.NAME: "statsmodels software_spec",
    client.software_specifications.ConfigurationMetaNames.DESCRIPTION: "Software specification for statsmodels",
    client.software_specifications.ConfigurationMetaNames.BASE_SOFTWARE_SPECIFICATION: {
        "guid": base_sw_spec_id
    },
}

sw_spec_details = client.software_specifications.store(meta_props=meta_prop_sw_spec)
sw_spec_id = client.software_specifications.get_id(sw_spec_details)

client.software_specifications.add_package_extension(sw_spec_id, pkg_extn_id)
SUCCESS
'SUCCESS'

Get the details of created software specification

client.software_specifications.get_details(sw_spec_id)
{'metadata': {'name': 'statsmodels software_spec', 'asset_id': 'a3445242-780d-4350-8d19-ff5ac6030191', 'href': '/v2/software_specifications/a3445242-780d-4350-8d19-ff5ac6030191', 'asset_type': 'software_specification', 'created_at': '2025-05-07T09:00:52Z', 'life_cycle': {'since_version': '5.2.0'}}, 'entity': {'software_specification': {'type': 'derived', 'display_name': 'statsmodels software_spec', 'package_extensions': [{'metadata': {'space_id': '8a13841b-df99-4b4d-bf2a-161ad2e33980', 'usage': {'last_updated_at': '2025-05-07T09:00:48Z', 'last_updater_id': '1000331001', 'last_update_time': 1746608448593, 'last_accessed_at': '2025-05-07T09:00:48Z', 'last_access_time': 1746608448593, 'last_accessor_id': '1000331001', 'access_count': 0}, 'rov': {'mode': 0, 'collaborator_ids': {}, 'member_roles': {'1000331001': {'user_iam_id': '1000331001', 'roles': ['OWNER']}}}, 'is_linked_with_sub_container': False, 'is_primary_attachment_downloadable': True, 'name': 'statsmodels env', 'description': 'Environment with statsmodels', 'tags': ['dsx'], 'asset_type': 'package_extension', 'origin_country': 'us', 'rating': 0, 'total_ratings': 0, 'catalog_id': 'd396e465-1b0f-4eaf-a6ea-78291dcae56f', 'created': 1746608446840, 'created_at': '2025-05-07T09:00:46Z', 'owner_id': '1000331001', 'size': 19, 'version': 0, 'asset_state': 'available', 'asset_attributes': ['package_extension'], 'asset_id': 'e58d185f-39e4-446e-96af-c3a412085e2c', 'asset_category': 'USER', 'creator_id': '1000331001'}, 'entity': {'package_extension': {'type': 'requirements_txt', 'object_key': 'package_extension/statsmodels_env_dHDcIs5ZM.txt', 'interpreter': 'pip'}}}], 'base_software_specification': {'guid': 'f47ae1c3-198e-5718-b59d-2ea471561e9e', 'href': '/v2/software_specifications/f47ae1c3-198e-5718-b59d-2ea471561e9e'}, 'software_configuration': {'included_packages': [{'name': 'Python', 'version': '3.12'}, {'name': 'Open CE', 'version': '2.0'}], 'platform': {'name': 'python', 'version': '3.12', 'facilities': ['pyvenv']}}}}}

Store the function

meta_props = {
    client.repository.FunctionMetaNames.NAME: "statsmodels function",
    client.repository.FunctionMetaNames.SOFTWARE_SPEC_ID: sw_spec_id,
}

function_details = client.repository.store_function(
    meta_props=meta_props, function=deployable_callable
)
function_id = client.repository.get_function_id(function_details)

Get function details

client.repository.get_details(function_id)
{'metadata': {'name': 'statsmodels function', 'space_id': '8a13841b-df99-4b4d-bf2a-161ad2e33980', 'resource_key': '5d669d9c-4f17-47ff-a708-ca513ad1aaf5', 'id': 'eb782a4f-db22-458b-9d15-9db52b5ff7f4', 'created_at': '2025-05-07T09:17:48Z', 'rov': {'member_roles': {'1000331001': {'user_iam_id': '1000331001', 'roles': ['OWNER']}}}, 'owner': '1000331001'}, 'entity': {'software_spec': {'id': 'a3445242-780d-4350-8d19-ff5ac6030191'}, 'type': 'python'}}

Note: You can see that function is successfully stored in watsonx.ai Runtime service.

client.repository.list_functions()

4. Create online deployment

You can use commands bellow to create online deployment for stored function (web service).

Create online deployment of a python function

metadata = {
    client.deployments.ConfigurationMetaNames.NAME: "Deployment of statsmodels function",
    client.deployments.ConfigurationMetaNames.ONLINE: {},
}

function_deployment = client.deployments.create(function_id, meta_props=metadata)
###################################################################################### Synchronous deployment creation for id: 'eb782a4f-db22-458b-9d15-9db52b5ff7f4' started ###################################################################################### initializing Note: online_url is deprecated and will be removed in a future release. Use serving_urls instead. ..... ready ----------------------------------------------------------------------------------------------- Successfully finished deployment creation, deployment_id='ad172150-7d99-410c-b899-b1357b9b0478' ----------------------------------------------------------------------------------------------- 
client.deployments.list()

Get deployment id.

deployment_id = client.deployments.get_id(function_deployment)
deployment_id
'ad172150-7d99-410c-b899-b1357b9b0478'

5. Scoring

You can send new scoring records to web-service deployment using score method.

scoring_payload = {"input_data": [{"values": data}]}

predictions = client.deployments.score(deployment_id, scoring_payload)
print(data_description["predictions"][0]["values"])
0 1 2 3 4 \ nobs 10.000000 10.000000 10.000000 10.000000 10.000000 missing 0.000000 0.000000 0.000000 0.000000 0.000000 mean 0.109419 -0.396759 0.230355 -0.300086 -0.404532 std_err 0.366493 0.353975 0.292359 0.310328 0.303524 upper_ci 0.827732 0.297020 0.803369 0.308147 0.190363 lower_ci -0.608894 -1.090538 -0.342659 -0.908318 -0.999428 std 1.158952 1.119368 0.924521 0.981344 0.959826 iqr 1.658504 0.502628 0.783272 1.306470 1.403269 iqr_normal 1.229451 0.372599 0.580640 0.968488 1.040245 mad 0.914476 0.643780 0.667696 0.786667 0.797205 mad_normal 1.146126 0.806858 0.836833 0.985941 0.999148 coef_var 10.591883 -2.821279 4.013468 -3.270214 -2.372679 range 3.488891 4.035260 3.012839 3.218347 3.073067 max 2.444000 0.805701 2.083121 1.060559 1.365585 min -1.044890 -3.229559 -0.929718 -2.157787 -1.707481 skew 0.863106 -1.599627 0.876143 -0.527975 0.502819 kurtosis 2.533978 5.397330 2.830348 2.352143 2.165138 jarque_bera 1.332076 6.659342 1.291370 0.639479 0.711792 jarque_bera_pval 0.513740 0.035805 0.524303 0.726338 0.700546 mode -1.044890 -3.229559 -0.929718 -2.157787 -1.707481 mode_freq 0.100000 0.100000 0.100000 0.100000 0.100000 median -0.235313 -0.331667 0.045002 -0.005845 -0.711699 1% -1.037849 -2.997851 -0.894753 -2.074665 -1.664691 5% -1.009683 -2.071020 -0.754893 -1.742177 -1.493531 10% -0.974476 -0.912480 -0.580069 -1.326566 -1.279581 25% -0.755378 -0.489836 -0.388339 -1.032802 -1.038413 50% -0.235313 -0.331667 0.045002 -0.005845 -0.711699 75% 0.903126 0.012792 0.394933 0.273668 0.364856 90% 1.385126 0.805457 1.547382 0.661974 0.605617 95% 1.914563 0.805579 1.815251 0.861267 0.985601 99% 2.338113 0.805676 2.029547 1.020701 1.289589 5 6 7 8 9 nobs 10.000000 10.000000 10.000000 10.000000 10.000000 missing 0.000000 0.000000 0.000000 0.000000 0.000000 mean 0.148414 0.137150 -0.123837 -0.223200 -0.598800 std_err 0.279302 0.271738 0.339934 0.212003 0.273558 upper_ci 0.695836 0.669747 0.542420 0.192318 -0.062636 lower_ci -0.399008 -0.395448 -0.790095 -0.638718 -1.134963 std 0.883231 0.859312 1.074964 0.670412 0.865066 iqr 0.959306 1.166451 1.045893 0.463584 1.196036 iqr_normal 0.711135 0.864691 0.775321 0.343656 0.886623 mad 0.694898 0.746995 0.816197 0.469066 0.706356 mad_normal 0.870925 0.936219 1.022952 0.587887 0.885287 coef_var 5.951126 6.265513 -8.680468 -3.003640 -1.444666 range 2.738575 2.362325 3.659138 2.252933 2.769094 max 1.015273 1.015773 1.390455 0.693056 0.683264 min -1.723302 -1.346552 -2.268684 -1.559877 -2.085830 skew -1.037692 -0.459174 -0.370394 -0.663394 -0.148528 kurtosis 2.954114 1.790141 2.809943 2.841604 2.054778 jarque_bera 1.795553 0.961300 0.243704 0.743939 0.409036 jarque_bera_pval 0.407475 0.618381 0.885280 0.689375 0.815040 mode -1.723302 -1.346552 -2.268684 -1.559877 -2.085830 mode_freq 0.100000 0.100000 0.100000 0.100000 0.100000 median 0.495079 0.284466 -0.362117 -0.139210 -0.597810 1% -1.647391 -1.308704 -2.139648 -1.512734 -2.029969 5% -1.343747 -1.157311 -1.623503 -1.324160 -1.806522 10% -0.964192 -0.968070 -0.978322 -1.088442 -1.527213 25% -0.170491 -0.307490 -0.498677 -0.368516 -1.101316 50% 0.495079 0.284466 -0.362117 -0.139210 -0.597810 75% 0.788815 0.858961 0.547216 0.095068 0.094719 90% 0.836263 0.954100 1.242465 0.530940 0.326458 95% 0.925768 0.984937 1.316460 0.611998 0.504861 99% 0.997372 1.009606 1.375656 0.676844 0.647583

6. Clean up

If you want to clean up all created assets:

  • experiments

  • trainings

  • pipelines

  • model definitions

  • models

  • functions

  • deployments

please follow up this sample notebook.

7. Summary and next steps

You successfully completed this notebook! You learned how to use watsonx.ai for function deployment and scoring with custom software_spec.

Check out our Online Documentation for more samples, tutorials, documentation, how-tos, and blog posts.

Authors

Jan Sołtysik, Software Engineer Intern at watsonx.ai.

Rafał Chrzanowski, Software Engineer Intern at watsonx.ai.

Copyright © 2020-2025 IBM. This notebook and its source code are released under the terms of the MIT License.