Book a Demo!
CoCalc Logo Icon
StoreFeaturesDocsShareSupportNewsAboutPoliciesSign UpSign In
ibm
GitHub Repository: ibm/watson-machine-learning-samples
 Path: blob/master/cloud/notebooks/python_sdk/deployments/custom_software_spec/Use custom software spec to create statsmodels function.ipynb
9223 views
Kernel: .venv_watsonx_ai_samples_py_312

Use custom software spec to create statsmodels function describing data with ibm-watsonx-ai

This notebook demonstrates how to deploy in watsonx.ai Runtime service a python function with statsmodel which requires to create custom software specification using conda yaml file with all required libraries. Some familiarity with bash is helpful. This notebook uses Python 3.12.

Learning goals

The learning goals of this notebook are:

  • Working with the watsonx.ai Runtime instance

  • Creating custom software specification

  • Online deployment of python function

  • Scoring data using deployed function

Contents

This notebook contains the following parts:

  1. Setup

  2. Function creation

  3. Function upload

  4. Web service creation

  5. Scoring

  6. Clean up

  7. Summary and next steps

1. Set up the environment

Before you use the sample code in this notebook, you must perform the following setup tasks:

Install and import the ibm-watsonx-ai and dependencies

Note: ibm-watsonx-ai documentation can be found here.

%pip install -U ibm-watsonx-ai | tail -n 1
%pip install statsmodels | tail -n 1
Successfully installed anyio-4.12.1 cachetools-6.2.4 certifi-2026.1.4 charset_normalizer-3.4.4 h11-0.16.0 httpcore-1.0.9 httpx-0.28.1 ibm-cos-sdk-2.14.3 ibm-cos-sdk-core-2.14.3 ibm-cos-sdk-s3transfer-2.14.3 ibm-watsonx-ai-1.5.0 idna-3.11 jmespath-1.0.1 lomond-0.3.3 numpy-2.4.1 pandas-2.2.3 pytz-2025.2 requests-2.32.5 tabulate-0.9.0 typing_extensions-4.15.0 tzdata-2025.3 urllib3-2.6.3 Successfully installed patsy-1.0.2 scipy-1.17.0 statsmodels-0.14.6

Connection to watsonx.ai Runtime

Authenticate the watsonx.ai Runtime service on IBM Cloud. You need to provide platform api_key and instance location.

You can use IBM Cloud CLI to retrieve platform API Key and instance location.

API Key can be generated in the following way:

ibmcloud login
ibmcloud iam api-key-create API_KEY_NAME

In result, get the value of api_key from the output.

Location of your watsonx.ai Runtime instance can be retrieved in the following way:

ibmcloud login --apikey API_KEY -a https://cloud.ibm.com
ibmcloud resource service-instance INSTANCE_NAME

In result, get the value of location from the output.

Tip: Your Cloud API key can be generated by going to the Users section of the Cloud console. From that page, click your name, scroll down to the API Keys section, and click Create an IBM Cloud API key. Give your key a name and click Create, then copy the created key and paste it below. You can also get a service specific url by going to the Endpoint URLs section of the watsonx.ai Runtime docs. You can check your instance location in your watsonx.ai Runtime Service instance details.

You can also get service specific apikey by going to the Service IDs section of the Cloud Console. From that page, click Create, then copy the created key and paste it below.

Action: Enter your url and api_key in the following cell.

import getpass

from ibm_watsonx_ai import Credentials

credentials = Credentials(
    url="https://us-south.ml.cloud.ibm.com",
    api_key=getpass.getpass("Enter your watsonx.ai api key and hit enter: "),
)

from ibm_watsonx_ai import APIClient

client = APIClient(credentials)

Working with spaces

First, create a space that will be used for your work. If you do not have space already created, you can use Deployment Spaces Dashboard to create one.

  • Click New Deployment Space

  • Create an empty space

  • Select Cloud Object Storage

  • Select watsonx.ai Runtime instance and press Create

  • Copy space_id and paste it below

Tip: You can also use SDK to prepare the space for your work. More information can be found here.

Action: Assign space ID below

space_id = "PASTE YOUR SPACE ID HERE"

You can use list method to print all existing spaces.

client.spaces.list(limit=10)

To be able to interact with all resources available in watsonx.ai Runtime, you need to set space which you will be using.

client.set.default_space(space_id)
'SUCCESS'

2. Create function

In this section you will learn how to create deployable function with statsmodels module calculating describition of a given data.

Create deploayable callable which uses stsmodels library

def deployable_callable():
    """
    Deployable python function with score
    function implemented.
    """
    try:
        from statsmodels.stats.descriptivestats import describe
    except ModuleNotFoundError as e:
        print(f"statsmodels not installed: {str(e)}")

    def score(payload):
        """
        Score method.
        """
        try:
            data = payload["input_data"][0]["values"]
            return {"predictions": [{"values": str(describe(data))}]}
        except Exception as e:
            return {"predictions": [{"values": [repr(e)]}]}

    return score

Test callable locally

import numpy as np

score_function = deployable_callable()

data = np.random.randn(10, 10)
data_description = score_function({"input_data": [{"values": data}]})

print(data_description["predictions"][0]["values"])
0 1 2 3 4 \ nobs 10.000000 10.000000 10.000000 10.000000 10.000000 missing 0.000000 0.000000 0.000000 0.000000 0.000000 mean -0.142165 -0.071109 -0.071099 0.076253 -0.518539 std_err 0.242244 0.308282 0.346994 0.340890 0.240426 upper_ci 0.332625 0.533114 0.608997 0.744386 -0.047313 lower_ci -0.616955 -0.675331 -0.751196 -0.591879 -0.989765 std 0.766043 0.974874 1.097293 1.077989 0.760294 iqr 0.944193 1.125893 1.290023 1.552004 0.551361 iqr_normal 0.699931 0.834626 0.956296 1.150502 0.408724 mad 0.593256 0.815466 0.846527 0.929699 0.581861 mad_normal 0.743537 1.022035 1.060964 1.165205 0.729255 coef_var -5.388403 -13.709627 -15.433254 14.136924 -1.466223 range 2.196459 3.071821 3.304794 3.212782 2.306719 max 1.107941 1.292705 1.157987 1.741964 0.331775 min -1.088517 -1.779116 -2.146807 -1.470818 -1.974945 skew 0.522918 -0.280316 -0.590590 0.046517 -1.049022 kurtosis 2.066502 2.041076 2.312824 1.693809 2.723417 jarque_bera 0.818830 0.514101 0.778083 0.714496 1.865951 jarque_bera_pval 0.664039 0.773329 0.677706 0.699599 0.393381 mode -1.088517 -1.779116 -2.146807 -1.470818 -1.974945 mode_freq 0.100000 0.100000 0.100000 0.100000 0.100000 median -0.167697 0.044103 -0.134118 -0.052250 -0.190067 1% -1.069107 -1.720200 -2.081429 -1.445076 -1.953344 5% -0.991467 -1.484538 -1.819915 -1.342106 -1.866941 10% -0.894416 -1.189959 -1.493023 -1.213394 -1.758938 25% -0.761740 -0.622794 -0.417795 -0.588030 -0.676190 50% -0.167697 0.044103 -0.134118 -0.052250 -0.190067 75% 0.182453 0.503099 0.872228 0.963974 -0.124829 90% 1.046135 1.024313 1.032291 1.133485 0.050975 95% 1.077038 1.158509 1.095139 1.437724 0.191375 99% 1.101760 1.265866 1.145417 1.681116 0.303695 5 6 7 8 9 nobs 10.000000 10.000000 10.000000 10.000000 10.000000 missing 0.000000 0.000000 0.000000 0.000000 0.000000 mean -0.479405 -0.009196 -0.689914 0.272393 0.296501 std_err 0.342700 0.348680 0.224534 0.298939 0.269834 upper_ci 0.192274 0.674204 -0.249834 0.858304 0.825366 lower_ci -1.151084 -0.692595 -1.129993 -0.313517 -0.232364 std 1.083711 1.102622 0.710040 0.945329 0.853290 iqr 0.814246 1.120643 0.545772 0.893136 0.926532 iqr_normal 0.603602 0.830734 0.404581 0.662083 0.686840 mad 0.842396 0.885369 0.526453 0.711562 0.673928 mad_normal 1.055787 1.109645 0.659811 0.891811 0.844643 coef_var -2.260535 -119.907503 -1.029173 3.470456 2.877870 range 3.521834 3.597965 2.297516 3.271928 2.824689 max 1.483028 1.741251 0.020551 1.576077 1.506523 min -2.038806 -1.856714 -2.276965 -1.695851 -1.318165 skew 0.611241 -0.296957 -1.300669 -0.638095 -0.258680 kurtosis 2.478605 2.256262 3.546913 3.040379 2.501796 jarque_bera 0.735964 0.377450 2.944196 0.679288 0.214945 jarque_bera_pval 0.692130 0.828014 0.229444 0.712024 0.898101 mode -2.038806 -1.856714 -2.276965 -1.695851 -1.318165 mode_freq 0.100000 0.100000 0.100000 0.100000 0.100000 median -0.890204 0.361908 -0.446802 0.487204 0.271910 1% -1.979978 -1.824084 -2.211912 -1.583614 -1.228881 5% -1.744667 -1.693563 -1.951702 -1.134670 -0.871743 10% -1.450529 -1.530412 -1.626440 -0.573489 -0.425321 25% -0.947748 -0.593498 -0.794263 -0.222327 -0.186629 50% -0.890204 0.361908 -0.446802 0.487204 0.271910 75% -0.133502 0.527145 -0.248491 0.670809 0.739904 90% 1.131071 0.998147 -0.152756 1.372305 1.410924 95% 1.307049 1.369699 -0.066102 1.474191 1.458723 99% 1.447833 1.666941 0.003221 1.555700 1.496963

3. Upload python function

In this section you will learn how to upload the python function to the Cloud.

Custom software_specification

Create new software specification based on default Python 3.12 environment extended by statsmodels package.

requirements_txt_content = "statsmodels"

with open("requirements.txt", "w") as file:
    file.write(requirements_txt_content)

base_sw_spec_id = client.software_specifications.get_id_by_name("runtime-25.1-py3.12")

The requirements.txt file describes details of package extension. Now you need to store new package extension using APIClient.

meta_prop_pkg_extn = {
    client.package_extensions.ConfigurationMetaNames.NAME: "statsmodels env",
    client.package_extensions.ConfigurationMetaNames.DESCRIPTION: "Environment with statsmodels",
    client.package_extensions.ConfigurationMetaNames.TYPE: "requirements_txt",
}

pkg_extn_details = client.package_extensions.store(
    meta_props=meta_prop_pkg_extn, file_path="requirements.txt"
)
pkg_extn_id = client.package_extensions.get_id(pkg_extn_details)
pkg_extn_url = client.package_extensions.get_href(pkg_extn_details)
Creating package extension SUCCESS

Create new software specification and add created package extention to it.

meta_prop_sw_spec = {
    client.software_specifications.ConfigurationMetaNames.NAME: "statsmodels software_spec",
    client.software_specifications.ConfigurationMetaNames.DESCRIPTION: "Software specification for statsmodels",
    client.software_specifications.ConfigurationMetaNames.BASE_SOFTWARE_SPECIFICATION: {
        "guid": base_sw_spec_id
    },
}

sw_spec_details = client.software_specifications.store(meta_props=meta_prop_sw_spec)
sw_spec_id = client.software_specifications.get_id(sw_spec_details)

client.software_specifications.add_package_extension(sw_spec_id, pkg_extn_id)
SUCCESS
'SUCCESS'

Get the details of created software specification

import json

print(json.dumps(client.software_specifications.get_details(sw_spec_id), indent=2))

Store the function

meta_props = {
    client.repository.FunctionMetaNames.NAME: "statsmodels function",
    client.repository.FunctionMetaNames.SOFTWARE_SPEC_ID: sw_spec_id,
}

function_details = client.repository.store_function(
    meta_props=meta_props, function=deployable_callable
)
function_id = client.repository.get_function_id(function_details)

Get function details

print(json.dumps(client.repository.get_details(function_id), indent=2))
{ "metadata": { "name": "statsmodels function", "space_id": "fb3d528a-bf16-460e-bcf6-06f05d8ba57c", "resource_key": "3d1f35e1-9750-4dee-8d73-df73908b47c9", "id": "96cfe121-3088-4550-aae9-fe7a53a99b70", "created_at": "2026-01-15T10:24:23Z", "rov": { "member_roles": { "IBMid-696000GJGB": { "user_iam_id": "IBMid-696000GJGB", "roles": [ "OWNER" ] } } }, "owner": "IBMid-696000GJGB" }, "entity": { "software_spec": { "id": "cb1dd3a0-2d7a-42ae-953d-06d24dccdf3c" }, "type": "python" } }

Note: You can see that function is successfully stored in watsonx.ai Runtime Service.

client.repository.list_functions()

4. Create online deployment

You can use commands bellow to create online deployment for stored function (web service).

Create online deployment of a python function

metadata = {
    client.deployments.ConfigurationMetaNames.NAME: "Deployment of statsmodels function",
    client.deployments.ConfigurationMetaNames.ONLINE: {},
}

function_deployment = client.deployments.create(function_id, meta_props=metadata)
###################################################################################### Synchronous deployment creation for id: '96cfe121-3088-4550-aae9-fe7a53a99b70' started ###################################################################################### initializing Note: online_url and serving_urls are deprecated and will be removed in a future release. Use inference instead. ................................ ready ----------------------------------------------------------------------------------------------- Successfully finished deployment creation, deployment_id='496c75d6-6483-4289-956d-36e418b8b937' ----------------------------------------------------------------------------------------------- 
client.deployments.list()

Get deployment id.

deployment_id = client.deployments.get_id(function_deployment)
print(deployment_id)
496c75d6-6483-4289-956d-36e418b8b937

5. Scoring

You can send new scoring records to web-service deployment using score method.

scoring_payload = {"input_data": [{"values": data}]}

predictions = client.deployments.score(deployment_id, scoring_payload)
print(data_description["predictions"][0]["values"])
0 1 2 3 4 \ nobs 10.000000 10.000000 10.000000 10.000000 10.000000 missing 0.000000 0.000000 0.000000 0.000000 0.000000 mean -0.142165 -0.071109 -0.071099 0.076253 -0.518539 std_err 0.242244 0.308282 0.346994 0.340890 0.240426 upper_ci 0.332625 0.533114 0.608997 0.744386 -0.047313 lower_ci -0.616955 -0.675331 -0.751196 -0.591879 -0.989765 std 0.766043 0.974874 1.097293 1.077989 0.760294 iqr 0.944193 1.125893 1.290023 1.552004 0.551361 iqr_normal 0.699931 0.834626 0.956296 1.150502 0.408724 mad 0.593256 0.815466 0.846527 0.929699 0.581861 mad_normal 0.743537 1.022035 1.060964 1.165205 0.729255 coef_var -5.388403 -13.709627 -15.433254 14.136924 -1.466223 range 2.196459 3.071821 3.304794 3.212782 2.306719 max 1.107941 1.292705 1.157987 1.741964 0.331775 min -1.088517 -1.779116 -2.146807 -1.470818 -1.974945 skew 0.522918 -0.280316 -0.590590 0.046517 -1.049022 kurtosis 2.066502 2.041076 2.312824 1.693809 2.723417 jarque_bera 0.818830 0.514101 0.778083 0.714496 1.865951 jarque_bera_pval 0.664039 0.773329 0.677706 0.699599 0.393381 mode -1.088517 -1.779116 -2.146807 -1.470818 -1.974945 mode_freq 0.100000 0.100000 0.100000 0.100000 0.100000 median -0.167697 0.044103 -0.134118 -0.052250 -0.190067 1% -1.069107 -1.720200 -2.081429 -1.445076 -1.953344 5% -0.991467 -1.484538 -1.819915 -1.342106 -1.866941 10% -0.894416 -1.189959 -1.493023 -1.213394 -1.758938 25% -0.761740 -0.622794 -0.417795 -0.588030 -0.676190 50% -0.167697 0.044103 -0.134118 -0.052250 -0.190067 75% 0.182453 0.503099 0.872228 0.963974 -0.124829 90% 1.046135 1.024313 1.032291 1.133485 0.050975 95% 1.077038 1.158509 1.095139 1.437724 0.191375 99% 1.101760 1.265866 1.145417 1.681116 0.303695 5 6 7 8 9 nobs 10.000000 10.000000 10.000000 10.000000 10.000000 missing 0.000000 0.000000 0.000000 0.000000 0.000000 mean -0.479405 -0.009196 -0.689914 0.272393 0.296501 std_err 0.342700 0.348680 0.224534 0.298939 0.269834 upper_ci 0.192274 0.674204 -0.249834 0.858304 0.825366 lower_ci -1.151084 -0.692595 -1.129993 -0.313517 -0.232364 std 1.083711 1.102622 0.710040 0.945329 0.853290 iqr 0.814246 1.120643 0.545772 0.893136 0.926532 iqr_normal 0.603602 0.830734 0.404581 0.662083 0.686840 mad 0.842396 0.885369 0.526453 0.711562 0.673928 mad_normal 1.055787 1.109645 0.659811 0.891811 0.844643 coef_var -2.260535 -119.907503 -1.029173 3.470456 2.877870 range 3.521834 3.597965 2.297516 3.271928 2.824689 max 1.483028 1.741251 0.020551 1.576077 1.506523 min -2.038806 -1.856714 -2.276965 -1.695851 -1.318165 skew 0.611241 -0.296957 -1.300669 -0.638095 -0.258680 kurtosis 2.478605 2.256262 3.546913 3.040379 2.501796 jarque_bera 0.735964 0.377450 2.944196 0.679288 0.214945 jarque_bera_pval 0.692130 0.828014 0.229444 0.712024 0.898101 mode -2.038806 -1.856714 -2.276965 -1.695851 -1.318165 mode_freq 0.100000 0.100000 0.100000 0.100000 0.100000 median -0.890204 0.361908 -0.446802 0.487204 0.271910 1% -1.979978 -1.824084 -2.211912 -1.583614 -1.228881 5% -1.744667 -1.693563 -1.951702 -1.134670 -0.871743 10% -1.450529 -1.530412 -1.626440 -0.573489 -0.425321 25% -0.947748 -0.593498 -0.794263 -0.222327 -0.186629 50% -0.890204 0.361908 -0.446802 0.487204 0.271910 75% -0.133502 0.527145 -0.248491 0.670809 0.739904 90% 1.131071 0.998147 -0.152756 1.372305 1.410924 95% 1.307049 1.369699 -0.066102 1.474191 1.458723 99% 1.447833 1.666941 0.003221 1.555700 1.496963

6. Clean up

If you want to clean up all created assets:

  • experiments

  • trainings

  • pipelines

  • model definitions

  • models

  • functions

  • deployments

see the steps in this sample notebook.

7. Summary and next steps

You successfully completed this notebook! You learned how to use watsonx.ai Runtime for function deployment and scoring with custom software_spec. Check out our Online Documentation for more samples, tutorials, documentation, how-tos, and blog posts.

Authors

Jan Sołtysik, Software Engineer Intern at watsonx.ai.

Rafał Chrzanowski, Software Engineer at watsonx.ai.

Copyright © 2020-2026 IBM. This notebook and its source code are released under the terms of the MIT License.