Use AutoAI RAG and Milvus to create a pattern about IBM.ipynb

Kernel: watsonx-ai-samples-py-312

Use AutoAI RAG and Milvus database to work with ibm-watsonx-ai SDK documentation.

Disclaimers

• Use only Spaces that are available in the watsonx context.

Notebook content

This notebook contains the steps and code to demonstrate the usage of IBM AutoAI RAG. The AutoAI RAG experiment conducted in this notebook uses data scraped from the ibm-watsonx-ai SDK documentation.

Some familiarity with Python is helpful. This notebook uses Python 3.12.

Learning goal

The learning goals of this notebook are:

• Create an AutoAI RAG job that will find the best RAG pattern based on provided data

Contents

This notebook contains the following parts:

• Setup

• Define the RAG Optimizer

• Run the RAG Experiment

• Compare and test RAG Patterns

• Historical runs

• Clean up

• Summary and next steps

Set up the environment

Before you use the sample code in this notebook, you must perform the following setup task:

• Contact your Cloud Pak for Data administrator and ask them for your account credentials

Install dependencies

Note: ibm-watsonx-ai documentation can be found here.

%pip install -U wget | tail -n 1
%pip install -U "ibm-watsonx-ai[rag]>=1.2.4" | tail -n 1

Define credentials

Authenticate the watsonx.ai Runtime service on IBM Cloud Pak for Data. You need to provide the admin's username and the platform url.

username = "PASTE YOUR USERNAME HERE"
url = "PASTE THE PLATFORM URL HERE"

Use the admin's api_key to authenticate watsonx.ai Runtime services:

import getpass
from ibm_watsonx_ai import Credentials

credentials = Credentials(
    username=username,
    api_key=getpass.getpass("Enter your watsonx.ai API key and hit enter: "),
    url=url,
    instance_id="openshift",
    version="5.2",
)

Alternatively you can use the admin's password:

import getpass
from ibm_watsonx_ai import Credentials

if "credentials" not in locals() or not credentials.api_key:
    credentials = Credentials(
        username=username,
        password=getpass.getpass("Enter your watsonx.ai password and hit enter: "),
        url=url,
        instance_id="openshift",
        version="5.2",
    )

Create APIClient instance

from ibm_watsonx_ai import APIClient

client = APIClient(credentials)

Working with spaces

First, you need to create a space for your work. If you do not have a space already created, you can use {PLATFORM_URL}/ml-runtime/spaces?context=icp4data to create one.

• Click New Deployment Space

• Create an empty space

• Go to the space Settings tab

• Copy the space_id and paste it below

Tip: You can also use SDK to prepare the space for your work. Find more information in the Space Management sample notebook.

Action: Assign the space ID below

space_id = "PASTE YOUR SPACE ID HERE"

To print all existing spaces, use the list method.

client.spaces.list(limit=10)

To be able to interact with all resources available in watsonx.ai, you need to set the space which you will be using.

client.set.default_space(space_id)

'SUCCESS'

RAG Optimizer definition

Define a connection to the training data

Define connection information to access the COS bucket and the file that contains the training data. This example uses ibm_watsonx_ai SDK documentation content.

The following code cell downloads the ibm_watsonx_ai Python SDK compressed file from GitHub (if not already downloaded), and extracts its contents to a specified folder.

import wget, zipfile, os

archive_name = "watsonx-ai-python-sdk"
archive_zip = "watsonx-ai-python-sdk.zip"

if not os.path.isfile(archive_zip):
    wget.download(
        "https://github.com/IBM/watsonx-ai-python-sdk/archive/refs/heads/gh-pages.zip",
        out=archive_zip,
    )

with zipfile.ZipFile(archive_zip, "r") as zip_ref:
    zip_ref.extractall(archive_name)

Create a connection to COS.

datasource_name = "bluemixcloudobjectstorage"

# Provide COS credentials
bucket_name = "PASTE YOUR BUCKET NAME HERE"
access_key = "PASTE YOUR ACCESS KEY HERE"
secret_key = "PASTE YOUR SECRET KEY HERE"
url = "PASTE YOUR URL HERE"

conn_meta_props = {
    client.connections.ConfigurationMetaNames.NAME: f"Connection to Database - {datasource_name} ",
    client.connections.ConfigurationMetaNames.DATASOURCE_TYPE: client.connections.get_datasource_type_id_by_name(
        datasource_name
    ),
    client.connections.ConfigurationMetaNames.DESCRIPTION: "Connection to external Database",
    client.connections.ConfigurationMetaNames.PROPERTIES: {
        "bucket": bucket_name,
        "access_key": access_key,
        "secret_key": secret_key,
        "iam_url": "https://iam.cloud.ibm.com/identity/token",
        "url": url,
    },
}

conn_details = client.connections.create(meta_props=conn_meta_props)
connection_id = client.connections.get_id(conn_details)

Creating connections...
SUCCESS

Create a Data Connection that represents input data references.

from ibm_watsonx_ai.helpers import DataConnection, S3Location

data_connection = DataConnection(
    connection_asset_id=connection_id,
    location=S3Location(bucket=bucket_name, path=archive_name),
)
input_data_references = [data_connection]
input_data_references[0].set_client(client)

Filter documents with the .html extension and save them to the COS bucket.

html_docs_files = []
for root, dirs, files in os.walk(archive_name):
    if root != f"{archive_name}/watsonx-ai-python-sdk-gh-pages":
        continue

    for file in filter(lambda x: x.endswith(".html"), files):
        file_path = os.path.join(root, file)
        html_docs_files.append(file_path)

Writing all SDK documents might take around 3 minutes.

for i, html_docs_file in enumerate(html_docs_files):
    data_connection.write(html_docs_file, remote_name=html_docs_file.split("/")[-1])
    print(
        f"Progress: {'✓' * (i+1)}{'.' * (len(html_docs_files)-i-1)}",
        end="\r",
        flush=True,
    )

Define a connection to the test data

Upload a json file that you want to use as a benchmark to COS and then define a connection to the file. This example uses content from the ibm_watsonx_ai SDK documentation.

benchmarking_data_IBM_page_content = [
    {
        "question": "How to install ibm-watsonx-ai library?",
        "correct_answer": "pip install ibm-watsonx-ai",
        "correct_answer_document_ids": ["install.html"],
    },
    {
        "question": "What is Credentials class parameters?",
        "correct_answer": "url, api_key, name, iam_serviceid_crn, token, projects_token, username, password, instance_id, version, bedrock_url, proxies, verify",
        "correct_answer_document_ids": ["base.html"],
    },
    {
        "question": "How to get AutoAI pipeline with number 3?",
        "correct_answer": "get_pipeline(pipeline_name='Pipeline_3')",
        "correct_answer_document_ids": ["autoai_working_with_class_and_optimizer.html"],
    },
    {
        "question": "How to get list of Embedding Models?",
        "correct_answer": "client.foundation_models.EmbeddingModels",
        "correct_answer_document_ids": ["fm_embeddings.html"],
    },
    {
        "question": "How to retrieve the list of model lifecycle data?",
        "correct_answer": "get_model_lifecycle(url='https://us-south.ml.cloud.ibm.com', model_id='ibm/granite-13b-instruct-v2')",
        "correct_answer_document_ids": ["fm_helpers.html"],
    },
    {
        "question": "What is path to ModelInference class?",
        "correct_answer": "ibm_watsonx_ai.foundation_models.inference.ModelInference",
        "correct_answer_document_ids": ["fm_model_inference.html"],
    },
    {
        "question": "What is method for get model inference details?",
        "correct_answer": "get_details()",
        "correct_answer_document_ids": ["fm_model_inference.html"],
    },
]

Upload the benchmark testing data to the bucket as a json file.

import json

test_filename = "benchmarking_data_ibm_watson_ai.json"

if not os.path.isfile(test_filename):
    with open(test_filename, "w") as json_file:
        json.dump(benchmarking_data_IBM_page_content, json_file, indent=4)

test_asset_details = client.data_assets.create(
    name=test_filename, file_path=test_filename
)

test_asset_id = client.data_assets.get_id(test_asset_details)
test_asset_id

Creating data asset...
SUCCESS

'43bcc9c3-04d6-4fc6-870e-fa997a179741'

Define connection information to the testing data.

from ibm_watsonx_ai.helpers import DataConnection

test_data_references = [DataConnection(data_asset_id=test_asset_id)]

Set up connectivity information to Milvus

This notebook focuses on a self-managed Milvus cluster using IBM watsonx.data.

The following cell retrieves the Milvus username, password, host, and port from the environment (if available) and prompts you to provide them manually in case of failure.

You can provide a connection asset ID to read all required connection data from it. Before doing so, make sure that a connection asset was created in your space.

import os
import getpass

milvus_connection_id = input(
    "Provide connection asset ID in your space. Skip this, if you wish to type credentials by hand and hit enter: "
)

if not milvus_connection_id:
    try:
        username = os.environ["USERNAME"]
    except KeyError:
        username = input("Please enter your Milvus user name and hit enter: ")

    try:
        password = os.environ["PASSWORD"]
    except KeyError:
        password = getpass.getpass("Please enter your Milvus password and hit enter: ")

    try:
        host = os.environ["HOST"]
    except KeyError:
        host = input("Please enter your Milvus hostname and hit enter: ")

    try:
        port = os.environ["PORT"]
    except KeyError:
        port = input("Please enter your Milvus port number and hit enter: ")

    try:
        ssl = os.environ["SSL"]
    except:
        ssl = bool(
            input(
                "Please enter ('y'/anything) if your Milvus instance has SSL enabled. Skip if it is not: "
            )
        )

    # Create connection
    milvus_data_source_type_id = client.connections.get_datasource_type_uid_by_name(
        "milvus"
    )
    details = client.connections.create(
        {
            client.connections.ConfigurationMetaNames.NAME: "Milvus Connection",
            client.connections.ConfigurationMetaNames.DESCRIPTION: "Connection created by the sample notebook",
            client.connections.ConfigurationMetaNames.DATASOURCE_TYPE: milvus_data_source_type_id,
            client.connections.ConfigurationMetaNames.PROPERTIES: {
                "host": host,
                "port": port,
                "username": username,
                "password": password,
                "ssl": ssl,
            },
        }
    )

    milvus_connection_id = client.connections.get_id(details)

Define connection information to vector store references.

vector_store_references = [DataConnection(connection_asset_id=milvus_connection_id)]

RAG Optimizer configuration

Provide the input information for the AutoAI RAG optimizer:

• name - experiment name

• description - experiment description

• max_number_of_rag_patterns - maximum number of RAG patterns to create

• optimization_metrics - target optimization metrics

from ibm_watsonx_ai.experiment import AutoAI

experiment = AutoAI(credentials, space_id=space_id)

rag_optimizer = experiment.rag_optimizer(
    name="AutoAI RAG ibm-watsonx-ai SDK documentation",
    description="AutoAI RAG experiemnt trainded on ibm-watsonx-ai SDK documentataion",
    max_number_of_rag_patterns=6,
    optimization_metrics=[AutoAI.RAGMetrics.ANSWER_CORRECTNESS],
)

To retrieve the configuration parameters, use get_params().

rag_optimizer.get_params()

{'name': 'AutoAI RAG ibm-watsonx-ai SDK documentation',
 'description': 'AutoAI RAG experiemnt trainded on ibm-watsonx-ai SDK documentataion',
 'max_number_of_rag_patterns': 6,
 'optimization_metrics': ['answer_correctness']}

Run the RAG Experiment

Call the run() method to trigger the AutoAI RAG experiment. Choose one of two modes:

• To use the interactive mode (synchronous job), specify background_mode=False

• To use the background mode (asynchronous job), specify background_mode=True

run_details = rag_optimizer.run(
    input_data_references=input_data_references,
    test_data_references=test_data_references,
    vector_store_references=vector_store_references,
    background_mode=False,
)

##############################################

Running '38d8ffb0-9d93-4956-9a50-aaef8068e299'

##############################################


pending..............
running.............................................................................................................................................................................................................
completed
Training of '38d8ffb0-9d93-4956-9a50-aaef8068e299' finished successfully.

To monitor the AutoAI RAG jobs in background mode, use the get_run_status() method.

rag_optimizer.get_run_status()

'completed'

Compare and test of RAG Patterns

You can list the trained patterns and information on evaluation metrics in the form of a Pandas DataFrame by calling the summary() method. Use the DataFrame to compare all discovered patterns and select the one you want for further testing.

summary = rag_optimizer.summary()
summary

Additionally, you can pass the scoring parameter to the summary method to filter RAG patterns, starting with the best.

summary = rag_optimizer.summary(scoring="faithfulness")

Get the selected pattern

Get the RAGPattern object from the RAG Optimizer experiment. By default, the RAGPattern of the best pattern is returned.

best_pattern_name = summary.index.values[0]
print("Best pattern is:", best_pattern_name)

best_pattern = rag_optimizer.get_pattern()

Best pattern is: Pattern4

To retrieve the pattern details, use the get_pattern_details method.

rag_optimizer.get_pattern_details(pattern_name="Pattern2")

Query the RAGPattern locally to test it.

questions = ["How to use new approach of providing credentials to APIClient?"]

payload = {
    client.deployments.ScoringMetaNames.INPUT_DATA: [
        {"values": questions, "access_token": client.token}
    ]
}

resp = best_pattern.inference_function()(payload)

print(resp["predictions"][0]["values"][0][0])

According to the provided document, the new approach of providing credentials to APIClient is through the Credentials class. You can create a Credentials object using various methods, such as:

* Using an API key: `credentials = Credentials(url="https://us-south.ml.cloud.ibm.com", api_key="***********")`
* Using a token: `credentials = Credentials(url="https://us-south.ml.cloud.ibm.com", token="***********")`
* Using username and password: `credentials = Credentials(url="<URL>", username="<USERNAME>", password="<PASSWORD>")`
* Using username and API key: `credentials = Credentials(url="<URL>", username="<USERNAME>", api_key="<API_KEY>")`

Once you have created a Credentials object, you can pass it to the APIClient constructor: `client = APIClient(credentials=credentials)`.

Deploy the RAGPattern

To deploy the RAGPattern, store the defined RAG function and then create a deployed asset.

deployment_details = best_pattern.deploy(
    name="AutoAI RAG deployment - ibm_watsonx_ai documentataion", space_id=space_id
)

######################################################################################

Synchronous deployment creation for id: 'a2e3e943-efc8-4f24-b574-6ca25c168941' started

######################################################################################


initializing
Note: online_url is deprecated and will be removed in a future release. Use serving_urls instead.
.............
ready


-----------------------------------------------------------------------------------------------
Successfully finished deployment creation, deployment_id='6d1e9447-85fa-4a9c-8703-180565c3e83b'
-----------------------------------------------------------------------------------------------

Test the deployed function

The RAG service is now deployed in our space. To test the solution, run the cell below. Questions have to be provided in the payload. Their format is provided below.

deployment_id = client.deployments.get_id(deployment_details)

score_response = client.deployments.score(deployment_id, payload)
score_response

{'predictions': [{'fields': ['answer', 'reference_documents'],
   'values': [['\n\nAccording to the provided document, the new approach of providing credentials to APIClient is through the Credentials class. You can create a Credentials object using various methods, such as:\n\n* Using an API key: `credentials = Credentials(url="https://us-south.ml.cloud.ibm.com", api_key="***********")`\n* Using a token: `credentials = Credentials(url="https://us-south.ml.cloud.ibm.com", token="***********")`\n* Using username and password: `credentials = Credentials(url="<URL>", username="<USERNAME>", password="<PASSWORD>")`\n* Using username and API key: `credentials = Credentials(url="<URL>", username="<USERNAME>", api_key="<API_KEY>")`\n\nOnce you have created a Credentials object, you can pass it to the APIClient constructor: `client = APIClient(credentials=credentials)`.',
     [{'page_content': 'Example of create Credentials object\n\nIBM watsonx.ai for IBM Cloud\n\nfrom ibm_watsonx_ai import Credentials\n\n# Example of creating the credentials using an API key:\ncredentials = Credentials(\n    url = "https://us-south.ml.cloud.ibm.com",\n    api_key = "***********"\n)\n\n# Example of creating the credentials using a token:\ncredentials = Credentials(\n    url = "https://us-south.ml.cloud.ibm.com",\n    token = "***********"\n)\n\n\n\nIBM watsonx.ai software\n\nimport os\nfrom ibm_watsonx_ai import Credentials\n\n# Example of creating the credentials using username and password:\ncredentials = Credentials(\n    url = "<URL>",\n    username = "<USERNAME>",\n    password = "<PASSWORD>",\n    instance_id = "openshift",\n    version = "5.0"\n)\n\n# Example of creating the credentials using username and apikey:\ncredentials = Credentials(\n    url = "<URL>",\n    username = "<USERNAME>",\n    api_key = "<API_KEY>",\n    instance_id = "openshift",\n    version = "5.0"\n)\n\n# Example of creating the credentials using a token:\naccess_token = os.environ[\'USER_ACCESS_TOKEN\']\ncredentials = Credentials(\n    url = "<URL>",\n    token = access_token,\n    instance_id = "openshift"\n    version = "5.0"\n)\n\n\n\n\nstatic from_dict(credentials, _verify=None)[source]¶\nCreate a Credentials object from dictionary.\n\nParameters:\ncredentials (dict) – credentials in the dictionary\n\nReturns:\ninitialised credentials object\n\nReturn type:\nCredentials\n\n\nExample:\nfrom ibm_watsonx_ai import Credentials\n\ncredentials = Credentials.from_dict({\n    \'url\': "<url>",\n    \'apikey\': "<api_key>"\n})\n\n\n\n\n\nto_dict()[source]¶\nGet dictionary from the Credentials object.\n\nReturns:\ndictionary with credentials\n\nReturn type:\ndict\n\n\nExample:\nfrom ibm_watsonx_ai import Credentials\n\ncredentials = Credentials.from_dict({\n    \'url\': "<url>",\n    \'apikey\': "<api_key>"\n})\n\ncredentials_dict = credentials.to_dict()\n\n\n\n\n\n\n\n\n\n\n\n\n\nNext\n\nCore\n\n\n\n\n\n\n\nPrevious\n\nAPI\n\n\n\n\n\n\n                Copyright © 2023-2024, IBM\n            \n            Made with Sphinx and @pradyunsg\'s\n            \n            Furo\n\n\n\n\n\n\n\n\n\n\n            On this page\n          \n\n\n\n\nBase\nAPIClient\nAPIClient\nAPIClient.set_headers()\nAPIClient.set_token()\n\n\n\n\nCredentials\nCredentials\nCredentials.from_dict()\nCredentials.to_dict()',
       'metadata': {'sequence_number': [10, 11, 12, 13],
        'document_id': 'base.html'}},
      {'page_content': 'import os\nfrom ibm_watsonx_ai import Credentials\n\n# Example of creating the credentials using username and password:\ncredentials = Credentials(\n    url = "<URL>",\n    username = "<USERNAME>",\n    password = "<PASSWORD>",\n    instance_id = "openshift",\n    version = "5.0"\n)\n\n# Example of creating the credentials using username and apikey:\ncredentials = Credentials(\n    url = "<URL>",\n    username = "<USERNAME>",\n    api_key = "<API_KEY>",\n    instance_id = "openshift",\n    version = "5.0"\n)\n\n# Example of creating the credentials using a token:\naccess_token = os.environ[\'USER_ACCESS_TOKEN\']\ncredentials = Credentials(\n    url = "<URL>",\n    token = access_token,\n    instance_id = "openshift"\n    version = "5.0"\n)\n\n\n\n\nstatic from_dict(credentials, _verify=None)[source]¶\nCreate a Credentials object from dictionary.\n\nParameters:\ncredentials (dict) – credentials in the dictionary\n\nReturns:\ninitialised credentials object\n\nReturn type:\nCredentials\n\n\nExample:\nfrom ibm_watsonx_ai import Credentials\n\ncredentials = Credentials.from_dict({\n    \'url\': "<url>",\n    \'apikey\': "<api_key>"\n})\n\n\n\n\n\nto_dict()[source]¶\nGet dictionary from the Credentials object.\n\nReturns:\ndictionary with credentials\n\nReturn type:\ndict\n\n\nExample:\nfrom ibm_watsonx_ai import Credentials\n\ncredentials = Credentials.from_dict({\n    \'url\': "<url>",\n    \'apikey\': "<api_key>"\n})\n\ncredentials_dict = credentials.to_dict()\n\n\n\n\n\n\n\n\n\n\n\n\n\nNext\n\nCore\n\n\n\n\n\n\n\nPrevious\n\nAPI\n\n\n\n\n\n\n                Copyright © 2023-2024, IBM\n            \n            Made with Sphinx and @pradyunsg\'s\n            \n            Furo\n\n\n\n\n\n\n\n\n\n\n            On this page\n          \n\n\n\n\nBase\nAPIClient\nAPIClient\nAPIClient.set_headers()\nAPIClient.set_token()\n\n\n\n\nCredentials\nCredentials\nCredentials.from_dict()\nCredentials.to_dict()',
       'metadata': {'sequence_number': [11, 12, 13],
        'document_id': 'base.html'}},
      {'page_content': 'Toggle table of contents sidebar\n\n\n\n\n\nBase¶\n\nAPIClient¶\n\n\nclass client.APIClient(credentials=None, project_id=None, space_id=None, verify=None, **kwargs)[source]¶\nThe main class of ibm_watsonx_ai. The very heart of the module. APIClient contains objects that manage the service reasources.\n\nTo explore how to use APIClient, refer to:\nSetup - to check correct initialization of APIClient for a specific environment.\nCore - to explore core properties of an APIClient object.\n\n\n\n\nParameters:\n\nurl (str) – URL of the service\ncredentials (Credentials) – credentials used to connect with the service\nproject_id (str, optional) – ID of the project that is used\nspace_id (str, optional) – ID of deployment space that is used\nverify (bool, optional) – certificate verification flag, deprecated, use Credentials(verify=…) to set verify\n\n\n\nExample:\nfrom ibm_watsonx_ai import APIClient, Credentials\n\ncredentials = Credentials(\n    url = "<url>",\n    api_key = "<api_key>"\n)\n\nclient = APIClient(credentials, space_id="<space_id>")\n\nclient.models.list()\nclient.deployments.get_details()\n\nclient.set.default_project("<project_id>")\n\n...\n\n\n\n\nset_headers(headers)[source]¶\nMethod which allows refresh/set new User Request Headers.\n\nParameters:\nheaders (dict) – User Request Headers\n\n\nExamples\nheaders = {\n    \'Authorization\': \'Bearer <USER AUTHORIZATION TOKEN>\',\n    \'User-Agent\': \'ibm-watsonx-ai/1.0.1 (lang=python; arch=x86_64; os=darwin; python.version=3.10.13)\',\n    \'X-Watson-Project-ID\': \'<PROJECT ID>\',\n    \'Content-Type\': \'application/json\'\n}\n\nclient.set_headers(headers)\n\n\n\n\n\nset_token(token)[source]¶\nMethod which allows refresh/set new User Authorization Token.\n\nParameters:\ntoken (str) – User Authorization Token\n\n\nExamples\nclient.set_token("<USER AUTHORIZATION TOKEN>")\n\n\n\n\n\n\nCredentials¶\n\n\nclass credentials.Credentials(*, url=None, api_key=None, name=None, iam_serviceid_crn=None, token=None, projects_token=None, username=None, password=None, instance_id=None, version=None, bedrock_url=None, platform_url=None, proxies=None, verify=None)[source]¶\nThis class encapsulate passed credentials and additional params.\n\nParameters: url (str) – URL of the service\napi_key (str, optional) – service API key used in API key authentication\nname (str, optional) – service name used during space creation for a Cloud environment\niam_serviceid_crn (str, optional) – service CRN used during space creation for a Cloud environment\ntoken (str, optional) – service token, used in token authentication\nprojects_token (str, optional) – service projects token used in token authentication\nusername (str, optional) – username, used in username/password or username/api_key authentication, applicable for ICP only\npassword (str, optional) – password, used in username/password authentication, applicable for ICP only\ninstance_id (str, optional) – instance ID, mandatory for ICP\nversion (str, optional) – ICP version, mandatory for ICP\nbedrock_url (str, optional) – Bedrock URL, applicable for ICP only\nproxies (dict, optional) – dictionary of proxies, containing protocol and URL mapping (example: { “https”: “https://example.url.com” })',
       'metadata': {'sequence_number': [4, 5, 6, 7, 8],
        'document_id': 'base.html'}},
      {'page_content': 'url (str) – URL of the service\napi_key (str, optional) – service API key used in API key authentication\nname (str, optional) – service name used during space creation for a Cloud environment\niam_serviceid_crn (str, optional) – service CRN used during space creation for a Cloud environment\ntoken (str, optional) – service token, used in token authentication\nprojects_token (str, optional) – service projects token used in token authentication\nusername (str, optional) – username, used in username/password or username/api_key authentication, applicable for ICP only\npassword (str, optional) – password, used in username/password authentication, applicable for ICP only\ninstance_id (str, optional) – instance ID, mandatory for ICP\nversion (str, optional) – ICP version, mandatory for ICP\nbedrock_url (str, optional) – Bedrock URL, applicable for ICP only\nproxies (dict, optional) – dictionary of proxies, containing protocol and URL mapping (example: { “https”: “https://example.url.com” })\nverify (bool, optional) – certificate verification flag Example of create Credentials object\n\nIBM watsonx.ai for IBM Cloud\n\nfrom ibm_watsonx_ai import Credentials\n\n# Example of creating the credentials using an API key:\ncredentials = Credentials(\n    url = "https://us-south.ml.cloud.ibm.com",\n    api_key = "***********"\n)\n\n# Example of creating the credentials using a token:\ncredentials = Credentials(\n    url = "https://us-south.ml.cloud.ibm.com",\n    token = "***********"\n)\n\n\n\nIBM watsonx.ai software\n\nimport os\nfrom ibm_watsonx_ai import Credentials\n\n# Example of creating the credentials using username and password:\ncredentials = Credentials(\n    url = "<URL>",\n    username = "<USERNAME>",\n    password = "<PASSWORD>",\n    instance_id = "openshift",\n    version = "5.0"\n)\n\n# Example of creating the credentials using username and apikey:\ncredentials = Credentials(\n    url = "<URL>",\n    username = "<USERNAME>",\n    api_key = "<API_KEY>",\n    instance_id = "openshift",\n    version = "5.0"\n)\n\n# Example of creating the credentials using a token:\naccess_token = os.environ[\'USER_ACCESS_TOKEN\']\ncredentials = Credentials(\n    url = "<URL>",\n    token = access_token,\n    instance_id = "openshift"\n    version = "5.0"\n)\n\n\n\n\nstatic from_dict(credentials, _verify=None)[source]¶\nCreate a Credentials object from dictionary.\n\nParameters:\ncredentials (dict) – credentials in the dictionary\n\nReturns:\ninitialised credentials object\n\nReturn type:\nCredentials\n\n\nExample:\nfrom ibm_watsonx_ai import Credentials\n\ncredentials = Credentials.from_dict({\n    \'url\': "<url>",\n    \'apikey\': "<api_key>"\n})\n\n\n\n\n\nto_dict()[source]¶\nGet dictionary from the Credentials object.\n\nReturns:\ndictionary with credentials\n\nReturn type:\ndict\n\n\nExample:\nfrom ibm_watsonx_ai import Credentials\n\ncredentials = Credentials.from_dict({\n    \'url\': "<url>",\n    \'apikey\': "<api_key>"\n})\n\ncredentials_dict = credentials.to_dict()\n\n\n\n\n\n\n\n\n\n\n\n\n\nNext\n\nCore\n\n\n\n\n\n\n\nPrevious\n\nAPI',
       'metadata': {'sequence_number': [8, 9, 10, 11, 12],
        'document_id': 'base.html'}},
      {'page_content': 'password (str, optional) – password, used in username/password authentication, applicable for ICP only\ninstance_id (str, optional) – instance ID, mandatory for ICP\nversion (str, optional) – ICP version, mandatory for ICP\nbedrock_url (str, optional) – Bedrock URL, applicable for ICP only\nproxies (dict, optional) – dictionary of proxies, containing protocol and URL mapping (example: { “https”: “https://example.url.com” })\nverify (bool, optional) – certificate verification flag Example of create Credentials object\n\nIBM watsonx.ai for IBM Cloud\n\nfrom ibm_watsonx_ai import Credentials\n\n# Example of creating the credentials using an API key:\ncredentials = Credentials(\n    url = "https://us-south.ml.cloud.ibm.com",\n    api_key = "***********"\n)\n\n# Example of creating the credentials using a token:\ncredentials = Credentials(\n    url = "https://us-south.ml.cloud.ibm.com",\n    token = "***********"\n)\n\n\n\nIBM watsonx.ai software\n\nimport os\nfrom ibm_watsonx_ai import Credentials\n\n# Example of creating the credentials using username and password:\ncredentials = Credentials(\n    url = "<URL>",\n    username = "<USERNAME>",\n    password = "<PASSWORD>",\n    instance_id = "openshift",\n    version = "5.0"\n)\n\n# Example of creating the credentials using username and apikey:\ncredentials = Credentials(\n    url = "<URL>",\n    username = "<USERNAME>",\n    api_key = "<API_KEY>",\n    instance_id = "openshift",\n    version = "5.0"\n)\n\n# Example of creating the credentials using a token:\naccess_token = os.environ[\'USER_ACCESS_TOKEN\']\ncredentials = Credentials(\n    url = "<URL>",\n    token = access_token,\n    instance_id = "openshift"\n    version = "5.0"\n)\n\n\n\n\nstatic from_dict(credentials, _verify=None)[source]¶\nCreate a Credentials object from dictionary.\n\nParameters:\ncredentials (dict) – credentials in the dictionary\n\nReturns:\ninitialised credentials object\n\nReturn type:\nCredentials\n\n\nExample:\nfrom ibm_watsonx_ai import Credentials\n\ncredentials = Credentials.from_dict({\n    \'url\': "<url>",\n    \'apikey\': "<api_key>"\n})\n\n\n\n\n\nto_dict()[source]¶\nGet dictionary from the Credentials object.\n\nReturns:\ndictionary with credentials\n\nReturn type:\ndict\n\n\nExample:\nfrom ibm_watsonx_ai import Credentials\n\ncredentials = Credentials.from_dict({\n    \'url\': "<url>",\n    \'apikey\': "<api_key>"\n})\n\ncredentials_dict = credentials.to_dict()\n\n\n\n\n\n\n\n\n\n\n\n\n\nNext\n\nCore\n\n\n\n\n\n\n\nPrevious\n\nAPI\n\n\n\n\n\n\n                Copyright © 2023-2024, IBM\n            \n            Made with Sphinx and @pradyunsg\'s\n            \n            Furo\n\n\n\n\n\n\n\n\n\n\n            On this page\n          \n\n\n\n\nBase\nAPIClient\nAPIClient\nAPIClient.set_headers()\nAPIClient.set_token()\n\n\n\n\nCredentials\nCredentials\nCredentials.from_dict()\nCredentials.to_dict()',
       'metadata': {'sequence_number': [9, 10, 11, 12, 13],
        'document_id': 'base.html'}}]]]}]}

print(score_response["predictions"][0]["values"][0][0])

According to the provided document, the new approach of providing credentials to APIClient is through the Credentials class. You can create a Credentials object using various methods, such as:

* Using an API key: `credentials = Credentials(url="https://us-south.ml.cloud.ibm.com", api_key="***********")`
* Using a token: `credentials = Credentials(url="https://us-south.ml.cloud.ibm.com", token="***********")`
* Using username and password: `credentials = Credentials(url="<URL>", username="<USERNAME>", password="<PASSWORD>")`
* Using username and API key: `credentials = Credentials(url="<URL>", username="<USERNAME>", api_key="<API_KEY>")`

Once you have created a Credentials object, you can pass it to the APIClient constructor: `client = APIClient(credentials=credentials)`.

Historical runs

In this section, you will learn how to work with historical RAG Optimizer jobs (runs).

To list historical runs, use the list() method and provide the 'rag_optimizer' filter.

experiment.runs(filter="rag_optimizer").list()

run_id = run_details["metadata"]["id"]
run_id

'38d8ffb0-9d93-4956-9a50-aaef8068e299'

Get the executed optimizer's configuration parameters

experiment.runs.get_rag_params(run_id=run_id)

{'name': 'AutoAI RAG ibm-watsonx-ai SDK documentation',
 'description': 'AutoAI RAG experiemnt trainded on ibm-watsonx-ai SDK documentataion',
 'max_number_of_rag_patterns': 6,
 'optimization_metrics': ['answer_correctness']}

Get the historical rag_optimizer instance and training details

historical_opt = experiment.runs.get_rag_optimizer(run_id)

List trained patterns for the selected optimizer

historical_opt.summary()

Clean up

To delete the current experiment, use the cancel_run(hard_delete=True) method.

Warning: Be careful: once you delete an experiment, you will no longer be able to refer to it.

rag_optimizer.cancel_run(hard_delete=True)

'SUCCESS'

To delete the deployment, use the delete method.

Warning: If you keep the deployment active, it might lead to unnecessary consumption of Compute Unit Hours (CUHs).

client.deployments.delete(deployment_id)

'SUCCESS'

To clean up all of the created assets:

• experiments

• trainings

• pipelines

• model definitions

• models

• functions

• deployments

follow the steps in this sample notebook.

Summary and next steps

You successfully completed this notebook!

You learned how to use ibm-watsonx-ai to run AutoAI RAG experiments.

Check out our Online Documentation for more samples, tutorials, documentation, how-tos, and blog posts.

Authors

Mateusz Szewczyk, Software Engineer at watsonx.ai

Copyright © 2024-2025 IBM. This notebook and its source code are released under the terms of the MIT License.