GitHub Repository: ibm/watson-machine-learning-samples
Path: blob/master/cloud/notebooks/python_sdk/experiments/autoai_rag/Use AutoAI RAG and Chroma to create a pattern about IBM.ipynb
⁶⁴⁰⁵ views

Kernel: autoai_rag

Use AutoAI RAG and Chroma to create a pattern and get information from `ibm-watsonx-ai` SDK documentation

Disclaimers

Use only Projects and Spaces that are available in watsonx context.

Notebook content

This notebook contains the steps and code to demonstrate the usage of IBM AutoAI RAG. The AutoAI RAG experiment conducted in this notebook uses data scraped from the ibm-watsonx-ai SDK documentation.

Some familiarity with Python is helpful. This notebook uses Python 3.11.

Learning goal

The learning goals of this notebook are:

Create an AutoAI RAG job that will find the best RAG pattern based on provided data

This notebook contains the following parts:

Set up the environment

Before you use the sample code in this notebook, you must perform the following setup tasks:

Create a watsonx.ai Runtime Service instance (a free plan is offered and information about how to create the instance can be found here).

Install and import the required modules and dependencies

In [ ]:

!pip install -U 'ibm-watsonx-ai[rag]>=1.3.26' | tail -n 1
!pip install -U "langchain_community>=0.3,<0.4" | tail -n 1

Defining the watsonx.ai credentials

This cell defines the credentials required to work with the watsonx.ai Runtime service.

Action: Provide the IBM Cloud user API key. For details, see documentation.

In [ ]:

import getpass
from ibm_watsonx_ai import Credentials

credentials = Credentials(
    url="https://us-south.ml.cloud.ibm.com",
    api_key=getpass.getpass("Please enter your watsonx.ai api key (hit enter): "),
)

Defining the project id

The foundation model requires a project id that provides the context for the call. We will try to obtain the id directly from the project in which this notebook runs. If this fails, you'll have to provide the project id.

In [2]:

import os

try:
    project_id = os.environ["PROJECT_ID"]
except KeyError:
    project_id = input("Please enter your project_id (hit enter): ")

Create an instance of APIClient with authentication details.

In [3]:

from ibm_watsonx_ai import APIClient

client = APIClient(credentials=credentials, project_id=project_id)

RAG Optimizer definition

Defining a connection to training data

Upload training data to a COS bucket and then define a connection to this file. This example uses the Base description from the ibm_watsonx_ai documentation.

The code in the next cell uploads training data to the bucket.

In [4]:

import os
import requests

url = "https://ibm.github.io/watsonx-ai-python-sdk/base.html"

document_filename = "base.html"

response = requests.get(url)

response.raise_for_status()

if not os.path.isfile(document_filename):
    with open(document_filename, "w", encoding="utf-8") as file:
        file.write(response.text)

document_asset_details = client.data_assets.create(name=document_filename, file_path=document_filename)

document_asset_id = client.data_assets.get_id(document_asset_details)
document_asset_id

Out[4]:

Creating data asset...
SUCCESS

'3735c60e-6ead-40a1-a73a-aa7f02ba850d'

Define a connection to training data.

In [5]:

from ibm_watsonx_ai.helpers import DataConnection

input_data_references = [DataConnection(data_asset_id=document_asset_id)]

Defining a connection to test data

Upload a json file that will be used for benchmarking to COS and then define a connection to this file. This example uses content from the ibm_watsonx_ai SDK documentation.

In [6]:

benchmarking_data_IBM_page_content = [
    {
        "question": "How can you set or refresh user request headers using the APIClient class?",
        "correct_answer": "client.set_headers({'Authorization': 'Bearer <token>'})",
        "correct_answer_document_ids": [
            "base.html"
        ]
    },
    {
        "question": "How to initialise Credentials object with api_key",
        "correct_answer": "credentials = Credentials(url = 'https://us-south.ml.cloud.ibm.com', api_key = '***********')",
        "correct_answer_document_ids": [
            "base.html"
        ]
    }
]

The code in the next cell uploads testing data to the bucket as a json file.

In [7]:

import json

test_filename = "benchmarking_data_Base.json"

if not os.path.isfile(test_filename):
    with open(test_filename, "w") as json_file:
        json.dump(benchmarking_data_IBM_page_content, json_file, indent=4)

test_asset_details = client.data_assets.create(name=test_filename, file_path=test_filename)

test_asset_id = client.data_assets.get_id(test_asset_details)
test_asset_id

Out[7]:

Creating data asset...
SUCCESS

'6eeb3620-8933-4592-8b5b-0d19ac6f2133'

Define connection information to testing data.

In [8]:

test_data_references = [DataConnection(data_asset_id=test_asset_id)]

RAG Optimizer configuration

Provide the input information for AutoAI RAG optimizer:

name - experiment name
description - experiment description
max_number_of_rag_patterns - maximum number of RAG patterns to create
optimization_metrics - target optimization metrics

In [12]:

from ibm_watsonx_ai.experiment import AutoAI

experiment = AutoAI(credentials, project_id=project_id)

rag_optimizer = experiment.rag_optimizer(
    name='AutoAI RAG run - Base documentation',
    description="AutoAI RAG Optimizer on ibm_watsonx_ai Base documentation",
    foundation_models=["mistralai/mistral-small-3-1-24b-instruct-2503"],
    embedding_models=["ibm/slate-125m-english-rtrvr"],
    retrieval_methods=["simple"],
    chunking=[
        {
            "chunk_size": 512,
            "chunk_overlap": 64,
            "method": "recursive"
        }
    ],
    max_number_of_rag_patterns=4,
    optimization_metrics=[AutoAI.RAGMetrics.ANSWER_CORRECTNESS]
)

Configuration parameters can be retrieved via get_params().

In [13]:

rag_optimizer.get_params()

Out[13]:

{'name': 'AutoAI RAG run - Base documentation',
 'description': 'AutoAI RAG Optimizer on ibm_watsonx_ai Base documentation',
 'chunking': [{'chunk_size': 512, 'chunk_overlap': 64, 'method': 'recursive'}],
 'embedding_models': ['ibm/slate-125m-english-rtrvr'],
 'retrieval_methods': ['simple'],
 'generation': {'foundation_models': [{'model_id': 'mistralai/mistral-small-3-1-24b-instruct-2503'}]},
 'max_number_of_rag_patterns': 4,
 'optimization_metrics': ['answer_correctness']}

RAG Experiment run

Call the run() method to trigger the AutoAI RAG experiment. You can either use interactive mode (synchronous job) or background mode (asynchronous job) by specifying background_mode=True.

In [14]:

run_details = rag_optimizer.run(
    input_data_references=input_data_references,
    test_data_references=test_data_references,
    background_mode=False
)

Out[14]:

##############################################

Running 'b4b57086-916a-4228-af2d-e420159bcf08'

##############################################


pending.............
running......
completed
Training of 'b4b57086-916a-4228-af2d-e420159bcf08' finished successfully.

You can use the get_run_status() method to monitor AutoAI RAG jobs in background mode.

In [15]:

rag_optimizer.get_run_status()

Out[15]:

'completed'

Comparison and testing of RAG Patterns

You can list the trained patterns and information on evaluation metrics in the form of a Pandas DataFrame by calling the summary() method. You can use the DataFrame to compare all discovered patterns and select the one you like for further testing.

In [16]:

summary = rag_optimizer.summary()
summary

Out[16]:

Additionally, you can pass the scoring parameter to the summary method, to filter RAG patterns starting with the best.

summary = rag_optimizer.summary(scoring="faithfulness")

In [17]:

rag_optimizer.get_run_details()

Out[17]:

{'entity': {'hardware_spec': {'id': 'a6c4923b-b8e4-444c-9f43-8a7ec3020110',
   'name': 'L'},
  'input_data_references': [{'location': {'href': '/v2/assets/3735c60e-6ead-40a1-a73a-aa7f02ba850d?project_id=eac8bfe2-a00b-43ca-846b-305af5cc6395',
     'id': '3735c60e-6ead-40a1-a73a-aa7f02ba850d'},
    'type': 'data_asset'}],
  'parameters': {'constraints': {'chunking': [{'chunk_overlap': 64,
      'chunk_size': 512,
      'method': 'recursive'}],
    'embedding_models': ['ibm/slate-125m-english-rtrvr'],
    'generation': {'foundation_models': [{'model_id': 'mistralai/mistral-small-3-1-24b-instruct-2503'}]},
    'max_number_of_rag_patterns': 4,
    'retrieval_methods': ['simple']},
   'optimization': {'metrics': ['answer_correctness']},
   'output_logs': True},
  'results': [{'context': {'iteration': 0,
     'max_combinations': 2,
     'rag_pattern': {'composition_steps': ['model_selection',
       'chunking',
       'embeddings',
       'retrieval',
       'generation'],
      'duration_seconds': 9,
      'location': {'evaluation_results': 'default_autoai_rag_out/b4b57086-916a-4228-af2d-e420159bcf08/Pattern1/evaluation_results.json',
       'indexing_notebook': 'default_autoai_rag_out/b4b57086-916a-4228-af2d-e420159bcf08/Pattern1/indexing_inference_notebook.ipynb',
       'inference_notebook': 'default_autoai_rag_out/b4b57086-916a-4228-af2d-e420159bcf08/Pattern1/indexing_inference_notebook.ipynb',
       'inference_service_code': 'default_autoai_rag_out/b4b57086-916a-4228-af2d-e420159bcf08/Pattern1/inference_ai_service.gz',
       'inference_service_metadata': 'default_autoai_rag_out/b4b57086-916a-4228-af2d-e420159bcf08/Pattern1/inference_service_metadata.json'},
      'name': 'Pattern1',
      'settings': {'chunking': {'chunk_overlap': 64,
        'chunk_size': 512,
        'method': 'recursive'},
       'embeddings': {'model_id': 'ibm/slate-125m-english-rtrvr',
        'truncate_input_tokens': 512,
        'truncate_strategy': 'left'},
       'generation': {'context_template_text': '{document}',
        'model_id': 'mistralai/mistral-small-3-1-24b-instruct-2503',
        'parameters': {'decoding_method': 'greedy',
         'max_new_tokens': 1000,
         'max_sequence_length': 131072,
         'min_new_tokens': 1},
        'prompt_template_text': "<s>[INST] <<SYS>>\nYou are a helpful, respectful and honest assistant. Always answer as helpfully as possible, while being safe. Your answers should not include any harmful, unethical, racist, sexist, toxic, dangerous, or illegal content. Please ensure that your responses are socially unbiased and positive in nature.\n\nIf a question does not make any sense, or is not factually coherent, explain why instead of answering something not correct. If you don't know the answer to a question, please don't share false information.\n\n<</SYS>>\n\nGenerate the next agent response by answering the question. You are provided several documents with titles. If the answer comes from different documents please mention all possibilities and use the titles of documents to separate between topics or domains. If you cannot base your answer on the given documents, please state that you do not have an answer. {reference_documents}\n\nRespond exclusively in the language of the question, regardless of any other language used in the provided context. Ensure that your entire response is in the same language as the question.\n\n{question} [/INST]",
        'word_to_token_ratio': 2.1957},
       'retrieval': {'method': 'simple', 'number_of_chunks': 3},
       'vector_store': {'datasource_type': 'chroma',
        'distance_metric': 'cosine',
        'index_name': 'autoai_rag_b4b57086_20250626072109',
        'operation': 'upsert',
        'schema': {'fields': [{'description': 'text field',
           'name': 'text',
           'role': 'text',
           'type': 'string'},
          {'description': 'document name field',
           'name': 'document_id',
           'role': 'document_name',
           'type': 'string'},
          {'description': 'chunk starting token position in the source document',
           'name': 'start_index',
           'role': 'start_index',
           'type': 'number'},
          {'description': 'chunk number per document',
           'name': 'sequence_number',
           'role': 'sequence_number',
           'type': 'number'},
          {'description': 'vector embeddings',
           'name': 'vector',
           'role': 'vector_embeddings',
           'type': 'array'}],
         'id': 'autoai_rag_1.0',
         'name': 'Document schema using open-source loaders',
         'type': 'struct'}}},
      'settings_importance': {'chunking': [{'importance': 0.125,
         'parameter': 'chunk_size'},
        {'importance': 0.125, 'parameter': 'chunk_overlap'}],
       'embeddings': [{'importance': 0.125, 'parameter': 'embedding_model'}],
       'generation': [{'importance': 0.125, 'parameter': 'foundation_model'}],
       'retrieval': [{'importance': 0.125, 'parameter': 'retrieval_method'},
        {'importance': 0.125, 'parameter': 'window_size'},
        {'importance': 0.125, 'parameter': 'number_of_chunks'}]}},
     'software_spec': {'name': 'autoai-rag_rt24.1-py3.11'}},
    'metrics': {'test_data': [{'ci_high': 0.6667,
       'ci_low': 0.25,
       'mean': 0.4583,
       'metric_name': 'answer_correctness'},
      {'ci_high': 0.4086,
       'ci_low': 0.1529,
       'mean': 0.2808,
       'metric_name': 'faithfulness'},
      {'mean': 1.0, 'metric_name': 'context_correctness'}]}},
   {'context': {'iteration': 1,
     'max_combinations': 2,
     'rag_pattern': {'composition_steps': ['model_selection',
       'chunking',
       'embeddings',
       'retrieval',
       'generation'],
      'duration_seconds': 6,
      'location': {'evaluation_results': 'default_autoai_rag_out/b4b57086-916a-4228-af2d-e420159bcf08/Pattern2/evaluation_results.json',
       'indexing_notebook': 'default_autoai_rag_out/b4b57086-916a-4228-af2d-e420159bcf08/Pattern2/indexing_inference_notebook.ipynb',
       'inference_notebook': 'default_autoai_rag_out/b4b57086-916a-4228-af2d-e420159bcf08/Pattern2/indexing_inference_notebook.ipynb',
       'inference_service_code': 'default_autoai_rag_out/b4b57086-916a-4228-af2d-e420159bcf08/Pattern2/inference_ai_service.gz',
       'inference_service_metadata': 'default_autoai_rag_out/b4b57086-916a-4228-af2d-e420159bcf08/Pattern2/inference_service_metadata.json'},
      'name': 'Pattern2',
      'settings': {'chunking': {'chunk_overlap': 64,
        'chunk_size': 512,
        'method': 'recursive'},
       'embeddings': {'model_id': 'ibm/slate-125m-english-rtrvr',
        'truncate_input_tokens': 512,
        'truncate_strategy': 'left'},
       'generation': {'context_template_text': '{document}',
        'model_id': 'mistralai/mistral-small-3-1-24b-instruct-2503',
        'parameters': {'decoding_method': 'greedy',
         'max_new_tokens': 1000,
         'max_sequence_length': 131072,
         'min_new_tokens': 1},
        'prompt_template_text': "<s>[INST] <<SYS>>\nYou are a helpful, respectful and honest assistant. Always answer as helpfully as possible, while being safe. Your answers should not include any harmful, unethical, racist, sexist, toxic, dangerous, or illegal content. Please ensure that your responses are socially unbiased and positive in nature.\n\nIf a question does not make any sense, or is not factually coherent, explain why instead of answering something not correct. If you don't know the answer to a question, please don't share false information.\n\n<</SYS>>\n\nGenerate the next agent response by answering the question. You are provided several documents with titles. If the answer comes from different documents please mention all possibilities and use the titles of documents to separate between topics or domains. If you cannot base your answer on the given documents, please state that you do not have an answer. {reference_documents}\n\nRespond exclusively in the language of the question, regardless of any other language used in the provided context. Ensure that your entire response is in the same language as the question.\n\n{question} [/INST]",
        'word_to_token_ratio': 2.1957},
       'retrieval': {'method': 'simple', 'number_of_chunks': 5},
       'vector_store': {'datasource_type': 'chroma',
        'distance_metric': 'cosine',
        'index_name': 'autoai_rag_b4b57086_20250626072109',
        'operation': 'upsert',
        'schema': {'fields': [{'description': 'text field',
           'name': 'text',
           'role': 'text',
           'type': 'string'},
          {'description': 'document name field',
           'name': 'document_id',
           'role': 'document_name',
           'type': 'string'},
          {'description': 'chunk starting token position in the source document',
           'name': 'start_index',
           'role': 'start_index',
           'type': 'number'},
          {'description': 'chunk number per document',
           'name': 'sequence_number',
           'role': 'sequence_number',
           'type': 'number'},
          {'description': 'vector embeddings',
           'name': 'vector',
           'role': 'vector_embeddings',
           'type': 'array'}],
         'id': 'autoai_rag_1.0',
         'name': 'Document schema using open-source loaders',
         'type': 'struct'}}},
      'settings_importance': {'chunking': [{'importance': 0.0,
         'parameter': 'chunk_size'},
        {'importance': 0.0, 'parameter': 'chunk_overlap'}],
       'embeddings': [{'importance': 0.0, 'parameter': 'embedding_model'}],
       'generation': [{'importance': 0.0, 'parameter': 'foundation_model'}],
       'retrieval': [{'importance': 0.0, 'parameter': 'retrieval_method'},
        {'importance': 0.0, 'parameter': 'window_size'},
        {'importance': 1.0, 'parameter': 'number_of_chunks'}]}},
     'software_spec': {'name': 'autoai-rag_rt24.1-py3.11'}},
    'metrics': {'test_data': [{'ci_high': 0.75,
       'ci_low': 0.6667,
       'mean': 0.7083,
       'metric_name': 'answer_correctness'},
      {'ci_high': 0.3868,
       'ci_low': 0.1984,
       'mean': 0.2926,
       'metric_name': 'faithfulness'},
      {'mean': 1.0, 'metric_name': 'context_correctness'}]}}],
  'results_reference': {'location': {'path': 'default_autoai_rag_out',
    'training': 'default_autoai_rag_out/b4b57086-916a-4228-af2d-e420159bcf08',
    'training_status': 'default_autoai_rag_out/b4b57086-916a-4228-af2d-e420159bcf08/training-status.json',
    'training_log': 'default_autoai_rag_out/b4b57086-916a-4228-af2d-e420159bcf08/output.log',
    'assets_path': 'default_autoai_rag_out/b4b57086-916a-4228-af2d-e420159bcf08/assets'},
   'type': 'container'},
  'status': {'completed_at': '2025-06-26T07:21:42.639Z',
   'message': {'level': 'info',
    'text': 'AAR019I: AutoAI execution completed.'},
   'running_at': '2025-06-26T07:21:01.000Z',
   'state': 'completed',
   'step': 'generation'},
  'test_data_references': [{'location': {'href': '/v2/assets/6eeb3620-8933-4592-8b5b-0d19ac6f2133?project_id=eac8bfe2-a00b-43ca-846b-305af5cc6395',
     'id': '6eeb3620-8933-4592-8b5b-0d19ac6f2133'},
    'type': 'data_asset'}],
  'timestamp': '2025-06-26T07:21:45.185Z'},
 'metadata': {'created_at': '2025-06-26T07:19:49.317Z',
  'description': 'AutoAI RAG Optimizer on ibm_watsonx_ai Base documentation',
  'id': 'b4b57086-916a-4228-af2d-e420159bcf08',
  'modified_at': '2025-06-26T07:21:42.823Z',
  'name': 'AutoAI RAG run - Base documentation',
  'project_id': 'eac8bfe2-a00b-43ca-846b-305af5cc6395',
  'tags': ['autorag.264ab5bc-2eb8-4478-bd2b-db4d8dbe1e7c']}}

Get selected pattern

Get the RAGPattern object from the RAG Optimizer experiment. By default, the RAGPattern of the best pattern is returned.

In [19]:

best_pattern_name = summary.index.values[0]
print('Best pattern is:', best_pattern_name)

best_pattern = rag_optimizer.get_pattern(pattern_name="Pattern1")

Out[19]:

Best pattern is: Pattern2

The pattern details can be retrieved by calling the get_pattern_details method:

rag_optimizer.get_pattern_details(pattern_name='Pattern2')

Create the index/collection

Build solution on the best pattern, with additional document indexing.

You can check which index_name you are working on:

best_pattern.vector_store._index_name

In [20]:

from langchain_community.document_loaders import WebBaseLoader

urls = [
    "https://ibm.github.io/watsonx-ai-python-sdk/fm_embeddings.html",
    "https://ibm.github.io/watsonx-ai-python-sdk/fm_custom_models.html",
    "https://ibm.github.io/watsonx-ai-python-sdk/fm_text_extraction.html"
]
docs_list = WebBaseLoader(urls).load()
doc_splits = best_pattern.chunker.split_documents(docs_list)

Out[20]:

USER_AGENT environment variable not set, consider setting it to identify your requests.

In [21]:

best_pattern.indexing_function(doc_splits)

Out[21]:

/var/folders/js/prfjw1gx7tqb554jl4sqw9pw0000gn/T/ipykernel_96767/1260819839.py:1: DeprecationWarning: `indexing_function` is deprecated and will be removed in a future release.
  best_pattern.indexing_function(doc_splits)

['8591131e50033fba603cbb386f8cdb553dfb1ce78e1ae5efebd66f6232b9ea8c',
 'c6ded0d72ad8bb1b2b31adb52e6de57f1e48a57d0103cfa5ade1714eb0b5ebd8',
 '32381dd4b918ae2c4bc88ce20bf314ce57fd8d94aeac88ac069a2b0f65d05dd9',
 '63bd9001dd1b0d3a96843ec90789cfa0539185e503b807cf7c4e9c810f0cf59a',
 '31d0e7b15a6d50e4074ba326bbc215adaafe1d417e93150ada3adf5d652fbb85',
 'bf834893766f79c474a698d7e74704e179929110b36c6d71f9290fc721470274',
 '73a6da22da234d2b58e62d9292ad0d28830110ecd4664716874c526a217f91f2',
 'ad36cf5048f49bae789428a05f3d6d12fb1c268fd01ac2047c562151d1ba0f47',
 '719535595c7836d36822bc567246061dc3c6ba4facb3acf46c0e7a9253feab3f',
 '5a9d36496ee5a13b95ee835cdbd2fc2b0b82635d86267e64a07973b42e780f3a',
 'b53ef0e8415975ac57f0cb6e63a10063528bc9394186dd0d0f81a27c774fdf71',
 '17ceba8a4fe063ec7751a5d2ebb33664f1128a7e5670da0a0df900bace3f98bd',
 '23386fbdbdfe78d502f274099641d9cbebf244c8134c0dbb9c87ed7a73f3ab53',
 '4d8c3685f7bae330b9ab35d3d592d3c86a68d7cd5805158d96354f16003644b3',
 '9a3ce2e2b6cd84b0da2cf497396f86cf00aaae028bab20455585af25d114c607',
 '9608a6a30c2656d6d88d680fd6c5184e7bdc86bdb793521bc8bdf744dd77be99',
 'a2a3e81be9fc27aa53ba3d0a2db29d9b4ef33c5a101fa13b48df5bb6f257bd9f',
 '4ce3320d8dd747dc734d6f08fd49cd70f78b05cbc784ea501f1491b2320f1e13',
 '1e64d334f89d59521735ad242c152ffa7083eed9f8ca93ca92cbbc6ae5909029',
 'f4a4c7f17fceaf052bc3d047c5a2c8cf1b1393bd7fbf137f085fa90d9b003942',
 '73345010c0320d8e4db06c86047f21d0a8ee56724ca89b4082587dc4c8cc9a8f',
 '0ba6a30a96369c88f57eca4ad56d8d404fd5d0266f1495cfd5adf3a117dc9330',
 'bb2a4bf58371dfbd2543b4729313551d12c64ce0a466b769643a86335ac89788',
 'cf86a13a3da9874b920b22aea801cb2e6de5339c023e2b78cf90a51329c0274f',
 'f65d8dc93b3fb3f28350b6cfc002bb97042d7a94e7b9947152e77094244b8b35',
 '8798b9360cc7c72f685eb73672df9ba2e4251dd27e4e48910ee4c3ab1c143530',
 'f24d534fb83fe29270dfe29723da8bfc5ae08210504bb7e0fade91172c23ced0',
 '9455b5cd09800ee64d2446eb9d8f95512288f8203d351967541a466d6186ee19',
 'be8a9a649260c0b51e9b82504ae21b7d86afc8a40759f775c6b80240ea545a8b',
 '8f652154e6c8be7cdc3fcd3e0ab36c0c199ce69f68c6f7759a78981785d70c65',
 'a329e2a0951fda8535daaf83bdc711f1a95f1b70aa1ebbac36159cfc7402c00e',
 '75a76b423b7bc2289b13ce3e05b42254048398a36f498334045ad8379c28fccd',
 'd0f7ea1caa72e585cdcbb6228487b07159a65fea44fd20770f9474334253a65f',
 '8c9a37922cbde00438e063a336e96063d82b5662f35f6ef6eec49aa766388192',
 'ff747908f764af18e9ce7eb116dfb8ff3678566674e687add25db98a43de6307',
 '15613d7525864d7c08fc3b2429e146ff9d1393bf1fe445d2451e990061868e3b',
 '27998b5b25bbe734180b6def9c790af07bf50cc24cb3d166fe38d4605b1ef05b',
 'a5f9db83b122eb05c9e055ab1239e56aa206534226ddc270a07ce1c4294ed823',
 '4c16fab26fd2c5e987106103ec0455b5cba31eeb325da7576f40e80181d12e7b',
 'b9eb3fc532ef8eb82f9e0161d91f4c9d3fec4ee22631b9473d87ea4ba52a54f5',
 'ec3e3b2337613af9ba3b95724c66e0911718abbb450cd9d382364e5e200f7a8b',
 '7d234b9b2c08fa04172c3cc60636c4b32b1822f4872fb109b40f8e2ea57d085b',
 '717bf62a26df8fc51a713ae2b4361a5e502986c33476f3b817147a886989f2b5',
 '026e2a7004df0159716838f95b40011565e52bb44e7dbd36af2335fab9791551',
 '2ccd8515a02fe5217844e47739df93b5e3e20bf603f91b7da6b64782e2688036',
 '822d11a3e04e34513ea697e2b61b0f9a6d42723f688e499b9476ed3faec72351',
 '74e29c3609ffc7b6a9f67146eb10294d4acf0146b831cfd23458de97f4f81fc1',
 '3b1975ef1d0eefd53aa71a1737004e389505448cd14b05aa93bfec11d8bb34a4',
 'e7adc05e1ca1f66f134572eb203b5a8d888aeb1ef2fc5a1407afdf7a368c0f51',
 'feedd690f80d1601486cb7ca9dcc06d17c43213e1fbc7521834a1912f5eccd3e',
 'c0cdfd45c84ee8edb888c5a6256c5ba158da80336a791b0bdcd1b7507b246aad',
 '8350e305a234e4ebf0b7e9a73b9f0c21c71a786cb7c18798c597d403023ebab3',
 '54b0ae12530962f157fce6fbe0c0f68944bec6fb95c021d81b01d693037880c5',
 '006ef2170dc731be45e251dc2e3180b995b90879d55b6f44dcbcd39d7b38911f',
 'c2ef512b5931f0f72070eafd8f0ca3d1a204d5dd0babd85d3d4f17686daa42d8',
 '7e1ddd18d69e528237f2794d75f595d461dc9819e2a6a00ce5e9786c9bac0bbe',
 '89ac7a1c8ea149a43cbc8c7757f23776fc025d820a0dcd09c681482d3f4aba6b',
 'fe21ec70e23983960e17ecec90af0df35175b105b5f831c34fb4429c2d4ea38b',
 '268a80b79727f6a61b4d5fbae2532aab15e825bf0b3e8e24660e9dfee34e2db9',
 '3f3788e66628e666cd4539e5904541cbca0e28832d800b8e2e0a5072cd1b37ba',
 'e4c2737c5e59eb5e49e4d5ab8e3e97b1f4cd3c3241a8d0a16bb110c9809f42c7',
 '082dba365980f5d4e4563c90c488dd0eef1f1dcd54e3963299cf60eb3770985f',
 '2a797c219679f94a1a4805150aae8b6ac226d7a881275dcfaf80e1af147d5e28',
 '9b912a699c65c4ef603b3fb1fe41989f17358cf6ac2827a5f71a0dd2bbfaadf6',
 'ad53edda03070f01102cbf48b53056a049b81e190c233e70abfbf07b10eb855f',
 '74d4d274b62129efd976ec8734bfecdff70a17243e2839aae9728084b2507487',
 '83fe7c0d3b303cfdf44a036414709bcda9f46e607ab5a78844b92491357f336b',
 '8700f06c0a703491ad70ac633042e8af4958f0fcbcbc180ce9aece783cae8d24',
 '60fd7c2fd25af21fec55fe891c700fb329d78f751db7d8efe0938048e49bb95c',
 '4beca166cc67a7d530d678798aa76bdcaeb65c0b3d24a34de00cd941420dc3ed',
 '80294c9fb6b3ae65ba3c98a496a1ce856f48643529dc9ae5a78b0ccb2c977023',
 '9a68dc81fe71acf969d7361dbc1867d186bca12184f850ad89be9fb041fcd0e1',
 '39046b7148aba69d05ec6946d4e615ddfd7ed994cf53fd3b581553512681f942',
 '13c1bd11ff79e5b825ba168014ece524be07d9b28901b2dd2f1477a43d80ff14',
 'ea4a4776b794432b22b5dd005880ac24b662a9f471f8ed961934c46d21f6d6cc',
 '22af0bc3885f076154a508e23376b8dbd3c8ee7d9a1fc063af9043d0d392e7c0',
 '6ac5a58223a24d75270393133e95196b9dcd7f40983200b4fb0e1f55ba605162',
 '1ea0d12a407f944e7eef72ffba93c8bcfe9258c1cf5954dfa870799c82616dcc',
 '7948e7a1672aa4a292e3244545393ddbdceaf01a027e06e3915bc160840b2a18',
 '06bb2d254ec7e22863e74b80086f84aca8cf2aaa69e3636e5b49c548fdfaaa5b',
 'e7ef106bc3c77a598841fc628b7ba5938f24b3c0a06884eddb6a9b7f132f25a0',
 '237e83354aafd17c0bc1fa79423b6d7597ec68aac2214b88d40e9bf29a77645d',
 'b64dd94f3be7e43814d67d36d134f08b64513a8f5e7bbffdccdcca614665e304',
 '8aacec8e75b21488b2c9c9c778d77b58ef5733faa0150423c169794f7d4bd772',
 'd0edfdab936fdaa4556adb31954473968cebdd487f42396610bfdcf01d1b2ebc',
 'a2ec3cabfc0dd97bf1f91be38e78f888594ec1b8c213af3fefe22c8fbfd9bb54',
 'f49c9caccfac6a97c8fc7d530013bb952cf0c6d256b797ce9c93a3ccd0184c9e',
 '7fc77d9d4890adb4adc824802a1edec98d42c2b01439895920bffdf7e6c3b645',
 '6b1a482a06866e01314fcad1c864820d2334c3c1a04c8a0bec6a9ae8ad32bc74',
 '9f188007fd7980e451b897b7afccd48d400182ae01339533d49c0a3ada323c79',
 '4f7b557b1899fce18132a30a06980dc72f17707e06aa1b0c0e097673ffaac74d',
 '22ceefffd7b8701f731bab3d4c20b8cd98c83745874088fc26d4c17d5cf65bad',
 '3d10d8c183dc08f3c70f4a7aca53c65810e4af8f908f2b60cf2ef09601e72e89',
 'fd5d2a1e2dd6e3c5ad556b9c5a42f1f76f9361e0709f4a7f3e983c9b53bd19ac',
 '22ede1c9491e0dc3e7e90809c1d295cb0df8a2a7082945fbc034a1d617f61f46',
 '8cfbbb1b443ce877fa85f24d89d623cfa5f3177f97979fe475f00664d392d482',
 'e5cfb31339cd605743ebf82ced58a7c4a3cb0e87b1638d4769d746b994a1ca81']

Query the RAGPattern locally, to test it.

In [22]:

from ibm_watsonx_ai.deployments import RuntimeContext

runtime_context = RuntimeContext(api_client=client)
inference_service_function = best_pattern.inference_service(runtime_context)[0]

In [25]:

question = "How to add Task Credentials?"

context = RuntimeContext(
    api_client=client,
    request_payload_json={"messages": [{"role": "user", "content": question}]},
)

inference_service_function(context)

Out[25]:

{'body': {'choices': [{'index': 0,
    'message': {'role': 'assistant',
     'content': ' </s>\n\nTo add Task Credentials, you need to follow these steps:\n\n**IBM watsonx.ai for IBM Cloud**\n\n1. **Create Credentials Object**: You can create a credentials object using either an API key or a token. Here are examples of both methods:\n\n   - **Using an API Key**:\n     ```python\n     from ibm_watsonx_ai import Credentials\n\n     credentials = Credentials(\n         url="https://us-south.ml.cloud.ibm.com",\n         api_key="YOUR_API_KEY"\n     )\n     ```\n\n   - **Using a Token**:\n     ```python\n     from ibm_watsonx_ai import Credentials\n\n     credentials = Credentials(\n         url="https://us-south.ml.cloud.ibm.com",\n         token="YOUR_TOKEN"\n     )\n     ```\n\n2. **Store Task Credentials**: Once you have the credentials object, you can store the task credentials using the `store` method:\n   ```python\n   client.task_credentials.store()\n   ```\n\n3. **List Available Task Credentials**: To list the available task credentials, use the `list` method:\n   ```python\n   client.task_credentials.list()\n   ```\n\n4. **Set Default Project or Space**: You can set the default project or space using the `set` method:\n   ```python\n   client.set.default_project(project_id="YOUR_PROJECT_ID")\n   # or\n   client.set.default_space(space_id="YOUR_SPACE_ID")\n   ```\n\n**IBM watsonx.ai software**\n\n1. **Import Necessary Modules**: Import the required modules from the `ibm_watsonx_ai` package.\n   ```python\n   import os\n   from ibm_watsonx_ai import Credentials\n   ```\n\n2. **Create Credentials Object**: Similar to the IBM watsonx.ai for IBM Cloud, you can create a credentials object using an API key or a token.\n\n3. **Store Task Credentials**: Use the `store` method to store the task credentials.\n\n4. **List Available Task Credentials**: Use the `list` method to list the available task credentials.\n\n5. **Set Default Project or Space**: Use the `set` method to set the default project or space.\n\nFor more detailed information, you can refer to the documentation on [Adding task credentials](https://www.ibm.com/docs/en/watsonx-ai?topic=credentials-adding-task).'},
    'reference_documents': [{'page_content': 'If the list is empty, you can create new task credentials with the store method:\nclient.task_credentials.store()\n\n\nTo get the status of available task credentials, use the get_details method:\nclient.task_credentials.get_details()',
      'metadata': {'document_id': '-2359849608442110910',
       'language': 'en',
       'sequence_number': 8,
       'source': 'https://ibm.github.io/watsonx-ai-python-sdk/fm_custom_models.html',
       'start_index': 0,
       'title': 'Custom models - IBM watsonx.ai'}},
     {'page_content': 'client = APIClient(credentials)\nclient.set.default_project(project_id=project_id)\n# or client.set.default_space(space_id=space_id)\n\n\n\n\nAdd Task Credentials¶\n\nWarning\nIf not already added, Task Credentials are required on IBM watsonx.ai for IBM Cloud to make a deployment.\n\nWith task credentials, you can deploy a custom foundation model and avoid token expiration issues.\nFor more details, see Adding task credentials.\nTo list available task credentials, use the list method:\nclient.task_credentials.list()',
      'metadata': {'document_id': '-2359849608442110910',
       'language': 'en',
       'sequence_number': 7,
       'source': 'https://ibm.github.io/watsonx-ai-python-sdk/fm_custom_models.html',
       'start_index': 0,
       'title': 'Custom models - IBM watsonx.ai'}},
     {'page_content': 'Example of create Credentials object\n\nIBM watsonx.ai for IBM Cloud\n\nfrom ibm_watsonx_ai import Credentials\n\n# Example of creating the credentials using an API key:\ncredentials = Credentials(\n    url = "https://us-south.ml.cloud.ibm.com",\n    api_key = IAM_API_KEY\n)\n\n# Example of creating the credentials using a token:\ncredentials = Credentials(\n    url = "https://us-south.ml.cloud.ibm.com",\n    token = "***********"\n)\n\n\n\nIBM watsonx.ai software\n\nimport os\nfrom ibm_watsonx_ai import Credentials',
      'metadata': {'document_id': 'base.html',
       'sequence_number': 26,
       'start_index': 9171}}]}]}}

Historical runs

In this section you learn to work with historical RAG Optimizer jobs (runs).

To list historical runs use the list() method and provide the 'rag_optimizer' filter.

In [27]:

experiment.runs(filter='rag_optimizer').list()

Out[27]:

In [29]:

run_id = run_details['metadata']['id']
run_id

Out[29]:

'b4b57086-916a-4228-af2d-e420159bcf08'

Get executed optimizer's configuration parameters

In [30]:

experiment.runs.get_rag_params(run_id=run_id)

Out[30]:

{'name': 'AutoAI RAG run - Base documentation',
 'description': 'AutoAI RAG Optimizer on ibm_watsonx_ai Base documentation',
 'chunking': [{'chunk_overlap': 64, 'chunk_size': 512, 'method': 'recursive'}],
 'embedding_models': ['ibm/slate-125m-english-rtrvr'],
 'retrieval_methods': ['simple'],
 'max_number_of_rag_patterns': 4,
 'generation': {'foundation_models': [{'model_id': 'mistralai/mistral-small-3-1-24b-instruct-2503'}]},
 'optimization_metrics': ['answer_correctness']}

Get historical rag_optimizer instance and training details

In [31]:

historical_opt = experiment.runs.get_rag_optimizer(run_id)

List trained patterns for selected optimizer

In [32]:

historical_opt.summary()

Out[32]:

Clean up

To delete the current experiment, use the cancel_run method.

Warning: Be careful: once you delete an experiment, you will no longer be able to refer to it.

In [29]:

rag_optimizer.cancel_run()

Out[29]:

'SUCCESS'

If you want to clean up all created assets:

experiments
trainings
pipelines
model definitions
models
functions
deployments

please follow up this sample notebook.

Summary and next steps

You successfully completed this notebook!

You learned how to use ibm-watsonx-ai to run AutoAI RAG experiments.

Check out our Online Documentation for more samples, tutorials, documentation, how-tos, and blog posts.

Authors

Michał Steczko, Software Engineer watsonx.ai

Use AutoAI RAG and Chroma to create a pattern and get information from `ibm-watsonx-ai` SDK documentation

Disclaimers

Notebook content

Learning goal

Contents

Set up the environment

Install and import the required modules and dependencies

Defining the watsonx.ai credentials

Defining the project id

RAG Optimizer definition

Defining a connection to training data

Defining a connection to test data

RAG Optimizer configuration

RAG Experiment run

Comparison and testing of RAG Patterns

Get selected pattern

Create the index/collection

Historical runs

Get executed optimizer's configuration parameters

Get historical rag_optimizer instance and training details

List trained patterns for selected optimizer

Clean up

Summary and next steps

Authors

Product

Resources

Company

Use AutoAI RAG and Chroma to create a pattern and get information from ibm-watsonx-ai SDK documentation

Disclaimers

Notebook content

Learning goal

Contents

Set up the environment

Install and import the required modules and dependencies

Defining the watsonx.ai credentials

Defining the project id

RAG Optimizer definition

Defining a connection to training data

Defining a connection to test data

RAG Optimizer configuration

RAG Experiment run

Comparison and testing of RAG Patterns

Get selected pattern

Create the index/collection

Historical runs

Get executed optimizer's configuration parameters

Get historical rag_optimizer instance and training details

List trained patterns for selected optimizer

Clean up

Summary and next steps

Authors

Use AutoAI RAG and Chroma to create a pattern and get information from `ibm-watsonx-ai` SDK documentation