GitHub Repository: ibm/watson-machine-learning-samples
Path: blob/master/cpd5.2/notebooks/python_sdk/deployments/foundation_models/Use watsonx, and LangChain to make a series of calls to a language model.ipynb
⁶⁴¹² views

Kernel: watsonx-ai-samples-py-312

Use watsonx, and LangChain to make a series of calls to a language model

Disclaimers

Use only Projects and Spaces that are available in watsonx context.

Notebook content

This notebook contains the steps and code to demonstrate Simple Sequential Chain using langchain integration with watsonx models.

Some familiarity with Python is helpful. This notebook uses Python 3.12.

Learning goal

The goal of this notebook is to demonstrate how to chain meta-llama/llama-3-3-70b-instruct and mistralai/ministral-8b-instruct models to generate a sequence of creating a random question on a given topic and an answer to that question and also to make the user friends with LangChain framework, using simple chain (LLMChain) and the extended chain (SimpleSequentialChain) with the WatsonxLLM.

This notebook contains the following parts:

Set up the environment

Before you use the sample code in this notebook, you must perform the following setup tasks:

Contact with your Cloud Pak for Data administrator and ask them for your account credentials

Install dependencies

Note: ibm-watsonx-ai documentation can be found here.

In [ ]:

%pip install -U "langchain>=0.3.25,<0.4" | tail -n 1
%pip install -U "langchain_ibm>=0.3.10,<0.4" | tail -n 1
%pip install -U ibm_watsonx-ai | tail -n 1

Successfully installed PyYAML-6.0.2 SQLAlchemy-2.0.40 annotated-types-0.7.0 anyio-4.9.0 certifi-2025.4.26 charset-normalizer-3.4.2 h11-0.16.0 httpcore-1.0.9 httpx-0.28.1 idna-3.10 jsonpatch-1.33 jsonpointer-3.0.0 langchain-0.3.25 langchain-core-0.3.59 langchain-text-splitters-0.3.8 langsmith-0.3.42 orjson-3.10.18 packaging-24.2 pydantic-2.11.4 pydantic-core-2.33.2 requests-2.32.3 requests-toolbelt-1.0.0 sniffio-1.3.1 tenacity-9.1.2 typing-extensions-4.13.2 typing-inspection-0.4.0 urllib3-2.4.0 zstandard-0.23.0
Successfully installed ibm-cos-sdk-2.14.0 ibm-cos-sdk-core-2.14.0 ibm-cos-sdk-s3transfer-2.14.0 ibm-watsonx-ai-1.3.13 jmespath-1.0.1 langchain_ibm-0.3.10 lomond-0.3.3 numpy-2.2.5 pandas-2.2.3 pytz-2025.2 requests-2.32.2 tabulate-0.9.0 tzdata-2025.2
Successfully installed anyio-4.9.0 h11-0.16.0 httpcore-1.0.9 httpx-0.28.1 ibm-cos-sdk-2.14.0 ibm-cos-sdk-core-2.14.0 ibm-cos-sdk-s3transfer-2.14.0 ibm-watsonx-ai-1.3.13 jmespath-1.0.1 lomond-0.3.3 requests-2.32.2 sniffio-1.3.1 tabulate-0.9.0

Define credentials

Authenticate the watsonx.ai Runtime service on IBM Cloud Pak for Data. You need to provide the admin's username and the platform url.

In [ ]:

username = "PASTE YOUR USERNAME HERE"
url = "PASTE THE PLATFORM URL HERE"

Use the admin's api_key to authenticate watsonx.ai Runtime services:

In [ ]:

import getpass
from ibm_watsonx_ai import Credentials

credentials = Credentials(
    username=username,
    api_key=getpass.getpass("Enter your watsonx.ai API key and hit enter: "),
    url=url,
    instance_id="openshift",
    version="5.2",
)

Alternatively you can use the admin's password:

In [3]:

import getpass
from ibm_watsonx_ai import Credentials

if "credentials" not in locals() or not credentials.api_key:
    credentials = Credentials(
        username=username,
        password=getpass.getpass("Enter your watsonx.ai password and hit enter: "),
        url=url,
        instance_id="openshift",
        version="5.2",
    )

Working with projects

First of all, you need to create a project that will be used for your work. If you do not have a project created already, follow the steps below:

Open IBM Cloud Pak main page
Click all projects
Create an empty project
Copy project_id from url and paste it below

Action: Assign project ID below

In [4]:

import os

try:
    project_id = os.environ["PROJECT_ID"]
except KeyError:
    project_id = input("Please enter your project_id (hit enter): ")

Create `APIClient` instance

In [5]:

from ibm_watsonx_ai import APIClient

client = APIClient(credentials, project_id)

Foundation Models on `watsonx.ai`

List available models

In [6]:

for model in client.foundation_models.TextModels:
    print(f"- {model}")

Out[6]:

- ibm/granite-guardian-3-2b
- ibm/granite-guardian-3-8b
- meta-llama/llama-3-3-70b-instruct
- mistralai/ministral-8b-instruct

You need to specify model_id's that will be used for inferencing:

In [7]:

model_id_1 = client.foundation_models.TextModels.LLAMA_3_3_70B_INSTRUCT
model_id_2 = client.foundation_models.TextModels.MINISTRAL_8B_INSTRUCT

Defining the model parameters

You might need to adjust model parameters for different models or tasks, to do so please refer to documentation under GenTextParamsMetaNames class.

Action: If any complications please refer to the documentation.

In [8]:

from ibm_watsonx_ai.metanames import GenTextParamsMetaNames as GenParams
from ibm_watsonx_ai.foundation_models.utils.enums import DecodingMethods

parameters = {
    GenParams.DECODING_METHOD: DecodingMethods.SAMPLE,
    GenParams.MAX_NEW_TOKENS: 100,
    GenParams.MIN_NEW_TOKENS: 1,
    GenParams.TEMPERATURE: 0.5,
    GenParams.TOP_K: 50,
    GenParams.TOP_P: 1,
}

Initialize the model

Initialize the ModelInference class with previously set params.

In [9]:

from ibm_watsonx_ai.foundation_models import ModelInference

llama_3_3_70b_instruct_model = ModelInference(
    model_id=model_id_1,
    params=parameters,
    credentials=credentials,
    project_id=project_id,
)

ministral_8b_instruct_model = ModelInference(
    model_id=model_id_2, credentials=credentials, project_id=project_id
)

WatsonxLLM interface

WatsonxLLM is a wrapper around watsonx.ai models that provide chain integration around the models.

Action: For more details about CustomLLM check the LangChain documentation

Initialize the `WatsonxLLM` class.

In [10]:

from langchain_ibm import WatsonxLLM

llama_3_3_70b_instruct_llm = WatsonxLLM(watsonx_model=llama_3_3_70b_instruct_model)
ministral_8b_instruct_llm = WatsonxLLM(watsonx_model=ministral_8b_instruct_model)

Hint: To use Chain interface from LangChain with watsonx.ai models you must call model.to_langchain() method.

It returns WatsonxLLM wrapper compatible with LangChain CustomLLM specification.

In [11]:

llama_3_3_70b_instruct_model.to_langchain()

Out[11]:

WatsonxLLM(model_id='meta-llama/llama-3-3-70b-instruct', project_id='34296cd2-f609-4dc8-83cc-aecb19d52199', params={'decoding_method': <DecodingMethods.SAMPLE: 'sample'>, 'max_new_tokens': 100, 'min_new_tokens': 1, 'temperature': 0.5, 'top_k': 50, 'top_p': 1}, watsonx_model=<ibm_watsonx_ai.foundation_models.inference.model_inference.ModelInference object at 0x103b42390>)

You can print all set data about the WatsonxLLM object using the dict() method.

In [12]:

llama_3_3_70b_instruct_llm.dict()

Out[12]:

{'model_id': 'meta-llama/llama-3-3-70b-instruct',
 'deployment_id': None,
 'params': {'decoding_method': <DecodingMethods.SAMPLE: 'sample'>,
  'max_new_tokens': 100,
  'min_new_tokens': 1,
  'temperature': 0.5,
  'top_k': 50,
  'top_p': 1},
 'project_id': '34296cd2-f609-4dc8-83cc-aecb19d52199',
 'space_id': None,
 '_type': 'IBM watsonx.ai'}

Simple Sequential Chain experiment

The simplest type of sequential chain is called a SimpleSequentialChain, in which each step has a single input and output and the output of one step serves as the input for the following step.

The experiment will consist in generating a random question about any topic and answer the following question.

An object called PromptTemplate assists in generating prompts using a combination of user input, additional non-static data, and a fixed template string.

In our case we would like to create two PromptTemplate objects which will be responsible for creating a random question and answering it.

In [13]:

from langchain import PromptTemplate

prompt_1 = PromptTemplate(
    input_variables=["topic"],
    template="Generate a random question about {topic}. Question: ",
)
prompt_2 = PromptTemplate(
    input_variables=["question"],
    template="Answer the following question: {question}. Answer: ",
)

We would like to add functionality around language models using LLMChain chain.

prompt_to_llama chain formats the prompt template whose task is to generate random question, passes the formatted string to LLM and returns the LLM output.

Hint: To use Chain interface from LangChain with watsonx.ai models you must call model.to_langchain() method.

It returns WatsonxLLM wrapper compatible with LangChain CustomLLM specification.

In [14]:

prompt_to_llama = prompt_1 | llama_3_3_70b_instruct_model.to_langchain()

llama_to_ministral chain formats the prompt template whose task is to answer the question we got from prompt_to_llama chain, passes the formatted string to LLM and returns the LLM output.

In [15]:

llama_to_ministral = prompt_2 | ministral_8b_instruct_model.to_langchain()

This is the overall chain where we run prompt_to_llama and llama_to_ministral chains in sequence.

In [16]:

qa = prompt_to_llama | llama_to_ministral

Generate random question and answer to topic.

In [17]:

qa.invoke("cars")

Out[17]:

' The most popular car color in the United States is white.'

Summary and next steps

You successfully completed this notebook!

You learned how to use Simple Sequential Chain using custom LLM WastonxLLM.

Check out our Online Documentation for more samples, tutorials, documentation, how-tos, and blog posts.

Authors

Mateusz Szewczyk, Software Engineer at watsonx.ai.

Use watsonx, and LangChain to make a series of calls to a language model

Disclaimers

Notebook content

Learning goal

Contents

Set up the environment

Install dependencies

Define credentials

Working with projects

Create `APIClient` instance

Foundation Models on `watsonx.ai`

List available models

Defining the model parameters

Initialize the model

WatsonxLLM interface

Initialize the `WatsonxLLM` class.

Simple Sequential Chain experiment

Summary and next steps

Authors

Product

Resources

Company

Use watsonx, and LangChain to make a series of calls to a language model

Disclaimers

Notebook content

Learning goal

Contents

Set up the environment

Install dependencies

Define credentials

Working with projects

Create APIClient instance

Foundation Models on watsonx.ai

List available models

Defining the model parameters

Initialize the model

WatsonxLLM interface

Initialize the WatsonxLLM class.

Simple Sequential Chain experiment

Summary and next steps

Authors

Create `APIClient` instance

Foundation Models on `watsonx.ai`

Initialize the `WatsonxLLM` class.