Use watsonx, and Meta `llama-3-3-70b-instruct` to answer question about an article.ipynb

Kernel: clear_env

Use watsonx, and Meta llama-3-3-70b-instruct to answer question about an article

Disclaimers

• Use only Projects and Spaces that are available in watsonx context.

Notebook content

This notebook contains the steps and code to demonstrate support for question answering in watsonx. It introduces commands for defining prompt and model testing.

Some familiarity with Python is helpful. This notebook uses Python 3.11.

Learning goal

The goal of this notebook is to demonstrate how to use llama-3-3-70b-instruct model to answer question about provided article.

Contents

This notebook contains the following parts:

• Setup

• Foundation Models on watsonx

• Model testing

• Summary

Set up the environment

Before you use the sample code in this notebook, you must perform the following setup tasks:

• Create a watsonx.ai Runtime Service instance (a free plan is offered and information about how to create the instance can be found here).

Install and import dependencies

Note: ibm-watsonx-ai documentation can be found here.

%pip install -U ibm-watsonx-ai | tail -n 1

Successfully installed anyio-4.11.0 cachetools-6.2.1 certifi-2025.10.5 charset_normalizer-3.4.4 h11-0.16.0 httpcore-1.0.9 httpx-0.28.1 ibm-cos-sdk-2.14.3 ibm-cos-sdk-core-2.14.3 ibm-cos-sdk-s3transfer-2.14.3 ibm-watsonx-ai-1.4.4 idna-3.11 jmespath-1.0.1 lomond-0.3.3 numpy-2.3.4 pandas-2.2.3 pytz-2025.2 requests-2.32.5 sniffio-1.3.1 tabulate-0.9.0 tzdata-2025.2 urllib3-2.5.0

Defining the watsonx.ai credentials

This cell defines the watsonx.ai credentials required to work with watsonx Foundation Model inferencing.

Action: Provide the IBM Cloud user API key. For details, see documentation.

import getpass

from ibm_watsonx_ai import Credentials

credentials = Credentials(
    url="https://us-south.ml.cloud.ibm.com",
    api_key=getpass.getpass("Please enter your watsonx.ai api key (hit enter): "),
)

Defining the project id

The Foundation Model requires project id that provides the context for the call. We will obtain the id from the project in which this notebook runs. Otherwise, please provide the project id.

import os

try:
    project_id = os.environ["PROJECT_ID"]
except KeyError:
    project_id = input("Please enter your project_id (hit enter): ")

from ibm_watsonx_ai import APIClient

api_client = APIClient(credentials=credentials, project_id=project_id)

Foundation Models on watsonx.ai

List available models

All avaliable models are presented under TextModels class. For more information refer to documentation.

print([model.name for model in api_client.foundation_models.TextModels])

['DEFENCE_GRANITE', 'GRANITE_3_2_8B_INSTRUCT', 'GRANITE_3_2B_INSTRUCT', 'GRANITE_3_3_8B_INSTRUCT', 'GRANITE_3_8B_INSTRUCT', 'GRANITE_4_H_SMALL', 'GRANITE_8B_CODE_INSTRUCT', 'GRANITE_GUARDIAN_3_8B', 'GRANITE_VISION_3_2_2B', 'LLAMA_3_2_11B_VISION_INSTRUCT', 'LLAMA_3_2_90B_VISION_INSTRUCT', 'LLAMA_3_3_70B_INSTRUCT', 'LLAMA_3_405B_INSTRUCT', 'LLAMA_4_MAVERICK_17B_128E_INSTRUCT_FP8', 'LLAMA_GUARD_3_11B_VISION', 'MISTRAL_MEDIUM_2505', 'MISTRAL_SMALL_3_1_24B_INSTRUCT_2503', 'GPT_OSS_120B', 'ALLAM_1_13B_INSTRUCT']

You need to specify model_id that will be used for inferencing:

model_id = api_client.foundation_models.TextModels.LLAMA_3_3_70B_INSTRUCT

Defining the model parameters

You might need to adjust model parameters for different models or tasks, to do so please refer to documentation.

from ibm_watsonx_ai.metanames import GenTextParamsMetaNames as GenParams

parameters = {
    GenParams.DECODING_METHOD: "greedy",
    GenParams.MAX_NEW_TOKENS: 100,
    GenParams.STOP_SEQUENCES: ["\n\n"],
}

Initialize the model

Initialize the ModelInference class with previous set params.

from ibm_watsonx_ai.foundation_models import ModelInference

model = ModelInference(
    model_id=model_id, params=parameters, credentials=credentials, project_id=project_id
)

Model's details

model.get_details()

Answer the question about provided article

Define instructions for the model with few-shot example.

instruction = """
Answer the following question using only information from the article. If there is no good answer in the article, say "I don't know".

Article: 
###
Tomatoes are one of the most popular plants for vegetable gardens. Tip for success: If you select varieties that are resistant to disease and pests, growing tomatoes can be quite easy. For experienced gardeners looking for a challenge, there are endless heirloom and specialty varieties to cultivate. Tomato plants come in a range of sizes. There are varieties that stay very small, less than 12 inches, and grow well in a pot or hanging basket on a balcony or patio. Some grow into bushes that are a few feet high and wide, and can be grown is larger containers. Other varieties grow into huge bushes that are several feet wide and high in a planter or garden bed. Still other varieties grow as long vines, six feet or more, and love to climb trellises. Tomato plants do best in full sun. You need to water tomatoes deeply and often. Using mulch prevents soil-borne disease from splashing up onto the fruit when you water. Pruning suckers and even pinching the tips will encourage the plant to put all its energy into producing fruit.
###

Question: Is growing tomatoes easy?
Answer: Yes, if you select varieties that are resistant to disease and pests.

Question: What varieties of tomatoes are there?
Answer: There are endless heirloom and specialty varieties.
"""

Prepare question for the model.

question = "Question: Why should you use mulch when growing tomatoes?"

Answer the question using Meta llama-3-3-70b-instruct model.

Inter the model to answer the question, according to provided instruction.

result = model.generate_text(" ".join([instruction, question]))

Explore model output.

print(result)

 
Answer: Using mulch prevents soil-borne disease from splashing up onto the fruit when you water. 

Summary and next steps

You successfully completed this notebook!

You learned how to answer questions about body of text with Meta's llama-3-3-70b-instruct on watsonx.

Check out our Online Documentation for more samples, tutorials, documentation, how-tos, and blog posts.

Authors

Daniel Ryszka, watsonx.ai & Watson Machine Learning

Rafał Chrzanowski, Software Engineer at watsonx.ai

Copyright © 2023-2026 IBM. This notebook and its source code are released under the terms of the MIT License.