Use watsonx to analyze car rental customer satisfaction from text.ipynb

Kernel: .venv_watsonx_ai_samples_py_312

Use watsonx to analyze car rental customer satisfaction from text

Disclaimers

• Use only Projects and Spaces that are available in watsonx context.

Notebook content

This notebook contains the steps and code to demonstrate support of text sentiment analysis in watsonx. It introduces commands for data retrieval, model testing and scoring.

Some familiarity with Python is helpful. This notebook uses Python 3.12.

Learning goal

The goal of this notebook is to demonstrate how to use watsonx.ai model to analyze customer satisfaction from text.

Contents

This notebook contains the following parts:

• Setup

• Data loading

• Foundation Models on watsonx

• Model testing

• Score

• Summary

Set up the environment

Before you use the sample code in this notebook, you must perform the following setup tasks:

• Create a watsonx.ai Runtime Service instance (a free plan is offered and information about how to create the instance can be found here).

Install and import the datasets and dependecies

%pip install wget | tail -n 1
%pip install httpx | tail -n 1
%pip install pandas | tail -n 1
%pip install datasets | tail -n 1
%pip install ibm-cloud-sdk-core | tail -n 1
%pip install "scikit-learn==1.6.1" | tail -n 1

import getpass
import json
import os

import httpx
import pandas as pd
import wget
from ibm_cloud_sdk_core import IAMTokenManager
from sklearn.model_selection import train_test_split

Inferencing class

This cell defines a class that makes a REST API call to the watsonx Foundation Model inferencing API that we will use to generate output from the provided input. The class takes the access token created in the previous step, and uses it to make a REST API call with input, model id and model parameters. The response from the API call is returned as the cell output.

Action: Provide watsonx.ai Runtime url to work with watsonx.ai.

endpoint_url = input("Please enter your watsonx.ai Runtime endpoint url (hit enter): ")

Initialize the PromptClient class.

Hint: Your authentication token might expire, if so please regenerate the access_token and reinitialize the PromptClient class.

class PromptClient:
    def __init__(self, access_token: str, project_id: str, endpoint_url: str):
        self.project_id = project_id
        self.url = f"{endpoint_url.rstrip('/')}/ml/v1/text/chat"
        self.headers = {
            "Authorization": f"Bearer {access_token}",
            "Content-Type": "application/json",
        }

    def chat(self, model_id: str, messages: list[str], **params):
        payload = {
            "model_id": model_id,
            "messages": messages,
            "project_id": self.project_id,
            **params,
        }

        response = httpx.post(
            self.url,
            params={"version": "2024-03-19"},
            json=payload,
            headers=self.headers,
            timeout=30,
        )
        if response.status_code == 200:
            return response.json()
        else:
            raise RuntimeError(response.text)

watsonx API connection

This cell defines the credentials required to work with watsonx API for Foundation Model inferencing.

Action: Provide the IBM Cloud user API key. For details, see documentation.

access_token = IAMTokenManager(
    apikey=getpass.getpass("Please enter your watsonx.ai api key (hit enter): "),
    url="https://iam.cloud.ibm.com/identity/token",
).get_token()

Defining the project id

The API requires project id that provides the context for the call. We will obtain the id from the project in which this notebook runs. Otherwise, please provide the project id.

try:
    project_id = os.environ["PROJECT_ID"]
except KeyError:
    project_id = input("Please enter your project_id (hit enter): ")

Data loading

Download the car_rental_training_data dataset. The dataset provides insight about customers opinions on car rental. It has a label that consists of values: unsatisfied, satisfied.

car_rental_dataset = "car_rental_training_data.csv"

url = f"https://raw.githubusercontent.com/IBM/watsonx-ai-samples/master/cloud/data/cars-4-you/{car_rental_dataset}"

if not os.path.isfile(car_rental_dataset):
    wget.download(url, out=car_rental_dataset)

data = pd.read_csv(car_rental_dataset, sep=";")

Examine donwloaded data.

data.head()

Define label map.

label_map = {0: "unsatisfied", 1: "satisfied"}

Inspect data labels distribution.

data["Satisfaction"].value_counts()

Satisfaction
1    274
0    212
Name: count, dtype: int64

Prepare train and test sets.

data_train, data_test, y_train, y_test = train_test_split(
    data.Customer_Service,
    data.Satisfaction,
    test_size=0.3,
    random_state=33,
    stratify=data.Satisfaction,
)
data_train = pd.DataFrame(data_train)
data_test = pd.DataFrame(data_test)

data_train["satisfaction"] = list(map(label_map.get, y_train))
data_test["satisfaction"] = list(map(label_map.get, y_test))

Foundation Models on watsonx

List available chat models

models_json = httpx.get(
    endpoint_url + "/ml/v1/foundation_model_specs",
    headers={
        "Authorization": f"Bearer {access_token}",
        "Content-Type": "application/json",
        "Accept": "application/json",
    },
    params={
        "limit": 50,
        "version": "2024-03-19",
        "filters": "function_text_chat,!lifecycle_withdrawn:and",
    },
).json()

models_ids = [m["model_id"] for m in models_json["resources"]]
models_ids

['ibm/granite-3-2-8b-instruct',
 'ibm/granite-3-3-8b-instruct',
 'ibm/granite-3-3-8b-instruct-np',
 'ibm/granite-3-8b-instruct',
 'ibm/granite-4-h-small',
 'ibm/granite-guardian-3-8b',
 'meta-llama/llama-3-2-11b-vision-instruct',
 'meta-llama/llama-3-2-90b-vision-instruct',
 'meta-llama/llama-3-3-70b-instruct',
 'meta-llama/llama-3-405b-instruct',
 'meta-llama/llama-4-maverick-17b-128e-instruct-fp8',
 'meta-llama/llama-guard-3-11b-vision',
 'mistral-large-2512',
 'mistralai/mistral-medium-2505',
 'mistralai/mistral-small-3-1-24b-instruct-2503',
 'openai/gpt-oss-120b']

You need to specify model_id that will be used for inferencing from the list above:

model_id = "meta-llama/llama-3-3-70b-instruct"

Analyze the sentiment

Prepare model inputs - build zero-shot examples from the test set.

zero_shot_inputs = [text for text in data_test.Customer_Service.values]

print(json.dumps(zero_shot_inputs[:5], indent=2))

[
  "Provide more convenient car pickup from the airport parking.",
  "They could really try work harder.",
  "the rep was friendly but it was so loud in there that I could not hear what she was saying. I HATE having to walk across a big lot with all of my bags in search of my car which is always in the furthest corner.",
  "The agents were not friendly when I checked in initially, that was annoying because I had just spent 3 hours on a plane and wanted to be greeted with a better attitude.",
  "It was not as bad as it usually is."
]

Prepare model inputs - build few-shot examples. To build a few-shot example few instances of training data phrases are passed together with the reference sentiment and then appended with a test data phrase.

In this notebook, training phrases are stratified over all possible sentiments for each test case.

singleoutput = []
few_shot_inputs = []

for test_phrase in data_test.Customer_Service.values:
    for train_phrase, sentiment in (
        data_train.groupby("satisfaction", group_keys=False)
        .sample(2)[["Customer_Service", "satisfaction"]]
        .values
    ):
        singleoutput.append(
            f"\tsentence:\t{train_phrase}\n\tsatisfaction: {sentiment}\n"
        )

    singleoutput.append(f"\tsentence:\t{test_phrase}\n\tsatisfaction:")
    few_shot_inputs.append("".join(singleoutput))
    singleoutput = []

Inspect an exemplary few-shot prompt.

print(json.dumps(print(few_shot_inputs[0]), indent=2))

	sentence:	I never had to deal with customer service. The bus dropped me off at my car, when I returned the car, I had my receipt at that moment. I then got on the bus and left. Never having to wait in a line for customer service.
	satisfaction: satisfied
	sentence:	My last rental experience for leisure was fine -- no service issues to speak of.
	satisfaction: satisfied
	sentence:	I was penalty charged because vehicle was not returned with full tank.
	satisfaction: unsatisfied
	sentence:	slow and stupid
	satisfaction: unsatisfied
	sentence:	Provide more convenient car pickup from the airport parking.
	satisfaction:
null

Analyze the satisfaction using selected model.

Note: You might need to adjust model parameters for different models or tasks, to do so please refer to documentation.

Initialize the PromptClient class.

Hint: Your authentication token might expire, if so please regenerate the access_token reinitialize the PromptClient class.

prompt_client = PromptClient(access_token, project_id, endpoint_url)

Define instructions for the model and analyze the sentiment for a sample of zero-shot inputs from the test set.

results = []
instruction = "Classify the satisfaction expressed in this sentence using: satisfied, unsatisfied.\n"

for inp in zero_shot_inputs[:5]:
    results.append(
        prompt_client.chat(
            messages=[
                {"role": "system", "content": instruction},
                {"role": "user", "content": inp},
            ],
            model_id=model_id,
            guided_choice=["satisfied", "unsatisfied"],
        )
    )

Explore model output.

print(json.dumps(results, indent=2))

[
  {
    "id": "chatcmpl-84988f69ab3f5290f0949a699faaec55---de4206f8-2026-43ba-893a-0254ca5f6606",
    "object": "chat.completion",
    "model_id": "meta-llama/llama-3-3-70b-instruct",
    "model": "meta-llama/llama-3-3-70b-instruct",
    "choices": [
      {
        "index": 0,
        "message": {
          "role": "assistant",
          "content": "unsatisfied"
        },
        "finish_reason": "stop"
      }
    ],
    "created": 1768907136,
    "model_version": "3.3.0",
    "created_at": "2026-01-20T11:05:37.627Z",
    "usage": {
      "completion_tokens": 3,
      "prompt_tokens": 60,
      "total_tokens": 63
    },
    "system": {
      "warnings": [
        {
          "message": "This model is a Non-IBM Product governed by a third-party license that may impose use restrictions and other obligations. By using this model you agree to its terms as identified in the following URL.",
          "id": "disclaimer_warning",
          "more_info": "https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/fm-models.html?context=wx"
        },
        {
          "message": "The value of 'max_tokens' for this model was set to value 1024",
          "id": "unspecified_max_token",
          "additional_properties": {
            "limit": 0,
            "new_value": 1024,
            "parameter": "max_tokens",
            "value": 0
          }
        }
      ]
    }
  },
  {
    "id": "chatcmpl-f8943e11d2d58b9acfb20dac077e1c97---67bd8a4e-520d-4f5b-aeb4-f0579e81f9f0",
    "object": "chat.completion",
    "model_id": "meta-llama/llama-3-3-70b-instruct",
    "model": "meta-llama/llama-3-3-70b-instruct",
    "choices": [
      {
        "index": 0,
        "message": {
          "role": "assistant",
          "content": "unsatisfied"
        },
        "finish_reason": "stop"
      }
    ],
    "created": 1768907138,
    "model_version": "3.3.0",
    "created_at": "2026-01-20T11:05:39.076Z",
    "usage": {
      "completion_tokens": 3,
      "prompt_tokens": 57,
      "total_tokens": 60
    },
    "system": {
      "warnings": [
        {
          "message": "This model is a Non-IBM Product governed by a third-party license that may impose use restrictions and other obligations. By using this model you agree to its terms as identified in the following URL.",
          "id": "disclaimer_warning",
          "more_info": "https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/fm-models.html?context=wx"
        },
        {
          "message": "The value of 'max_tokens' for this model was set to value 1024",
          "id": "unspecified_max_token",
          "additional_properties": {
            "limit": 0,
            "new_value": 1024,
            "parameter": "max_tokens",
            "value": 0
          }
        }
      ]
    }
  },
  {
    "id": "chatcmpl-47ac124fc031aba780e7542c75ed3f07---f2d6f8f8-b4c5-400c-a242-f780c6311a79",
    "object": "chat.completion",
    "model_id": "meta-llama/llama-3-3-70b-instruct",
    "model": "meta-llama/llama-3-3-70b-instruct",
    "choices": [
      {
        "index": 0,
        "message": {
          "role": "assistant",
          "content": "unsatisfied"
        },
        "finish_reason": "stop"
      }
    ],
    "created": 1768907139,
    "model_version": "3.3.0",
    "created_at": "2026-01-20T11:05:40.718Z",
    "usage": {
      "completion_tokens": 3,
      "prompt_tokens": 100,
      "total_tokens": 103
    },
    "system": {
      "warnings": [
        {
          "message": "This model is a Non-IBM Product governed by a third-party license that may impose use restrictions and other obligations. By using this model you agree to its terms as identified in the following URL.",
          "id": "disclaimer_warning",
          "more_info": "https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/fm-models.html?context=wx"
        },
        {
          "message": "The value of 'max_tokens' for this model was set to value 1024",
          "id": "unspecified_max_token",
          "additional_properties": {
            "limit": 0,
            "new_value": 1024,
            "parameter": "max_tokens",
            "value": 0
          }
        }
      ]
    }
  },
  {
    "id": "chatcmpl-01efaf952d3c90487206eb1f55a363e1---e1a5efe6-501a-41db-89e4-616a40da9e1b",
    "object": "chat.completion",
    "model_id": "meta-llama/llama-3-3-70b-instruct",
    "model": "meta-llama/llama-3-3-70b-instruct",
    "choices": [
      {
        "index": 0,
        "message": {
          "role": "assistant",
          "content": "unsatisfied"
        },
        "finish_reason": "stop"
      }
    ],
    "created": 1768907141,
    "model_version": "3.3.0",
    "created_at": "2026-01-20T11:05:42.251Z",
    "usage": {
      "completion_tokens": 3,
      "prompt_tokens": 85,
      "total_tokens": 88
    },
    "system": {
      "warnings": [
        {
          "message": "This model is a Non-IBM Product governed by a third-party license that may impose use restrictions and other obligations. By using this model you agree to its terms as identified in the following URL.",
          "id": "disclaimer_warning",
          "more_info": "https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/fm-models.html?context=wx"
        },
        {
          "message": "The value of 'max_tokens' for this model was set to value 1024",
          "id": "unspecified_max_token",
          "additional_properties": {
            "limit": 0,
            "new_value": 1024,
            "parameter": "max_tokens",
            "value": 0
          }
        }
      ]
    }
  },
  {
    "id": "chatcmpl-e4998af14bbf2083eac4fe103e76ca70---5c1e119a-48bd-4508-b589-9ed6f09efa41",
    "object": "chat.completion",
    "model_id": "meta-llama/llama-3-3-70b-instruct",
    "model": "meta-llama/llama-3-3-70b-instruct",
    "choices": [
      {
        "index": 0,
        "message": {
          "role": "assistant",
          "content": "satisfied"
        },
        "finish_reason": "stop"
      }
    ],
    "created": 1768907142,
    "model_version": "3.3.0",
    "created_at": "2026-01-20T11:05:43.333Z",
    "usage": {
      "completion_tokens": 3,
      "prompt_tokens": 60,
      "total_tokens": 63
    },
    "system": {
      "warnings": [
        {
          "message": "This model is a Non-IBM Product governed by a third-party license that may impose use restrictions and other obligations. By using this model you agree to its terms as identified in the following URL.",
          "id": "disclaimer_warning",
          "more_info": "https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/fm-models.html?context=wx"
        },
        {
          "message": "The value of 'max_tokens' for this model was set to value 1024",
          "id": "unspecified_max_token",
          "additional_properties": {
            "limit": 0,
            "new_value": 1024,
            "parameter": "max_tokens",
            "value": 0
          }
        }
      ]
    }
  }
]

Score the model

Note: To run the Score section for model scoring on the whole car rental customer satisfaction dataset please transform following markdown cells to code cells. Have in mind that scoring model on the whole test set can take significant amount of time.

Get the true labels.

y_true = [label for label in data_test.satisfaction[:5]]
y_true

Get the sentiment labels returned by the selected model.

y_pred = [res["choices"][0]["message"]["content"] for res in results]
y_pred

Calculate the accuracy score.

from sklearn.metrics import accuracy_score

print(accuracy_score(y_pred, y_true))

Summary and next steps

You successfully completed this notebook!

You learned how to analyze car rental customer satisfaction with LLM on watsonx.

Check out our Online Documentation for more samples, tutorials, documentation, how-tos, and blog posts.

Authors

Mateusz Szewczyk, Software Engineer at watsonx.ai.

Copyright © 2026 IBM. This notebook and its source code are released under the terms of the MIT License.