GitHub Repository: ibm/watson-machine-learning-samples
Path: blob/master/cloud/notebooks/rest_api/deployments/foundation_models/Use watsonx, and Google `flan-t5-xxl` to generate advertising.ipynb
⁶⁴⁰⁵ views

Kernel: Python 3 (ipykernel)

Use watsonx, and google `flan-t5-xxl` to generate advertising

Disclaimers

Use only Projects and Spaces that are available in watsonx context.

Notebook content

This notebook contains the steps and code to demonstrate support of text generation in watsonx. It introduces commands for data generation and model testing.

Some familiarity with Python is helpful. This notebook uses Python 3.11.

Learning goal

The goal of this notebook is to demonstrate how to use flan-t5-xxl model to generate mail advertising

This notebook contains the following parts:

Set up the environment

Before you use the sample code in this notebook, you must perform the following setup tasks:

Create a watsonx.ai Runtime Service instance (a free plan is offered and information about how to create the instance can be found here).

Install and import the `datasets` and dependecies

In [ ]:

!pip install requests | tail -n 1
!pip install wget | tail -n 1
!pip install ibm-cloud-sdk-core | tail -n 1
!pip install wget | tail -n 1

In [1]:

import os, getpass, json
import requests
from ibm_cloud_sdk_core import IAMTokenManager

Inferencing class

This cell defines a class that makes a REST API call to the watsonx Foundation Model inferencing API that we will use to generate output from the provided input. The class takes the access token created in the previous step, and uses it to make a REST API call with input, model id and model parameters. The response from the API call is returned as the cell output.

Action: Provide watsonx.ai Runtime url to work with watsonx.ai.

In [2]:

endpoint_url = input("Please enter your watsonx.ai Runtime endpoint url (hit enter): ")

Define a Prompt class for prompts generation.

In [3]:

class Prompt:
    def __init__(self, access_token, project_id):
        self.access_token = access_token
        self.project_id = project_id

    def generate(self, input, model_id, parameters):
        wml_url = f"{endpoint_url}/ml/v1/text/generation?version=2024-03-19"
        Headers = {
            "Authorization": "Bearer " + self.access_token,
            "Content-Type": "application/json",
            "Accept": "application/json"
        }
        data = {
            "model_id": model_id,
            "input": input,
            "parameters": parameters,
            "project_id": self.project_id
        }
        response = requests.post(wml_url, json=data, headers=Headers)
        if response.status_code == 200:
            return response.json()["results"][0]
        else:
            return response.text

watsonx API connection

This cell defines the credentials required to work with watsonx API for Foundation Model inferencing.

Action: Provide the IBM Cloud user API key. For details, see documentation.

In [4]:

access_token = IAMTokenManager(
    apikey = getpass.getpass("Please enter your watsonx.ai api key (hit enter): "),
    url = "https://iam.cloud.ibm.com/identity/token"
).get_token()

Defining the project id

The API requires project id that provides the context for the call. We will obtain the id from the project in which this notebook runs. Otherwise, please provide the project id.

In [5]:

try:
    project_id = os.environ["PROJECT_ID"]
except KeyError:
    project_id = getpass.getpass("Please enter your project_id (hit enter): ")

Advertising prompts

Prepare advertising prompts.

Prepare credit card with cashback advertising.

In [6]:

ads = []

ads.append("Generate banking advertising mail of a credit card with 10% cashback.")

Prepare savings account advertising.

In [7]:

ads.append("Generate banking advertising mail of a saving account with 8 percent interest.")

Prepare educational course advertising.

In [8]:

ads.append("Generate advertising mail of an online JAVA programming course.")

Prepare photo editing software advertising.

In [9]:

ads.append("Generate advertising mail of paid (10$ a month) photo editing software for professionals.")

Prepare healthy food catering advertising.

In [10]:

ads.append("Generate advertising mail of healthy food catering with a free first meal.")

Foundation Models on watsonx

List available models

In [11]:

models_json = requests.get(endpoint_url + '/ml/v1/foundation_model_specs?version=2024-03-19&limit=50',
                           headers={
                                    'Authorization': f'Bearer {access_token}',
                                    'Content-Type': 'application/json',
                                    'Accept': 'application/json'
                            }).json()
models_ids = [m['model_id'] for m in models_json['resources']]
models_ids

Out[11]:

['bigcode/starcoder',
 'bigscience/mt0-xxl',
 'codellama/codellama-34b-instruct-hf',
 'eleutherai/gpt-neox-20b',
 'google/flan-t5-xl',
 'google/flan-t5-xxl',
 'google/flan-ul2',
 'ibm-mistralai/mixtral-8x7b-instruct-v01-q',
 'ibm/granite-13b-chat-v1',
 'ibm/granite-13b-chat-v2',
 'ibm/granite-13b-instruct-v1',
 'ibm/granite-13b-instruct-v2',
 'ibm/granite-20b-multilingual',
 'ibm/mpt-7b-instruct2',
 'meta-llama/llama-2-13b-chat',
 'meta-llama/llama-2-70b-chat']

You need to specify model_id that will be used for inferencing:

In [12]:

model_id = "google/flan-t5-xxl"

Generate mail advertising

Defining the model parameters

We need to provide a set of model parameters that will influence the result:

In [13]:

parameters =  {"decoding_method": "sample",
               "temperature": 0.3,
               "top_p": 1,
               "top_k": 50,
               "random_seed": 33,
               "repetition_penalty": 2,
               "min_new_tokens": 50,
               "max_new_tokens": 1024
}

Generate mail advertising using `flan-t5-xxl` model.

Note: You might need to adjust model parameters for different models or tasks, to do so please refer to documentation.

Initialize the Prompt class.

Hint: Your authentication token might expire, if so please regenerate the access_token and reinitialize the Prompt class.

In [14]:

prompt = Prompt(access_token, project_id)

Get the docs summaries.

In [15]:

results_low_temperature = []

for instruction in ads:
    results_low_temperature.append(prompt.generate(instruction, model_id, parameters))

Explore model outputs.

In [16]:

print(json.dumps(results_low_temperature, indent=2))

Out[16]:

[
  {
    "generated_text": "The Bank of America Cash Rewards Credit Card offers a 10% cash back on all purchases. Earn points for every $1 you spend. Use your rewards to get travel, merchandise, and gift cards. Get rewarded for shopping online, at the gas pump, or anywhere else with a Visa credit card.",
    "generated_token_count": 62,
    "input_token_count": 16,
    "stop_reason": "eos_token",
    "seed": 33
  },
  {
    "generated_text": "a saving account with 8 percent interest. if you're interested, please call us at 1-800-447-2337 or visit any of our banking centers. id#123456. cc#5feb3d",
    "generated_token_count": 53,
    "input_token_count": 16,
    "stop_reason": "eos_token",
    "seed": 33
  },
  {
    "generated_text": "Dear Sir/Madam, We are offering an online JAVA programming course. The course is designed by a team of experienced and qualified professionals. You can learn to program in Java and get certified in the process. Please visit our website for more information. Thank you,",
    "generated_token_count": 58,
    "input_token_count": 14,
    "stop_reason": "eos_token",
    "seed": 33
  },
  {
    "generated_text": "i have been using photoshop for years and it is the most powerful, yet easy to use program i have ever used. however, i am always looking for something better. i am not sure if you are interested in trying out my new software, but if you are, please let me know. it is called \"photoscape\" and it is the best thing i have ever bought. i think you will love it. i have attached a few screen shots of what it can do. i hope you like it. please feel free to contact me if you have any questions. i would be happy to help. i look forward to hearing from you.",
    "generated_token_count": 146,
    "input_token_count": 19,
    "stop_reason": "eos_token",
    "seed": 33
  },
  {
    "generated_text": "Healthy Food Catering is a new healthy food catering company. We are offering a free first meal to anyone who signs up for our service. For more information, please visit our website at www.healthyfoodcatering.com. Thank you for your time.",
    "generated_token_count": 55,
    "input_token_count": 16,
    "stop_reason": "eos_token",
    "seed": 33
  }
]

You might also try to change model parameters to see whether it provides better ads. In following cell we changed the model temperature to 1.0 to let the model select words more creatively.

In [17]:

parameters_temp =  {"decoding_method": "sample",
                    "temperature": 1.0,
                    "top_p": 1,
                    "top_k": 50,
                    "random_seed": 33,
                    "repetition_penalty": 2,
                    "min_new_tokens": 50,
                    "max_new_tokens": 1024
}

In [18]:

results_high_temperature = []

for instruction in ads:
    results_high_temperature.append(prompt.generate(instruction, model_id, parameters_temp))

Explore model outputs.

In [19]:

print(json.dumps(results_high_temperature, indent=2))

Out[19]:

[
  {
    "generated_text": "From now on, enjoy some amazing advantages with Sainsbury's Premier 1 Cashback Card! Redeem your card at any one of our 200+ retailers and get 10% cash back! You need to apply for this incredible offer here. Don't delay",
    "generated_token_count": 53,
    "input_token_count": 16,
    "stop_reason": "eos_token",
    "seed": 33
  },
  {
    "generated_text": "From the Credit Union Board of Directors: All members who have been with our credit union for a year or more are eligible for an 8.00% interest checking account. To open your account, stop by any of our locations, apply on-line at www.rbcusa.com or by calling 1-800-457-5555. For over 100 years we've made banking easy.",
    "generated_token_count": 81,
    "input_token_count": 16,
    "stop_reason": "eos_token",
    "seed": 33
  },
  {
    "generated_text": "From the makers of JAVA: \"Avail yourself of our online course and learn the programming language from a pro-teacher and the best technical team on the web\". We have now launched an improved version of JAVA that has been designed by experts, with interactive lessons, assignments and quizzes to keep you in touch with JAVA as well as the latest enhancements in Oracle Java SE 6 Beta. The new java has many exciting features that I will describe to you but the most important one is security and support for future technologies. This version is the last release of JAVA 5.x and is intended for those JAVA programmers who have already become adept at using the new language and would like to update their knowledge. To enroll in this upgrade, please email : sales3(at)iknowitall (dot) com with your request. You can expect to receive the confirmation by return email. For more details on upgrade, click here : Upgrade page.",
    "generated_token_count": 209,
    "input_token_count": 14,
    "stop_reason": "eos_token",
    "seed": 33
  },
  {
    "generated_text": "Hi, The free version of our photo editor is great. But most people want to edit their photos in a professional way. So we decided to introduce you a photo editing software called FX Photo Studio Pro for Windows, which requires that you order an installation disc, if you do not have one yet.",
    "generated_token_count": 65,
    "input_token_count": 19,
    "stop_reason": "eos_token",
    "seed": 33
  },
  {
    "generated_text": "From the makers of Fresh-n-Easy, now available in your area: The new Fresh-n-Easy Healthy Catering line is an evolution in quality fresh food. Our healthy meals taste great, are delivered to you on time and leave a positive impact on the environment by using green options that are in season year round. For free, get your first meal. For us, we believe eating well should be affordable, too. Call 1-877-696-0999 today for more details!",
    "generated_token_count": 106,
    "input_token_count": 16,
    "stop_reason": "eos_token",
    "seed": 33
  }
]

You can also try changing the prompt and/or parameters to generate more detailed advertising. In the next cell, the photo editing software advertising prompt was transformed to provide more accurate results.

In [20]:

photo_software_prompt = "Write an advertising e-mail of paid (9$ a month, first month free) photo editing software COOLSOFTWARE targeted for early professionals."

In [21]:

parameters_photo =  {"decoding_method": "sample",
                    "temperature": 1.0,
                    "top_p": 1,
                    "top_k": 50,
                    "random_seed": 33,
                    "repetition_penalty": 2,
                    "min_new_tokens": 200,
                    "max_new_tokens": 1024
}

In [22]:

results_photo_soft = prompt.generate(photo_software_prompt, model_id, parameters_photo)

In [23]:

print(json.dumps(results_photo_soft, indent=2))

Out[23]:

{
  "generated_text": "Cool Software is dedicated to helping photographers worldwide make the most of their photo-editing software investment by providing high quality, easy to use and affordable graphics software. Whether you're an amateur or professional photographer in need of new tools to enhance your work, our highly advanced, easy-to-use photo editing programs will let you see your creative vision come to life. Isn't it time you had a cool piece of professional software too? Well, at $8.99 each month (free for first month) Cool Software seems to fit that bill perfectly. This is the best part about our company: We only offer support through email. We haven't tossed your photos out to fend for themselves in a year because our program didn't save enough time or effort for the photo. At Cool Software we don't want to give you \"quick and dirty\" editing features either. Instead we want the very best in software, from one application to another. That means using different applications across the board to make your pictures look perfect. With that in mind we offer you a wide range of choices, not just from what works well now but what has worked well in the past. As well as other applications to fit your photography. You have the free trail version, so what better way to try before you buy than to test drive Cool Software. When you sign up simply send us an e-mail, and choose from our variety of products. It's that easy.",
  "generated_token_count": 308,
  "input_token_count": 34,
  "stop_reason": "eos_token",
  "seed": 33
}

Summary and next steps

You successfully completed this notebook!

You learned how to generate mail advertising with Google's flan-t5-xxl on watsonx.

Check out our Online Documentation for more samples, tutorials, documentation, how-tos, and blog posts.

Authors

Mateusz Szewczyk, Software Engineer at watsonx.ai.

Use watsonx, and google `flan-t5-xxl` to generate advertising

Disclaimers

Notebook content

Learning goal

Contents

Set up the environment

Install and import the `datasets` and dependecies

Inferencing class

watsonx API connection

Defining the project id

Advertising prompts

Prepare advertising prompts.

Foundation Models on watsonx

List available models

Generate mail advertising

Defining the model parameters

Generate mail advertising using `flan-t5-xxl` model.

Summary and next steps

Authors

Product

Resources

Company

Use watsonx, and google flan-t5-xxl to generate advertising

Disclaimers

Notebook content

Learning goal

Contents

Set up the environment

Install and import the datasets and dependecies

Inferencing class

watsonx API connection

Defining the project id

Advertising prompts

Prepare advertising prompts.

Foundation Models on watsonx

List available models

Generate mail advertising

Defining the model parameters

Generate mail advertising using flan-t5-xxl model.

Summary and next steps

Authors

Use watsonx, and google `flan-t5-xxl` to generate advertising

Install and import the `datasets` and dependecies

Generate mail advertising using `flan-t5-xxl` model.