Use watsonx and BigCode `starcoder-15.5b` to generate code based on instruction.ipynb

Kernel: rt241

Use watsonx and BigCode starcoder-15.5b to generate code based on instruction

Disclaimers

• Use only Projects and Spaces that are available in watsonx context.

Notebook content

This notebook contains the steps and code to demonstrate support for code generating in Watsonx. It introduces commands for defining prompt and model testing.

Some familiarity with Python is helpful. This notebook uses Python 3.11.

Learning goal

The goal of this notebook is to demonstrate how to generate code using BigCode starcoder-15.5b Watsonx model based on instruction provided by the user.

Contents

This notebook contains the following parts:

• Setup

• Foundation Models on Watsonx

• Generate code based on instruction

• Generated code testing

• Summary

Set up the environment

Before you use the sample code in this notebook, you must perform the following setup tasks:

• Contact with your Cloud Pack for Data administrator and ask him for your account credentials

Install and import the ibm-watsonx-ai and dependecies

Note: ibm-watsonx-ai documentation can be found here.

!pip install -U ibm-watsonx-ai | tail -n 1

Connection to WML

Authenticate the Watson Machine Learning service on IBM Cloud Pack for Data. You need to provide platform url, your username and api_key.

username = 'PASTE YOUR USERNAME HERE'
api_key = 'PASTE YOUR API_KEY HERE'
url = 'PASTE THE PLATFORM URL HERE'

from ibm_watsonx_ai import Credentials

credentials = Credentials(
    username=username,
    api_key=api_key,
    url=url,
    instance_id="openshift",
    version="5.0"
)

Alternatively you can use username and password to authenticate WML services.

credentials = Credentials(
    username=***,
    password=***,
    url=***,
    instance_id="openshift",
    version="5.0"
)

Defining the project id

The Foundation Model requires project id that provides the context for the call. We will obtain the id from the project in which this notebook runs. Otherwise, please provide the project id.

import os

try:
    project_id = os.environ["PROJECT_ID"]
except KeyError:
    project_id = input("Please enter your project_id (hit enter): ")

Foundation Models on watsonx.ai

List available models

All avaliable models are presented under ModelTypes class. For more information refer to documentation.

from ibm_watsonx_ai.foundation_models.utils.enums import ModelTypes

print([model.name for model in ModelTypes])

['FLAN_T5_XXL', 'FLAN_UL2', 'MT0_XXL', 'GPT_NEOX', 'MPT_7B_INSTRUCT2', 'STARCODER', 'LLAMA_2_70B_CHAT', 'LLAMA_2_13B_CHAT', 'GRANITE_13B_INSTRUCT', 'GRANITE_13B_CHAT', 'FLAN_T5_XL', 'GRANITE_13B_CHAT_V2', 'GRANITE_13B_INSTRUCT_V2', 'ELYZA_JAPANESE_LLAMA_2_7B_INSTRUCT', 'MIXTRAL_8X7B_INSTRUCT_V01_Q', 'CODELLAMA_34B_INSTRUCT_HF', 'GRANITE_20B_MULTILINGUAL', 'MERLINITE_7B', 'GRANITE_20B_CODE_INSTRUCT', 'GRANITE_34B_CODE_INSTRUCT', 'GRANITE_3B_CODE_INSTRUCT', 'GRANITE_7B_LAB', 'GRANITE_8B_CODE_INSTRUCT', 'LLAMA_3_70B_INSTRUCT', 'LLAMA_3_8B_INSTRUCT', 'MIXTRAL_8X7B_INSTRUCT_V01']

You need to specify model_id that will be used for inferencing:

model_id = ModelTypes.STARCODER

Defining the model parameters

You might need to adjust model parameters for different models or tasks, to do so please refer to documentation.

from ibm_watsonx_ai.metanames import GenTextParamsMetaNames as GenParams

parameters = {
    GenParams.DECODING_METHOD: "greedy",
    GenParams.MAX_NEW_TOKENS: 100,
    GenParams.STOP_SEQUENCES: ["<end·of·code>"]
}

Initialize the model

Initialize the Model class with previous set params.

from ibm_watsonx_ai.foundation_models import ModelInference

model = ModelInference(
    model_id=model_id, 
    params=parameters, 
    credentials=credentials,
    project_id=project_id)

Model's details

model.get_details()

Generate code based on instruction

Define instructions for the model with at-least one example.

instruction = """Using the directions below, generate Python code for the given task.

Input:
# Write a Python function that prints 'Hello World!' string 'n' times.

Output:
def print_n_times(n):
    for i in range(n):
        print("Hello World!")

<end of code>
"""

Prepare question for the model.

question = """Input:
# Write a Python function, which generates sequence of prime numbers.
# The function 'primes' will take the argument 'n', an int. It will return a list which contains all primes less than 'n'."""

Generat the code using BigCode starcoder-15.5b model.

Inter the model to generate the code, according to provided instruction.

result = model.generate_text(" ".join([instruction, question]))

Formatting the text to get the function itself

code_as_text = result.split('Output:')[1].split('<end of code>')[0]

Generated code testing

The resulting code looks as below.

print(code_as_text)

def primes(n):
    primes = []
    for i in range(2, n):
        for j in range(2, i):
            if i % j == 0:
                break
        else:
            primes.append(i)
    return primes

Use generated code to make it as function.

exec(code_as_text)

Define the number 'n' for which the primes() function should process prime numbers.

n = 25

Test and run the generated function.

primes(n)

[2, 3, 5, 7, 11, 13, 17, 19, 23]

Summary and next steps

You successfully completed this notebook!.

You learned how to generate code based on instuction with BigCode starcoder-15.5b on watsonx.ai.

Check out our Online Documentation for more samples, tutorials, documentation, how-tos, and blog posts.

Authors:

Mateusz Szewczyk, Software Engineer at Watson Machine Learning.

Copyright © 2023-2025 IBM. This notebook and its source code are released under the terms of the MIT License.