CPDaaS-04 Services.ipynb

Kernel: Python 3.10

Cloud Pak for Data as a Service services management

Make sure to set the API_key variable in the next cell before executing it

This notebook covers:

• Get a partial list of the services in the catalog

• List the services (resources) in the account

• Create a service

• Adding a service to the current project

• Removing the service from the project

• Deleting the service

References:

• Watson Data API

• Resource Controller API

import requests
import json
import warnings
import os, sys
from datetime import datetime
import inspect

import zipfile
from io import BytesIO

API_key = "<Insert API key here>" # from cloud.ibm.com...

# Should not use verify=False but I don't want to deal with SSL
verify=False
warnings.filterwarnings("ignore") # one of "error", "ignore", "always", "default", "module", or "once"

Support functions

instructions:

• Select the next empty cell as the current cell

• From the code snipet '</>' tab on the right, use Read data, Select data from project

• Then click on Data asset, and finally on cpdalllibs.zip

• Make sure the "Load as" selection is set to StreamingBody object, and click on Insert code to cell

• Make sure the inserted code references 'streaming_body_1' in a line like:

streaming_body_1 = cos_client.get_object(Bucket=bucket, Key=object_key)['Body']

# Load the python support functions
!rm -rf cpdalllibs
myzip = zipfile.ZipFile(BytesIO(streaming_body_1.read()))

myzip.extractall('.')

sys.path.append(".")
from cpdalllibs.cpdaaslibfns import *
importcpdaas()

# Test if we have access
help(getUsers)
print("\nShow the source of a function:\n")
print(inspect.getsource(getRoles))

Get an access token

resp = getToken(API_key)
if resp.status_code > 200 :
    print("getToken status code: {}, reason: {}".format(resp.status_code,resp.reason))
resp_json = resp.json()
access_token = resp_json['access_token']
print("Got a token at {} GMT".format(datetime.now().time().isoformat("seconds")))

# Header to use in subsequent queries
headersAPI = {
        'accept': 'application/json',
        'Content-type': 'application/json',
        'Authorization': 'Bearer ' + access_token,
        'cache-control': 'no-cache'
}
# Get the detail to extract the account_id
resp = apikeyDetails(API_key, access_token)
if resp.status_code > 200 :
    print("apikeyDetails status code: {}, reason: {}".format(resp.status_code,resp.reason))
key_details_json = resp.json()

account_id = key_details_json['account_id']
iam_id = key_details_json['iam_id']

Get a partial list of the services in the catalog

To get a list of all services, we could use a library but since we're not going deep into the catalog entries,it does not save us any work. Here is how we can do the same as above using the library:

!pip install --upgrade ibm-platform-services >/dev/null 2>&1
from ibm_platform_services import global_catalog_v1 
from ibm_cloud_sdk_core.authenticators.no_auth_authenticator import NoAuthAuthenticator

service = global_catalog_v1.GlobalCatalogV1(authenticator=NoAuthAuthenticator() )
resp = service.list_catalog_entries(limit=50, q="tag:watson kind:service")
res = resp.get_result()
print("Number of entries: {}\n".format(resp_json['count']))
print("{:<30}  {:<35}   {}".format("Display Name", "Entry Name","ID"))
print("\n".join(["{:<30}: {:<35} | {}".format(entry['overview_ui']['en']['display_name'],entry['name'],entry['id']) 
                     for entry in res['resources']]) )

We can get a maximum of 200 entries for each call. If we expect more, we need to loop using the offset parameter. Using the library, the code would be:

offset = 0
limit = 200 # max available limit
looping = True
print("{:<30}  {:<35}   {}".format("Display Name", "Entry Name","ID"))
while looping :
    resp = service.list_catalog_entries(offset=offset,limit=limit, q="tag:watson kind:service")
    res = resp.get_result()
    offset = offset + limit
    print("\n".join(["{:<30}: {:<35} | {}".format(entry['overview_ui']['en']['display_name'],entry['name'],entry['id']) 
                     for entry in res['resources']]) )
    if (offset > res['count']):
        looping = False

resp = listGlobalCatalog(headersAPI, 200)
if resp.status_code > 202 :
    print("Status code: {}, reason: {}\n".format(resp.status_code,resp.reason))
else :
    resp_json = resp.json()
    resources_json = [item for item in resp_json['resources'] ]
    found_names = [item['name'] for item in resources_json]

    print("Number of entries: {}".format(len(resources_json)))
    print("{:<30}  {:<35}   {}".format("Display Name", "Entry Name","ID"))
    print("\n".join(["{:<30} | {:<35} | {}".format(entry['overview_ui']['en']['display_name'],entry['name'],entry['id']) 
                         for entry in resources_json]) )

List one complete entry

#list one entry: Machine Learning
entry = [entry for entry in resp_json['resources'] if entry['name'] == "text-to-speech"][0]
print(json.dumps(entry, indent=2, sort_keys=True))

List the services (resources) in the account

# List resources (services) in the account
instances_json = getServiceInstances(headersAPI)
print("Number of instances: {}".format(len(instances_json['resources'])))
# print(json.dumps(instances_json, indent=2, sort_keys=False))
print("{:<24} | {}".format("Service name", "guid"))
print("\n".join(sorted((["{:<24} | {}".format(item['name'], item['guid']) for item in instances_json['resources']])) ))

Create a service

Display a summary of the available plans

# we have the resources_json global variable created previously.

# We need to find the info on the "service". Get the entry from resources_json
resource = [item for item in resources_json if item['name'] == 'text-to-speech'][0]
# Get the resource id so we can retrieve the child entries
resource_id = resource['id']
print("{}: {}\n".format(resource['name'],resource_id))

# Get the information the service and pick the resource_plan_id: GET /{id}/{kind}
# We call al kinds but in this case, we could limit ourselves to 'plan' since openscale only has plans
resp = requests.get(GLOBAL_CATALOG_ENDPOINT + '/{}/{}?include=metadata'.format(resource_id,'*'), headers=headersAPI)
if resp.status_code > 202 : # if error
    print("Status code: {}, reason: {}\n".format(resp.status_code,resp.reason))
    print(json.dumps(resp_json, indent=2, sort_keys=True))
else :
    resource_json = resp.json()
    print("{:<36} | {:<12} | {}".format("Plan id", "Plan name", "Available geos"))
    print("\n".join(["{} | {:<12} | {}".format(item['id'], item['name'],",".join(item['geo_tags'])) 
                 for item in resource_json['resources'] if item['kind'] == "plan"]))

Get available resource groups

There is at least the "default" group

# List the resources groups
resp = requests.get(RESOURCE_ENDPOINT + '/v2/resource_groups', headers=headersAPI)
if resp.status_code > 202 : # if error
    print("Status code: {}, reason: {}\n".format(resp.status_code,resp.reason))
else :
    resources_groups_json = resp.json()
    # print(json.dumps(resources_groups_json, indent=2, sort_keys=True))
    group_names = [item['name'] for item in resources_groups_json['resources']]
    print("\n".join(group_names))

Create the service

# Extract the plan ID we want from the aiopenscale resources (see the display summary cell above)
plan_id = [item['id'] for item in resource_json['resources'] if item['name'] == 'lite'][0]
# Same for the resource group
resource_group = [item['id'] for item in resources_groups_json['resources'] if item['name'] == 'default'][0]

payload = {
    'name': "text to speech az", # What we want to name our service
    'target': "us-south", # One value from the available geos
    'resource_group': resource_group, # Resource group ID for Outcomes Project Management
    'resource_plan_id': plan_id, # Standard
    'tags': ['demo', 'temp']
}

resp = requests.post(RESOURCE_ENDPOINT + '/v2/resource_instances', json=payload, headers=headersAPI)
if resp.status_code > 202 : # if error
    print("Status code: {}, reason: {}\n".format(resp.status_code,resp.reason))
else :
    print("{} created:".format(payload['name']))
T2S_svc = resp.json()
print(json.dumps(T2S_svc, indent=2, sort_keys=True))

Adding a service to the current project

WATSON_DATA_ENDPOINT="https://api.dataplatform.cloud.ibm.com"
# We need the current compute as to not lose associated services like WML
resp = getProject(headersAPI, os.environ["PROJECT_ID"])
prj_json=resp.json()
compute = prj_json['entity']['compute']
compute.append({
    # type allowable values: [analytics_engine,spark,machine_learning,streaming_analytics,watson,data_replication]
    'type': "analytics_engine", # From doc 
    'guid': T2S_svc['guid'],
    'name': T2S_svc['name'],
    'credentials': {},
    'crn': T2S_svc['crn'] # optional
})
payload = {
    'compute': compute
}
resp = requests.patch(WATSON_DATA_ENDPOINT + '/v2/projects/{}'.format(os.environ['PROJECT_ID']), 
                      json=payload, headers=headersAPI)
if resp.status_code > 204 : # if error
    print("Status code: {}, reason: {}\n".format(resp.status_code,resp.reason))

Listing the project services

resp = getProject(headersAPI, os.environ["PROJECT_ID"])
print("Status code: {}, reason: {}\n".format(resp.status_code,resp.reason))
prj_json=resp.json()
print("\n".join(["{} - {}".format(item['type'], item['name']) for item in prj_json['entity']['compute']]))

Removing the language translator service

• Remove the service from the project

• Delete the service IDs and the service

• Delete the service

Remove the service from the project

Removing is like adding a list minus the elements we remove

resp = getProject(headersAPI, os.environ["PROJECT_ID"])
prj_json=resp.json()
compute = [item for item in prj_json['entity']['compute'] if item['name'] != "text to speech az"]

payload = {
    'compute': compute
}
resp = requests.patch(WATSON_DATA_ENDPOINT + '/v2/projects/{}'.format(os.environ['PROJECT_ID']), 
                      json=payload, headers=headersAPI)
if resp.status_code > 204 : # if error
    print("Status code: {}, reason: {}\n".format(resp.status_code,resp.reason))

# Prove that the service is gone from the project
resp = getProject(headersAPI, os.environ["PROJECT_ID"])
prj_json=resp.json()
print("\n".join(["{} - {}".format(item['type'], item['name']) for item in prj_json['entity']['compute']]))

Delete the service

resp = requests.delete(RESOURCE_ENDPOINT + '/v2/resource_instances/{}?recursive=true'.format(T2S_svc["guid"]), headers=headersAPI)
if resp.status_code > 204 :
    print("Status code: {}, reason: {}\n".format(resp.status_code,resp.reason))
else :
    print("'{}' deleted".format(T2S_svc['name']))

Author

Jacques Roy is a member of the IBM Enablement for Data and AI

Copyright © 2023. This notebook and its source code are released under the terms of the MIT License.