Provisioning DSVM.ipynb

Kernel: Python 3.6

How To: Provisioning Data Science Virtual Machine (DSVM)

Notebook Version: 1.0
Python Version: Python 3.6 (including Python 3.6 - AzureML)
Required Packages: azure 4.0.0, azure-cli-profile 2.1.4
Platforms Supported:
- Azure Notebooks Free Compute - Azure Notebooks DSVM Data Source Required:
- no

Description

The sample notebook shows how to provision a Azure DSVM as an alternate computing resource for hosting Azure Notebooks.

Azure Notebooks provides Free Compute as the default computing resource, which is free of charge. However, sometimes you do want to have a powerful computing environment, and you don't want to go through Direct Compute route which requires JupyterHub installation on Linux machines, then Data Science Virtual Machine (DSVM) becomes a vital choice.

Table of Contents

1. How to create a new DSVM

2. How to use DSVM

3. Things to know about using DSVM

1. How to create a new DSVM

Using the Azure Portal

Follow this article for details.

You should select Linux Ubuntu DSVM. And keep in mind that on Azure DSVM, if you want to use Python 3.6 which is required by Microsoft Sentinel notebooks, you need to select Python 3.6 - AzureML.

Using the Azure CLI

# only run once
!pip install --upgrade Azure-Sentinel-Utilities

# please enter your tenant domain below, for Microsoft, using: microsoft.onmicrosoft.com
!az login --tenant ''

# User Input for creating a new DSVM
vm_size = 'Standard_DS3_v2'

# replace [[your_subcription_id]] with 'real subscription id'
!az account set --subscription [[your_subcription_id]]

# replace all [[your_stuff]] with 'real values'
!az group deployment create \
    --resource-group [[your_subcription_id]] \
    --template-uri https://raw.githubusercontent.com/Azure/DataScienceVM/master/Scripts/CreateDSVM/Ubuntu/azuredeploy.json \
    --parameters \
    '{ \
      "$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentParameters.json#",\
      "contentVersion": "1.0.0.0",\
      "parameters": {\
         "adminUsername": { "value" : "[[your_admin_id]]"},\
         "adminPassword": { "value" : "[[your_admin_password]]"},\
         "vmName": { "value" : "[[vm_name]]"},\
         "vmSize": { "value" : "Standard_DS3_v2"}\
      }\
    }'

*** Please go to the project page to select the VM that you just created as your new computing platform (Run on ...) to continue ...

2. How to use DSVM

1. Now that you have a DSVM, when you login to https://notebooks.azure.com, you can see you DSVM on the drop down list under Free Compute and Direct Compute.
   
   

2. Of course you will select DSVM, it will ask you to validate your JIT credentials.
   
   

3. Once you pick a notebook to run, you may encounter the following warning:
   
   As you may see, [Python 3.6 - AzureML] is the correct answer.

3. Things to know about using DSVM

1. The most important thing to know about Azure Notebooks on DSVM is that: Azure Notebooks project home directory is not mounted on the DSVM. So any references to Azure Notebooks folders / files will incur File/folder not found exception. In other words, each ipynb notebook need to be independent of other files.

2. There are work-around solutions:
   a. Data files can be stored on Azure Blob storage and blobfufe
   b. Python files can be added to the notebook by using the Jupyter magic, you can find an example here: %%writefile
   c. Configuration files are a bit more complicated. Using our Microsoft Sentinel config.json as an example, it is generated when you import Microsoft Sentinel Jupyter project from GitHub repo through Azure portal. The configuration JSON is Azure Log Analytics workspace specific file, so you clone one project for one Log Analytics workspace. You can find the config.json file at the root of the project home directory. Get Start.jpynb section 1 demonstrates how to set the configuration settings manually.