Senserva Connections Graph Notebook.ipynb

Kernel: Python 3.8 - AzureML

---

Welcome to the Senserva Connections Graph Notebook!

This notebook serves to visualize connections that exist between Azure Active Directory objects.

The connections will be an interaction between the objects in the form of membership, ownership, permission, etc. They are visualized using a Graph. The Graph is an ideal tool to visualize as one object can have many different kinds of connections. The nodes of the Graph will be sized according to number of connections (more connections = bigger size) and colored according to type of node (Low Risk = Green, High Risk = Red,...).

The data set for the Graph will be fetched from the Log Analytics Workspace that your Senserva Scanner is set up to use. More information about that setup process can be found here.

Setup

This section will setup our Python imports and Global variables

Several processing and visualization libraries are used in this notebook. In this notebook we will use the Pyserva Python library that our experts at Senserva have made to visualize your data. More information about the library and the source code can be found on the Github page

# If you need to know what Python modules are available, you may run this:
# help("modules")

# Install the Pyserva library
%pip install Pyserva

# Load Python libraries that will be used in this notebook to fetch Azure Sentinel data
from azure.loganalytics.models import QueryBody
from azure.mgmt.loganalytics import LogAnalyticsManagementClient
from azure.loganalytics import LogAnalyticsDataClient

# # Processing Helpers
import Pyserva
from Pyserva import JupyterNotebookHelper as Senserva
from Pyserva.StandaloneApiHelper import getTreeTemplate
from IPython.display import display, HTML, Markdown
import csv
import pandas as pd
import requests
import numpy as np

# Globals
# We use a file for data set storage so that we only have to grab once
defaultFilter = '(no filter applied)'
filename = 'edge_data.csv'
ueba_filename = 'ueba_edge_data.csv'
together_filename = 'together_edge_data.csv'
dropdown = [defaultFilter]

# Calling the above function to populate Sentinel workspace parameters
# The file, config.json, was generated by the system, however, you may modify the values, or manually set the variables
tenant_id, subscription_id, resource_group, workspace_id, workspace_name, user_alias, user_object_id = Senserva.read_config_values('config.json');

Authenticate to Azure

This section will authenticate our session for Azure to get data

We will use the configs defined above to get data from the Log Analytics workspace. The Pyserva library will use the data to analyze and render the data in later steps.

# If you need to install MsticPy, run the following line
# %pip install msticpy

from msticpy.data.data_providers import QueryProvider
from msticpy.common.wsconfig import WorkspaceConfig

# Use KqlMagic from MsticPy to authenticate to our workspace
# Note: This will be an interactive login using a device code
qry_prov = QueryProvider('LogAnalytics')
wkspace = WorkspaceConfig()
qry_prov.connect(wkspace.code_connect_str)

Gather and Process our Data

This section will gather the Senserva Scanner data from the Log Analytics Workspace and process it for display

We gather the dataset and the Pyserva library will process it for the relevant data points. The data points will be saved in a CSV file to your system. The data points are stored in the CSV so you can see them and to reduce the amount of web calls. A filter dropdown is generated after processing of all objects found. Select an object and the visualizations in the following steps will use the dropdown value to filter the data before rendering.

# This is the default table name of the Senserva Scanner, but you can change to any alias you may have
table_name = 'SenservaPro_CL'

kql_query = Senserva.SenservaPermissionQuery(table_name)
query = "{0}".format(kql_query)

# Execute the query
query_result = qry_prov.exec_query(query)

with open(filename, 'w', newline='') as csvfile:
    filewriter = csv.writer(csvfile, delimiter=',',
                        quotechar='|', quoting=csv.QUOTE_MINIMAL)

    # Pluck our data from the Pandas Series
    # Displays a dropdown menu that allows for selection of node to filter by
    edge_object_dropdown = Senserva.PluckDataFromQueryResults(query_result, dropdown, filewriter)    
    

ueba_edges = []
with open(filename, newline='') as csvfile:
    file_df = pd.read_csv(csvfile,delimiter=',')
    
    ueba_kql_query = Senserva.SenservaPermissionUebaQuery()
    ueba_query = "{0}".format(ueba_kql_query)

    # Execute the query
    ueba_result = qry_prov.exec_query(ueba_query)

    for index, row in file_df.iterrows():
        if(row['SourceId'] is not np.nan):
            for value in ueba_result.iterrows():
                source_id = value[1][0]
                if(row['SourceId'] == source_id):
                    rbac_name = value[1][1]
                    rbac_id = value[1][2]
                    rbac_object = value[1][3]
                    rbac_role = value[1][4]
                    relation = "{0} {1}".format(rbac_object, rbac_role)
                    ueba_edges.append([rbac_name, rbac_id, rbac_object, row['Source'], source_id, row['SourceType'], row['SourceWeight'], row['SourceWeight'], relation, relation, row['Risk'], row['UserMail'], row['UserManagerMail']])

with open(ueba_filename, 'w', newline='') as csvfile:
    filewriter = csv.writer(csvfile, delimiter=',',
                        quotechar='|', quoting=csv.QUOTE_MINIMAL)
            
    filewriter.writerow(['Source', 'SourceId', 'SourceType', 'Target', 'TargetId', 'TargetType', 'SourceWeight', 'TargetWeight', 'Relationship', 'Reason', 'Risk', 'UserMail', 'UserManagerMail'])
    for edge in ueba_edges:
        filewriter.writerow(edge)

Gather, Process, and Visualize the Data

This section will use the gathered data and render it in several ways

Data will be visualized as:

• A Graph

• A Tree

Data will be classified

• Active Directory (AD)

• Role Based Access Control (RBAC)

• Together (AD + RBAC)

If you change the filter in the above dropdown, the visualizations will need to be rerun to take effect

Active Directory (AD)

Key Features of the Table

• Risk - A score representing the suspicion of an object

• Weight - Impact an object can have on a network

• Overall - Using the Risk and Weight, a score to indicate what objects are most likely to be compromised

items = []

with open(filename, newline='') as csvfile:
    file_df = pd.read_csv(csvfile,delimiter=',')
    
    # Render our data as a Graph
    # The graph will apply a filter from the previous cell's
    # dropdown value. If a new filter needs to be applied,
    # select it from the dropdown and then re-run this cell
    items = Senserva.filterDataFrameAndCreateList(edge_object_dropdown, file_df, defaultFilter)


HTML(getTreeTemplate().format(inputData = Senserva.htmlTreeParser(items)))

Key Features of the Graph

• Color - Represents the risk of an object, green/yellow/red

• Size - Represents the number of connections, bigger size = more exposure

• Shape - Represents the type of object, see legend

• Relationships - Represents what interactions an object has the network

Legend

• ○ - User

• ◊ - Group

• △ - Application

• ☆ - Service Principal

• ➕ - Disabled User

• □ - Role

• ⬡ - PIM Role

• ⦻ - Default

with open(filename, newline='') as csvfile:
    file_df = pd.read_csv(csvfile,delimiter=',')
    
    # Render our data as a Graph
    # The graph will apply a filter from the previous cell's
    # dropdown value. If a new filter needs to be applied,
    # select it from the dropdown and then re-run this cell
    graph = Senserva.RenderGraphData(Senserva.filterHelper(edge_object_dropdown, file_df, defaultFilter))
    Senserva.show(graph)
        

Role Based Access Control (RBAC)

Key Features of the Table

• Risk - A score representing the suspicion of an object

• Weight - Impact an object can have on a network

• Overall - Using the Risk and Weight, a score to indicate what objects are most likely to be compromised

items = []

with open(ueba_filename, newline='') as csvfile:
    file_df = pd.read_csv(csvfile,delimiter=',')
    
    # Render our data as a Graph
    # The graph will apply a filter from the previous cell's
    # dropdown value. If a new filter needs to be applied,
    # select it from the dropdown and then re-run this cell
    items = Senserva.filterDataFrameAndCreateList(edge_object_dropdown, file_df, defaultFilter)

HTML(getTreeTemplate().format(inputData = Senserva.htmlTreeParser(items)))

Key Features of the Graph

• Color - Represents the risk of an object, green/yellow/red

• Size - Represents the number of connections, bigger size = more exposure

• Shape - Represents the type of object, see legend

• Relationships - Represents what interactions an object has the network

Legend

• ○ - User

• ◊ - Group

• △ - Application

• ☆ - Service Principal

• ➕ - Disabled User

• □ - Role

• ⬡ - PIM Role

• ⦻ - Default

with open(ueba_filename, newline='') as csvfile:
    file_df = pd.read_csv(csvfile,delimiter=',')
    
    # Render our data as a Graph
    # The graph will apply a filter from the previous cell's
    # dropdown value. If a new filter needs to be applied,
    # select it from the dropdown and then re-run this cell
    ueba_graph = Senserva.RenderGraphData(Senserva.filterHelper(edge_object_dropdown, file_df, defaultFilter))
    Senserva.show(ueba_graph)

Together (AD + RBAC)

Key Features of the Table

• Risk - A score representing the suspicion of an object

• Weight - Impact an object can have on a network

• Overall - Using the Risk and Weight, a score to indicate what objects are most likely to be compromised

items = []
ad_file_df = []
ueba_file_df = []

with open(filename, newline='') as csvfile:
    ad_file_df = pd.read_csv(csvfile,delimiter=',')
with open(ueba_filename, newline='') as csvfile:
    ueba_file_df = pd.read_csv(csvfile,delimiter=',')

together_file_df = ad_file_df.append(ueba_file_df, ignore_index=True)
items = Senserva.filterDataFrameAndCreateList(edge_object_dropdown, together_file_df, defaultFilter)
HTML(getTreeTemplate().format(inputData = Senserva.htmlTreeParser(items)))

Key Features of the Graph

• Color - Represents the risk of an object, green/yellow/red

• Size - Represents the number of connections, bigger size = more exposure

• Shape - Represents the type of object, see legend

• Relationships - Represents what interactions an object has the network

Legend

• ○ - User

• ◊ - Group

• △ - Application

• ☆ - Service Principal

• ➕ - Disabled User

• □ - Role

• ⬡ - PIM Role

• ⦻ - Default

together_graph = Senserva.RenderGraphData(Senserva.filterHelper(edge_object_dropdown, together_file_df, defaultFilter))
Senserva.show(together_graph)

Wrap-Up

Seeing the connections of your Azure Active Directory objects helps give an encompassing view not currently available through Azure. Using this information, you can more effectively investigate how these objects interact with each other. These visuals save time in how a huge network like Azure Active Directory is used.