Guided Investigation - Solarwinds Post Compromise Activity.ipynb

Kernel: Python 3.10 - SDK v2

Table of Contents

• Guided Hunting - Solarwinds Post Compromise
  • Notebook initialization
  • Setup
    • Configuration
      • msticpyconfig.yaml configuration File
• IOC Hunt
  • TearDrop Memory Only Dropper
  • SUNBURST and SUPERNOVA backdoor hashes
  • SUNBURST network beacons
  • SUNBURST suspicious SolarWinds child processes
• Checking for Solarwinds Servers
  • Asset Inventory
• Lateral Movement
  • • Hosts with New Logons
    • Rare RDP Connections
    • RDP Nesting
    • Rare application consent
    • Rare activity by High Value Account
  • Suspicious Logons
• Privilege Escalation
  • New Account Creation and Addition to Privileged groups
  • User Added to Azure Active Directory Privileged Groups
  • ADFS Hunting
    • Uncommon processes
    • Rare processes run by Service Accounts
    • Fireeye Red Team Tools
• Azure Active Directory Hunting
  • • Domain Federation trust Settings modification
    • Addition of New Key Credentials to an Application or Service Principal
    • Suspicious Application Consent
• Data Acccess and Exfiltration
  • • Azure Active Directory PowerShell to access non-Active Directory Resources
    • Anomalous increase in Exchange Mail Items Accesed
• Resources

Guided Hunting - Solarwinds Post Compromise

Details...

Notebook Version: 1.0
Python Version: Python 3.10 (including Python 3.10 - SDK v2 - AzureML)
Required Packages: kqlmagic, msticpy, pandas, numpy, matplotlib, networkx, ipywidgets, ipython, scikit_learn, dnspython, ipwhois, folium, holoviews

Data Sources Required:

• Log Analytics

  • Heartbeat

  • SecurityAlert

  • SecurityEvent

  • DeviceProcessEvents

  • DeviceNetworkEvents

  • DeviceFileEvents

  • SigninEvents

  • AuditLogs

  • AzureNetworkAnalytics_CL

• (Optional)

  • VirusTotal (with API key)

  • Alienvault OTX (with API key)

  • IBM Xforce (with API key)

  • CommonSecurityLog

This Notebook assists defenders in hunting for Solarwinds Post compromise Tactics , Tools and Procedures (TTPs) across different environments both on-prem and cloud data sources.

You can read more about the attack in below technical blogs

References :

• https://msrc-blog.microsoft.com/2020/12/13/customer-guidance-on-recent-nation-state-cyber-attacks/

• https://www.fireeye.com/blog/threat-research/2020/12/evasive-attacker-leverages-solarwinds-supply-chain-compromises-with-sunburst-backdoor.html

How to use:
Run the cells in this Notebook in order, at various points in the Notebook flow you will be prompted to enter or select options relevant to the scope of your triage.
This Notebook presumes you have Microsoft Sentinel Workspace settings and Threat Intelligence providers configured in a config file. If you do not have this in place please refer https://msticpy.readthedocs.io/en/latest/data_acquisition/TIProviders.html#configuration-file to https://github.com/Azure/Azure-Sentinel-Notebooks/blob/master/ConfiguringNotebookEnvironment.ipynb and to set this up.

---

Notebook initialization

The next cell:

• Checks for the correct Python version

• Checks versions and optionally installs required packages

• Imports the required packages into the notebook

• Sets a number of configuration options.

This should complete without errors. If you encounter errors or warnings look at the following two notebooks:

• TroubleShootingNotebooks

• ConfiguringNotebookEnvironment

If you are running in the Microsoft Sentinel Notebooks environment (Azure Notebooks or Azure ML) you can run live versions of these notebooks:

• Run TroubleShootingNotebooks

• Run ConfiguringNotebookEnvironment

You may also need to do some additional configuration to successfully use functions such as Threat Intelligence service lookup and Geo IP lookup. There are more details about this in the ConfiguringNotebookEnvironment notebook and in these documents:

• msticpy configuration

• Threat intelligence provider configuration

Setup

from pathlib import Path
from IPython.display import display, HTML

REQ_PYTHON_VER = "3.10"
REQ_MSTICPY_VER = "2.12.0"


display(HTML("<h3>Starting Notebook setup...</h3>"))


# If not using Azure Notebooks, install msticpy with
# %pip install msticpy
import msticpy as mp

additional_packages = [
    "tldextract", "IPWhois", "python-whois"
]
mp.init_notebook(
    namespace=globals(),
    additional_packages=additional_packages,
);

from datetime import date
from functools import lru_cache
from msticpy.vis.foliummap import get_map_center, get_center_ip_entities
from pandas import json_normalize
from ruamel.yaml import YAML
from tqdm.notebook import tqdm
import IPython
import csv
import dns
import glob
import io
import json
import re
import requests
import tldextract
import whois
import zipfile
from bokeh.plotting import figure
import ipywidgets as widgets

# See if we have a Microsoft Sentinel Workspace defined in our config file.
# If not, let the user specify Workspace and Tenant IDs

ws_config = mp.WorkspaceConfig()
if not ws_config.config_loaded:
    ws_config.prompt_for_ws()

qry_prov = mp.QueryProvider(data_environment="AzureSentinel")
print("done")

ti = TILookup()

Configuration

msticpyconfig.yaml configuration File

You can configure primary and secondary TI providers and any required parameters in the msticpyconfig.yaml file. This is read from the current directory or you can set an environment variable (MSTICPYCONFIG) pointing to its location.

To configure this file see the ConfigureNotebookEnvironment notebook

# Authenticate to Microsoft Sentinel workspace
qry_prov.connect(ws_config)

def get_solarwinds_queries_from_github(git_url, outputdir):
    r = requests.get(git_url)

    repo_zip = io.BytesIO(r.content)

    archive = zipfile.ZipFile(repo_zip, mode="r")

    # Only extract Detections and Hunting Queries Folder
    for file in archive.namelist():
        if file.startswith(
            (
                "Azure-Sentinel-master/Detections/DeviceEvents/SolarWinds_TEARDROP_Process-IOCs.yaml",
                "Azure-Sentinel-master/Detections/DeviceNetworkEvents/SolarWinds_SUNBURST_Network-IOCs.yaml",
                "Azure-Sentinel-master/Detections/DeviceProcessEvents/SolarWinds_SUNBURST_Process-IOCs.yaml",
                "Azure-Sentinel-master/Detections/DeviceFileEvents/SolarWinds_SUNBURST_&_SUPERNOVA_File-IOCs.yaml",
                "Azure-Sentinel-master/Detections/AuditLogs/MaliciousOAuthApp_O365AttackToolkit.yaml",
                "Azure-Sentinel-master/Detections/AuditLogs/MaliciousOAuthApp_PwnAuth.yaml",
                "Azure-Sentinel-master/Detections/AuditLogs/UseraddedtoPrivilgedGroups.yaml",
                "Azure-Sentinel-master/Detections/AuditLogs/ADFSDomainTrustMods.yaml",
                "Azure-Sentinel-master/Detections/AuditLogs/RareApplicationConsent.yaml",
                "Azure-Sentinel-master/Detections/SigninLogs/AzureAADPowerShellAnomaly.yaml",
                "Azure-Sentinel-master/Detections/OfficeActivity/MailItemsAccessedTimeSeries.yaml",
                "Azure-Sentinel-master/Detections/SecurityEvent/RDP_RareConnection.yaml",
                "Azure-Sentinel-master/Detections/SecurityEvent/RDP_Nesting.yaml",
                "Azure-Sentinel-master/Detections/SecurityEvent/UserCreatedAddedToBuiltinAdmins_1d.yaml",
                "Azure-Sentinel-master/Hunting Queries/SecurityEvent/HostsWithNewLogons.yaml",
                "Azure-Sentinel-master/Hunting Queries/MultipleDataSources/TrackingPrivAccounts.yaml",
                "Azure-Sentinel-master/Hunting Queries/SecurityEvent/ProcessEntropy.yaml",
                "Azure-Sentinel-master/Hunting Queries/SecurityEvent/RareProcbyServiceAccount.yaml",
                "Azure-Sentinel-master/Hunting Queries/SecurityEvent/uncommon_processes.yaml"
            )
        ):
            archive.extract(file, path=outputdir)
    print("Downloaded and Extracted Files successfully")

def_path = Path.joinpath(Path(os.getcwd()))
path_wgt = widgets.Text(value=str(def_path),
                        description='Path to extract to zipped repo files: ',
                        layout=widgets.Layout(width='50%'),
                        style={'description_width': 'initial'})
path_wgt

# Download the Microsoft Sentinel Github repo as ZIP
azsentinel_git_url = 'https://github.com/Azure/Azure-Sentinel/archive/master.zip'

get_solarwinds_queries_from_github(git_url=azsentinel_git_url, outputdir=path_wgt.value)

QUERIES_PATH = 'Azure-Sentinel-master'
sentinel_root = Path(path_wgt.value) / QUERIES_PATH

display(HTML("<h3>Listings under Detections...</h2>"))
%ls '{sentinel_root}/Detections/'

display(HTML("<h3>Listings under Hunting Queries...</h2>"))
%ls '{sentinel_root}/Hunting Queries/'

def parse_yaml(parent_dir, child_dir):

    sentinel_repourl = "https://github.com/Azure/Azure-Sentinel/blob/master"

    # Collect list of files recusrively uinder a folder
    yaml_queries = glob.glob(f"{parent_dir}/{child_dir}/**/*.yaml", recursive=True)
    df = pd.DataFrame()

    # Parse and load yaml
    parsed_yaml = YAML(typ="safe")

    # Recursively load yaml Files and append to dataframe
    for query in yaml_queries:
        with open(query, "r", encoding="utf-8", errors="ignore") as f:
            parsed_yaml_df = json_normalize(parsed_yaml.load(f))
            parsed_yaml_df["DetectionURL"] = query.replace(parent_dir, sentinel_repourl)
            df = df.append(parsed_yaml_df, ignore_index=True, sort=True)

    if child_dir == "Detections":
        df["DetectionType"] = "Analytics"
    elif child_dir == "Hunting Queries":
        df["DetectionType"] = "Hunting"

    df["DetectionService"] = "Microsoft Sentinel Community Github"

    return df

base_dir = path_wgt.value + "/Azure-Sentinel-master"
detections_df = parse_yaml(parent_dir=base_dir, child_dir="Detections")
hunting_df = parse_yaml(parent_dir=base_dir, child_dir="Hunting Queries")

frames = [detections_df, hunting_df]
sentinel_github_df = pd.concat(frames).reset_index(drop=True)
sentinel_github_df = sentinel_github_df.copy()
sentinel_github_df["DetectionURL"] = sentinel_github_df["DetectionURL"].str.replace(
    " ", "%20", regex=True
)
sentinel_github_df["IngestedDate"] = date.today()

# Displaying basic statistics of yaml files
display(HTML("<h3>Microsoft Sentinel Github Stats...</h3>"))
print(
    f"""Total Queries in Microsoft Sentinel Github:: {len(sentinel_github_df)}
    No of Detections :: {len(detections_df)}
    No of Hunting Queries:: {len(hunting_df)}
    """
)

display(sentinel_github_df.head())

IOC Hunt

TearDrop Memory Only Dropper

query = sentinel_github_df.loc[
    sentinel_github_df["name"] == "TEARDROP memory-only dropper"
]["query"].reset_index(drop=True)[0]
print("Collecting data...")
teardrop_df = qry_prov.exec_query(query)
teardrop_df

SUNBURST and SUPERNOVA backdoor hashes

query = sentinel_github_df.loc[
    sentinel_github_df["name"] == "SUNBURST and SUPERNOVA backdoor hashes"
]["query"].reset_index(drop=True)[0]
print("Collecting data...")
backdoor_df = qry_prov.exec_query(query)
backdoor_df

SUNBURST network beacons

query = sentinel_github_df.loc[
    sentinel_github_df["name"] == "SUNBURST network beacons"
]["query"].reset_index(drop=True)[0]
print("Collecting data...")
nwbeacon_df = qry_prov.exec_query(query)
nwbeacon_df

SUNBURST suspicious SolarWinds child processes

query = sentinel_github_df.loc[
    sentinel_github_df["name"] == "SUNBURST suspicious SolarWinds child processes"
]["query"].reset_index(drop=True)[0]
print("Collecting data...")
swprocess_df = qry_prov.exec_query(query)
swprocess_df

Checking for Solarwinds Servers

To hunt for similar TTPs used in this attack, a good place to start is to build an inventory of the machines that have SolarWinds Orion components. Organizations might already have a software inventory management system to indicate hosts where the SolarWinds application is installed. Alternatively, Microsoft Sentinel could be leveraged to run a simple query to gather similar details. Below query will pull the hosts with SolarWinds process running in last 30 days based on Process execution/Sysmon logs either via host onboarded to Sentinel or onboarded via Microsoft Defender for Endpoints (MDE)

Asset Inventory

# Gather Solarwinds details based on Process execution Logs from diverse data sources
solarwinds_query = f"""
let timeframe = 30d;
(union isfuzzy=true
  (
  SecurityEvent
  | where TimeGenerated >= ago(timeframe)
  | where EventID == '4688'
  | where tolower(NewProcessName) has 'solarwinds'
  | extend MachineName = Computer , Process = NewProcessName
  ),
  (
  DeviceProcessEvents
  | where TimeGenerated >= ago(timeframe)
  | where tolower(InitiatingProcessFolderPath) has 'solarwinds'
  | extend MachineName = DeviceName , Process = InitiatingProcessFolderPath
  ),
  (
  Event
  | where TimeGenerated >= ago(timeframe)
  | where Source == "Microsoft-Windows-Sysmon"
  | where EventID == 1
  | extend Image = EventDetail.[4].["#text"]
  | where tolower(Image) has 'solarwinds'
  | extend MachineName = Computer , Process = Image
  )
  )
  | summarize StartTime = min(TimeGenerated), EndTime = max(TimeGenerated), make_set(Process) by MachineName
"""
print("Collecting data...")
solarwinds_assets = qry_prov.exec_query(solarwinds_query)
md(f'Solarwinds Server Details Gathered', styles=["bold", "large"])
display(solarwinds_assets)

Lateral Movement

Hosts with New Logons

query = sentinel_github_df.loc[
    sentinel_github_df["name"] == "Hosts with new logons"
]["query"].reset_index(drop=True)[0]
print("Collecting data...")
hostswithnewlogons_df = qry_prov.exec_query(query)
hostswithnewlogons_df

Rare RDP Connections

query = sentinel_github_df.loc[
    sentinel_github_df["name"] == "Rare RDP Connections"
]["query"].reset_index(drop=True)[0]
print("Collecting data...")
rarerdp_df = qry_prov.exec_query(query)
rarerdp_df

RDP Nesting

query = sentinel_github_df.loc[
    sentinel_github_df["name"] == "RDP Nesting"
]["query"].reset_index(drop=True)[0]
print("Collecting data...")
rdpnest_df = qry_prov.exec_query(query)
rdpnest_df

Rare application consent

query = sentinel_github_df.loc[
    sentinel_github_df["name"] == "Rare application consent"
]["query"].reset_index(drop=True)[0]
print("Collecting data...")
rareappconsent_df = qry_prov.exec_query(query)
rareappconsent_df

Rare activity by High Value Account

query = sentinel_github_df.loc[
    sentinel_github_df["name"] == "Rare processes run by Service accounts"
]["query"].reset_index(drop=True)[0]
print("Collecting data...")
serviceaccproc_df = qry_prov.exec_query(query)
serviceaccproc_df

Suspicious Logons

If you have suspicious Netblocks not expected in your organization(e.g.VPS Netblocks or Proxy severs) that you want to monior , define it in IP_Data and run below query using ipv4_lookup kql function

query = """

SigninLogs 
| where TimeGenerated > ago(360d) 
| where ResultType == 0 
| extend additionalDetails = tostring(Status.additionalDetails) 
| evaluate ipv4_lookup(IP_Data, IPAddress, network, return_unmatched = false) 
| summarize make_set(additionalDetails), min(TimeGenerated), max(TimeGenerated) by IPAddress, UserPrincipalName 
| where array_length(set_additionalDetails) == 2 
| where (set_additionalDetails[1] == "MFA requirement satisfied by claim in the token" and set_additionalDetails[0] == "MFA requirement satisfied by claim provided by external provider") or (set_additionalDetails[0] == "MFA requirement satisfied by claim in the token" and set_additionalDetails[1] == "MFA requirement satisfied by claim provided by external provider") 
| project IPAddress, UserPrincipalName, min_TimeGenerated, max_TimeGenerated
"""
print("Collecting data...")
suspmfalogons_df = qry_prov.exec_query(query)
suspmfalogons_df

Privilege Escalation

New Account Creation and Addition to Privileged groups

query = sentinel_github_df.loc[
    sentinel_github_df["name"] == "New user created and added to the built-in administrators group"
]["query"].reset_index(drop=True)[0]
print("Collecting data...")
newaccprivgroups_df = qry_prov.exec_query(query)
newaccprivgroups_df

User Added to Azure Active Directory Privileged Groups

query = sentinel_github_df.loc[
    sentinel_github_df["name"] == "User added to Azure Active Directory Privileged Groups"
]["query"].reset_index(drop=True)[0]
print("Collecting data...")
aadprivgroups_df = qry_prov.exec_query(query)
aadprivgroups_df

ADFS Hunting

Below Queries produces results from all Servers. Its is advisable to filter the results for ADFS servers in scope.

Uncommon processes

query = sentinel_github_df.loc[
    sentinel_github_df["name"] == "Uncommon processes - bottom 5%"
]["query"].reset_index(drop=True)[0]
print("Collecting data...")
domain_trust_df = qry_prov.exec_query(query)
domain_trust_df

Rare processes run by Service Accounts

query = sentinel_github_df.loc[
    sentinel_github_df["name"] == "Rare processes run by Service accounts"
]["query"].reset_index(drop=True)[0]
print("Collecting data...")
serviceaccproc_df = qry_prov.exec_query(query)
serviceaccproc_df

Fireeye Red Team Tools

Check out KQL queries published by Sentinel Community at

• https://github.com/FalconForceTeam/FalconFriday/blob/master/Uncategorized/FireEye_red_team_tool_countermeasures.md

Azure Active Directory Hunting

Domain Federation trust Settings modification

query = sentinel_github_df.loc[
    sentinel_github_df["name"] == "Modified domain federation trust settings"
]["query"].reset_index(drop=True)[0]
print("Collecting data...")
domain_trust_df = qry_prov.exec_query(query)
domain_trust_df

Addition of New Key Credentials to an Application or Service Principal

query = """
  let auditLookback = 1h;
  AuditLogs
  | where TimeGenerated > ago(auditLookback)
  | where OperationName has_any ("Add service principal", "Certificates and secrets management") // captures "Add service principal", "Add service principal credentials", and "Update application – Certificates and secrets management" events
  | extend targetDisplayName = TargetResources[0].displayName
  | extend targetId = TargetResources[0].id
  | extend targetType = TargetResources[0].type
  | extend keyEvents = TargetResources[0].modifiedProperties
  | where keyEvents has "KeyIdentifier=" and keyEvents has "KeyUsage=Verify"
  | where Result =~ "success"
  | mv-expand keyEvents
  | where keyEvents.displayName =~ "KeyDescription"
  | parse keyEvents.newValue with * "KeyIdentifier=" keyIdentifier:string ",KeyType=" keyType:string ",KeyUsage=" keyUsage:string ",DisplayName=" keyDisplayName:string "]" *
  | parse keyEvents.oldValue with * "KeyIdentifier=" keyIdentifierOld:string "," *
  | where keyEvents.oldValue == "[]" or keyIdentifier != keyIdentifierOld
  | where keyUsage == "Verify"
  | extend UserAgent = iff(AdditionalDetails[0].key == "User-Agent",AdditionalDetails[0].value,"")
  | extend InitiatingUser = iff(isnotempty(InitiatedBy.user.userPrincipalName),InitiatedBy.user.userPrincipalName, InitiatedBy.app.displayName)
  | extend InitiatingIpAddress = iff(isnotempty(InitiatedBy.user.ipAddress), InitiatedBy.user.ipAddress, InitiatedBy.app.ipAddress)
  //
  // Adding the below filter for detection-quality events; Microsoft Sentinel users can comment out this line and tune additional service principal events for their environment
  | where targetType =~ "Application"
"""
print("Collecting data...")
newkeycreds_df = qry_prov.exec_query(query)
newkeycreds_df

Suspicious Application Consent

query = sentinel_github_df.loc[
    sentinel_github_df["name"] == "Suspicious application consent similar to O365 Attack Toolkit"
]["query"].reset_index(drop=True)[0]
print("Collecting data...")
o365toolkit_df = qry_prov.exec_query(query)
o365toolkit_df

query = sentinel_github_df.loc[
    sentinel_github_df["name"] == "Suspicious application consent similar to PwnAuth"
]["query"].reset_index(drop=True)[0]
print("Collecting data...")
o365pwnauth_df = qry_prov.exec_query(query)
o365pwnauth_df

Data Acccess and Exfiltration

Azure Active Directory PowerShell to access non-Active Directory Resources

query = sentinel_github_df.loc[
    sentinel_github_df["name"] == "Azure Active Directory PowerShell accessing non-AAD resources"
]["query"].reset_index(drop=True)[0]
print("Collecting data...")
adpowershell_df = qry_prov.exec_query(query)
adpowershell_df

Anomalous increase in Exchange Mail Items Accesed

query = sentinel_github_df.loc[
    sentinel_github_df["id"] == "b4ceb583-4c44-4555-8ecf-39f572e827ba"
]["query"].reset_index(drop=True)[0]
print("Collecting data...")
timeseriesmail_df = qry_prov.exec_query(query)
timeseriesmail_df

Resources

• Read the Get Started notebook.

• View sample notebooks in the Sample-Notebooks folder

• View How-tos and Troubleshooting in the HowTos folder

• https://msticpy.readthedocs.io/en/latest/GettingStarted.html

• https://msticnb.readthedocs.io/en/latest/notebooklets.html#current-notebooklets

• Recent Nation-State Cyber Attacks

• Behavior:Win32/Solorigate.C!dha threat description - Microsoft Security Intelligence

• Customer guidance on recent nation-state cyberattacks

• FireEye Advisory: Highly Evasive Attacker Leverages SolarWinds Supply Chain to Compromise Multiple Global Victims With SUNBURST Backdoor

• FireEye GitHub page: Sunburst Countermeasures 

• DHS Directive

• SolarWinds Security Advisory

• FalconFriday – Fireeye Red Team Tool Countermeasures KQL Queries

• microsoft/Microsoft-365-Defender-Hunting-Queries: Sample queries for Advanced hunting in Microsoft 365 Defender (github.com)

• Microsoft Sentinel SolarWinds Post Compromise Hunting Workbook

• [https://security.microsoft.com/threatanalytics3/2b74f636-146e-48dd-94f6-5cb5132467ca/overview]