Book a Demo!
CoCalc Logo Icon
StoreFeaturesDocsShareSupportNewsAboutPoliciesSign UpSign In
Azure
GitHub Repository: Azure/Azure-Sentinel-Notebooks
 Path: blob/master/tutorials-and-examples/feature-tutorials/Sumologic-DataConnector.ipynb
3253 views
Kernel: Python 3

Sumologic - Data Connector

Description

The data provider module of msticpy provides functions to allow for the defining of data sources, connectors to them and queries for them as well as the ability to return query result from the defined data sources.

For more information on Data Propviders, check documentation

In this notebooks we will demonstrate Sumologic data connector feature of msticpy. This feature is built on-top of the [Sumologic SDK for Python] (https://github.com/SumoLogic/sumologic-python-sdk) with some customizations and enhancements.

Installation

# Only run first time to install/upgrade msticpy to latest version
#%pip install --upgrade msticpy

Authentication

Authentication for the Sumologic data provider is handled by specifying credentials (accessid and accesskey) directly in the connect call or specifying the credentials in msticpy config file.

For more information on how to create credentials, follow Sumologic Docs Access Keys and Users and Roles. The user should have permission to at least run its own searches or more depending upon the actions to be performed by user.

Once you created user account with the appropriate roles, you will require the following details to specify while connecting

  • url = "https://api.us2.sumologic.com/api" (Sumologic url endpoint depending on which region is used)

  • accessid = "xxx" (as created in Sumologic user preferences)

  • accesskey = "xxx" (same)

Once you have details, you can specify it in msticpyconfig.yaml as shown in below example

DataProviders:
  Sumologic:
    Args:
      connection_str: "{Sumologic url endpoint}"
      accessid: "{accessid with search permissions to connect}"
      accesskey: "{accesskey of the user specified}"

#Check we are running Python 3.6
import sys
MIN_REQ_PYTHON = (3,6)
if sys.version_info < MIN_REQ_PYTHON:
    print('Check the Kernel->Change Kernel menu and ensure that Python 3.6')
    print('or later is selected as the active kernel.')
    sys.exit("Python %s.%s or later is required.\n" % MIN_REQ_PYTHON)

#imports
import pandas as pd
import msticpy.nbtools as nbtools
from datetime import datetime,timedelta

#data library imports
from msticpy.data.data_providers import QueryProvider
print('Imports Complete')
Imports Complete

Instantiating a query provider

You can instantiate a data provider for Sumologic by specifying the credentials in connect or in msticpy config file.
If the details are correct and authentication is successful, it will show connected.

URL endpoints are referenced on Sumo Logic Endpoints and Firewall Security

sumologic_prov = QueryProvider('Sumologic')
#sumologic_prov.connect(url=<url>, accessid=<accessid>, accesskey=<accesskey>)
sumologic_prov.connect()
connected

Running a Ad-hoc Sumologic query

You can define your own sumologic query and run it via sumologic provider via QUERY_PROVIDER.exec_query(<queryname>)

For more information, check documentation Running and Ad-hoc Query

sumologic_query = '''
*
| formatDate(_messageTime,"yyyy/dd/MM HH:mm:ss") as date
| first(date), last(date) by _sourceCategory
| count _sourceCategory,_first,_last
| sort -_count
'''
df = sumologic_prov.exec_query(sumologic_query, days=0.0005, verbosity=3)
df.head()

sumologic_query = '''_index=WINDOWS | count _sourceCategory,hostname'''
df = sumologic_prov.exec_query(sumologic_query, start_time=datetime.now() - timedelta(days=6.001), end_time=datetime.now() - timedelta(days=6))
df.head()