Key Python Libraries.ipynb

Kernel: Python

Python library is a collection of functions and methods that allows you to perform lots of actions without writing your own code.Here, a ‘library’ loosely describes a collection of core modules.

Data Science and Machine Learning

• Numpy and Scipy – Fundamental Scientific Computing

• Pandas – Data Manipulation and Analysis

• StatsModels – Statistical Modeling, Testing, and Analysis

• Matplotlib – Plotting and Visualization

• Seaborn – For Statistical Data Visualization

• Scikit-learn – Machine Learning and Data Mining

• Keras- Machine learning

• PyTorch- Machine Learning Tensor Flow

• LightGBM- Machine Learning(Model Optimization)

• Tensor Flow - Machine Learning

• Eli5- Machine learning models(Debugging)

• Theano- AI and ML

• NLTK - Natural Language Processing with Python

• SQLAlchemy- Database access

Web Development Frameworks and GUI

• Tkinter – GUI applications

• Requests – HTTP requests

• Django- Web framework

• Scrapy- Web crawling frameworks

• BeautifulSoup - web crawling and data scraping

• Flask - Micro-web framework

Others

• Pyspark - Support Python with Spark, Apache Spark community released a tool, PySpark

• TabPy - TabPy (the Tableau Python Server) is an Analytics Extension implementation which expands Tableau’s capabilities by allowing users to execute Python scripts and saved functions via Tableau’s table calculations

• Jython- Jython an implementation of the Python programming language designed to run on the Java platform

import numpy as np
L1=np.array([1,2,3,4])
L1

array([1, 2, 3, 4])

scipy.special package contains numerous functions of mathematical physics.

SciPy special function includes Cubic Root, Exponential, Log sum Exponential, Lambert, Permutation and Combinations, Gamma, Bessel, hypergeometric, Kelvin, beta, parabolic cylinder, Relative Error Exponential, etc..

import scipy.special 
import numpy as np
import scipy.stats
x = np.arange(10, 20)
y = np.array([2, 1, 4, 5, 8, 12, 18, 25, 96, 48])
result = scipy.stats.linregress(x, y)
print("slope=" ,result.slope)
print("Intercept= ",result.intercept)
#help(scipy.special)

slope= 7.4363636363636365
Intercept=  -85.92727272727274

import pandas as pd
pd.Series((1,2,34,35))

0     1
1     2
2    34
3    35
dtype: int64

pd.DataFrame(((1,"A"),(2,"B"),(3,"C")))

chicago_data = pd.read_csv('https://data.cityofchicago.org/resource/jcxq-k9xf.csv')
chicago_data

chicago_data.describe()

chicago_data.isnull().sum()

ca                                             1
community_area_name                            0
percent_of_housing_crowded                     0
percent_households_below_poverty               0
percent_aged_16_unemployed                     0
percent_aged_25_without_high_school_diploma    0
percent_aged_under_18_or_over_64               0
per_capita_income_                             0
hardship_index                                 1
dtype: int64

chicago_data.corr(method ='pearson')

#pip install pandas-profiling
import pandas_profiling as pro
profile = pro.ProfileReport(chicago_data)
profile

Summarize dataset:   0%|          | 0/22 [00:00<?, ?it/s]

Generate report structure:   0%|          | 0/1 [00:00<?, ?it/s]

Render HTML:   0%|          | 0/1 [00:00<?, ?it/s]

#pip install sweetviz
import sweetviz as sv
sweet_report = sv.analyze(chicago_data)
sweet_report.show_html('sweet_report.html')

                                             |          | [  0%]   00:00 -> (? left)

Report sweet_report.html was generated! NOTEBOOK/COLAB USERS: the web browser MAY not pop up, regardless, the report IS saved in your notebook/colab files.

df1 = sv.compare(chicago_data[0:],chicago_data [:3])
df1.show_html('Compare.html')

                                             |          | [  0%]   00:00 -> (? left)

Report Compare.html was generated! NOTEBOOK/COLAB USERS: the web browser MAY not pop up, regardless, the report IS saved in your notebook/colab files.

<sweetviz.dataframe_report.DataframeReport at 0x7f814b7c5d30>

#pip install autoviz
from autoviz.AutoViz_Class import AutoViz_Class
AV = AutoViz_Class()

viz = AV.AutoViz("imdb.csv")

File encoding decoder utf-8 does not work for this file
File encoding decoder iso-8859-11 does not work for this file
File encoding decoder cpl252 does not work for this file
Shape of your Data Set: (1000, 12)
############## C L A S S I F Y I N G  V A R I A B L E S  ####################
Classifying variables in data set...
    Number of Numeric Columns =  3
    Number of Integer-Categorical Columns =  2
    Number of String-Categorical Columns =  2
    Number of Factor-Categorical Columns =  0
    Number of String-Boolean Columns =  0
    Number of Numeric-Boolean Columns =  0
    Number of Discrete String Columns =  2
    Number of NLP String Columns =  0
    Number of Date Time Columns =  1
    Number of ID Columns =  2
    Number of Columns to Delete =  0
    12 Predictors classified...
        This does not include the Target column(s)
        4 variables removed since they were ID or low-information variables
Number of All Scatter Plots = 6

Time to run AutoViz (in seconds) = 31.061

 ###################### VISUALIZATION Completed ########################

import matplotlib
%matplotlib inline

#pip install autoviz