GitHub Repository: suyashi29/python-su
Path: blob/master/Key Python Libraries/Key Python Libraries - Day 1.ipynb
³⁰⁷⁴ views

Kernel: Python 3 (ipykernel)

Python library is a collection of functions and methods that allows you to perform lots of actions without writing your own code.Here, a ‘library’ loosely describes a collection of core modules.

Data Science and Machine Learning

Data Analysis

Numpy and Scipy – Fundamental Scientific Computing
Pandas – Data Manipulation and Analysis
StatsModels – Statistical Modeling, Testing, and Analysis

Data Visualization

Matplotlib: is an easy-to-use, low-level data visualization library
Seaborn: is a high-level interface built on top of the Matplotlib
Bokeh: is mainly famous for its interactive charts visualization.
Plotly: has hover tool capabilities that allow us to detect any outliers or anomalies in numerous data points. - It allows more customization. - It makes the graph visually more attractive

Machine Learning

Scikit-learn – Machine Learning and Data Mining
Keras- Machine learning
PyTorch- Machine Learning Tensor Flow
LightGBM- Machine Learning(Model Optimization)
Tensor Flow - Machine Learning
Eli5- Machine learning models(Debugging)
Theano- AI and ML
NLTK - Natural Language Processing with Python
SQLAlchemy- Database access

Web Development Frameworks and GUI

Tkinter – GUI applications
Requests – HTTP requests
Django- Web framework
Scrapy- Web crawling frameworks
BeautifulSoup - web crawling and data scraping
Flask - Micro-web framework

Others

Pyspark - Support Python with Spark, Apache Spark community released a tool, PySpark
TabPy - TabPy (the Tableau Python Server) is an Analytics Extension implementation which expands Tableau’s capabilities by allowing users to execute Python scripts and saved functions via Tableau’s table calculations
Jython- Jython an implementation of the Python programming language designed to run on the Java platform

python --version

Module,lib and package

Any Python file is a module, its name being the file's base name without the .py extension. A module is basically a bunch of related code saved in a file with the extension .py. You may choose to define functions, classes, or variables in a module. It’s also fine to include runnable code in modules.
A package is a collection of Python modules: while a module is a single Python file, a package is a directory of Python modules containing an additional init.py file, to distinguish a package from a directory that just happens to contain a bunch of Python scripts. Packages can be nested to any depth, provided that the corresponding directories contain their own init.py file.Python packages are basically a directory of a collection of modules.
A library is an umbrella term referring to a reusable chunk of code. Usually, a Python library contains a collection of related modules and packages. Actually, this term is often used interchangeably with “Python package” because packages can also contain modules and other packages (subpackages).

However, it is often assumed that while a package is a collection of modules, a library is a collection of packages.

[1,2,3]+[1,2,3] 
[1,2]*[1,2]
[1,2,"a"]
loops 
for i

In [3]:

import numpy as np
a=np.array([1,23])

In [5]:

a=[1,2,3]
a+a

Out[5]:

[1, 2, 3, 1, 2, 3]

In [7]:

a1=np.array(a)
a1+a1

Out[7]:

array([2, 4, 6])

In [9]:

x = np.arange(20, 10,-2)
x

Out[9]:

array([20, 18, 16, 14, 12])

scipy.special package contains numerous functions of mathematical physics.

SciPy special function includes Cubic Root, Exponential, Log sum Exponential, Lambert, Permutation and Combinations, Gamma, Bessel, hypergeometric, Kelvin, beta, parabolic cylinder, Relative Error Exponential, etc..

y=mx+c
Profit = 10(Sales)+30

In [10]:

##import scipy.special 
import numpy as np
import scipy.stats
x = np.arange(10, 20) 
y = np.array([2, 1, 4, 5, 8, 12, 18, 25, 96, 48])
result = scipy.stats.linregress(x, y)
print("slope=" ,result.slope)
print("Intercept= ",result.intercept)
#help(scipy.special)

Out[10]:

slope= 7.4363636363636365
Intercept=  -85.92727272727274

y=7.43x-86

In [11]:

import pandas as pd
pd.Series((1,2,34,35),index=["one","two","three","four"])

Out[11]:

one       1
two       2
three    34
four     35
dtype: int64

In [12]:

pd.DataFrame(((1,"A"),(2,"B"),(3,"C")))

Out[12]:

In [13]:

chicago_data = pd.read_csv("Chicago.csv")
chicago_data.head(2)

Out[13]:

In [14]:

chicago_data.describe()

Out[14]:

In [15]:

chicago_data.describe(include="object")

Out[15]:

In [16]:

chicago_data.isnull().sum()

Out[16]:

ID                        0
CASE_NUMBER               0
DATE                      0
BLOCK                     0
IUCR                      0
PRIMARY_TYPE              0
DESCRIPTION               0
LOCATION_DESCRIPTION      0
ARREST                    0
DOMESTIC                  0
BEAT                      0
DISTRICT                  0
WARD                     43
COMMUNITY_AREA_NUMBER    43
FBICODE                   0
X_COORDINATE              4
Y_COORDINATE              4
YEAR                      0
UPDATEDON                 0
LATITUDE                  4
LONGITUDE                 4
LOCATION                  4
dtype: int64

pip install wordcloud --trusted-host pypi.org --trusted-host files.pythonhosted.org sweetviz

##pip install pandas-profiling
#!pip install "http://localhost:8888/tree/Documents/PythonML/lib_python/pandas-profiling-3.0.0.tar.gz"
#3pip install wordcloud --trusted-host pypi.org --trusted-host files.pythonhosted.org Pandas profiling

pip install wordcloud --trusted-host pypi.org --trusted-host files.pythonhosted.org pandas profiling

! pip install "http://localhost:8888/tree/Documents/PythonML/lib_python/autoviz-0.0.83.tar.gz"

import pandas_profiling as pro
profile = pro.ProfileReport(chicago_data)
profile

In [ ]:

import pandas_profiling as pro
profile = pro.ProfileReport(chicago_data)
profile
profile.to_notebook_iframe() ## Utilizing iframe() to set up a frame inside the window ()

In [18]:

profile.to_file(output_file='Pandas ProfilingReport.html')

Out[18]:

Export report to file:   0%|          | 0/1 [00:00<?, ?it/s]

pip install sweetviz

In [19]:

#pip install sweetviz
import sweetviz as sv
sweet_report = sv.analyze(chicago_data)
sweet_report.show_html('sweet_report.html')

Out[19]:

                                             |                                             | [  0%]   00:00 ->…

Report sweet_report.html was generated! NOTEBOOK/COLAB USERS: the web browser MAY not pop up, regardless, the report IS saved in your notebook/colab files.

In [ ]:

df1 = sv.compare(chicago_data[0:],chicago_data [:3])
df1.show_html('Compare.html')

In [ ]:

!pip

##pip install autoviz
from autoviz.AutoViz_Class import AutoViz_Class
AV = AutoViz_Class()

viz = AV.AutoViz("chicago_data.csv")

In [20]:

import matplotlib
%matplotlib inline

In [22]:

plt = chicago_data.ARREST.value_counts().plot(kind='bar',color="green")
plt.set_xlabel('ARREST OR DIED')
plt.set_ylabel(' Count')
plt.legend(fontsize='x-large')

Out[22]:

<matplotlib.legend.Legend at 0x175ded1f2e0>

In [23]:

import seaborn as sns #Visualization
sns.set()
sns.swarmplot(x='ARREST', y='WARD', data=chicago_data)
##sns.violinplot('SoloPassenger','Survived',hue='Pclass',data=)

Out[23]:

<AxesSubplot:xlabel='ARREST', ylabel='WARD'>