GitHub Repository: suyashi29/python-su
Path: blob/master/Data Visualization using Python/ Numpy.ipynb
³⁰⁷⁴ views

Kernel: Python 3 (ipykernel)

Numerical analysis , Data prepartion

Convert some data points into arrays for numerical analysis
optimize our data rounding , mathemtheical implementation Raw data - Information- Visualization- Model Mathematical - Points- Actions: A and B supply: A and B Bigdata , Data Analytics, Machine Learning, AI(Deep learning, NLP)

In [ ]:

Numbers (Int, Float, Complex)
Strings ("a","asb")

Data Containers 

List=[1,2,3],[[1,2],[2,3]]
1-[1,"a",3.5]

In [ ]:

a="abcd"
l=["1",2]
l[0]
l[1]=9
l
l.append(8)

In [ ]:

d={}
d={"A":1}
d

In [ ]:

x=[1,2,3]
y=[22,22,22]

x+y

Numpy:Introduction

NumPy is a Open Source Python package. It stands for Numerical Python. It is a library consisting of multidimensional array objects and a collection of routines for processing of array.
NumPy is the fundamental package for scientific computing with Python , having following important functionalities:
- A powerful N-dimensional array object
- A sophisticated (broadcasting) functions
- Contains tools for integrating C/C++ and Fortran code
- Have useful linear algebra, Fourier transform, and random number capabilities

Why NUMpy?

Mathematical and logical operations on arrays.
Efficient storage and manipulation of numerical arrays is which is fundamental in the process of data science.
NumPy arrays form the core of nearly the entire ecosystem of data science tools in Python,

Installation

Anaconda: A free distribution of Python with scientific packages. Supports Linux, Windows and Mac
```
  To install numpy type : conda install -c anaconda numpy
```
pip:Most major projects upload official packages to the Python Package index. They can be installed on most operating systems using Python’s standard pip package manager.

You can install packages via one of the following Commands:

 python -m pip install --user numpy scipy matplotlib ipython jupyter pandas sympy nose
 
 pip install numpy

import numpy 
help()

Importing Numpy & Checking Version

In [1]:

import numpy 
numpy.__version__

Out[1]:

'1.21.5'

Bounding numpy package to local variable

The numpy package is bound to the local variable numpy. The import as syntax simply allows you to bind the import to the local variable name of your choice (usually to avoid name collisions, shorten verbose module names, or standardize access to modules with compatible APIs).

In [2]:

import numpy as np
a=np.array((2,3,4,5))
a

Out[2]:

array([2, 3, 4, 5])

In [ ]:

Numpy : Array
Pandas: Data Frames 
Analysis

import time 
import numpy as np
size_1=1000000
def pure_python_version():
    time_python =time.time()
    list1=range(size_1)
    list2=range(size_1)
    sum_list=[list1[i]+list2[i] for i in range(len(list1))]
    return time.time() - time_python
def numpy_version():
    time_numpy = time.time()
    array1 = np.arange(size_1)
    array2 =  np.arange(size_1)
    sum_array = array1 + array2
    return time.time() - time_numpy
python_time = pure_python_version()
numpy_time  = numpy_version()

print("pure python version=",format(python_time))
print("numpy =",format(numpy_time))

Creating Arrays

The basic ndarray is created using an array function in NumPy which creates an ndarray from any object exposing array interface, or from any method that returns an array.

a=[] - index  - 1D
b= [[1,2]] - R*C - 2D
c=[[[1,2]],[[1,2]] - 3D : A*R*C

In [5]:

import numpy as np
m=[2,3,4,6,7,8] #1D 
d1=np.array(m)
d1 
n =(1,2,6)
n1=np.array(n)
d1
n1

Out[5]:

array([1, 2, 6])

In [6]:

a = [[1,2,3],[4,5,6],[8,-2,3]] # 2D ,R*C
b=[[1,2,4],[2,3,66]]
d2= np.array(a)
d2
n2=np.array(b)
d2
n2,d2

Out[6]:

(array([[ 1,  2,  4],
        [ 2,  3, 66]]),
 array([[ 1,  2,  3],
        [ 4,  5,  6],
        [ 8, -2,  3]]))

In [7]:

b=[[[1,2,3],[1,-2,0],[0,1,1]],[[4,6,3],[6,8,15],[1,0,-2]],[[11,-2,-3],[5,-6,14],[1,-1,0]]]
d3=np.array(b)#3D A*R*C 
d3
print(d3)

##c=[[[2,1],[3,2]],[[2,3,4],[3,4,5]],[[1,3],[4,5]]]
##n3=np.array(c,dtype=int)
#3n3

Out[7]:

[[[ 1  2  3]
  [ 1 -2  0]
  [ 0  1  1]]

 [[ 4  6  3]
  [ 6  8 15]
  [ 1  0 -2]]

 [[11 -2 -3]
  [ 5 -6 14]
  [ 1 -1  0]]]

Acess the members

In [8]:

d1

Out[8]:

array([2, 3, 4, 6, 7, 8])

In [11]:

d1[-1]
d1[2:4]

Out[11]:

array([4, 6])

In [12]:

d2

Out[12]:

array([[ 1,  2,  3],
       [ 4,  5,  6],
       [ 8, -2,  3]])

In [13]:

## Access member
d2[1,2]

Out[13]:

6

In [14]:

d2[0,0], d2[1,1],d2[2,2]

Out[14]:

(1, 5, 3)

In [15]:

#Access Row
d2[2]

Out[15]:

array([ 8, -2,  3])

In [16]:

 ##AcCessing Columns
d2[:,2]

Out[16]:

array([3, 6, 3])

In [18]:

d2[2:]

Out[18]:

array([[ 8, -2,  3]])

In [19]:

## Indexing in 3d array[axis,Row,Column]
d3

Out[19]:

array([[[ 1,  2,  3],
        [ 1, -2,  0],
        [ 0,  1,  1]],

       [[ 4,  6,  3],
        [ 6,  8, 15],
        [ 1,  0, -2]],

       [[11, -2, -3],
        [ 5, -6, 14],
        [ 1, -1,  0]]])

In [20]:

d3[0]

Out[20]:

array([[ 1,  2,  3],
       [ 1, -2,  0],
       [ 0,  1,  1]])

In [21]:

d3[1,2]

Out[21]:

array([ 1,  0, -2])

In [22]:

d3[2,:,1]

Out[22]:

array([-2, -6, -1])

In [23]:

d3[1,1,1]

Out[23]:

8

In [ ]:

d3[2,1,1]

Array Atrributes

Numpy Arrays are conveinent and fast as compared to Python Lists. 1.Shape(Elements,rowcolumn,axisrow*column)

In [24]:

print(d2.shape) #Rows*Column
print(d1.shape)#1D
print(d3.shape)#axis*Rows*Coulmn

Out[24]:

(3, 3)
(6,)
(3, 3, 3)

In [25]:

print(d2.ndim )#dimension of array
print(d3.ndim )
print(d1.ndim)

Out[25]:

2
3
1

ndim for checking dimension of array

dtype for checking data type of array
d2.size : FOR NUMBER OF ELEMENTS

In [27]:


print(d2.size,d3.size,d1.size,sep=",") # total number of elements

Out[27]:

9,27,6

In [26]:

d2.dtype
a=np.array([3,1.3])
a.dtype

Out[26]:

dtype('float64')

Array Initilization

In [28]:

a=list(range(10,1,-2))#:start,stop-1,interval
a

Out[28]:

[10, 8, 6, 4, 2]

In [29]:

np.arange(1,20,3)

Out[29]:

array([ 1,  4,  7, 10, 13, 16, 19])

In [ ]:

#Create an array of odd numbers between 1-20
np.arange(1,20,2) #(start,end-1,space) 
##Create an array of even numbers between 30-10
np.arange(30,10,-2)

In [30]:

np.zeros((2,2) ,dtype = int)

Out[30]:

array([[0, 0],
       [0, 0]])

In [ ]:

np.zeros((6),dtype=int)#null matrix

In [ ]:

np.zeros((2,3,2)) #A*R*C

In [33]:

np.full((2,2),fill_value=5,dtype=None)

Out[33]:

array([[5, 5],
       [5, 5]])

In [31]:

np.ones((3,2))#unit matrix

Out[31]:

array([[1., 1.],
       [1., 1.],
       [1., 1.]])

In [34]:

## eye function only works for  2D square matrix : Return a 2-D array with ones on the diagonal and zeros elsewhere
np.eye(2)

Out[34]:

array([[1., 0.],
       [0., 1.]])

In [36]:

np.linspace(10,20,5)#(start,stop,number of elements)

Out[36]:

array([10. , 12.5, 15. , 17.5, 20. ])

Array Initilization with Random numbers

In various applications( like assigning weights in Artificial Neural Networks) arrays need to be initialised randomly. for this purpose there are various predefined functions in Numpy(reshape and random)

In [43]:

np.random.rand(2,2)

Out[43]:

array([[0.0202184 , 0.83261985],
       [0.77815675, 0.87001215]])

In [58]:

## Use seed to fix random generated values
np.random.seed(0)
x=np.random.rand(2,2)
#np.random.rand(2,2)
x
print(x)

Out[58]:

[[0.5488135  0.71518937]
 [0.60276338 0.54488318]]

In [68]:

x=np.random.randint(1,10)
x

Out[68]:

7

In [71]:

y=np.array([1,2,3,4,56,78,6,7,7])
a=y.reshape(3,3)
a

Out[71]:

array([[ 1,  2,  3],
       [ 4, 56, 78],
       [ 6,  7,  7]])

In [72]:

d3

Out[72]:

array([[[ 1,  2,  3],
        [ 1, -2,  0],
        [ 0,  1,  1]],

       [[ 4,  6,  3],
        [ 6,  8, 15],
        [ 1,  0, -2]],

       [[11, -2, -3],
        [ 5, -6, 14],
        [ 1, -1,  0]]])

In [73]:

z=d3.flatten() #transforms 
z

Out[73]:

array([ 1,  2,  3,  1, -2,  0,  0,  1,  1,  4,  6,  3,  6,  8, 15,  1,  0,
       -2, 11, -2, -3,  5, -6, 14,  1, -1,  0])

Mathematical and stats functions

In [74]:

import numpy as np
a=[[1,2,13,0],[4,5,6,78],[-8,8,-6,9]]
z2=np.array(a)
z2

Out[74]:

array([[ 1,  2, 13,  0],
       [ 4,  5,  6, 78],
       [-8,  8, -6,  9]])

In [76]:

z2[:,0].sum()

Out[76]:

-3

1,3,-1,8,5,6 -1,1,3,5,6,8,9

,A,A,A,B,C,D

z2.sum()
z2.mean()
z2.std()
z2.var()
np.median(z2)

In [ ]:

np.median(z2)

In [ ]:

z2[1].std()  
z2[1].var()

In [ ]:

np.median(z2[2])

In [87]:

z2

Out[87]:

array([[ 1,  2, 13,  0],
       [ 4,  5,  6, 78],
       [-8,  8, -6,  9]])

In [90]:

np.argmin(z2,axis=0) # returns the indices of min element  along colums

Out[90]:

array([1, 2, 0, 1], dtype=int64)

In [89]:

np.argmin(z2,axis=1)

Out[89]:

array([3, 0, 0], dtype=int64)

In [ ]:

## axis=0 means that the operation is performed down the columns of a 2D array a in turn.
z2
np.argmin(z2,axis=1) # returns the indices of min element  along rows
np.argmax(z2,axis=1)# returns the indices of max element along rows
# On the other hand, axis=1 means that the operation is performed across the rows of array
np.argmin(z,axis=1) # returns the indices of min element  along rows
np.argmax(z,axis=0) # returns the indices of max element  along columns

In [ ]:

z2

In [ ]:

np.argmin(z2,axis=0)

Transpose doesn't change the number of dimensions, just reverses their order

In [77]:

z2

Out[77]:

array([[ 1,  2, 13,  0],
       [ 4,  5,  6, 78],
       [-8,  8, -6,  9]])

In [78]:

z2.T

Out[78]:

array([[ 1,  4, -8],
       [ 2,  5,  8],
       [13,  6, -6],
       [ 0, 78,  9]])

a=np.array([[1,2],[3,2]])
z2
a+z2

In [79]:

import numpy as np
A = np.array([ [4, 10, 11], [21, 22, 23], [31, 32, 33] ])
B = np.ones((3,1))
print("Matrix Multiplication")
print(np.dot (A,B))
np.dot(A,B)

Out[79]:

Matrix Multiplication
[[25.]
 [66.]
 [96.]]

array([[25.],
       [66.],
       [96.]])

In [80]:

z2*z2

Out[80]:

array([[   1,    4,  169,    0],
       [  16,   25,   36, 6084],
       [  64,   64,   36,   81]])

In [82]:

np.dot(A,B)

Out[82]:

array([[25.],
       [66.],
       [96.]])

In [83]:

np.sqrt(z2)

Out[83]:

C:\Users\suyashi144893\AppData\Local\Temp\1\ipykernel_12340\1777213199.py:1: RuntimeWarning: invalid value encountered in sqrt
  np.sqrt(z2)

array([[1.        , 1.41421356, 3.60555128, 0.        ],
       [2.        , 2.23606798, 2.44948974, 8.83176087],
       [       nan, 2.82842712,        nan, 3.        ]])

In [84]:

np.exp(A)

Out[84]:

array([[5.45981500e+01, 2.20264658e+04, 5.98741417e+04],
       [1.31881573e+09, 3.58491285e+09, 9.74480345e+09],
       [2.90488497e+13, 7.89629602e+13, 2.14643580e+14]])

In [85]:


a = np.array([0,30,45,60,90]) 

print ('Sine of different angles:' )
# Convert to radians by multiplying with pi/180 
print (np.sin(a*np.pi/180) )
print ("\n") 

print ('Cosine values for angles in array:')
print (np.cos(a*np.pi/180) )
print ("\n") 

print ('Tangent values for given angles:' )
print (np.tan(a*np.pi/180) )

Out[85]:

Sine of different angles:
[0.         0.5        0.70710678 0.8660254  1.        ]


Cosine values for angles in array:
[1.00000000e+00 8.66025404e-01 7.07106781e-01 5.00000000e-01
 6.12323400e-17]


Tangent values for given angles:
[0.00000000e+00 5.77350269e-01 1.00000000e+00 1.73205081e+00
 1.63312394e+16]

In [86]:

a=np.array([[12.267111,23.662],[33.21,45.887]])
#print (np.around(a)) 
print (np.around(a, decimals = 1))
print (np.around(a, decimals = 3))

Out[86]:

[[12.3 23.7]
 [33.2 45.9]]
[[12.267 23.662]
 [33.21  45.887]]

help()

Binary Universal Functions

In [91]:

import numpy as np 
A=[[1,3,3],[3,4,3]]
B=[[3,3,4],[1,2,0]]
a=np.array(A)
b=np.array(B)
np.greater_equal(a,b)#greater ,less ,equal
#np.all(A) ## If all members ahave non- null values then True

Out[91]:

array([[False,  True, False],
       [ True,  True,  True]])

In [93]:

np.all(B)

Out[93]:

False

In [ ]:

import numpy as np
np.all(b) #returns true for all non-zero elements

Concatenating Arrays

In [95]:

import numpy as np
A=np.array([[1,2],[2,3]]) 
B=np.array([[1,2]])

In [97]:

A , B

Out[97]:

(array([[1, 2],
        [2, 3]]),
 array([[1, 2]]))

In [101]:

X=np.ones((2,3), dtype=int)
X

## [1,2,3],[2,3,4]= [1,2,3,2,3,4]= number of coulmns 
## [1,2],[[[1,2],[[3,2]]]= [[1,2],[1,2],[3,2]]= numbers of rows

Out[101]:

array([[1, 1, 1],
       [1, 1, 1]])

In [102]:

a=np.array([[3,2]])
a

Out[102]:

array([[1, 2]])

In [105]:

np.concatenate([a,B],axis=0) # On basis of row axis=1

Out[105]:

array([[1, 2],
       [1, 2]])

In [ ]:

A=np.array([[1,2],[2,3]]) ##- 2*2
B=np.array([[1,2],[2,6]])  ## 2*2

In [ ]:

np.concatenate([A,B],axis=1)

Structured Array Creation

import numpy as np
data_type = [('Emp ID', 'S15'), ('Jb role', "S14"), ('Salary', float)]
Emp_data = [('101A', 'Manager', 980000.0), ('101B', 'Analysts', 52000.0),('101C', 'SALE EXECUTIVE', 45000.0), ('101D', 'VP', 200000.0)]
Emp = np.array(Emp_data)
Emp

Broadcasting

Broadcasting is a method to overcome size of the smaller array by duplicacy so that it is the dimensionality and size as the larger array.

Why Broadcasting?

To solve the problem of arithmetic with arrays with different sizes.

In [106]:

z2  #3*4

Out[106]:

array([[ 1,  2, 13,  0],
       [ 4,  5,  6, 78],
       [-8,  8, -6,  9]])

In [114]:

z1=np.array([1,2,3,6]) #1*3

In [115]:

z1*z2

Out[115]:

array([[  1,   4,  39,   0],
       [  4,  10,  18, 468],
       [ -8,  16, -18,  54]])

n*m = Higher size array
1*m= Number of colums of higher array and lower array should be same
There should be single row in lower array

[1,2,3] -[[1,2,3],[6,7,8]]

case1: First array should have 1*C order, C is num of colum in second array
case2: int, float
first array gets duplicated according to the shape of higher array

In [ ]:

a=np.array([[2,1,3]]) #1*3
b=np.array([[1,2,4],[4,6,4],[1,2,3]]) #3*3
a*b

1- Lower array should be in (1*C)

In [ ]:

import numpy as np
d1=np.array([[1,2,1]]) #1*3
d2=np.array([[1,2,3],[1,-2,0],[2,3,1]]) #3*3

d1+d2

In [ ]:

a = np.array([[1 ,2 ,3] ,[4,5,6],[2,3,4]]) - 3*3
b = np.array([[1,1,1]]) - 2*3
a+b

Array with different shapes

In [ ]:

a=np.array([[[1,2,3]]])
np.ndim(a)

In [ ]:

a1=np.array([1,2,3])
a2=np.array([[1,2,4],[1,1,1],[1,1,1]])
np.dot(a2,a1)

In [ ]:


a = np.array([[0.0,0.0,0.0],[10.0,10.0,10.0],[20.0,20.0,20.0],[30.0,30.0,30.0]]) 
b = np.array([0.0,1.0,2.0])  
   
print ('First array:\n',a )
print ('Second array:\n',b )
print ("\n")   
print ('First Array + Second Array=\n',a+b)

In [ ]:

a=np.array([[1,2],[0,8]])
b=np.array([[1,2,3,-5],[2,2,7,-1]])
b.shape
a.shape

File Input Output

In [116]:

## File Input Output

#The np.save stores the input array in a disk file with npy extension.

a = np.array([14,20,30,4,5,13]) 
np.save('demo',a)
# To reconstruct array from demo.npy, use load() function.
b = np.load('demo.npy')
b

## he storage and retrieval of array data in 
#simple text file format is done with savetxt() and loadtxt() functions.
np.savetxt('demo.txt',a) 
b = np.loadtxt('demo.txt') 
print (b)

Out[116]:

[14. 20. 30.  4.  5. 13.]

In [ ]: