CoCalc -- ShortVersion.ipynb

Real-time collaboration for Jupyter Notebooks, Linux Terminals, LaTeX, VS Code, R IDE, and more,
all in one place. Commercial Alternative to JupyterHub.

Views: ¹³⁶

Kernel: Python 3 (Anaconda 5)

In [1]:

import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
from datetime import timedelta

from sklearn.model_selection import train_test_split
from sklearn.linear_model import LogisticRegression

from sklearn.metrics import confusion_matrix

In [2]:

df_all_weather = pd.read_csv('2008-2018 Weather Data.csv')

In [3]:

df_all_weather.head()

Out[3]:

In [4]:

df_beaches = pd.read_csv('DOHMH_Beach_Water_Quality_Data.csv')

In [5]:

df_beaches.head()

Out[5]:

In [6]:

df_beaches.fillna(0,inplace=True)
df_beaches['Sample Date'] = pd.to_datetime(df_beaches['Sample Date'])

In [7]:

df_beaches.head()

Out[7]:

In [8]:

df_man = df_beaches[df_beaches['Beach Name'] == 'MANHATTAN BEACH']

In [9]:

df_man.head()

Out[9]:

In [10]:

df_man = df_man[['Sample Date','Enterococci Results']]

In [11]:

df_man.head()

Out[11]:

In [12]:

df_man.columns = ['DATE','Enterococci']

In [13]:

df_man.head()

Out[13]:

In [14]:

df_man['DATE'] = pd.to_datetime(df_man['DATE'])

In [15]:

df_man.sort_values(by='DATE',inplace=True)

In [16]:

df_man.head()

Out[16]:

In [17]:

df_man.index = range(len(df_man.index))

In [18]:

df_man.head()

Out[18]:

In [19]:

df_jfk = df_all_weather[df_all_weather.NAME == 'JFK INTERNATIONAL AIRPORT, NY US']

In [20]:

df_jfk.head()

Out[20]:

In [21]:

df_jfk.columns

Out[21]:

Index(['STATION', 'NAME', 'LATITUDE', 'LONGITUDE', 'ELEVATION', 'DATE', 'AWND',
       'DAPR', 'FMTM', 'MDPR', 'MDSF', 'PGTM', 'PRCP', 'SNOW', 'SNWD', 'TAVG',
       'TMAX', 'TMIN', 'TOBS', 'TSUN', 'WDF2', 'WDF5', 'WESD', 'WESF', 'WSF2',
       'WSF5', 'WT01', 'WT02', 'WT03', 'WT04', 'WT05', 'WT06', 'WT07', 'WT08',
       'WT09', 'WT11', 'WT13', 'WT14', 'WT15', 'WT16', 'WT17', 'WT18', 'WT19',
       'WT21', 'WT22'],
      dtype='object')

In [22]:

df_jfk = df_jfk[['DATE','PRCP']]

In [23]:

df_jfk['DATE'] = pd.to_datetime(df_jfk['DATE'])

In [24]:

df_jfk.sort_values(by='DATE',inplace=True)

In [25]:

df_merged = pd.merge(df_man, df_jfk, how='inner', on='DATE')

In [26]:

df_merged.head()

Out[26]:

In [27]:

df_merged.head()

Out[27]:

In [28]:

def eWarn(m):
    if m > 104:
        return 1
    else:
        return 0

In [29]:

df_merged['Warning'] = df_merged.Enterococci.apply(eWarn)

In [30]:

df_merged.head()

Out[30]:

In [31]:

df_merged.Warning.value_counts()

Out[31]:

0    649
1     38
Name: Warning, dtype: int64

In [32]:

df_merged.plot.scatter(x='PRCP',y='Warning',color='black')
plt.title('Warning vs. PRCP')

Out[32]:

Text(0.5,1,'Warning vs. PRCP')

In [33]:

X = np.array(df_merged.PRCP)
y = df_merged.Warning

In [34]:

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.33, random_state=101)

In [35]:

logmodel = LogisticRegression(solver='liblinear')

In [36]:

logmodel.fit(X_train.reshape(-1,1),y_train)

Out[36]:

LogisticRegression(C=1.0, class_weight=None, dual=False, fit_intercept=True,
          intercept_scaling=1, max_iter=100, multi_class='warn',
          n_jobs=None, penalty='l2', random_state=None, solver='liblinear',
          tol=0.0001, verbose=0, warm_start=False)

In [37]:

b0 = logmodel.intercept_
b1 = logmodel.coef_

In [38]:

X2 = sorted(X)
df_merged.plot.scatter(x='PRCP',y='Warning',color='black')
plt.plot(X2,1/(1+np.exp(-b0-b1*X2)).reshape(-1,1),'r')
plt.title('Warning vs. PRCP')

Out[38]:

Text(0.5,1,'Warning vs. PRCP')

In [39]:

predictions = logmodel.predict(X_test.reshape(-1,1))

In [40]:

print(confusion_matrix(y_test,logmodel.predict(X_test.reshape(-1, 1))))

Out[40]:

[[213   0]
 [ 14   0]]

Real-time collaboration for Jupyter Notebooks, Linux Terminals, LaTeX, VS Code, R IDE, and more,
all in one place. Commercial Alternative to JupyterHub.

Product

Resources

Company

Real-time collaboration for Jupyter Notebooks, Linux Terminals, LaTeX, VS Code, R IDE, and more, all in one place. Commercial Alternative to JupyterHub.

Real-time collaboration for Jupyter Notebooks, Linux Terminals, LaTeX, VS Code, R IDE, and more,
all in one place. Commercial Alternative to JupyterHub.