CoCalc -- IsolateCO2&Concatenating.ipynb

GEP475GROUPINEEDANAP

Justin Hoijer - GEP 475 SP2018 Course

InfiltrationHUB / Methods / 1.RawDataFixing_Concat_CO2.isolation / IsolateCO2&Concatenating.ipynb

³⁸⁹⁵ views

Kernel: Python 3 (Anaconda)

Creating New "CO2 only" csv files, from orginal Netatmo csvs

making sure both files have same index and same column name for CO2

In [13]:

import numpy as np
import pandas as pd

In [2]:

year1 = pd.read_csv('../DataFiles/Raw/NetAtmo_2016.csv', parse_dates=True, index_col=1)
year1.head()

Out[2]:

In [3]:

year1.describe()

Out[3]:

In [0]:

In [4]:

year2 = pd.read_csv('NetAtmo_2017.csv', parse_dates=True, index_col=1)
year2.head()

Out[4]:

above you can see, the 2016 csv (year1) did not have the same index as the 2017 file.

How to change:

In [5]:

year1.index.names = ['Time']
year1.head()

Out[5]:

Now that we have the same index, we'll go ahead and create the new files.

isolating the CO2 data

In [6]:

firstyear = year1['CO2']
#firstyear.head()

In [7]:

secondyear = year2['CO2']
#secondyear.head()

using to_csv

In [8]:

firstyear.to_csv('Netatmo2016CO2ONLY.csv')

In [9]:

column_names = ['Time','ppm']
year1CO2 = pd.read_csv('Netatmo2016CO2ONLY.csv', parse_dates=True, index_col=0, names=column_names)
year1CO2.head()

Out[9]:

In [10]:

#year1CO2.describe()

In [11]:

secondyear.to_csv('Netatmo2017CO2ONLY.csv')

In [12]:

column_names = ['Time','ppm']
year2CO2= pd.read_csv('Netatmo2017CO2ONLY.csv', parse_dates=True, index_col=0, names=column_names)
year2CO2.head()

Out[12]:

Next, I want to create another csv file that has the column names already inside. (I dont know a better way)*

In [14]:

year1CO2.to_csv('2016CO2ONLY.csv')
year1 =  pd.read_csv('2016CO2ONLY.csv', parse_dates=True, index_col=0,)
#year1.head()

In [15]:

year2CO2.to_csv('2017CO2ONLY.csv')
year2 =  pd.read_csv('2017CO2ONLY.csv', parse_dates=True, index_col=0,)
year2.head()

Out[15]:

Now we are left with the last two files, 2016 and 2017 CO2ONLY, and need to combine them

Using Concatenate

In [16]:

df1 = pd.read_csv('2016CO2ONLY.csv', parse_dates=True)
#df1.head()
df2 = pd.read_csv('2017CO2ONLY.csv', parse_dates=True)
#df2.head()

In [17]:

combined = pd.concat([df1,df2])

These cells are just to convince myself the data was combined correctly

In [18]:

combined.describe()

Out[18]:

In [19]:

combined.head()

Out[19]:

In [20]:

#df1.describe()

In [21]:

#df2.describe()

In [22]:

#df1.head()

In [23]:

#combined.head()

In [24]:

#df2.tail()

In [25]:

#combined.tail()

Now, finally, we make the last csv file. It will include all the CO2 data from years 2016 and 2017

In [26]:

combined.to_csv('Combinedwithcolumnnanes.csv')

In [27]:

netatmodata = pd.read_csv('Combinedwithcolumnnanes.csv', parse_dates=True, index_col=1)

netatmodata.head()

Out[27]:

here, I'd like to make another csv without the extra columb.. HELP SOTO

In [28]:

both = netatmodata['ppm']

In [29]:

both.head()

Out[29]:

Time
2016-02-19 13:26:00      NaN
2016-02-19 13:27:00    718.0
2016-02-19 13:27:00      NaN
2016-02-19 13:31:00    337.0
2016-02-19 13:36:00    332.0
Name: ppm, dtype: float64

In [30]:

both.to_csv('Netatmo2016_2017CO2ppm.csv')

In [31]:

netatmodata = pd.read_csv('Netatmo2016_2017CO2ppm.csv', parse_dates=True, index_col=0, )

netatmodata.head()

Out[31]:

In [0]:

Creating New "CO2 only" csv files, from orginal Netatmo csvs

above you can see, the 2016 csv (year1) did not have the same index as the 2017 file.

How to change:

Now that we have the same index, we'll go ahead and create the new files.

using to_csv

Next, I want to create another csv file that has the column names already inside. (I dont know a better way)*

Now we are left with the last two files, 2016 and 2017 CO2ONLY, and need to combine them

These cells are just to convince myself the data was combined correctly

Now, finally, we make the last csv file. It will include all the CO2 data from years 2016 and 2017

here, I'd like to make another csv without the extra columb.. HELP SOTO

Product

Resources

Company