Heatmap of monthly mean Snow Water Equivalent (SWE) over 2 decades period at Mores Creek Summit (MCS)

Share This Post

This is an embedded juipyter notebook file that I created to make a heatmap of SWE over 2 decades period at Mores Creek Summit, Boise. The data used in this article can be downloaded from USDA.

As usual, the first thing is to import all required packages.
# import modules
import numpy as np
import matplotlib.pyplot as plt
import pandas as pd
import glob
import os

import matplotlib.pyplot as plt
import seaborn as sns
#Print the current working directory
os.getcwd()
Out[ ]:
'/home/naheemadebisi/PhD/snow-analytics/Lowman'
# use the glob function of the glob modeule to return a list of all csv files

data_dir = glob.glob('MCS_SNOWTEL/SWE/*.csv')
data_dir
Out[ ]:
['MCS_SNOWTEL/SWE/637_26_WATERYEAR=2015.csv',
 'MCS_SNOWTEL/SWE/637_26_WATERYEAR=2021.csv',
 'MCS_SNOWTEL/SWE/637_26_WATERYEAR=2009.csv',
 'MCS_SNOWTEL/SWE/637_26_WATERYEAR=2019.csv',
 'MCS_SNOWTEL/SWE/637_26_WATERYEAR=2017.csv',
 'MCS_SNOWTEL/SWE/637_26_WATERYEAR=2006.csv',
 'MCS_SNOWTEL/SWE/637_26_WATERYEAR=2002.csv',
 'MCS_SNOWTEL/SWE/637_26_WATERYEAR=2008.csv',
 'MCS_SNOWTEL/SWE/637_26_WATERYEAR=2012.csv',
 'MCS_SNOWTEL/SWE/637_26_WATERYEAR=2018.csv',
 'MCS_SNOWTEL/SWE/637_26_WATERYEAR=2005.csv',
 'MCS_SNOWTEL/SWE/637_26_WATERYEAR=2007.csv',
 'MCS_SNOWTEL/SWE/637_26_WATERYEAR=2000.csv',
 'MCS_SNOWTEL/SWE/637_26_WATERYEAR=2014.csv',
 'MCS_SNOWTEL/SWE/637_26_WATERYEAR=2003.csv',
 'MCS_SNOWTEL/SWE/637_26_WATERYEAR=2013.csv',
 'MCS_SNOWTEL/SWE/637_26_WATERYEAR=2022.csv',
 'MCS_SNOWTEL/SWE/637_26_WATERYEAR=2016.csv',
 'MCS_SNOWTEL/SWE/637_26_WATERYEAR=2020.csv',
 'MCS_SNOWTEL/SWE/637_26_WATERYEAR=2004.csv',
 'MCS_SNOWTEL/SWE/637_26_WATERYEAR=2001.csv',
 'MCS_SNOWTEL/SWE/637_26_WATERYEAR=2011.csv',
 'MCS_SNOWTEL/SWE/637_26_WATERYEAR=2010.csv']
#read the csv files into a list of dataframes
dataframes = []
for file in data_dir:
    dataframes.append(pd.read_csv(file, header=1,index_col= 'Date', usecols= ['Date', 'WTEQ.I-1 (in) '], na_values = -99.9))

dataframes
    
Out[ ]:
[            WTEQ.I-1 (in) 
 Date                      
 2014-10-01             0.0
 2014-10-02             0.0
 2014-10-03             0.0
 2014-10-04             0.0
 2014-10-05             0.0
 ...                    ...
 2015-09-27             0.0
 2015-09-28             0.0
 2015-09-29             0.0
 2015-09-30             0.0
 2015-09-30             NaN
 
 [366 rows x 1 columns],
             WTEQ.I-1 (in) 
 Date                      
 2020-10-01             0.0
 2020-10-02             0.0
 2020-10-03             0.0
 2020-10-04             0.0
 2020-10-05             0.0
 ...                    ...
 2021-09-27             0.0
 2021-09-28             0.0
 2021-09-29             0.0
 2021-09-30             0.0
 2021-09-30             NaN
 
 [366 rows x 1 columns],
             WTEQ.I-1 (in) 
 Date                      
 2008-10-01             0.0
 2008-10-02             0.0
 2008-10-03             0.0
 2008-10-04             0.0
 2008-10-05             0.0
 ...                    ...
 2009-09-27             0.0
 2009-09-28             0.0
 2009-09-29             0.0
 2009-09-30             0.0
 2009-09-30             NaN
 
 [366 rows x 1 columns],
             WTEQ.I-1 (in) 
 Date                      
 2018-10-01             0.0
 2018-10-02             0.0
 2018-10-03             0.0
 2018-10-04             0.0
 2018-10-05             0.0
 ...                    ...
 2019-09-27             0.0
 2019-09-28             0.0
 2019-09-29             0.0
 2019-09-30             0.0
 2019-09-30             NaN
 
 [366 rows x 1 columns],
             WTEQ.I-1 (in) 
 Date                      
 2016-10-01             0.0
 2016-10-02             0.0
 2016-10-03             0.0
 2016-10-04             0.0
 2016-10-05             0.0
 ...                    ...
 2017-09-27             0.0
 2017-09-28             0.0
 2017-09-29             0.0
 2017-09-30             0.0
 2017-09-30             NaN
 
 [366 rows x 1 columns],
             WTEQ.I-1 (in) 
 Date                      
 2005-10-01             0.0
 2005-10-02             0.0
 2005-10-03             0.0
 2005-10-04             0.0
 2005-10-05             0.0
 ...                    ...
 2006-09-27             0.0
 2006-09-28             0.0
 2006-09-29             0.0
 2006-09-30             0.0
 2006-09-30             NaN
 
 [366 rows x 1 columns],
             WTEQ.I-1 (in) 
 Date                      
 2001-10-01             0.0
 2001-10-02             0.0
 2001-10-03             0.0
 2001-10-04             0.0
 2001-10-05             0.0
 ...                    ...
 2002-09-27             0.0
 2002-09-28             0.0
 2002-09-29             0.0
 2002-09-30             0.0
 2002-09-30             NaN
 
 [366 rows x 1 columns],
             WTEQ.I-1 (in) 
 Date                      
 2007-10-01             0.0
 2007-10-02             0.0
 2007-10-03             0.0
 2007-10-04             0.0
 2007-10-05             0.0
 ...                    ...
 2008-09-27             0.0
 2008-09-28             0.0
 2008-09-29             0.0
 2008-09-30             0.0
 2008-09-30             NaN
 
 [367 rows x 1 columns],
             WTEQ.I-1 (in) 
 Date                      
 2011-10-01             0.0
 2011-10-02             0.0
 2011-10-03             0.0
 2011-10-04             0.0
 2011-10-05             0.0
 ...                    ...
 2012-09-27             0.0
 2012-09-28             0.0
 2012-09-29             0.0
 2012-09-30             0.0
 2012-09-30             NaN
 
 [367 rows x 1 columns],
             WTEQ.I-1 (in) 
 Date                      
 2017-10-01             0.0
 2017-10-02             0.0
 2017-10-03             0.0
 2017-10-04             0.0
 2017-10-05             0.0
 ...                    ...
 2018-09-27             0.0
 2018-09-28             0.0
 2018-09-29             0.0
 2018-09-30             0.0
 2018-09-30             NaN
 
 [366 rows x 1 columns],
             WTEQ.I-1 (in) 
 Date                      
 2004-10-01             0.0
 2004-10-02             0.0
 2004-10-03             0.0
 2004-10-04             0.0
 2004-10-05             0.0
 ...                    ...
 2005-09-27             0.0
 2005-09-28             0.0
 2005-09-29             0.0
 2005-09-30             0.0
 2005-09-30             NaN
 
 [366 rows x 1 columns],
             WTEQ.I-1 (in) 
 Date                      
 2006-10-01             0.0
 2006-10-02             0.0
 2006-10-03             0.0
 2006-10-04             0.0
 2006-10-05             0.0
 ...                    ...
 2007-09-27             0.0
 2007-09-28             0.0
 2007-09-29             0.0
 2007-09-30             0.0
 2007-09-30             NaN
 
 [366 rows x 1 columns],
             WTEQ.I-1 (in) 
 Date                      
 1999-10-01             0.0
 1999-10-02             0.0
 1999-10-03             0.0
 1999-10-04             0.0
 1999-10-05             0.0
 ...                    ...
 2000-09-27             0.0
 2000-09-28             0.0
 2000-09-29             0.0
 2000-09-30             0.0
 2000-09-30             NaN
 
 [367 rows x 1 columns],
             WTEQ.I-1 (in) 
 Date                      
 2013-10-01             0.0
 2013-10-02             0.0
 2013-10-03             0.0
 2013-10-04             0.0
 2013-10-05             0.0
 ...                    ...
 2014-09-27             0.0
 2014-09-28             0.0
 2014-09-29             0.0
 2014-09-30             0.0
 2014-09-30             NaN
 
 [366 rows x 1 columns],
             WTEQ.I-1 (in) 
 Date                      
 2002-10-01             0.0
 2002-10-02             0.0
 2002-10-03             0.0
 2002-10-04             0.0
 2002-10-05             0.0
 ...                    ...
 2003-09-27             0.0
 2003-09-28             0.0
 2003-09-29             0.0
 2003-09-30             0.0
 2003-09-30             NaN
 
 [366 rows x 1 columns],
             WTEQ.I-1 (in) 
 Date                      
 2012-10-01             0.0
 2012-10-02             0.0
 2012-10-03             0.0
 2012-10-04             0.0
 2012-10-05             0.0
 ...                    ...
 2013-09-27             0.0
 2013-09-28             0.0
 2013-09-29             0.0
 2013-09-30             0.0
 2013-09-30             NaN
 
 [366 rows x 1 columns],
             WTEQ.I-1 (in) 
 Date                      
 2021-10-01             0.0
 2021-10-02             0.0
 2021-10-03             0.0
 2021-10-04             0.0
 2021-10-05             0.0
 ...                    ...
 2022-03-16            21.7
 2022-03-17            21.6
 2022-03-18            21.6
 2022-03-19            21.5
 2022-03-20            21.4
 
 [171 rows x 1 columns],
             WTEQ.I-1 (in) 
 Date                      
 2015-10-01             0.0
 2015-10-02             0.0
 2015-10-03             0.0
 2015-10-04             0.0
 2015-10-05             0.0
 ...                    ...
 2016-09-27             0.0
 2016-09-28             0.0
 2016-09-29             0.0
 2016-09-30             0.0
 2016-09-30             NaN
 
 [367 rows x 1 columns],
             WTEQ.I-1 (in) 
 Date                      
 2019-10-01             0.0
 2019-10-02             0.0
 2019-10-03             0.0
 2019-10-04             0.0
 2019-10-05             0.0
 ...                    ...
 2020-09-27             0.0
 2020-09-28             0.0
 2020-09-29             0.0
 2020-09-30             0.0
 2020-09-30             NaN
 
 [367 rows x 1 columns],
             WTEQ.I-1 (in) 
 Date                      
 2003-10-01             0.0
 2003-10-02             0.0
 2003-10-03             0.0
 2003-10-04             0.0
 2003-10-05             0.0
 ...                    ...
 2004-09-27             0.0
 2004-09-28             0.0
 2004-09-29             0.0
 2004-09-30             0.0
 2004-09-30             NaN
 
 [367 rows x 1 columns],
             WTEQ.I-1 (in) 
 Date                      
 2000-10-01             0.0
 2000-10-02             0.0
 2000-10-03             0.0
 2000-10-04             0.0
 2000-10-05             0.0
 ...                    ...
 2001-09-27             0.0
 2001-09-28             0.0
 2001-09-29             0.0
 2001-09-30             0.0
 2001-09-30             NaN
 
 [366 rows x 1 columns],
             WTEQ.I-1 (in) 
 Date                      
 2010-10-01             0.0
 2010-10-02             0.0
 2010-10-03             0.0
 2010-10-04             0.0
 2010-10-05             0.0
 ...                    ...
 2011-09-27             0.0
 2011-09-28             0.0
 2011-09-29             0.0
 2011-09-30             0.0
 2011-09-30             NaN
 
 [366 rows x 1 columns],
             WTEQ.I-1 (in) 
 Date                      
 2009-10-01             0.0
 2009-10-02             0.0
 2009-10-03             0.0
 2009-10-04             1.0
 2009-10-05             1.7
 ...                    ...
 2010-09-27             0.0
 2010-09-28             0.0
 2010-09-29             0.0
 2010-09-30             0.0
 2010-09-30             NaN
 
 [366 rows x 1 columns]]
#concatenate the dataframes into one dataframe and sort based on date index
df = pd.concat(dataframes).sort_index()
df
Out[ ]:
WTEQ.I-1 (in)
Date
1999-10-01 0.0
1999-10-02 0.0
1999-10-03 0.0
1999-10-04 0.0
1999-10-05 0.0
2022-03-16 21.7
2022-03-17 21.6
2022-03-18 21.6
2022-03-19 21.5
2022-03-20 21.4

8229 rows × 1 columns

df.columns
Out[ ]:
Index(['WTEQ.I-1 (in) '], dtype='object')
#display the data frame info
df.info()
<class 'pandas.core.frame.DataFrame'>
Index: 8229 entries, 1999-10-01 to 2022-03-20
Data columns (total 1 columns):
 #   Column          Non-Null Count  Dtype  
---  ------          --------------  -----  
 0   WTEQ.I-1 (in)   8207 non-null   float64
dtypes: float64(1)
memory usage: 128.6+ KB
#find the min and max of the dataframe
print('The min of values in the dataframe is {} \nThe max is {}'.format(df['WTEQ.I-1 (in) '].min(),df['WTEQ.I-1 (in) '].max()))
The min of values in the dataframe is 0.0 
The max is 42.6
# count the number of null values in the dataframe
df['WTEQ.I-1 (in) '].isnull().sum()
Out[ ]:
22

Since there are only 22 NAN values out of the 8k data, we can drop the null values.

# drop the null values
df = df.dropna()
df['WTEQ.I-1 (in) '].isnull().sum()
Out[ ]:
0
#Set to datetime format
df.index = pd.to_datetime(df.index)
df
Out[ ]:
WTEQ.I-1 (in)
Date
1999-10-01 0.0
1999-10-02 0.0
1999-10-03 0.0
1999-10-04 0.0
1999-10-05 0.0
2022-03-16 21.7
2022-03-17 21.6
2022-03-18 21.6
2022-03-19 21.5
2022-03-20 21.4

8207 rows × 1 columns

# calculate the monthly mean
df_monthly = df.resample('M').mean()
df_monthly
Out[ ]:
WTEQ.I-1 (in)
Date
1999-10-31 0.000000
1999-11-30 0.760000
1999-12-31 7.170968
2000-01-31 15.870968
2000-02-29 22.634483
2021-11-30 1.356667
2021-12-31 6.383871
2022-01-31 17.670968
2022-02-28 18.996429
2022-03-31 20.550000

270 rows × 1 columns

#change the unit of the SWE values to centimeter
df_monthly['SWE (cm)'] = df_monthly['WTEQ.I-1 (in) '] * 2.54
df_monthly
Out[ ]:
WTEQ.I-1 (in) SWE (cm)
Date
1999-10-31 0.000000 0.000000
1999-11-30 0.760000 1.930400
1999-12-31 7.170968 18.214258
2000-01-31 15.870968 40.312258
2000-02-29 22.634483 57.491586
2021-11-30 1.356667 3.445933
2021-12-31 6.383871 16.215032
2022-01-31 17.670968 44.884258
2022-02-28 18.996429 48.250929
2022-03-31 20.550000 52.197000

270 rows × 2 columns

# Create a column of the month and year
df_monthly['Month'] = df_monthly.index.month_name()
df_monthly['Year'] = df_monthly.index.year

df_monthly
Out[ ]:
WTEQ.I-1 (in) SWE (cm) Month Year
Date
1999-10-31 0.000000 0.000000 October 1999
1999-11-30 0.760000 1.930400 November 1999
1999-12-31 7.170968 18.214258 December 1999
2000-01-31 15.870968 40.312258 January 2000
2000-02-29 22.634483 57.491586 February 2000
2021-11-30 1.356667 3.445933 November 2021
2021-12-31 6.383871 16.215032 December 2021
2022-01-31 17.670968 44.884258 January 2022
2022-02-28 18.996429 48.250929 February 2022
2022-03-31 20.550000 52.197000 March 2022

270 rows × 4 columns

#pivot the dataframe to ctabularize the monthly mean
df_monthly_pivot = df_monthly.pivot("Month", "Year", "SWE (cm)") #drop column
df_monthly_pivot
Out[ ]:
Year 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2013 2014 2015 2016 2017 2018 2019 2020 2021 2022
Month
April NaN 65.311867 41.164933 75.793600 68.072000 70.459600 45.017267 104.495600 52.874333 85.301667 33.020000 80.484133 39.539333 63.965667 81.948867 46.007867 83.083400 60.790667 51.011667 NaN
August NaN 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 NaN
December 18.214258 18.066774 32.020387 14.945032 26.530710 19.426903 29.013355 23.138581 12.986774 13.593097 17.591548 22.966516 28.841290 18.353548 10.512323 14.437032 11.053097 21.082000 16.215032 NaN
February NaN 57.491586 34.181143 63.500000 46.645286 70.366759 31.532286 85.398429 47.643143 65.356828 27.803929 50.645786 50.019857 66.950897 74.403857 34.480500 55.526214 54.758897 55.535286 48.250929
January NaN 40.312258 28.513548 48.571355 36.920129 55.486710 28.185806 62.107097 40.533484 40.623613 20.705097 28.030129 41.762516 50.742645 45.752774 22.007871 29.816323 33.429677 39.624000 44.884258
July NaN 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 NaN
June NaN 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.033867 0.000000 0.550333 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 NaN
March NaN 73.938581 38.616194 76.216387 61.992387 79.100516 36.125355 94.586323 61.820323 75.593677 34.896323 73.045484 46.129677 77.560129 94.414258 50.414903 88.588645 61.123871 69.538645 52.197000
May NaN 18.853355 8.504903 33.929484 43.302903 15.715226 14.010968 50.111742 9.479935 49.882323 3.269226 40.476129 2.179484 10.831871 23.654774 2.548194 20.393742 7.792065 3.449484 NaN
November 1.930400 5.317067 2.726267 6.172200 6.468533 6.383867 12.352867 5.935133 1.049867 3.454400 5.240867 5.579533 4.597400 0.474133 5.850467 1.845733 1.735667 9.050867 3.445933 NaN
October 0.000000 0.991419 0.122903 0.000000 0.000000 1.229032 0.000000 0.000000 0.286774 0.073742 0.000000 0.024581 0.024581 0.106516 1.302774 0.000000 0.778387 0.000000 0.000000 NaN
September NaN 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 NaN

12 rows × 24 columns

#sort by string month column and drop the 1999 column
df_monthly_pivot.index = pd.Categorical(df_monthly_pivot.index,
                                        categories=['January', 'February', 'March', 'April', 'May', 'June', 'July', 'August', 'September', 'October', 'November', 'December'],
                                        ordered=True)
df_monthly_pivot.drop(1999, axis=1, inplace=True)
df_monthly_pivot = df_monthly_pivot.sort_index()

df_monthly_pivot
Out[ ]:
Year 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2013 2014 2015 2016 2017 2018 2019 2020 2021 2022
January 40.312258 28.513548 48.571355 36.920129 55.486710 28.185806 62.107097 40.533484 40.623613 39.951742 20.705097 28.030129 41.762516 50.742645 45.752774 22.007871 29.816323 33.429677 39.624000 44.884258
February 57.491586 34.181143 63.500000 46.645286 70.366759 31.532286 85.398429 47.643143 65.356828 45.411571 27.803929 50.645786 50.019857 66.950897 74.403857 34.480500 55.526214 54.758897 55.535286 48.250929
March 73.938581 38.616194 76.216387 61.992387 79.100516 36.125355 94.586323 61.820323 75.593677 61.254968 34.896323 73.045484 46.129677 77.560129 94.414258 50.414903 88.588645 61.123871 69.538645 52.197000
April 65.311867 41.164933 75.793600 68.072000 70.459600 45.017267 104.495600 52.874333 85.301667 70.366467 33.020000 80.484133 39.539333 63.965667 81.948867 46.007867 83.083400 60.790667 51.011667 NaN
May 18.853355 8.504903 33.929484 43.302903 15.715226 14.010968 50.111742 9.479935 49.882323 36.125355 3.269226 40.476129 2.179484 10.831871 23.654774 2.548194 20.393742 7.792065 3.449484 NaN
June 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.033867 0.000000 0.550333 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 NaN
July 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 NaN
August 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 NaN
September 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 NaN
October 0.991419 0.122903 0.000000 0.000000 1.229032 0.000000 0.000000 0.286774 0.073742 1.483032 0.000000 0.024581 0.024581 0.106516 1.302774 0.000000 0.778387 0.000000 0.000000 NaN
November 5.317067 2.726267 6.172200 6.468533 6.383867 12.352867 5.935133 1.049867 3.454400 4.055533 5.240867 5.579533 4.597400 0.474133 5.850467 1.845733 1.735667 9.050867 3.445933 NaN
December 18.066774 32.020387 14.945032 26.530710 19.426903 29.013355 23.138581 12.986774 13.593097 14.281355 17.591548 22.966516 28.841290 18.353548 10.512323 14.437032 11.053097 21.082000 16.215032 NaN

12 rows × 23 columns

# Set font size and font family
plt.rcParams.update({'font.size': 18})
plt.rcParams['font.family'] = 'serif'
plt.rcParams['font.serif'] = ['Times New Roman'] + plt.rcParams['font.serif']

#create a figure and axes elements
fig, ax = plt.subplots(figsize=(13,9), constrained_layout=True)

#create a heatmap of the monthly mean using seaborn
sns.heatmap(df_monthly_pivot, linewidth = .5, cmap = 'YlGnBu', ax=ax)

#set title
ax.set_title('Monthly Mean SWE (cm) at Mores Creek Summit, ID')


plt.show()

#save the figure
fig.savefig('Monthly_Mean_SWE.jpeg', dpi=500)
In [ ]:
 

 

 

More To Explore