HistogramKDE_precipitation_example_blank.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "8e08a443-3f4e-401d-83e5-bc1d667b8844",
   "metadata": {},
   "source": [
    "# EDS 220 Fall 2022\n",
    "# Visualizing histograms and kernel density estimates: 1D data\n",
    "\n",
    "In previous exercises, we worked with time series data to determine how to best read in data and clean up missing data/outliers.\n",
    "\n",
    "Now let's practice building some visualizations of data _distributions_: namely, histogram and kernel density estimate-based plots of the data's probability distribution function."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "97855ef3-5582-4b66-a8e9-38fe473e92c3",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Import necessary packages\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "import matplotlib.pyplot as plt\n",
    "import scipy\n",
    "import scipy.stats as stats"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fc916dfc-87c7-4f7e-a335-e99214e70a0b",
   "metadata": {},
   "source": [
    "### 1) Read in data using Pandas\n",
    "\n",
    "We'll use the same precipitation data that we worked with in the `Precipitation_QCexample_BoulderCO` notebook. This data comes from the [NOAA Climate Data Online](https://www.ncdc.noaa.gov/cdo-web/) service, and has been downloaded into the class repository (`Precip_BoulderCO_COOPstation.csv`). This file contains hourly precipitation averages calculated from a weather station in Boulder, and will be read in using Pandas:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "e8f7706d-85b1-4e83-b7e5-92e9291f9d4b",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Read in data file into a Pandas data frame, flagging missing values as NaN\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0a005d64-c1c6-48e2-8744-175abb580261",
   "metadata": {},
   "source": [
    "The contents of the data frame can be displayed using the `.head()` command. Additionally, information on the associated _metadata_ is available in the class repository:\n",
    "\n",
    "PRECIP_HLY_documentation.pdf"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "fbdc1c3f-26b1-4f62-88b7-d4418eac21dc",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Display header information for Pandas DataFrame\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "25f09b73-1b9d-42d0-a5c6-125d58a03562",
   "metadata": {},
   "source": [
    "### 2) Create histogram using Matplotlib\n",
    "\n",
    "There are several ways to create histograms in Python. One simple way is using Matplotlib; this is actually a wrapper around the Numpy histogram function, which operates under the hood.\n",
    "\n",
    "   More details on Matplotlib histograms can be found here:\n",
    "  [https://matplotlib.org/stable/gallery/statistics/hist.html](https://matplotlib.org/stable/gallery/statistics/hist.html)\n",
    "  \n",
    "\n",
    "The basic generation of the histogram is done using `plt.hist` where Matplotlib.pyplot has been imported as `plt`.\n",
    "\n",
    "In Matplotlib, the histogram function produces several different outputs, which I'll name as follows:\n",
    "* n: the count of values within each bin\n",
    "* bins: the edges of each bin\n",
    "* patches: an \"object\" referring to the histogram bars, which you can edit to alter the appearance of the plot\n",
    "\n",
    "Let's make a histogram of the \"HPCP\" field in the data frame above, using automatically generated bins:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "b407cc42-8a94-4b3d-a40b-cf61ece5d94d",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create histogram plot using Matplotlib\n",
    "\n",
    "\n",
    "# Label plot axes\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b3581a43-03fe-4e87-806b-6c2046b92dce",
   "metadata": {},
   "source": [
    "You'll see that most of the precipitation data is below 1 inch; we can get a better view by either\n",
    "- changing the x-limit of the plot\n",
    "- or changing the specification of the bins\n",
    "\n",
    "Let's try the latter! The Numpy `linspace` function is useful in this context, since it's an easy way to generate linearly spaced values over a certain range."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "59283667-45c6-4470-a30b-620e8c87c3ae",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Specify bins\n",
    "\n",
    "\n",
    "# Re-create histogram plot\n",
    "\n",
    "\n",
    "# Label plot axes\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d635fe00-349a-43a0-92f6-14c2e63e7fe6",
   "metadata": {},
   "source": [
    "Oftentimes when working with histograms, we don't care about the absolute number of occurrences within a particular bin. Instead, the _proportion_ of data falling into different bins is more useful. To convert the histogram from an absolute count to a fraction, use the `density` argument to hist:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "6ee21d31-e786-4a72-bdfc-a7ff5c7057d6",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Re-create histogram plot\n",
    "\n",
    "\n",
    "# Label plot axes\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d7970293-a2c7-47f7-a8ba-c378c158c656",
   "metadata": {},
   "source": [
    "In this new, normalized histogram, the sum of the bar areas will equal 100%."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fc653a67-eddb-4fcb-9ac3-cfc6fbd296bc",
   "metadata": {},
   "source": [
    "### 3) Kernel Density Estimation with Scipy\n",
    "\n",
    "The examples above give a good sense for the data distribution, but are limited by the need to specify either the number of bins or the values associated with bin edges. Using the kernel density estimator functionality available in Scipy, we can get a continuous estimate of the distribution which has a smoother appearance.\n",
    "\n",
    "The `gaussian_kde` function in Scipy will be used here; note that in order for this to work, we have to exclude all the NA values, otherwise the code will crash. This can be done using the `.dropna()` functionality in Pandas:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "47e0137c-e33d-4368-933c-f34bbb4a64ac",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Perform kernel density estimation on hourly precipitation values\n",
    "\n",
    "\n",
    "# Display plot of the distribution\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ec100c2e-d8b2-4da3-8341-550ef147a687",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}