Intro_slides.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Introduction to Python for Earth Scientists\n",
    "\n",
    "<img alt=\"xkcd 303 Compiling\n",
    "    The #1 Programmer Excuse for Legitimately Slacking Off: 'My code's compiling.'\n",
    "    [Two programmers are sword-fighting on office chairs in a hallway. An unseen manager calls them back to work through an open office door.]\n",
    "    Manager: Hey! Get back to work!\n",
    "    Cueball: Compiling!\n",
    "    Manager: Oh. Carry on.\"  align=\"right\" style=\"width:30%\" src=\"https://imgs.xkcd.com/comics/compiling.png\">\n",
    "\n",
    "## Motivation for these sessions\n",
    "\n",
    "1. Why should you want to learn to program?\n",
    "2. Why are we teaching you Python?\n",
    "3. Introduce you to some fundamentals of programming,\n",
    "4. Demonstrate why *DRY* is important, not just for reducing effort\n",
    "5. Get you set-up to work on your own machines! "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "\n",
    "<img alt=\"El servicing an ECLIPSE network seismometer site dressed in smart clothing.\" align=\"right\" style=\"width:15%\" src=\"images/el_service_smart.jpg\">\n",
    "\n",
    "## Dr. Eleanor (El) Mestel\n",
    "\n",
    "- Postdoctoral Fellow in Geophysics (Room CO503)\n",
    "- PhD from VUW: Taupō volcano seismology and co-production of research"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "### Using Python in my research\n",
    "- Learned Python for my Master's and used it throughout my research\n",
    "- Using large seismic datasets: waveforms, station metadata, earthquake catalogues, etc. \n",
    "- Helps me to understand the processes and software in my analysis (even those written in other programming languages!)\n",
    "- Makes analysis simpler, faster and more reproducible"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Why programming?!\n",
    "\n",
    "1. Short term:\n",
    "    - You need to know some programming to get through ESCI451 and other courses this year!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "2. Medium term:\n",
    "    - Your research projects probably involve working with data - programming can help you!\n",
    "    - You might want to make some pretty plots during your research - we can show you how."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "3. Long term:\n",
    "    - If you stay in research, analysing data in a *reproducible* style will probably be part of your life.\n",
    "    - For those that do not stay in academia, programming skills are among the most useful and transferable skills you will learn at university. A quick job search on the 19th February 2024 listed 114 jobs in Wellington related to Python alone."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Why programming (what's wrong with Excel!?)\n",
    "   \n",
    "<img alt=\"XKCD 2180 spreadsheets\n",
    "    [Cueball is at his computer. In the air on either side of him are an angel version of Cueball, with a halo and wings, and a devil version of Cueball, with horns and a pitchfork. The angel's dialogue appears in regular print, while the devil's dialogue appears in white print in black speech balloons.]\n",
    "    Angel: Don't use a spreadsheet! Do it right.\n",
    "    Devil: But a spreadsheet would be so easy.\n",
    "    Angel: In the long run you'll regret it!\n",
    "    [Closeup on Cueball, the angel, and the devil.]\n",
    "    Angel: Take the time to write real code.\n",
    "    Devil: Just paste the data! Tinker until it works!\n",
    "    Devil: Build a labyrinth of REGEXREPLACE() and ARRAYFORMULA()!\n",
    "    Devil: Feel the power!\n",
    "    [Closeup on the devil.]\n",
    "    Angel (off-panel): Fight the temptation!\n",
    "    Devil: Ever tried QUERY() in Google Sheets? It lets you treat a block of cells like a database and run SQL queries on them.\n",
    "    [Another shot of Cueball at his computer with the angel and devil at either side.]\n",
    "    Angel: Don't listen to\n",
    "    Angel: ...wait, really?\n",
    "    Devil: Yes, and let me tell you about IMPORTHTML()...\n",
    "    Angel: Oooh...\" align=\"center\" style=\"width:70%\" src=\"https://imgs.xkcd.com/comics/spreadsheets.png\">\n",
    "\n",
    "You've spent pretty much all your undergraduate career (in SGEES anyway) without much programming, so why do you need to learn now?\n",
    "\n",
    "**Reproducibility**, **Safety**,  **Speed**,  **Complexity**,  **Working with large data** "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Why programming (what's wrong with Excel!?)\n",
    "\n",
    "- **Reproducibility:** If someone can't replicate your \n",
    "  work, why should we trust it to be true?\n",
    "- **Safety:** Your data and your processing should not\n",
    "  overlap.  Your raw data should be sacred. [Poor use of Excel resulted in the loss of 16,000 COVID-19 records in the UK.](https://www.theguardian.com/politics/2020/oct/05/how-excel-may-have-caused-loss-of-16000-covid-tests-in-england)\n",
    "- **Speed:** You want a result, and you want it yesterday... Learn how to write good code \n",
    "    (and change the clock-speed on your computer) and you can...\n",
    "- **Complexity:** Being able to solve complex problems logically, in a way that others can follow\n",
    "    (and reproduce) is essential to natural sciences. *Hint: Writing good code is as much about the*\n",
    "    *quality of your documentation as it is about the quality of your code*.\n",
    "- **Data scale:** Data in natural sciences is noisy, and large. Ideally to understand the natural world\n",
    "    we would have data from every place at every time throughout the Earth. We don't have that, but\n",
    "    our datasets are growing..."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Data scale: seismology bias\n",
    "\n",
    "Seismographs deployed for:\n",
    "- long durations (multi-year);\n",
    "- multiple locations;\n",
    "- modest sampling rates.\n",
    "\n",
    "For example: [SAMBA](http://ds.iris.edu/mda/9F/?starttime=2008-01-01T00:00:00&endtime=2020-12-31T23:59:59) is\n",
    "the Southern Alps Microearthquake Borehole Array around Aoraki Mt. Cook has been recording since 2008.\n",
    "\n",
    "<img alt=\"Panorama including a SAMBA site (COVA) in the mountains\" align=\"right\" style=\"width:100%\" src=\"images/COVA_pano.jpg\">"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "SAMBA records at 200Hz (200 samples per second). How many seconds per day?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-02-19T03:20:12.508307Z",
     "iopub.status.busy": "2024-02-19T03:20:12.507723Z",
     "iopub.status.idle": "2024-02-19T03:20:12.515202Z",
     "shell.execute_reply": "2024-02-19T03:20:12.514807Z"
    },
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "There are 86400 seconds in a day\n"
     ]
    }
   ],
   "source": [
    "seconds_per_day = 60 * 60 * 24\n",
    "print(f\"There are {seconds_per_day} seconds in a day\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "How many samples per day?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-02-19T03:20:12.568678Z",
     "iopub.status.busy": "2024-02-19T03:20:12.568497Z",
     "iopub.status.idle": "2024-02-19T03:20:12.571175Z",
     "shell.execute_reply": "2024-02-19T03:20:12.570804Z"
    },
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "SAMBA records 17280000.0 samples per day\n"
     ]
    }
   ],
   "source": [
    "sampling_rate = 200.0\n",
    "samples_per_day = seconds_per_day * sampling_rate\n",
    "print(f\"SAMBA records {samples_per_day} samples per day\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "So, > 17 million samples per day.  But that is just for one channel: SAMBA seismographs\n",
    "have three channels, a vertical and two horizontals, so how many samples per day for one station?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-02-19T03:20:12.572868Z",
     "iopub.status.busy": "2024-02-19T03:20:12.572737Z",
     "iopub.status.idle": "2024-02-19T03:20:12.575316Z",
     "shell.execute_reply": "2024-02-19T03:20:12.574861Z"
    },
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "One station records 51840000.0 samples per day\n"
     ]
    }
   ],
   "source": [
    "number_of_channels = 3\n",
    "samples_per_day_per_station = samples_per_day * number_of_channels\n",
    "print(f\"One station records {samples_per_day_per_station} samples per day\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "Nearly 52 million samples per day per station. SAMBA is made up of 13 stations, so our dataset gets bigger\n",
    "still:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-02-19T03:20:12.577010Z",
     "iopub.status.busy": "2024-02-19T03:20:12.576854Z",
     "iopub.status.idle": "2024-02-19T03:20:12.579358Z",
     "shell.execute_reply": "2024-02-19T03:20:12.578921Z"
    },
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "SAMBA records 673920000.0 samples per day\n"
     ]
    }
   ],
   "source": [
    "number_of_stations = 13\n",
    "samples_per_day_total = samples_per_day_per_station * number_of_stations\n",
    "print(f\"SAMBA records {samples_per_day_total} samples per day\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "673 million samples per day across the network. So what is that over the first 10 years of operation?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-02-19T03:20:12.581003Z",
     "iopub.status.busy": "2024-02-19T03:20:12.580873Z",
     "iopub.status.idle": "2024-02-19T03:20:12.583491Z",
     "shell.execute_reply": "2024-02-19T03:20:12.583063Z"
    },
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "SAMBA records 246149280000.0 samples per year.\n",
      "In 10 years of operation SAMBA recorded 2.461493e+12 samples\n"
     ]
    }
   ],
   "source": [
    "days_per_year = 365.25 # Roughly\n",
    "samples_per_year = days_per_year * samples_per_day_total\n",
    "print(f\"SAMBA records {samples_per_year} samples per year.\")\n",
    "samples_per_decade = samples_per_year * 10\n",
    "print(f\"In 10 years of operation SAMBA recorded {samples_per_decade:e} samples\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "2 Trillion samples.\n",
    "\n",
    "Try working with that in a spreadsheet...\n",
    "\n",
    "And that is barely scraping the surface of dataset sizes in Earth Science. Consider thousands of seismic stations around the globe, or more recently the development of DAS fibre technology which generates **terabytes of data per day.**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Why Python?\n",
    "\n",
    "Our first notebook covers this a little more, but the key things for me are:\n",
    "\n",
    "1. Open-source, community driven (often free) software;\n",
    "2. Simple syntax, fast to make mistakes and helpful error messages;\n",
    "3. Community libraries to do lots of complex tasks \n",
    "   (e.g. [obspy](https://github.com/obspy/obspy/wiki) for seismology)\n",
    "4. Widely used: Python is in the top three most popular programming languages according to [TIOBE](https://www.tiobe.com/tiobe-index/), and possibly the most popular *interpreted* language.\n",
    "    1. There is a wealth of information online from others.\n",
    "    2. Python is useable across a huge range of tasks.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# How to use these notebooks:\n",
    "\n",
    "There are a series of Jupyter notebooks on here.  You can run them interactively in your\n",
    "browser.  You should run through them, change some values, see what works, try and play\n",
    "with variables and experiment.  There will be sections that you are expected to fill in\n",
    "marked as **Exercise:**.  Shout out if and when you have problems.\n",
    "\n",
    "Let us know if you want to play around with any other data.\n",
    "\n",
    "Remember that this course is supposed to be a brief look over some of the key ideas in Python\n",
    "and useful libraries, it is not complete!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Getting set-up on a CO501 computer\n",
    "1. Log into your Linux account\n",
    "2. Open a terminals and set-up your conda virtual environment\n",
    "```bash\n",
    "startconda esci451\n",
    "```\n",
    "3. Clone the notebook repository\n",
    "```bash\n",
    "git clone https://github.com/calum-chamberlain/ESCI451-Python.git\n",
    "```\n",
    "4. Start Jupyter:\n",
    "```bash\n",
    "cd ESCI451-Python  # Or where ever you made the folder containing the notebooks\n",
    "jupyter notebook\n",
    "```"
   ]
  }
 ],
 "metadata": {
  "celltoolbar": "Slideshow",
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}