1a_python1.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Python Basics I\n",
    "\n",
    "In this notebook, we will work with the following:\n",
    "\n",
    "- Python basic data types.\n",
    "- Python data structures."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "# imports\n",
    "import datetime\n",
    "\n",
    "import pandas as pd"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Python\n",
    "\n",
    "Below, we will cover:\n",
    "\n",
    "- Basic data types: `int`, `float`,`str`.\n",
    "- Data structures: lists and dictionaries.\n",
    "- Functions and methods.\n",
    "- Mutability.\n",
    "\n",
    "\n",
    "## Preliminary items\n",
    "\n",
    "By tradition, the first programming task for new learners is to print \"Hello, World!\" to the default output.\n",
    "So, in the cell below, type in `print('Hello, World!')` and run the cell, either by pressing Shift-Enter or by clicking the run button (looks like the \"play\" symbol) in the toolbar."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Type your \"Hello, World\" print statement below.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Basic data types\n",
    "\n",
    "### Integers"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "# To create an int, we can just assign a number without a decimal.\n",
    "a = 2\n",
    "b = 3"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Addition:        5\n",
      "Subtraction:    -1\n",
      "Multiplication:  6\n",
      "Exponentiation:  8\n"
     ]
    }
   ],
   "source": [
    "# Most operations preserve int types\n",
    "print(f'Addition:        {a + b}')\n",
    "print(f'Subtraction:    {a - b}')\n",
    "print(f'Multiplication:  {a * b}')\n",
    "print(f'Exponentiation:  {a ** b}')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Division:           1.5\n",
      "Floor/int Division: 1\n"
     ]
    }
   ],
   "source": [
    "# Division does not, but we can use floor division if that's what we want.\n",
    "print(f'Division:           {b / a}')\n",
    "print(f'Floor/int Division: {b // a}')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Division:           <class 'float'>\n",
      "Floor/int Division: <class 'int'>\n"
     ]
    }
   ],
   "source": [
    "# Printing types\n",
    "print(f'Division:           {type(b / a)}')\n",
    "print(f'Floor/int Division: {type(b // a)}')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Floats\n",
    "\n",
    "Anything with a float returns a float.\n",
    "Also, in the example below, note the imprecision we talked about earlier."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Addition:        5.1\n",
      "Subtraction:     2.9\n",
      "Even here:       2.9999999999999996\n"
     ]
    }
   ],
   "source": [
    "c = 4\n",
    "d = 1.1\n",
    "\n",
    "print(f'Addition:        {c + d}')\n",
    "print(f'Subtraction:     {c - d}')\n",
    "print(f'Even here:       {4.1 - d}')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "e:        123.456\n",
      "Too far: -inf\n"
     ]
    }
   ],
   "source": [
    "# Scientific notation works, too.\n",
    "e = 1.23456e02\n",
    "print(f'e:        {e}')\n",
    "print(f'Too far: {-1.23456e1000}')\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Strings\n",
    "\n",
    "We denote a string with single `'` or double `\"` quote characters."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "a_string = 'Hello!'\n",
    "b_string = \"Good way to contain 'single' quotes.\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Hello!\n",
      "Good way to contain 'single' quotes.\n"
     ]
    }
   ],
   "source": [
    "print(a_string)\n",
    "print(b_string)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Above, I used a new Python 3.6 feature called f strings.\n",
    "This kind of string can contain code that is evaluated when the string is created, and it is very helpful for generating messages or other output."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "My name is: Jason\n"
     ]
    }
   ],
   "source": [
    "# Replace the following with your name.\n",
    "my_name = 'Jason'\n",
    "print(f'My name is: {my_name}')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A common data handling issue is that we have something stored as strings that are actually numeric.\n",
    "We can convert these using the appropriate functions.\n",
    "We will see later that pandas can help us with this inside dataframes."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "6\n",
      "12340.0\n"
     ]
    }
   ],
   "source": [
    "f = '6'\n",
    "g = '1.234e4'\n",
    "print(int(f))\n",
    "print(float(g))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Boolean values\n",
    "\n",
    "Python has special values `True` and `False` that are boolean.\n",
    "They can be used directly, and they are also returned by comparisons.\n",
    "As we will see later, they are useful for conditional logic."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "h = True\n",
    "i = False"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "False"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# We can test values.\n",
    "h is False"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "i is False"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# We can use logical operators, too.\n",
    "h or i"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "False"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "h and i"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Exclusive OR operator.\n",
    "h ^ i"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Underneath, these are actually just the ints 0 and 1.\n",
    "# I wouldn't make a habit of using them this way.\n",
    "h + 1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## None type\n",
    "\n",
    "Python has a type called `None` that is used for null values.\n",
    "Often, we want to test whether something exists before trying to work with it, as a way of avoiding errors."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [],
   "source": [
    "j = None"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "False"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "h is None"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "j is None"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Datetime\n",
    "\n",
    "Dates and times are a big topic, and we are only scratching the surface.\n",
    "See the [documentation](https://docs.python.org/3/library/datetime.html) for more.\n",
    "Also, we will often with with dates in pandas dataframes, and it has datetime tools, too."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2019-09-25 12:36:50.509843\n"
     ]
    }
   ],
   "source": [
    "a_datetime = datetime.datetime.now()\n",
    "print(a_datetime)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2019\n",
      "9\n",
      "25\n"
     ]
    }
   ],
   "source": [
    "# We can get components.\n",
    "print(a_datetime.year)\n",
    "print(a_datetime.month)\n",
    "print(a_datetime.day)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2018-09-25 12:36:50.509843\n"
     ]
    }
   ],
   "source": [
    "# Using timedelta, we can add or subtract units of time.\n",
    "print(a_datetime - datetime.timedelta(days=365))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Data structures\n",
    "\n",
    "These are Python's built in data structures.\n",
    "They are simple, yet powerful, and they have pretty good performance.\n",
    "However, for things like dataframes, we use something designed for that use, like pandas.\n",
    "\n",
    "## Lists\n",
    "\n",
    "We're just scratching the surface with lists, but the [documentation](https://docs.python.org/3/tutorial/introduction.html#lists) is great."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['Lists', 'are', 'quite', 'helpful']\n"
     ]
    }
   ],
   "source": [
    "a_list = ['Lists', 'are', 'quite', 'helpful']\n",
    "print(a_list)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[1, 'two', 3.0, 4]\n"
     ]
    }
   ],
   "source": [
    "# We can mix types, though we often don't.\n",
    "b_list = [\n",
    "    1,\n",
    "    'two',\n",
    "    3.0,\n",
    "    4\n",
    "]\n",
    "print(b_list)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1\n",
      "['Lists', 'are']\n"
     ]
    }
   ],
   "source": [
    "# Access with indices and slicing\n",
    "print(b_list[0])\n",
    "print(a_list[:2])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Dictionaries\n",
    "\n",
    "Like lists, see the [documentation](https://docs.python.org/3/tutorial/datastructures.html#dictionaries).\n",
    "There are some common things, like the pandas [rename method](https://pandas.pydata.org/pandas-docs/stable/basics.html#basics-rename), that take dictionaries."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'A': 1, 'B': 2, 'C': 3}\n"
     ]
    }
   ],
   "source": [
    "a_dict = {'A': 1, 'B': 2, 'C': 3}\n",
    "print(a_dict)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Access by key.\n",
    "a_dict['B']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'A': 1, 'B': 2, 'C': 3, 'D': 4}\n"
     ]
    }
   ],
   "source": [
    "# We can add new keys after creating the dictionary.\n",
    "a_dict['D'] = 4\n",
    "print(a_dict)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'E': 5, 'F': 6}\n",
      "{'A': 1, 'B': 2, 'C': 3, 'D': 4, 'E': 5, 'F': 6}\n"
     ]
    }
   ],
   "source": [
    "# We can also combine dictionaries using the update method.\n",
    "b_dict = {'E': 5, 'F': 6}\n",
    "print(b_dict)\n",
    "a_dict.update(b_dict)\n",
    "print(a_dict)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Nested data structures\n",
    "\n",
    "Data structures can also contain other data structures (and potentially multiple levels deep).\n",
    "One common use for us is to represent a row of data as a dictionary, where the keys can be thought of as column names and the values are the values of variables.\n",
    "Then, we put those dicts into a list where each dict is an element of the list.\n",
    "Afterward, we can pass this structure to pandas (which we cover later), and it will make a nice dataframe for us."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[{'ticker': 'msft', 'founding_year': 1975}, {'ticker': 'aapl', 'founding_year': 1976}]\n"
     ]
    }
   ],
   "source": [
    "a_dict_list = [\n",
    "    {\n",
    "        'ticker': 'msft',\n",
    "        'founding_year': 1975\n",
    "    },\n",
    "    {\n",
    "        'ticker': 'aapl',\n",
    "        'founding_year': 1976\n",
    "    }\n",
    "]\n",
    "print(a_dict_list)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>ticker</th>\n",
       "      <th>founding_year</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>0</td>\n",
       "      <td>msft</td>\n",
       "      <td>1975</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>1</td>\n",
       "      <td>aapl</td>\n",
       "      <td>1976</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  ticker  founding_year\n",
       "0   msft           1975\n",
       "1   aapl           1976"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.DataFrame(a_dict_list)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Bonus topics\n",
    "\n",
    "We won't cover these directly, but here are two other topics that are good to know.\n",
    "\n",
    "1. **Reading and writing to files.** See the relevant Python [tutorial](https://docs.python.org/3/tutorial/inputoutput.html#reading-and-writing-files). Two things to note:\n",
    "    1. For most things we do, we don't need to be this low level. For example, pandas will easily read and write structured data like CSV, Stata, SAS. You should always prefer the higher-level tools if they can do what you want.\n",
    "    1. I usually use this for either reading a list of identifiers that I created at some other time (e.g., the gvkeys of all S&P 500 firms) or for recording which observations that I have already processed (e.g., when web scraping).\n",
    "1. **Handling errors.** See the relevant Python [tutorial](https://docs.python.org/3/tutorial/errors.html#handling-exceptions) on `try-except` clauses (which will handle most of what we tend to need in research). In particular, note what they say about using `except` without a specified exception (in other resources, this is often called a \"naked\" `except`). This will capture anything, including the exception that is raised when you use the Kernel:Interrupt Kernel menu item, and that isn't usually what you want."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}