ejercicios/notebooks/01_BrechaDigital_Analisis.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Proyecto 1: Brecha Digital\n",
    "### Análisis de datos\n",
    "\n",
    "#### Basado en el reporte [\"La Brecha Digital en California\"](https://www.ppic.org/wp-content/uploads/jtf-californias-digital-divide-in-spanish.pdf)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Pregunta(s) de Investigación:\n",
    "1. ¿Qué porcentaje de hogares en el estado X tiene acceso a internet de alta velocidad?\n",
    "2. ¿Varía este número según los grupos demográficos? (en este caso raza/etnia)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Meta:\n",
    "* Usar nuestro conjunto de datos `datos_analisis` (creado en el [notebook Preparación de Datos](00_BrechaDigital_PrepDatos.ipynb)) para contestar nuestras preguntas de investación."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Contexto:\n",
    "* Escribe tu propia descripción del contexto: En que parte del proyecto encaja este notebook, una pequeña descripción de los datos que vas a utilizar (_este conjunto de datos contiene información de X estado para el año YYYY)_."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "***"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Paso 1: Prepara tu entorno de trabajo.\n",
    "\n",
    "**Import**a las bibliotecas necesarias y crea objetos `Path` (ruta de archivo). Esto grarantiza reproducibilidad en distintos sistemas operativos (Windows utiliza `\\` en lugar de `/` para separar los nombres de archivos."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Necesitamos: \n",
    "1. `pandas` para trabajar con los datos.\n",
    "2. `pathlib`, y más específicamente su objeto `Path`, para trabajar con rutas de archivos. Esto asegura que nuestro código funcione en Windows (que utiliza `\\` en sus rutas) y MacOS/Linux (los cuales utilizan `/`).\n",
    "3. `datetime` - tip: Existen sistemas de control de versiones para datos pero etiquetar tus archivos de datos (cuando no son masivos) no es un mal primer paso si estás comenzando.\n",
    "4. `arbol` - para mostrar el árbol de directorios."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "06-05-19\n"
     ]
    }
   ],
   "source": [
    "# Preparando tu entorno de trabajo\n",
    "import _____ as pd\n",
    "from _____ import Path\n",
    "from herramientas import _________\n",
    "from ______ import _______ as dt\n",
    "hoy = dt.______()._______(\"%_-%_-%_\")\n",
    "\n",
    "print(hoy)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Directorio de datos y rutas\n",
    "RUTA_DATOS_EN_BRUTO = ____(\"../datos/brutos/\")\n",
    "RUTA_DATOS_________ = ____(\"../datos/interinos/\")\n",
    "RUTA_______________ = ____(\"../datos/procesados/\")\n",
    "___________________ = ____(\"../datos/finales/\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "arbol(RUTA_DATOS_INTERINOS)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "datos = pd.read_stata(RUTA_DATOS_INTERINOS / f'datos_para_analisis-{hoy}.dta')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(44816, 14)"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "datos._______"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>year</th>\n",
       "      <th>serial</th>\n",
       "      <th>hhwt</th>\n",
       "      <th>stateicp</th>\n",
       "      <th>countyfip</th>\n",
       "      <th>cinethh</th>\n",
       "      <th>cihispeed</th>\n",
       "      <th>pernum</th>\n",
       "      <th>perwt</th>\n",
       "      <th>relate</th>\n",
       "      <th>sex</th>\n",
       "      <th>age</th>\n",
       "      <th>race</th>\n",
       "      <th>hispan</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2017</td>\n",
       "      <td>953662</td>\n",
       "      <td>57</td>\n",
       "      <td>ohio</td>\n",
       "      <td>0</td>\n",
       "      <td>yes, with a subscription to an internet service</td>\n",
       "      <td>yes (cable modem, fiber optic or dsl service)</td>\n",
       "      <td>1</td>\n",
       "      <td>58</td>\n",
       "      <td>head/householder</td>\n",
       "      <td>female</td>\n",
       "      <td>48</td>\n",
       "      <td>white</td>\n",
       "      <td>not hispanic</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2017</td>\n",
       "      <td>953662</td>\n",
       "      <td>57</td>\n",
       "      <td>ohio</td>\n",
       "      <td>0</td>\n",
       "      <td>yes, with a subscription to an internet service</td>\n",
       "      <td>yes (cable modem, fiber optic or dsl service)</td>\n",
       "      <td>2</td>\n",
       "      <td>62</td>\n",
       "      <td>child</td>\n",
       "      <td>male</td>\n",
       "      <td>20</td>\n",
       "      <td>white</td>\n",
       "      <td>not hispanic</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2017</td>\n",
       "      <td>953662</td>\n",
       "      <td>57</td>\n",
       "      <td>ohio</td>\n",
       "      <td>0</td>\n",
       "      <td>yes, with a subscription to an internet service</td>\n",
       "      <td>yes (cable modem, fiber optic or dsl service)</td>\n",
       "      <td>3</td>\n",
       "      <td>78</td>\n",
       "      <td>child</td>\n",
       "      <td>female</td>\n",
       "      <td>9</td>\n",
       "      <td>white</td>\n",
       "      <td>not hispanic</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>2017</td>\n",
       "      <td>953668</td>\n",
       "      <td>140</td>\n",
       "      <td>ohio</td>\n",
       "      <td>61</td>\n",
       "      <td>yes, with a subscription to an internet service</td>\n",
       "      <td>yes (cable modem, fiber optic or dsl service)</td>\n",
       "      <td>1</td>\n",
       "      <td>140</td>\n",
       "      <td>head/householder</td>\n",
       "      <td>male</td>\n",
       "      <td>28</td>\n",
       "      <td>black/african american/negro</td>\n",
       "      <td>not hispanic</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>2017</td>\n",
       "      <td>953668</td>\n",
       "      <td>140</td>\n",
       "      <td>ohio</td>\n",
       "      <td>61</td>\n",
       "      <td>yes, with a subscription to an internet service</td>\n",
       "      <td>yes (cable modem, fiber optic or dsl service)</td>\n",
       "      <td>2</td>\n",
       "      <td>192</td>\n",
       "      <td>sibling</td>\n",
       "      <td>female</td>\n",
       "      <td>16</td>\n",
       "      <td>black/african american/negro</td>\n",
       "      <td>not hispanic</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   year  serial  hhwt stateicp  countyfip  \\\n",
       "0  2017  953662    57     ohio          0   \n",
       "1  2017  953662    57     ohio          0   \n",
       "2  2017  953662    57     ohio          0   \n",
       "3  2017  953668   140     ohio         61   \n",
       "4  2017  953668   140     ohio         61   \n",
       "\n",
       "                                           cinethh  \\\n",
       "0  yes, with a subscription to an internet service   \n",
       "1  yes, with a subscription to an internet service   \n",
       "2  yes, with a subscription to an internet service   \n",
       "3  yes, with a subscription to an internet service   \n",
       "4  yes, with a subscription to an internet service   \n",
       "\n",
       "                                       cihispeed  pernum  perwt  \\\n",
       "0  yes (cable modem, fiber optic or dsl service)       1     58   \n",
       "1  yes (cable modem, fiber optic or dsl service)       2     62   \n",
       "2  yes (cable modem, fiber optic or dsl service)       3     78   \n",
       "3  yes (cable modem, fiber optic or dsl service)       1    140   \n",
       "4  yes (cable modem, fiber optic or dsl service)       2    192   \n",
       "\n",
       "             relate     sex age                          race        hispan  \n",
       "0  head/householder  female  48                         white  not hispanic  \n",
       "1             child    male  20                         white  not hispanic  \n",
       "2             child  female   9                         white  not hispanic  \n",
       "3  head/householder    male  28  black/african american/negro  not hispanic  \n",
       "4           sibling  female  16  black/african american/negro  not hispanic  "
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "datos._______()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "Int64Index: 44816 entries, 0 to 44815\n",
      "Data columns (total 14 columns):\n",
      "year         44816 non-null category\n",
      "serial       44816 non-null int32\n",
      "hhwt         44816 non-null int16\n",
      "stateicp     44816 non-null category\n",
      "countyfip    44816 non-null int16\n",
      "cinethh      44816 non-null category\n",
      "cihispeed    44816 non-null category\n",
      "pernum       44816 non-null int8\n",
      "perwt        44816 non-null int16\n",
      "relate       44816 non-null category\n",
      "sex          44816 non-null category\n",
      "age          44816 non-null category\n",
      "race         44816 non-null category\n",
      "hispan       44816 non-null category\n",
      "dtypes: category(9), int16(3), int32(1), int8(1)\n",
      "memory usage: 1.2 MB\n"
     ]
    }
   ],
   "source": [
    "datos._____()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Nuestra **unidad de observación** es todavía una persona (ponderada) pero nos interesa datos a nivel **hogar**.\n",
    "\n",
    "De la [documentación](https://usa.ipums.org/usa-action/variables/HHWT#description_section) de IPUMS:\n",
    ">HHWT indica cuantos hogares de la población de EEUU estan siendo representados por cada hogar en una muestra de IPUMS.<br><br>\n",
    ">Es generalmente una buena idea usar HHWT cuando se está llevando a cabo análisis a nivel hogar de cualquier muestra de IPUMS. El uso de HHWT es opcional cuando se están analizando uno de las muestras \"planas\" o no ponderados de IPUMS. Muestras planas de IPUMS incluyen las muestras 1% de 1850-1930, todas las muestras de 1960, 1970 y 1980, las muestras no ponderadas 1% de 1990 y 2000, la muestra 10% de 2010, y cualquiera de los conjuntos de datos del censo de cuentas totales 100%. HHWT debe ser utilizada para obtener estadísticas representativas a nivel nacional para análisis a nivel hogar de cualquier muestra fuera de esas. <br><br>\n",
    ">**Usuarios deberían también asegurarse de seleccionar una persona (por ejemplo, PERNUM = 1) para representar el hogar entero.**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "***"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Paso 2: Elimina todas las observaciones donde `pernum` no sea igual a 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "mascara_pernum = (________ _= 1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "datos[mascara_pernum].shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Guarda tus datos en una variable con un nombre apropiado."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "hogares_del_estado = ____[_________]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "***"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Paso 3: Familiarizate con tus variables de interés"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "De la [documentación](https://usa.ipums.org/usa-action/variables/CINETHH#description_section) de IPUMS:\n",
    ">CINETHH reporta si algún miembro del hogar accede al internet. Aquí, \"acceso\" se refiere a si alguien en el hogar usa o se conecta al internet o no, sin importar si paga por el servicio o no."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# encuentra los totales de los valores de CINETHH\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "De la [documentación](https://usa.ipums.org/usa-action/variables/CIHISPEED#description_section) de IPUMS:\n",
    ">CIHISPEED reporta si la persona respondiendo o cualquier miembro de su hogar se subscribió al internet utilizando servicios de banda ancha (alta velocidad) tales como cable, fibra optica, o servicio DSL. <br><br>\n",
    ">Nota al usuario: La ACS de 2016 introdujo cambios a las preguntas que pertienen al uso de computadoras y acceso al internet. Mira la sección comparabilidad y el texto del cuestionario para más información. Información adicional proveniente del Buró del Censo sobre la alteración de estas preguntas esta disponible en su reporte: **ACS Content Test Shows Need to Update Terminology**."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# encuentra los totales de los valores de CIHISPEED\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<details>\n",
    "    <summary>¿Adivina que puede hacer <span style=\"font-family:monospace\">.value_counts()</span></summary>\n",
    "    Tiene la opción de activar su parametro <span style=\"font-family:monospace\">normalize</span> que normaliza tus series.\n",
    "</details>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "\u001b[0;31mSignature:\u001b[0m\n",
       "\u001b[0mpd\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mSeries\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mvalue_counts\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m\u001b[0m\n",
       "\u001b[0;34m\u001b[0m    \u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\n",
       "\u001b[0;34m\u001b[0m    \u001b[0mnormalize\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;32mFalse\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\n",
       "\u001b[0;34m\u001b[0m    \u001b[0msort\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;32mTrue\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\n",
       "\u001b[0;34m\u001b[0m    \u001b[0mascending\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;32mFalse\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\n",
       "\u001b[0;34m\u001b[0m    \u001b[0mbins\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;32mNone\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\n",
       "\u001b[0;34m\u001b[0m    \u001b[0mdropna\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;32mTrue\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\n",
       "\u001b[0;34m\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
       "\u001b[0;31mDocstring:\u001b[0m\n",
       "Return a Series containing counts of unique values.\n",
       "\n",
       "The resulting object will be in descending order so that the\n",
       "first element is the most frequently-occurring element.\n",
       "Excludes NA values by default.\n",
       "\n",
       "Parameters\n",
       "----------\n",
       "normalize : boolean, default False\n",
       "    If True then the object returned will contain the relative\n",
       "    frequencies of the unique values.\n",
       "sort : boolean, default True\n",
       "    Sort by values.\n",
       "ascending : boolean, default False\n",
       "    Sort in ascending order.\n",
       "bins : integer, optional\n",
       "    Rather than count values, group them into half-open bins,\n",
       "    a convenience for ``pd.cut``, only works with numeric data.\n",
       "dropna : boolean, default True\n",
       "    Don't include counts of NaN.\n",
       "\n",
       "Returns\n",
       "-------\n",
       "counts : Series\n",
       "\n",
       "See Also\n",
       "--------\n",
       "Series.count: Number of non-NA elements in a Series.\n",
       "DataFrame.count: Number of non-NA elements in a DataFrame.\n",
       "\n",
       "Examples\n",
       "--------\n",
       ">>> index = pd.Index([3, 1, 2, 3, 4, np.nan])\n",
       ">>> index.value_counts()\n",
       "3.0    2\n",
       "4.0    1\n",
       "2.0    1\n",
       "1.0    1\n",
       "dtype: int64\n",
       "\n",
       "With `normalize` set to `True`, returns the relative frequency by\n",
       "dividing all values by the sum of values.\n",
       "\n",
       ">>> s = pd.Series([3, 1, 2, 3, 4, np.nan])\n",
       ">>> s.value_counts(normalize=True)\n",
       "3.0    0.4\n",
       "4.0    0.2\n",
       "2.0    0.2\n",
       "1.0    0.2\n",
       "dtype: float64\n",
       "\n",
       "**bins**\n",
       "\n",
       "Bins can be useful for going from a continuous variable to a\n",
       "categorical variable; instead of counting unique\n",
       "apparitions of values, divide the index in the specified\n",
       "number of half-open bins.\n",
       "\n",
       ">>> s.value_counts(bins=3)\n",
       "(2.0, 3.0]      2\n",
       "(0.996, 2.0]    2\n",
       "(3.0, 4.0]      1\n",
       "dtype: int64\n",
       "\n",
       "**dropna**\n",
       "\n",
       "With `dropna` set to `False` we can also see NaN index values.\n",
       "\n",
       ">>> s.value_counts(dropna=False)\n",
       "3.0    2\n",
       "NaN    1\n",
       "4.0    1\n",
       "2.0    1\n",
       "1.0    1\n",
       "dtype: int64\n",
       "\u001b[0;31mFile:\u001b[0m      /anaconda3/envs/pycon/lib/python3.7/site-packages/pandas/core/base.py\n",
       "\u001b[0;31mType:\u001b[0m      function\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "pd.Series.value_counts?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# pruebalo en la serie cinethh\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# ahora en cihispeed\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "***"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Esto sería el final de nuestro análisis si no estuvieramos trabajando con datos **ponderados**. Datos **ponderados** significa que cada una de nuestras observaciones representa más de una persona u hogar.\n",
    "\n",
    "`perwt` = \"Peso de persona\"\n",
    "\n",
    "`hhwt` = \"Peso del hogar\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`.value_counts(normalize=True)` cuenta el número de observaciones de cada valor de tu Serie y lo divide por el total. Si cada una de nuestras observaciones fuera una persona/hogar, esta sería la respuesta a nuestra pregunta de investigación 1."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Lo que necesitamos hacer es **agregar** datos."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "***"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Paso 4: Agrupando y agregando datos\n",
    "\n",
    "La mecánica es más o menos la misma:\n",
    "1. Cuenta el número total de observaciones de cada uno de los valores de la Serie.\n",
    "2. Sumas **no el número de observaciones** sino el _peso_ de cada observación.\n",
    "3. Divide por el total."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Paso 4.1: Agrupa tus datos por sus valores correspondientes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<pandas.core.groupby.generic.DataFrameGroupBy object at 0x114b5a710>"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "hogares_del_estado.groupby(\"_________\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "De la [documentación](http://pandas.pydata.org/pandas-docs/stable/user_guide/groupby.html) de `Pandas`:\n",
    "\n",
    "> Una operación grupal (`groupby`) implica una combinación de dividir el\n",
    "objeto, **aplicar una función**, y combinar los resultados. Esto se puede usar \n",
    "para agrupar grandes cantidades de datos y operaciones de cómputo en estos grupos."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Lo que nos falta es **aplicar una función**.\n",
    "\n",
    "Prueba lo siguiente:\n",
    "```python\n",
    "hogares_del_estado.groupby(\"countyfip\").sum()\n",
    "```\n",
    "\n",
    "Podemos aplicar *casi* cualquier función.\n",
    "\n",
    "Prueba `.mean()`, `.max()`, `.min()`, `.std()`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "También puedes seleccionar columnas como lo harías en cualquier otro DataFrame."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "hogares_del_estado.groupby(\"________\")['hhwt']._____()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "***"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "n_hogares = hogares_del_estado.groupby(\"cihispeed\")['hhwt'].sum()[2]\n",
    "_estado = hogares_del_estado['statefip'].unique()[0]\n",
    "print(f\"\"\"\n",
    "Podemos ver ahora que {n_hogares:,} los hogares en {_estado} tienen acceso a Internet de alta velocidad. Pero, ¿de cuántos?\n",
    "\n",
    "Para hacer esto más fácil de seguir, guardemos nuestros resultados en una variable:\n",
    "\"\"\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "hogares_con_acceso_internet_alta_velocidad = ____________._____(\"_____\")[\"____\"].___()\n",
    "\n",
    "hogares_con_acceso_internet_alta_velocidad"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Esto se parece a cualquier 'pandas.Series' regular, ¿cómo encontramos la _suma_ total de una serie de elementos?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![math](../../static/math.png)\n",
    "\n",
    "<details>\n",
    "    <summary>respuesta</summary>\n",
    "    <pre>.sum()</pre>\n",
    "</details>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "¡Ese es nuestro denominador! \n",
    "\n",
    "![nice](../../static/nooice.gif)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "***"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Cuando _aplicas_ una operación a una Serie de `pandas` esta se le aplica a cada uno de sus elementos.\n",
    "\n",
    "Prueba lo siguiente:\n",
    "```python\n",
    "hogares_con_acceso_internet_alta_velocidad * 1_000_000\n",
    "```\n",
    "\n",
    "```python\n",
    "hogares_con_acceso_internet_alta_velocidad + 1_000_000\n",
    "```\n",
    "\n",
    "```python\n",
    "hogares_con_acceso_internet_alta_velocidad / 1_000_000\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<details>\n",
    "    <summary>tip</summary>\n",
    "    recuerda que puedes usar la tecla <strong>tab</strong> para autocompletar tus variables. Así no tienes que escribir todas las letras hogares_con_acceso_internet_alta_velocidad cada que lo necesites.\n",
    "</details>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Ahora que tienes el denominador de nuestra ecuación (¿cuántos hogares hay en total en el estado X), cómo encontraría cada uno de los 3 valores en su participación `hogares_con_acceso_internet_alta_velocidad` del total?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "***\n",
    "***"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Parte 2 del análisis: Creando variables derivadas"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Ahora que tienes la respuesta a la **Pregunta de Investigación 1**, podemos movernos a PI2:\n",
    ">_¿Varía este número según los grupos demográficos? (en este caso raza/etnia)._"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "La función `.groupby()` de `pandas` puede tomar una lista de columnas por las cuales agrupar tu DataFrame.\n",
    "\n",
    "Prueba lo siguiente:\n",
    "```python\n",
    "hogares_del_estado.groupby(['race', 'cihispeed'])[['hhwt']].sum()\n",
    "```\n",
    "\n",
    "_Notese que estoy utilizando_ `[[hhwt]]` _(una lista de un elemento) y no solo_ `[hhwt]` _prueba ambos tu y discutamos cual es la diferencia._"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<details>\n",
    "    <summary>La diferencia</summary>\n",
    "    Cuando pasas una lista de elementos (aunque sea de un solo elemento - <span style='font-family:monospace;background-color:#EEE'>[[hhwt]]</span>) a un objeto <span style='font-family:monospace;background-color:#EEE'>.groupby()</span> recibes de regreso un DataFrame. Si pasas una sola columna <span style='font-family:monospace;background-color:#EEE'>[hhwt]</span>, recibes una Serie.\n",
    "</details>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "***"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Paso 1: Define tus grupos"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "El indexador `.loc` de Pandas no solo sirve para dividir DataFrames, sino también para asignar nuevos valores a ciertos segmentos de DataFrames.\n",
    "\n",
    "Por ejemplo,\n",
    "```python\n",
    "mascara_datos_inventados = (data['columna_1'] == 'Sin Respuesta')\n",
    "datos.loc[mascara_datos_inventados, 'nueva_columna'] = 'esta fila no existia'\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "El código de arriba toma todas las filas que satisfacen la condición y luego busca `'nueva_columna'`, si no existe, lo creará y le asignará el valor `'esta fila no existia'` a todas las filas que satisfagan con la condición. El resto se rellenará con valores nulos (NaNs)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### Creemos nuestras máscaras"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "mascara_latino = \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "mascara_blanco = \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "mascara_negro = \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "mascara______ = \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "mascara______ = \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "mascara______ =\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<details>\n",
    "    <summary>Los 6 valores</summary>\n",
    "    En los círculos de investigación de Estados Unidos es común tener 5 grupos principales de raza/etnia: Latinos, Negros/Africano-Americanos, Blancos, Asiaticos/Isleños del Pacífico y Indio-Americanos/Nativos. <br>\n",
    "    Esto obviamente es una sobre-simplificación. Tu puedes utilizar los grupos que quieras pero para este análisis vamos a utilizar estos 5 grupos y un grupo \"Otros\" para capturar todos los hogares en nuestro conjunto de datos.\n",
    "</details>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Asignemos estos valores a una nueva columna `'razet'` por Raza/Etnia"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "hogares_del_estado.loc[mascara_latino, 'razet'] = 'Latino'\n",
    "hogares_del_estado.loc[mascara_blanco, 'razet'] = 'Blanco'\n",
    "hogares_del_estado.loc[mascara_negro, 'razet'] = 'Negro/Africano-Americano'\n",
    "hogares_del_estado.loc[mascara______, 'razet'] = '_______'\n",
    "hogares_del_estado.loc[mascara______, 'razet'] = '_______'\n",
    "hogares_del_estado.loc[mascara______, 'razet'] = '_______'\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Revisa tus resultados.\n",
    "\n",
    "Bajo tu nueva lógica, todos los valores de `race` deberían haber sido capturados en `razet` y no debería haber valores nulos, ¿verdad?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "La función `.isna()` de Pandas regresa una Serie que contiene o True o False (verdadero / falso) correspondiendo a cada valor de la Serie dependiendo de si es un valor nulo o no.\n",
    "\n",
    "Y además en `python`, True es igual a 1 y False a 0.\n",
    "\n",
    "¿Qué crees que pasaría si `.sum`as los valores de una Serie de pandas booleana?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "***"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### Multiples maneras de agrupar/agregar datos"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Ahora que ha derivado una variable de trabajo por raza/etnia, puedes agregar tus datos para responder **PI2**. En pandas, hay muchas maneras de hacer esto, algunas de ellas son:\n",
    "1. `.groupby()` como lo hemos hecho hasta ahora.\n",
    "2. `.pivot_table()`\n",
    "3. `pd.crosstabs()` <- este es un método `pandas`, no un método DataFrame. Más información más tarde."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### GroupBy"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "hogares_del_estado.groupby(['razet', '______'])[['______']]._____()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Guardemos eso en una variable con nombre apropiado ya que la usaremos más adelante."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "cihispeed_por_razet = hogares_del_estado.groupby(['razet', '______'])[['______']]._____()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Ahora, este DataFrame agrupado tiene el número total de hogares en cada uno de estos grupos razet-cihispeed.\n",
    "\n",
    "Necesitamos la proporción de los valores de `cihispeed` por grupo de razet.\n",
    "\n",
    "En nuestra ecuación,\n",
    "\n",
    "$$ \\frac{hogares\\ con\\ internet\\ de\\ alta\\ velocidad}{total\\ de\\ hogares\\ en\\ grupo\\ razet}$$\n",
    "\n",
    "Necesitamos encontrar nuestro denominador."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# encuentra el denominador\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# divide los grupos razet-cihispeed por el denominador\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# guardalo en una variable de nombre apropiado\n",
    "proporcion_cihispeed_por_razet = "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Este es un marco de datos de índice de múltiples niveles y hay algunas maneras de dividirlo. Probemos 3:\n",
    "1. un segmento clásico `.loc`\n",
    "2. una sección transversal (`.xs()`)\n",
    "3. El método `.reset_index()`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Clásico `.loc`**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "proporcion_cihispeed_por_razet.loc[FILTRO_PARA_LAS_FILAS, COLUMNAS]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Sección transversal**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "\u001b[0;31mSignature:\u001b[0m \u001b[0mshares_cihispeed_by_racen\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mxs\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mkey\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0maxis\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mlevel\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;32mNone\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mdrop_level\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;32mTrue\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
       "\u001b[0;31mDocstring:\u001b[0m\n",
       "Return cross-section from the Series/DataFrame.\n",
       "\n",
       "This method takes a `key` argument to select data at a particular\n",
       "level of a MultiIndex.\n",
       "\n",
       "Parameters\n",
       "----------\n",
       "key : label or tuple of label\n",
       "    Label contained in the index, or partially in a MultiIndex.\n",
       "axis : {0 or 'index', 1 or 'columns'}, default 0\n",
       "    Axis to retrieve cross-section on.\n",
       "level : object, defaults to first n levels (n=1 or len(key))\n",
       "    In case of a key partially contained in a MultiIndex, indicate\n",
       "    which levels are used. Levels can be referred by label or position.\n",
       "drop_level : bool, default True\n",
       "    If False, returns object with same levels as self.\n",
       "\n",
       "Returns\n",
       "-------\n",
       "Series or DataFrame\n",
       "    Cross-section from the original Series or DataFrame\n",
       "    corresponding to the selected index levels.\n",
       "\n",
       "See Also\n",
       "--------\n",
       "DataFrame.loc : Access a group of rows and columns\n",
       "    by label(s) or a boolean array.\n",
       "DataFrame.iloc : Purely integer-location based indexing\n",
       "    for selection by position.\n",
       "\n",
       "Notes\n",
       "-----\n",
       "`xs` can not be used to set values.\n",
       "\n",
       "MultiIndex Slicers is a generic way to get/set values on\n",
       "any level or levels.\n",
       "It is a superset of `xs` functionality, see\n",
       ":ref:`MultiIndex Slicers <advanced.mi_slicers>`.\n",
       "\n",
       "Examples\n",
       "--------\n",
       ">>> d = {'num_legs': [4, 4, 2, 2],\n",
       "...      'num_wings': [0, 0, 2, 2],\n",
       "...      'class': ['mammal', 'mammal', 'mammal', 'bird'],\n",
       "...      'animal': ['cat', 'dog', 'bat', 'penguin'],\n",
       "...      'locomotion': ['walks', 'walks', 'flies', 'walks']}\n",
       ">>> df = pd.DataFrame(data=d)\n",
       ">>> df = df.set_index(['class', 'animal', 'locomotion'])\n",
       ">>> df\n",
       "                           num_legs  num_wings\n",
       "class  animal  locomotion\n",
       "mammal cat     walks              4          0\n",
       "       dog     walks              4          0\n",
       "       bat     flies              2          2\n",
       "bird   penguin walks              2          2\n",
       "\n",
       "Get values at specified index\n",
       "\n",
       ">>> df.xs('mammal')\n",
       "                   num_legs  num_wings\n",
       "animal locomotion\n",
       "cat    walks              4          0\n",
       "dog    walks              4          0\n",
       "bat    flies              2          2\n",
       "\n",
       "Get values at several indexes\n",
       "\n",
       ">>> df.xs(('mammal', 'dog'))\n",
       "            num_legs  num_wings\n",
       "locomotion\n",
       "walks              4          0\n",
       "\n",
       "Get values at specified index and level\n",
       "\n",
       ">>> df.xs('cat', level=1)\n",
       "                   num_legs  num_wings\n",
       "class  locomotion\n",
       "mammal walks              4          0\n",
       "\n",
       "Get values at several indexes and levels\n",
       "\n",
       ">>> df.xs(('bird', 'walks'),\n",
       "...       level=[0, 'locomotion'])\n",
       "         num_legs  num_wings\n",
       "animal\n",
       "penguin         2          2\n",
       "\n",
       "Get values at specified column and axis\n",
       "\n",
       ">>> df.xs('num_wings', axis=1)\n",
       "class   animal   locomotion\n",
       "mammal  cat      walks         0\n",
       "        dog      walks         0\n",
       "        bat      flies         2\n",
       "bird    penguin  walks         2\n",
       "Name: num_wings, dtype: int64\n",
       "\u001b[0;31mFile:\u001b[0m      /anaconda3/envs/pycon/lib/python3.7/site-packages/pandas/core/generic.py\n",
       "\u001b[0;31mType:\u001b[0m      method\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "proporcion_cihispeed_por_razet.xs?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Solo necesitas una **key**, que es el valor que te interesa (por ejemplo \"yes (cable modem, fiber optic or dsl service)\" que es una respuesta \"Si\" a la pregunta \"Tienes acceso al internet\" del censo), y el nivel en el cual buscar esta **key** (en python empezamos a contar en 0)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "proporcion_cihispeed_por_razet.xs(key = '________', level = _)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**`.reset_index()`**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Otra forma de dividir un DataFrame de índice multinivel es convertirlo en un DataFrame de índice no multinivel. Para hacer eso necesitas _resetear_ tu índice. Después de eso, podemos dividirlo en la forma en que hemos estado segmentando nuestros DataFrames anteriormente."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "__________ = ____________._________()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "__________"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "mascara_si_cihispeed = (____________[_________] == '___________')\n",
    "_______[mascara_si_cihispeed]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "***"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### Tablas dinámicas (pivot tables)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "El segundo método para agregar nuestros datos es `.pivot_table()`s (tablas dinámicas).\n",
    "\n",
    "Si ha trabajado con Excel, es posible que ya esté familiarizado con lo que es una tabla dinámica.\n",
    "\n",
    "De [Wikipedia](https://es.wikipedia.org/wiki/Tabla_din%C3%A1mica):\n",
    ">las tablas dinámicas pueden de forma automática clasificar, contar, totalizar o dar la media de los datos almacenados en una tabla o una hoja de cálculo. Se muestran los resultados en una segunda tabla (llamada \"tabla dinámica\" o \"tabla pivote\") que muestra los datos resumidos."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "\u001b[0;31mSignature:\u001b[0m\n",
       "\u001b[0mstate_households\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mpivot_table\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m\u001b[0m\n",
       "\u001b[0;34m\u001b[0m    \u001b[0mvalues\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;32mNone\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\n",
       "\u001b[0;34m\u001b[0m    \u001b[0mindex\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;32mNone\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\n",
       "\u001b[0;34m\u001b[0m    \u001b[0mcolumns\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;32mNone\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\n",
       "\u001b[0;34m\u001b[0m    \u001b[0maggfunc\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;34m'mean'\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\n",
       "\u001b[0;34m\u001b[0m    \u001b[0mfill_value\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;32mNone\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\n",
       "\u001b[0;34m\u001b[0m    \u001b[0mmargins\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;32mFalse\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\n",
       "\u001b[0;34m\u001b[0m    \u001b[0mdropna\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;32mTrue\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\n",
       "\u001b[0;34m\u001b[0m    \u001b[0mmargins_name\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;34m'All'\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\n",
       "\u001b[0;34m\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
       "\u001b[0;31mDocstring:\u001b[0m\n",
       "Create a spreadsheet-style pivot table as a DataFrame. The levels in\n",
       "the pivot table will be stored in MultiIndex objects (hierarchical\n",
       "indexes) on the index and columns of the result DataFrame.\n",
       "\n",
       "Parameters\n",
       "----------\n",
       "values : column to aggregate, optional\n",
       "index : column, Grouper, array, or list of the previous\n",
       "    If an array is passed, it must be the same length as the data. The\n",
       "    list can contain any of the other types (except list).\n",
       "    Keys to group by on the pivot table index.  If an array is passed,\n",
       "    it is being used as the same manner as column values.\n",
       "columns : column, Grouper, array, or list of the previous\n",
       "    If an array is passed, it must be the same length as the data. The\n",
       "    list can contain any of the other types (except list).\n",
       "    Keys to group by on the pivot table column.  If an array is passed,\n",
       "    it is being used as the same manner as column values.\n",
       "aggfunc : function, list of functions, dict, default numpy.mean\n",
       "    If list of functions passed, the resulting pivot table will have\n",
       "    hierarchical columns whose top level are the function names\n",
       "    (inferred from the function objects themselves)\n",
       "    If dict is passed, the key is column to aggregate and value\n",
       "    is function or list of functions\n",
       "fill_value : scalar, default None\n",
       "    Value to replace missing values with\n",
       "margins : boolean, default False\n",
       "    Add all row / columns (e.g. for subtotal / grand totals)\n",
       "dropna : boolean, default True\n",
       "    Do not include columns whose entries are all NaN\n",
       "margins_name : string, default 'All'\n",
       "    Name of the row / column that will contain the totals\n",
       "    when margins is True.\n",
       "\n",
       "Returns\n",
       "-------\n",
       "table : DataFrame\n",
       "\n",
       "See Also\n",
       "--------\n",
       "DataFrame.pivot : Pivot without aggregation that can handle\n",
       "    non-numeric data.\n",
       "\n",
       "Examples\n",
       "--------\n",
       ">>> df = pd.DataFrame({\"A\": [\"foo\", \"foo\", \"foo\", \"foo\", \"foo\",\n",
       "...                          \"bar\", \"bar\", \"bar\", \"bar\"],\n",
       "...                    \"B\": [\"one\", \"one\", \"one\", \"two\", \"two\",\n",
       "...                          \"one\", \"one\", \"two\", \"two\"],\n",
       "...                    \"C\": [\"small\", \"large\", \"large\", \"small\",\n",
       "...                          \"small\", \"large\", \"small\", \"small\",\n",
       "...                          \"large\"],\n",
       "...                    \"D\": [1, 2, 2, 3, 3, 4, 5, 6, 7],\n",
       "...                    \"E\": [2, 4, 5, 5, 6, 6, 8, 9, 9]})\n",
       ">>> df\n",
       "     A    B      C  D  E\n",
       "0  foo  one  small  1  2\n",
       "1  foo  one  large  2  4\n",
       "2  foo  one  large  2  5\n",
       "3  foo  two  small  3  5\n",
       "4  foo  two  small  3  6\n",
       "5  bar  one  large  4  6\n",
       "6  bar  one  small  5  8\n",
       "7  bar  two  small  6  9\n",
       "8  bar  two  large  7  9\n",
       "\n",
       "This first example aggregates values by taking the sum.\n",
       "\n",
       ">>> table = pivot_table(df, values='D', index=['A', 'B'],\n",
       "...                     columns=['C'], aggfunc=np.sum)\n",
       ">>> table\n",
       "C        large  small\n",
       "A   B\n",
       "bar one      4      5\n",
       "    two      7      6\n",
       "foo one      4      1\n",
       "    two    NaN      6\n",
       "\n",
       "We can also fill missing values using the `fill_value` parameter.\n",
       "\n",
       ">>> table = pivot_table(df, values='D', index=['A', 'B'],\n",
       "...                     columns=['C'], aggfunc=np.sum, fill_value=0)\n",
       ">>> table\n",
       "C        large  small\n",
       "A   B\n",
       "bar one      4      5\n",
       "    two      7      6\n",
       "foo one      4      1\n",
       "    two      0      6\n",
       "\n",
       "The next example aggregates by taking the mean across multiple columns.\n",
       "\n",
       ">>> table = pivot_table(df, values=['D', 'E'], index=['A', 'C'],\n",
       "...                     aggfunc={'D': np.mean,\n",
       "...                              'E': np.mean})\n",
       ">>> table\n",
       "                  D         E\n",
       "               mean      mean\n",
       "A   C\n",
       "bar large  5.500000  7.500000\n",
       "    small  5.500000  8.500000\n",
       "foo large  2.000000  4.500000\n",
       "    small  2.333333  4.333333\n",
       "\n",
       "We can also calculate multiple types of aggregations for any given\n",
       "value column.\n",
       "\n",
       ">>> table = pivot_table(df, values=['D', 'E'], index=['A', 'C'],\n",
       "...                     aggfunc={'D': np.mean,\n",
       "...                              'E': [min, max, np.mean]})\n",
       ">>> table\n",
       "                  D   E\n",
       "               mean max      mean min\n",
       "A   C\n",
       "bar large  5.500000  9   7.500000   6\n",
       "    small  5.500000  9   8.500000   8\n",
       "foo large  2.000000  5   4.500000   4\n",
       "    small  2.333333  6   4.333333   2\n",
       "\u001b[0;31mFile:\u001b[0m      /anaconda3/envs/pycon/lib/python3.7/site-packages/pandas/core/frame.py\n",
       "\u001b[0;31mType:\u001b[0m      method\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "hogares_del_estado.pivot_table?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "What we need are four things:\n",
    "1. What variable will become our `index`?\n",
    "2. What variable will become our `columns`?\n",
    "3. What variable will become our `values`?\n",
    "4. How will we aggregate our values?\n",
    "\n",
    "Pandas is going to grab each unique value in the variables you choose and use those as rows in your `.index` or separate columns in your `.columns`. The `values` variable should be _quantitative_ in this case (but it doesn't have to be, necessarily). `.pivot_table` will by default find the `mean` of your `values` variable for each cell in your new table, in this case we don't care about the `mean`, we want to `sum` up the total number of households."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Prueba lo siguiente:\n",
    "\n",
    "```python\n",
    "hogares_del_estado.pivot_table(\n",
    "    index = '______',\n",
    "    columns = '______', \n",
    "    values = 'hhwt',\n",
    "    aggfunc = '___',\n",
    "    margins = True,\n",
    ")\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Guárdalo en una variable apropiadamente nombrada."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "hogares_tabla_pivote = hogares_del_estado.pivot_table(\n",
    "    index = '_____',\n",
    "    columns = '______',\n",
    "    ______ = '______',\n",
    "    ______ = '____',\n",
    "    _______ = True,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "¿Cúal crees que es el siguiente paso?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}