BBC_News_Articles_Neural_Network_Classifier.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "c83d1d7b",
   "metadata": {},
   "source": [
    "# Week 2: Diving deeper into the BBC News archive\n",
    "\n",
    "Welcome! In this assignment you will be revisiting the [BBC News Classification Dataset](https://www.kaggle.com/c/learn-ai-bbc/overview), which contains 2225 examples of news articles with their respective labels. \n",
    "\n",
    "This time you will not only work with the tokenization process but you will also create a classifier using specialized layers for text data such as Embedding and GlobalAveragePooling1D.\n",
    "\n",
    "Let's get started!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "c09324f3",
   "metadata": {
    "deletable": false,
    "editable": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[32mGrader metadata detected! You can proceed with the lab!\u001b[0m\r\n"
     ]
    }
   ],
   "source": [
    "# IMPORTANT: This will check your notebook's metadata for grading.\n",
    "# Please do not continue the lab unless the output of this cell tells you to proceed. \n",
    "!python add_metadata.py --filename C3W2_Assignment.ipynb"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5c94ba57",
   "metadata": {},
   "source": [
    "_**NOTE:** To prevent errors from the autograder, you are not allowed to edit or delete non-graded cells in this notebook . Please only put your solutions in between the `### START CODE HERE` and `### END CODE HERE` code comments, and also refrain from adding any new cells. **Once you have passed this assignment** and want to experiment with any of the non-graded code, you may follow the instructions at the bottom of this notebook._"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "ce7f3d12",
   "metadata": {
    "deletable": false,
    "editable": false,
    "id": "gnwiOnGyW5JK",
    "tags": [
     "graded"
    ]
   },
   "outputs": [],
   "source": [
    "# grader-required-cell\n",
    "\n",
    "import io\n",
    "import csv\n",
    "import tensorflow as tf\n",
    "import numpy as np\n",
    "from tensorflow.keras.preprocessing.text import Tokenizer\n",
    "from tensorflow.keras.preprocessing.sequence import pad_sequences\n",
    "import matplotlib.pyplot as plt"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bb1bdd29",
   "metadata": {},
   "source": [
    "You probably remember structure of the csv that contains the data from last week, but in case you need a reminder run the next cell:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "51d18e5b",
   "metadata": {
    "deletable": false,
    "editable": false,
    "tags": [
     "graded"
    ]
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "First line (header) looks like this:\n",
      "\n",
      "category,text\n",
      "\n",
      "Each data point looks like this:\n",
      "\n",
      "tech,tv future in the hands of viewers with home theatre systems  plasma high-definition tvs  and digital video recorders moving into the living room  the way people watch tv will be radically different in five years  time.  that is according to an expert panel which gathered at the annual consumer electronics show in las vegas to discuss how these new technologies will impact one of our favourite pastimes. with the us leading the trend  programmes and other content will be delivered to viewers via home networks  through cable  satellite  telecoms companies  and broadband service providers to front rooms and portable devices.  one of the most talked-about technologies of ces has been digital and personal video recorders (dvr and pvr). these set-top boxes  like the us s tivo and the uk s sky+ system  allow people to record  store  play  pause and forward wind tv programmes when they want.  essentially  the technology allows for much more personalised tv. they are also being built-in to high-definition tv sets  which are big business in japan and the us  but slower to take off in europe because of the lack of high-definition programming. not only can people forward wind through adverts  they can also forget about abiding by network and channel schedules  putting together their own a-la-carte entertainment. but some us networks and cable and satellite companies are worried about what it means for them in terms of advertising revenues as well as  brand identity  and viewer loyalty to channels. although the us leads in this technology at the moment  it is also a concern that is being raised in europe  particularly with the growing uptake of services like sky+.  what happens here today  we will see in nine months to a years  time in the uk   adam hume  the bbc broadcast s futurologist told the bbc news website. for the likes of the bbc  there are no issues of lost advertising revenue yet. it is a more pressing issue at the moment for commercial uk broadcasters  but brand loyalty is important for everyone.  we will be talking more about content brands rather than network brands   said tim hanlon  from brand communications firm starcom mediavest.  the reality is that with broadband connections  anybody can be the producer of content.  he added:  the challenge now is that it is hard to promote a programme with so much choice.   what this means  said stacey jolna  senior vice president of tv guide tv group  is that the way people find the content they want to watch has to be simplified for tv viewers. it means that networks  in us terms  or channels could take a leaf out of google s book and be the search engine of the future  instead of the scheduler to help people find what they want to watch. this kind of channel model might work for the younger ipod generation which is used to taking control of their gadgets and what they play on them. but it might not suit everyone  the panel recognised. older generations are more comfortable with familiar schedules and channel brands because they know what they are getting. they perhaps do not want so much of the choice put into their hands  mr hanlon suggested.  on the other end  you have the kids just out of diapers who are pushing buttons already - everything is possible and available to them   said mr hanlon.  ultimately  the consumer will tell the market they want.   of the 50 000 new gadgets and technologies being showcased at ces  many of them are about enhancing the tv-watching experience. high-definition tv sets are everywhere and many new models of lcd (liquid crystal display) tvs have been launched with dvr capability built into them  instead of being external boxes. one such example launched at the show is humax s 26-inch lcd tv with an 80-hour tivo dvr and dvd recorder. one of the us s biggest satellite tv companies  directtv  has even launched its own branded dvr at the show with 100-hours of recording capability  instant replay  and a search function. the set can pause and rewind tv for up to 90 hours. and microsoft chief bill gates announced in his pre-show keynote speech a partnership with tivo  called tivotogo  which means people can play recorded programmes on windows pcs and mobile devices. all these reflect the increasing trend of freeing up multimedia so that people can watch what they want  when they want.\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# grader-required-cell\n",
    "\n",
    "with open(\"./bbc-text.csv\", 'r') as csvfile:\n",
    "    print(f\"First line (header) looks like this:\\n\\n{csvfile.readline()}\")\n",
    "    print(f\"Each data point looks like this:\\n\\n{csvfile.readline()}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fa68de0d",
   "metadata": {},
   "source": [
    "As you can see, each data point is composed of the category of the news article followed by a comma and then the actual text of the article."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "993a2837",
   "metadata": {},
   "source": [
    "## Defining useful global variables\n",
    "\n",
    "Next, you will define some global variables that will be used in the unit tests after your solutions. **Please do not use these in the function body of the graded functions.**\n",
    "\n",
    "- `NUM_WORDS`: The maximum number of words to keep, based on word frequency. Defaults to 1000.\n",
    "\n",
    "\n",
    "- `EMBEDDING_DIM`: Dimensionality of the dense embedding, will be used in the embedding layer of the model. Defaults to 16.\n",
    "\n",
    "\n",
    "- `MAXLEN`: Maximum length of all sequences. Defaults to 120.\n",
    "\n",
    "\n",
    "- `PADDING`: Padding strategy (pad either before or after each sequence.). Defaults to 'post'.\n",
    "\n",
    "\n",
    "- `OOV_TOKEN`: Token to replace out-of-vocabulary words during text_to_sequence calls. Defaults to \"\\<OOV>\".\n",
    "\n",
    "    \n",
    "- `TRAINING_SPLIT`: Proportion of data used for training. Defaults to 0.8\n",
    "\n",
    "**For now leave them unchanged but after submitting your assignment for grading you are encouraged to come back here and play with these parameters to see the impact they have in the classification process**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "87fb4a2a",
   "metadata": {
    "deletable": false,
    "editable": false,
    "tags": [
     "graded"
    ]
   },
   "outputs": [],
   "source": [
    "# grader-required-cell\n",
    "\n",
    "NUM_WORDS = 1000\n",
    "EMBEDDING_DIM = 16\n",
    "MAXLEN = 120\n",
    "PADDING = 'post'\n",
    "OOV_TOKEN = \"<OOV>\"\n",
    "TRAINING_SPLIT = .8"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d0f8c052",
   "metadata": {},
   "source": [
    "## Loading and pre-processing the data\n",
    "\n",
    "Now you should code the functions to remove stopwords from text and to load the data from a csv file.\n",
    "\n",
    "Since you already coded these functions for the previous week, these are provided for you."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "b10206ea",
   "metadata": {
    "deletable": false,
    "editable": false,
    "tags": [
     "graded"
    ]
   },
   "outputs": [],
   "source": [
    "# grader-required-cell\n",
    "\n",
    "def remove_stopwords(sentence):\n",
    "    \"\"\"\n",
    "    Removes a list of stopwords\n",
    "    \n",
    "    Args:\n",
    "        sentence (string): sentence to remove the stopwords from\n",
    "    \n",
    "    Returns:\n",
    "        sentence (string): lowercase sentence without the stopwords\n",
    "    \"\"\"\n",
    "    # List of stopwords\n",
    "    stopwords = [\"a\", \"about\", \"above\", \"after\", \"again\", \"against\", \"all\", \"am\", \"an\", \"and\", \"any\", \"are\", \"as\", \"at\", \"be\", \"because\", \"been\", \"before\", \"being\", \"below\", \"between\", \"both\", \"but\", \"by\", \"could\", \"did\", \"do\", \"does\", \"doing\", \"down\", \"during\", \"each\", \"few\", \"for\", \"from\", \"further\", \"had\", \"has\", \"have\", \"having\", \"he\", \"he'd\", \"he'll\", \"he's\", \"her\", \"here\", \"here's\", \"hers\", \"herself\", \"him\", \"himself\", \"his\", \"how\", \"how's\", \"i\", \"i'd\", \"i'll\", \"i'm\", \"i've\", \"if\", \"in\", \"into\", \"is\", \"it\", \"it's\", \"its\", \"itself\", \"let's\", \"me\", \"more\", \"most\", \"my\", \"myself\", \"nor\", \"of\", \"on\", \"once\", \"only\", \"or\", \"other\", \"ought\", \"our\", \"ours\", \"ourselves\", \"out\", \"over\", \"own\", \"same\", \"she\", \"she'd\", \"she'll\", \"she's\", \"should\", \"so\", \"some\", \"such\", \"than\", \"that\", \"that's\", \"the\", \"their\", \"theirs\", \"them\", \"themselves\", \"then\", \"there\", \"there's\", \"these\", \"they\", \"they'd\", \"they'll\", \"they're\", \"they've\", \"this\", \"those\", \"through\", \"to\", \"too\", \"under\", \"until\", \"up\", \"very\", \"was\", \"we\", \"we'd\", \"we'll\", \"we're\", \"we've\", \"were\", \"what\", \"what's\", \"when\", \"when's\", \"where\", \"where's\", \"which\", \"while\", \"who\", \"who's\", \"whom\", \"why\", \"why's\", \"with\", \"would\", \"you\", \"you'd\", \"you'll\", \"you're\", \"you've\", \"your\", \"yours\", \"yourself\", \"yourselves\" ]\n",
    "    \n",
    "    # Sentence converted to lowercase-only\n",
    "    sentence = sentence.lower()\n",
    "\n",
    "    words = sentence.split()\n",
    "    no_words = [w for w in words if w not in stopwords]\n",
    "    sentence = \" \".join(no_words)\n",
    "\n",
    "    return sentence\n",
    "\n",
    "\n",
    "def parse_data_from_file(filename):\n",
    "    \"\"\"\n",
    "    Extracts sentences and labels from a CSV file\n",
    "    \n",
    "    Args:\n",
    "        filename (string): path to the CSV file\n",
    "    \n",
    "    Returns:\n",
    "        sentences, labels (list of string, list of string): tuple containing lists of sentences and labels\n",
    "    \"\"\"\n",
    "    sentences = []\n",
    "    labels = []\n",
    "    with open(filename, 'r') as csvfile:\n",
    "        reader = csv.reader(csvfile, delimiter=',')\n",
    "        next(reader)\n",
    "        for row in reader:\n",
    "            labels.append(row[0])\n",
    "            sentence = row[1]\n",
    "            sentence = remove_stopwords(sentence)\n",
    "            sentences.append(sentence)\n",
    "\n",
    "    return sentences, labels"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "7f92f7fc",
   "metadata": {
    "deletable": false,
    "editable": false,
    "tags": [
     "graded"
    ]
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "There are 2225 sentences in the dataset.\n",
      "\n",
      "First sentence has 436 words (after removing stopwords).\n",
      "\n",
      "There are 2225 labels in the dataset.\n",
      "\n",
      "The first 5 labels are ['tech', 'business', 'sport', 'sport', 'entertainment']\n"
     ]
    }
   ],
   "source": [
    "# grader-required-cell\n",
    "\n",
    "# Test the functions\n",
    "sentences, labels = parse_data_from_file(\"./bbc-text.csv\")\n",
    "\n",
    "print(f\"There are {len(sentences)} sentences in the dataset.\\n\")\n",
    "print(f\"First sentence has {len(sentences[0].split())} words (after removing stopwords).\\n\")\n",
    "print(f\"There are {len(labels)} labels in the dataset.\\n\")\n",
    "print(f\"The first 5 labels are {labels[:5]}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fcdffb0b",
   "metadata": {},
   "source": [
    "***Expected Output:***\n",
    "\n",
    "```\n",
    "There are 2225 sentences in the dataset.\n",
    "\n",
    "First sentence has 436 words (after removing stopwords).\n",
    "\n",
    "There are 2225 labels in the dataset.\n",
    "\n",
    "The first 5 labels are ['tech', 'business', 'sport', 'sport', 'entertainment']\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b147ad15",
   "metadata": {},
   "source": [
    "## Training - Validation Split\n",
    "\n",
    "Now you will code the `train_val_split()` function. Given the training split size, this function will split the full lists of sentences and labels into training and validation sentences and labels.\n",
    "\n",
    "**IMPORTANT NOTE: For all graded functions in this assignment, please do not use the global variables (e.g. `TRAINING_SPLIT`) in your solution. Instead, you should use the function parameters defined in the function header (e.g. `training_split`).**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "1e0158cb",
   "metadata": {
    "deletable": false,
    "tags": [
     "graded"
    ]
   },
   "outputs": [],
   "source": [
    "# grader-required-cell\n",
    "\n",
    "# GRADED FUNCTIONS: train_val_split\n",
    "def train_val_split(sentences, labels, training_split):\n",
    "    \"\"\"\n",
    "    Splits the dataset into training and validation sets\n",
    "    \n",
    "    Args:\n",
    "        sentences (list of string): lower-cased sentences without stopwords\n",
    "        labels (list of string): list of labels\n",
    "        training split (float): proportion of the dataset to convert to include in the train set\n",
    "    \n",
    "    Returns:\n",
    "        train_sentences, validation_sentences, train_labels, validation_labels - lists containing the data splits\n",
    "    \"\"\"\n",
    "    \n",
    "    ### START CODE HERE\n",
    "    \n",
    "    # Compute the number of sentences that will be used for training (should be an integer)\n",
    "    train_size = int(len(sentences) * training_split)\n",
    "\n",
    "    # Split the sentences and labels into train/validation splits\n",
    "    train_sentences = sentences[:train_size]\n",
    "    train_labels = labels[:train_size]\n",
    "\n",
    "    validation_sentences = sentences[train_size:]\n",
    "    validation_labels = labels[train_size:]\n",
    "    \n",
    "    ### END CODE HERE\n",
    "    \n",
    "    return train_sentences, validation_sentences, train_labels, validation_labels"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "cc283035",
   "metadata": {
    "deletable": false,
    "editable": false,
    "tags": [
     "graded"
    ]
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "There are 1780 sentences for training.\n",
      "\n",
      "There are 1780 labels for training.\n",
      "\n",
      "There are 445 sentences for validation.\n",
      "\n",
      "There are 445 labels for validation.\n"
     ]
    }
   ],
   "source": [
    "# grader-required-cell\n",
    "\n",
    "# Test your function\n",
    "train_sentences, val_sentences, train_labels, val_labels = train_val_split(sentences, labels, TRAINING_SPLIT)\n",
    "\n",
    "print(f\"There are {len(train_sentences)} sentences for training.\\n\")\n",
    "print(f\"There are {len(train_labels)} labels for training.\\n\")\n",
    "print(f\"There are {len(val_sentences)} sentences for validation.\\n\")\n",
    "print(f\"There are {len(val_labels)} labels for validation.\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cec2d944",
   "metadata": {},
   "source": [
    "***Expected Output:***\n",
    "\n",
    "```\n",
    "There are 1780 sentences for training.\n",
    "\n",
    "There are 1780 labels for training.\n",
    "\n",
    "There are 445 sentences for validation.\n",
    "\n",
    "There are 445 labels for validation.\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ecc1b2fb",
   "metadata": {},
   "source": [
    "## Tokenization - Sequences and padding\n",
    "\n",
    "Now that you have sets for training and validation it is time for you to begin the tokenization process.\n",
    "\n",
    "Begin by completing the `fit_tokenizer` function below. This function should return a [Tokenizer](https://www.tensorflow.org/api_docs/python/tf/keras/preprocessing/text/Tokenizer) that has been fitted to the training sentences."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "af18d135",
   "metadata": {
    "deletable": false,
    "lines_to_next_cell": 2,
    "tags": [
     "graded"
    ]
   },
   "outputs": [],
   "source": [
    "# grader-required-cell\n",
    "\n",
    "# GRADED FUNCTION: fit_tokenizer\n",
    "def fit_tokenizer(train_sentences, num_words, oov_token):\n",
    "    \"\"\"\n",
    "    Instantiates the Tokenizer class on the training sentences\n",
    "    \n",
    "    Args:\n",
    "        train_sentences (list of string): lower-cased sentences without stopwords to be used for training\n",
    "        num_words (int) - number of words to keep when tokenizing\n",
    "        oov_token (string) - symbol for the out-of-vocabulary token\n",
    "    \n",
    "    Returns:\n",
    "        tokenizer (object): an instance of the Tokenizer class containing the word-index dictionary\n",
    "    \"\"\"\n",
    "    \n",
    "    ### START CODE HERE\n",
    "    \n",
    "    # Instantiate the Tokenizer class, passing in the correct values for num_words and oov_token\n",
    "    tokenizer = Tokenizer(num_words=num_words, oov_token=oov_token)\n",
    "    \n",
    "    # Fit the tokenizer to the training sentences\n",
    "    tokenizer.fit_on_texts(train_sentences)\n",
    "    \n",
    "    ### END CODE HERE\n",
    "    \n",
    "    return tokenizer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "07772b6f",
   "metadata": {
    "deletable": false,
    "editable": false,
    "tags": [
     "graded"
    ]
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Vocabulary contains 27285 words\n",
      "\n",
      "<OOV> token included in vocabulary\n"
     ]
    }
   ],
   "source": [
    "# grader-required-cell\n",
    "\n",
    "# Test your function\n",
    "tokenizer = fit_tokenizer(train_sentences, NUM_WORDS, OOV_TOKEN)\n",
    "word_index = tokenizer.word_index\n",
    "\n",
    "print(f\"Vocabulary contains {len(word_index)} words\\n\")\n",
    "print(\"<OOV> token included in vocabulary\" if \"<OOV>\" in word_index else \"<OOV> token NOT included in vocabulary\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2d08234f",
   "metadata": {},
   "source": [
    "***Expected Output:***\n",
    "\n",
    "```\n",
    "Vocabulary contains 27285 words\n",
    "\n",
    "<OOV> token included in vocabulary\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "58c6d999",
   "metadata": {},
   "source": [
    "Now that the tokenizer has been fitted to the training data, you need a function that will convert each text data point into its padded sequence representation, for this complete the `seq_and_pad` function below:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "8eec1dd6",
   "metadata": {
    "deletable": false,
    "tags": [
     "graded"
    ]
   },
   "outputs": [],
   "source": [
    "# grader-required-cell\n",
    "\n",
    "# GRADED FUNCTION: seq_and_pad\n",
    "def seq_and_pad(sentences, tokenizer, padding, maxlen):\n",
    "    \"\"\"\n",
    "    Generates an array of token sequences and pads them to the same length\n",
    "    \n",
    "    Args:\n",
    "        sentences (list of string): list of sentences to tokenize and pad\n",
    "        tokenizer (object): Tokenizer instance containing the word-index dictionary\n",
    "        padding (string): type of padding to use\n",
    "        maxlen (int): maximum length of the token sequence\n",
    "    \n",
    "    Returns:\n",
    "        padded_sequences (array of int): tokenized sentences padded to the same length\n",
    "    \"\"\" \n",
    "    \n",
    "    ### START CODE HERE\n",
    "       \n",
    "    # Convert sentences to sequences\n",
    "    sequences = tokenizer.texts_to_sequences(sentences)\n",
    "    \n",
    "    # Pad the sequences using the correct padding and maxlen\n",
    "    padded_sequences = pad_sequences(sequences, padding=padding, maxlen=maxlen)\n",
    "    \n",
    "    ### END CODE HERE\n",
    "    \n",
    "    return padded_sequences"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "2b0ff145",
   "metadata": {
    "deletable": false,
    "editable": false,
    "tags": [
     "graded"
    ]
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Padded training sequences have shape: (1780, 120)\n",
      "\n",
      "Padded validation sequences have shape: (445, 120)\n"
     ]
    }
   ],
   "source": [
    "# grader-required-cell\n",
    "\n",
    "# Test your function\n",
    "train_padded_seq = seq_and_pad(train_sentences, tokenizer, PADDING, MAXLEN)\n",
    "val_padded_seq = seq_and_pad(val_sentences, tokenizer, PADDING, MAXLEN)\n",
    "\n",
    "print(f\"Padded training sequences have shape: {train_padded_seq.shape}\\n\")\n",
    "print(f\"Padded validation sequences have shape: {val_padded_seq.shape}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0ae14de6",
   "metadata": {},
   "source": [
    "***Expected Output:***\n",
    "\n",
    "```\n",
    "Padded training sequences have shape: (1780, 120)\n",
    "\n",
    "Padded validation sequences have shape: (445, 120)\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d99f4805",
   "metadata": {},
   "source": [
    "Finally you need to tokenize the labels. For this complete the `tokenize_labels` function below.\n",
    "\n",
    "A couple of things to note:\n",
    "- You should fit the tokenizer to all the labels to avoid the case of a particular label not being present in the validation set. Since you are dealing with labels there should never be an OOV label.\n",
    "\n",
    "\n",
    "- In the previous function you used the `pad_sequences` function which returns numpy arrays. Here you will not be using it since you don't need to pad the labels so you need to make the conversion to numpy arrays yourself.\n",
    "\n",
    "\n",
    "- The argument `split_labels` refers to the labels of a particular split (train or validation). This is because the function should work independently of the split being used.\n",
    "\n",
    "\n",
    "- Using Keras' Tokenizer yields values that start at 1 rather than at 0. This will present a problem when training since Keras usually expects the labels to start at 0. To work around this issue you could use an extra neuron in the last layer of your model. However this approach is rather hacky and not very clear. Instead you will substract 1 from every value of the labels that the function returns. Remember that when using numpy arrays you can simply do something like `np.array - 1` to accomplish this since numpy allows for vectorized operations."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "217c08ad",
   "metadata": {
    "deletable": false,
    "id": "XkWiQ_FKZNp2",
    "lines_to_next_cell": 2,
    "tags": [
     "graded"
    ]
   },
   "outputs": [],
   "source": [
    "# grader-required-cell\n",
    "\n",
    "# GRADED FUNCTION: tokenize_labels\n",
    "def tokenize_labels(all_labels, split_labels):\n",
    "    \"\"\"\n",
    "    Tokenizes the labels\n",
    "    \n",
    "    Args:\n",
    "        all_labels (list of string): labels to generate the word-index from\n",
    "        split_labels (list of string): labels to tokenize\n",
    "    \n",
    "    Returns:\n",
    "        label_seq_np (array of int): tokenized labels\n",
    "    \"\"\"\n",
    "    \n",
    "    ### START CODE HERE\n",
    "    \n",
    "    # Instantiate the Tokenizer (no additional arguments needed)\n",
    "    label_tokenizer = Tokenizer()\n",
    "    \n",
    "    # Fit the tokenizer on all the labels\n",
    "    label_tokenizer.fit_on_texts(all_labels)\n",
    "    \n",
    "     # Convert split labels to sequences\n",
    "    label_seq = label_tokenizer.texts_to_sequences(split_labels)\n",
    "    \n",
    "    # Convert sequences to a numpy array and reshape to be 2D\n",
    "    label_seq_np = np.array(label_seq).reshape(-1, 1)\n",
    "    \n",
    "    # Adjust the labels to be zero-indexed\n",
    "    label_seq_np -= 1\n",
    "    \n",
    "    ### END CODE HERE\n",
    "    \n",
    "    return label_seq_np"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "a4c970e3",
   "metadata": {
    "deletable": false,
    "editable": false,
    "tags": [
     "graded"
    ]
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "First 5 labels of the training set should look like this:\n",
      "[[3]\n",
      " [1]\n",
      " [0]\n",
      " [0]\n",
      " [4]]\n",
      "\n",
      "First 5 labels of the validation set should look like this:\n",
      "[[4]\n",
      " [3]\n",
      " [2]\n",
      " [0]\n",
      " [0]]\n",
      "\n",
      "Tokenized labels of the training set have shape: (1780, 1)\n",
      "\n",
      "Tokenized labels of the validation set have shape: (445, 1)\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# grader-required-cell\n",
    "\n",
    "# Test your function\n",
    "train_label_seq = tokenize_labels(labels, train_labels)\n",
    "val_label_seq = tokenize_labels(labels, val_labels)\n",
    "\n",
    "print(f\"First 5 labels of the training set should look like this:\\n{train_label_seq[:5]}\\n\")\n",
    "print(f\"First 5 labels of the validation set should look like this:\\n{val_label_seq[:5]}\\n\")\n",
    "print(f\"Tokenized labels of the training set have shape: {train_label_seq.shape}\\n\")\n",
    "print(f\"Tokenized labels of the validation set have shape: {val_label_seq.shape}\\n\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3f0aed04",
   "metadata": {},
   "source": [
    "***Expected Output:***\n",
    "\n",
    "```\n",
    "First 5 labels of the training set should look like this:\n",
    "[[3]\n",
    " [1]\n",
    " [0]\n",
    " [0]\n",
    " [4]]\n",
    "\n",
    "First 5 labels of the validation set should look like this:\n",
    "[[4]\n",
    " [3]\n",
    " [2]\n",
    " [0]\n",
    " [0]]\n",
    "\n",
    "Tokenized labels of the training set have shape: (1780, 1)\n",
    "\n",
    "Tokenized labels of the validation set have shape: (445, 1)\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "563c542a",
   "metadata": {},
   "source": [
    "## Selecting the model for text classification\n",
    "\n",
    "Now that the data is ready to be fed a Neural Network it is time for you to define the model that will classify each text as being part of a certain category. \n",
    "\n",
    "For this complete the `create_model` below. \n",
    "\n",
    "A couple of things to keep in mind:\n",
    "\n",
    "- Notice that this function has three parameters, all of which are meant to be passed to an [Embedding](https://www.tensorflow.org/api_docs/python/tf/keras/layers/Embedding) layer, which is what you will probably use as a first layer for your model.\n",
    "\n",
    "\n",
    "- The last layer should be a Dense layer with 5 units (since there are 5 categories) with a softmax activation.\n",
    "\n",
    "\n",
    "- You should also compile your model using an appropiate loss function and optimizer. Please choose these without needing to edit the following code cell that contains `model.fit()`\n",
    "\n",
    "\n",
    "- You can use any architecture you want but keep in mind that this problem doesn't need many layers to be solved successfully. You don't need any layers beside Embedding, [GlobalAveragePooling1D](https://www.tensorflow.org/api_docs/python/tf/keras/layers/GlobalAveragePooling1D) and Dense layers but feel free to try out different architectures.\n",
    "\n",
    "\n",
    "- **To pass this graded function your model should reach at least a 95% training accuracy and a 90% validation accuracy in under 30 epochs.**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "03a7fbc9",
   "metadata": {
    "deletable": false,
    "id": "HZ5um4MWZP-W",
    "tags": [
     "graded"
    ]
   },
   "outputs": [],
   "source": [
    "# grader-required-cell\n",
    "\n",
    "# GRADED FUNCTION: create_model\n",
    "def create_model(num_words, embedding_dim, maxlen):\n",
    "    \"\"\"\n",
    "    Creates a text classifier model\n",
    "    \n",
    "    Args:\n",
    "        num_words (int): size of the vocabulary for the Embedding layer input\n",
    "        embedding_dim (int): dimensionality of the Embedding layer output\n",
    "        maxlen (int): length of the input sequences\n",
    "    \n",
    "    Returns:\n",
    "        model (tf.keras Model): the text classifier model\n",
    "    \"\"\"\n",
    "    \n",
    "    tf.random.set_seed(123)\n",
    "    \n",
    "    ### START CODE HERE\n",
    "    \n",
    "    model = tf.keras.Sequential([\n",
    "        # Embedding layer to convert integer sequences into dense vectors\n",
    "        tf.keras.layers.Embedding(input_dim=num_words, \n",
    "                                  output_dim=embedding_dim, \n",
    "                                  input_length=maxlen),\n",
    "        \n",
    "        # GlobalAveragePooling1D to reduce the dimensionality\n",
    "        tf.keras.layers.GlobalAveragePooling1D(),\n",
    "        \n",
    "        # Dense layer with 5 units for 5 categories\n",
    "        tf.keras.layers.Dense(5, activation='softmax')\n",
    "    ])\n",
    "    \n",
    "    model.compile(loss='sparse_categorical_crossentropy',\n",
    "                  optimizer='adam',\n",
    "                  metrics=['accuracy']) \n",
    "\n",
    "    ### END CODE HERE\n",
    "\n",
    "    return model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "id": "aa115b83",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/30\n",
      "56/56 [==============================] - 1s 4ms/step - loss: 1.5959 - accuracy: 0.2882 - val_loss: 1.5818 - val_accuracy: 0.4472\n",
      "Epoch 2/30\n",
      "56/56 [==============================] - 0s 2ms/step - loss: 1.5613 - accuracy: 0.4478 - val_loss: 1.5453 - val_accuracy: 0.4562\n",
      "Epoch 3/30\n",
      "56/56 [==============================] - 0s 2ms/step - loss: 1.5152 - accuracy: 0.4607 - val_loss: 1.4965 - val_accuracy: 0.4809\n",
      "Epoch 4/30\n",
      "56/56 [==============================] - 0s 2ms/step - loss: 1.4552 - accuracy: 0.5140 - val_loss: 1.4343 - val_accuracy: 0.5371\n",
      "Epoch 5/30\n",
      "56/56 [==============================] - 0s 2ms/step - loss: 1.3825 - accuracy: 0.6107 - val_loss: 1.3628 - val_accuracy: 0.6247\n",
      "Epoch 6/30\n",
      "56/56 [==============================] - 0s 2ms/step - loss: 1.3001 - accuracy: 0.6961 - val_loss: 1.2819 - val_accuracy: 0.7281\n",
      "Epoch 7/30\n",
      "56/56 [==============================] - 0s 2ms/step - loss: 1.2110 - accuracy: 0.7893 - val_loss: 1.1974 - val_accuracy: 0.7955\n",
      "Epoch 8/30\n",
      "56/56 [==============================] - 0s 2ms/step - loss: 1.1194 - accuracy: 0.8545 - val_loss: 1.1134 - val_accuracy: 0.8449\n",
      "Epoch 9/30\n",
      "56/56 [==============================] - 0s 2ms/step - loss: 1.0283 - accuracy: 0.8843 - val_loss: 1.0306 - val_accuracy: 0.8652\n",
      "Epoch 10/30\n",
      "56/56 [==============================] - 0s 2ms/step - loss: 0.9407 - accuracy: 0.9084 - val_loss: 0.9516 - val_accuracy: 0.8876\n",
      "Epoch 11/30\n",
      "56/56 [==============================] - 0s 2ms/step - loss: 0.8579 - accuracy: 0.9242 - val_loss: 0.8774 - val_accuracy: 0.8944\n",
      "Epoch 12/30\n",
      "56/56 [==============================] - 0s 2ms/step - loss: 0.7820 - accuracy: 0.9309 - val_loss: 0.8102 - val_accuracy: 0.8944\n",
      "Epoch 13/30\n",
      "56/56 [==============================] - 0s 2ms/step - loss: 0.7130 - accuracy: 0.9371 - val_loss: 0.7494 - val_accuracy: 0.9056\n",
      "Epoch 14/30\n",
      "56/56 [==============================] - 0s 2ms/step - loss: 0.6512 - accuracy: 0.9444 - val_loss: 0.6950 - val_accuracy: 0.9079\n",
      "Epoch 15/30\n",
      "56/56 [==============================] - 0s 2ms/step - loss: 0.5965 - accuracy: 0.9472 - val_loss: 0.6462 - val_accuracy: 0.9056\n",
      "Epoch 16/30\n",
      "56/56 [==============================] - 0s 2ms/step - loss: 0.5476 - accuracy: 0.9478 - val_loss: 0.6035 - val_accuracy: 0.9079\n",
      "Epoch 17/30\n",
      "56/56 [==============================] - 0s 2ms/step - loss: 0.5040 - accuracy: 0.9522 - val_loss: 0.5660 - val_accuracy: 0.9101\n",
      "Epoch 18/30\n",
      "56/56 [==============================] - 0s 2ms/step - loss: 0.4659 - accuracy: 0.9522 - val_loss: 0.5322 - val_accuracy: 0.9124\n",
      "Epoch 19/30\n",
      "56/56 [==============================] - 0s 2ms/step - loss: 0.4317 - accuracy: 0.9579 - val_loss: 0.5022 - val_accuracy: 0.9146\n",
      "Epoch 20/30\n",
      "56/56 [==============================] - 0s 2ms/step - loss: 0.4015 - accuracy: 0.9596 - val_loss: 0.4753 - val_accuracy: 0.9191\n",
      "Epoch 21/30\n",
      "56/56 [==============================] - 0s 2ms/step - loss: 0.3746 - accuracy: 0.9590 - val_loss: 0.4526 - val_accuracy: 0.9169\n",
      "Epoch 22/30\n",
      "56/56 [==============================] - 0s 2ms/step - loss: 0.3502 - accuracy: 0.9618 - val_loss: 0.4304 - val_accuracy: 0.9191\n",
      "Epoch 23/30\n",
      "56/56 [==============================] - 0s 2ms/step - loss: 0.3282 - accuracy: 0.9612 - val_loss: 0.4115 - val_accuracy: 0.9191\n",
      "Epoch 24/30\n",
      "56/56 [==============================] - 0s 2ms/step - loss: 0.3087 - accuracy: 0.9629 - val_loss: 0.3938 - val_accuracy: 0.9191\n",
      "Epoch 25/30\n",
      "56/56 [==============================] - 0s 2ms/step - loss: 0.2906 - accuracy: 0.9657 - val_loss: 0.3781 - val_accuracy: 0.9213\n",
      "Epoch 26/30\n",
      "56/56 [==============================] - 0s 2ms/step - loss: 0.2741 - accuracy: 0.9685 - val_loss: 0.3643 - val_accuracy: 0.9213\n",
      "Epoch 27/30\n",
      "56/56 [==============================] - 0s 2ms/step - loss: 0.2593 - accuracy: 0.9697 - val_loss: 0.3515 - val_accuracy: 0.9258\n",
      "Epoch 28/30\n",
      "56/56 [==============================] - 0s 2ms/step - loss: 0.2456 - accuracy: 0.9685 - val_loss: 0.3394 - val_accuracy: 0.9258\n",
      "Epoch 29/30\n",
      "56/56 [==============================] - 0s 2ms/step - loss: 0.2329 - accuracy: 0.9702 - val_loss: 0.3289 - val_accuracy: 0.9258\n",
      "Epoch 30/30\n",
      "56/56 [==============================] - 0s 2ms/step - loss: 0.2212 - accuracy: 0.9697 - val_loss: 0.3191 - val_accuracy: 0.9258\n"
     ]
    }
   ],
   "source": [
    "# NOTE: Please do not edit this cell\n",
    "\n",
    "model = create_model(NUM_WORDS, EMBEDDING_DIM, MAXLEN)\n",
    "\n",
    "history = model.fit(train_padded_seq, train_label_seq, epochs=30, validation_data=(val_padded_seq, val_label_seq))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bde99681",
   "metadata": {},
   "source": [
    "Once training has finished you can run the following cell to check the training and validation accuracy achieved at the end of each epoch.\n",
    "\n",
    "**Remember that to pass this assignment your model should achieve a training accuracy of at least 95% and a validation accuracy of at least 90%. If your model didn't achieve these thresholds, try training again with a different model architecture.**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "357f1ba4",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYMAAAEJCAYAAAB2T0usAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deXxU1fn48c+TyUYCJIGELQGCssoOERGtCxTFDdwQqVp3aysu9WutWqtUrfXXahdba6VWK4oigli0VCuKCxWUBBAwLCIQkrAFyEJC9nl+f8wQhpCEAXIzmZnn/XrNa+beuXPnuRk4zz3n3HuOqCrGGGPCW0SgAzDGGBN4lgyMMcZYMjDGGGPJwBhjDJYMjDHGYMnAGGMMDiYDEXlJRHaLyNpG3hcReVZENonIahEZ4VQsxhhjmuZkzeCfwIQm3r8A6ON93AY872AsxhhjmhDp1I5V9TMRSW9ik0nATPXc9bZMRBJFpKuq7mhqv8nJyZqe3tRujTHG1JeVlbVHVVMae9+xZOCHVCDXZznPu67JZJCenk5mZqaTcRljTMgRkZym3g+KDmQRuU1EMkUks6CgINDhGGNMyAlkMsgHuvssp3nXHUFVZ6hqhqpmpKQ0WssxxhhznAKZDBYAP/ReVTQaKD5af4ExxhhnONZnICJvAOcAySKSBzwKRAGo6t+AhcCFwCbgAHCjU7EYY4xpmpNXE009yvsK3OHU9xtjjPFfUHQgG2OMcZYlA2OMMQG9z8AYYwKu1q1U1tRSWe2mssbteV3j9i7XHlpX7aZWlZhIFzGREZ5H1KHX0ZERnveiIoh2RVBZ46a8qpayqhoOVHqfq2ooq6w97LmqVomMEFwRcvizK+KI9UPTEklPjnfk72DJwBgTMmrdSnF5NfvKqig8UOV5Lqui8ED1Ycv7DlRRdMCzXXF5daDD9tuvLxtkycAYEzxUlbKqWgBccujsNiJCmvxcTa2bA9W1h86kvWfPB7xn2GWVNZ5C/IC3UC+rpuhAVd1yUXk1jU3rHhMZQcf4aJLio+kQH01aUhwd4qJIjIsmPsZ16Iw/KoJo16HXvusjRKjyqSk0XpNwExMZQVx0JPExLs9ztIu4mHrP0ZFEuQS3Qo3bTa1bqXErtbXeZ7cetj45Pqa5f6o6lgyMCXGqSmWN21OgVh4qWOs3XYjgbY44snki0nVofXWtu+6s+uDZdv3lwgNVVNceWSof+g4hMiKi7jvcqhyo8hSk/oh2RdAhPprEuCg6xEczoGt7OsR5C/q4KJLio0mK8xT6nnXRtIl2Nfefttm4BFwRgY3PkoExIWB/RTWbC8rYvKeU73Yfet5eXM6Bqlpq3Y2cLp+gCIHEuGiSvIVyjw5xDE1LJMlbULtEqHErblVqapVat9vnjPfQma8gxMW4iI+OJM57xtzQGXWbaBdJcdHERbsQabqWYY6NJQNjAkhV2VNaRV7hAfaVVfn1mVq3kl9UzncFhwr+XSWVde+7IoSeHeI4KSWe00/uSNuYyMMK2viYes/RkbSJdiFQr6D2FNyeQvxQ4e2KoO6su31s1FGbfkxwsGRgjIPcbmVPaSW5heXkF5WTV3iAvMJy7+MA+YXlfjeN1NcuNpKTU9pyZu8UTu4Uz8kpbTk5JZ4eHeKJjrSrxs2xsWRgjFetWyk64GnvLjzYBl52qHOy8EB13fL+ippDnXuHdfYd2elXv0PT03nZhv5d2vH9AZ1JTWxDWlIbOraNwZ+TbEHokhBLcttoayoxzcaSgQlLNbVuvt1dypq8YtbkF7M6v5h1O0qoauQsvU2U67AOy64JsUT6drS6juwUPfic0i6GtKQ40pLakJrUhrho+29nWh/7V2lCXk2tm+8KylidV8Rab8Gfvb2krnmmXUwkA1Pbc/3pPUlLiqsr8OuuRmnlV6IY0xwsGZiQoarsKqlkw679fLtrPxt27mfjrv1s3FVKebXnmvf4aBcDUxO4bnRPBqclMDg1gfSO8dYJasKeJQMTlPaVVbF+Zwnf7iplw679bPQW/CUVNXXbJLeNoV+Xtkwd1YPBae0ZnJrISclW8BvTEEsGplVzu5Wte8vI3lHCuh0lZG8vIXtHyWGXUia0iaJf53ZMHNaNvp3b1T06xEcHMHJjgoslA9Nq1NS6WZNfTLa30F+3o4T1O/dzwDusQWSE0LtTW844OZkBXdvTv6un0O/ULsauqjHNSxXKC6Fo26FHcS4c2BvYuIZdAyef68iuLRmYgCssq+KN5dt4dWkOO4orAM819Kd0bc9VGd05pVt7Tunanj6d2xITaR25ph5VqCqFiuLDH9UH/Pu82w2lOw8v+ItyoWr/4dtFt4P4ZM+YGoHSd4Jju7ZkYAJm4679vPy/LcxfmU9FtZsxJ3fkoQsHMKx7ImlJbexs3ymV+w8v9IpyoDgPXFEQm9D4I8b77IqCypIjC9+Keusqiz0FbXNS95EFf2WJZ/2JikmAxB6Q1At6ne15ndjd+9wDYhMDmwgcZsnAtCi3W1m8YTcv/28rSzbtISYygsuGp3LDGen079I+0OEFJ1WoqWigcC6Gsj3eQj/H08xRtM3T/OErMhYS0sBde+hzWnv88US39SaP9hDhQBET0xbad4NOAxpIWO0PvY6K87PwFmjbCdokNn+sQcSSgWkRpZU1vJWZyytfbGXr3gN0aR/Lz87vx9RRPayjV9XTpNHgWXZRw4W871lxRTHUNjGuUVTcobPbtFMh4eDZbk/Pc/2mD1WoKjt8/76Pmsomag/twWXFSjBy9FcTkQnAnwAX8KKqPlXv/Z7AS0AKsA+4VlXznIzJtKyyyhqe/ehbZn25jdLKGob3SOTe8/pxwaAuRLkCMH7OYe3L9Zs1SiAypoEzzUSIbe9pHml0n2VNF9oVRY00rXgf7pqG931QZBtvTO09McV1gA69GjkzTjy0HNfRs+2xNG+IeM6+Y9oCqf5/zgQ1x5KBiLiA54DxQB6wXEQWqGq2z2ZPAzNV9RURGQv8BrjOqZhMy/riuz38fN5q8grLuWRIN246sxfDurdQVbymErZ8Dhv+DflZh59xH28TSFT8oUL2sHZzP/YZFXd4oR2fAh17H96sEetbkCcevj7SuUlNjAFnawajgE2quhlARGYDkwDfZHAKcK/39WLgHQfjMS2krLKGp/6znleX5dArOZ45PzqdU9M7OP/F5UWwaRGsfw++XeS5GiQqHnqcBsn9GmjW8C2IEyGmnSeJ1G+CqX+GX1EMNVWQ0r+R5pL2Pvv0vo4M86Yw0+o5mQxSgVyf5TzgtHrbfA1cjqcp6TKgnYh0VNUAX8xrjpdvbeDmM3tx33n9nB3XpzgPNvwH1v8btn7uaW6J7wSDLof+F3muComKPcaddnckVGNas0D39NwH/EVEbgA+A/KBI+rbInIbcBtAjx49WjI+4yff2kB6x7hjqw1UHYDd2bBztedSR3/UVsHWJbBjlWe5Yx84fZonAaRmQISN52/MsXAyGeRz+ClWmnddHVXdjqdmgIi0Ba5Q1aL6O1LVGcAMgIyMDGfm7zPHzbc2cNMZvfjZ+Y3UBlShdBfsXHP4Y993h64TFxeIHwW5CHQdBt+fDv0ugpS+zXlIxoQdJ5PBcqCPiPTCkwSuBn7gu4GIJAP7VNUNPIjnyiITJPyqDVSWwrK/Qs4XnoL/wJ5D7yX2gC5DYNAV0GWw55HYI6Rv7DGmtXIsGahqjYhMAz7Ac2npS6r6jYg8BmSq6gLgHOA3IqJ4monucCoe07yycgq5582VTdcG1r0H/7kfSvI9hX7fCYcK/c4Dw/4mH2NaE9H6c/K1chkZGZqZmRnoMMLaW5m5/GL+WrokxPLMVUOPrA0U5XqSwIaF0GkgXPJH6D4qMMEaYwAQkSxVzWjs/UB3IJsgUlPr5jf/Wc8/lmzhzN7J/OUHw0mM87lksrYGvnweFv8GUBj/GIz+SeM3axljWg1LBsYvxeXV3PnGSj7bWMANY9J5+KIBRPreQZyXCe/eA7vWeJqDLvydp/3fGBMULBmYo/quoJRbX8kkt/AAT10+mKtH+RTy5UXw0WOQ+RK06wpXvQoDLrFOYGOCjCUD06RPNxYw7fUVRLsimHXLaEb18vYPqMLaefD+g54rhE67Hcb+wnMXrzEm6FgyMA1SVf6xZAtPLlxH387tePH6DNKS4jxvbl8J//2l547fbsPhmreg27DABmyMOSGWDMwRKmtqeXj+Wt7KymPCwC48c9VQ4mMiPUM/fPQ4rJ7tGQ3zwqch4yaIsNnHjAl2lgzMYQr2V3L7a1lk5RRy97g+3D2uDxHVpfDRH2HpXzzNQ2fcA9+71zMAmzEmJFgyMHUOVNVwxfNfsHt/Bc/9YAQXDUyBFf+ExU9C2W4YdCWMewSSegY6VGNMM7NkYOrM+Gwz2/Yd4I1bR3O6eyX87WEoWAfdR8PUNyCt0ftVjDFBzpKBAWB3SQUvfLqZW/pWcPoXt8B3H3smBr9qJgyYaJeKGhPiLBkYAJ7570Z6uXN4aPsjngnSz38STr3VJmUxJkxYMjCs21HCgqxNfJrwNyJc7eFHn0H7roEOyxjTgiwZGH7zn/U8HjOLThWb4dq3LREYE4ZsOqgw9+nGAtpuepcrWeS5ZLT3uECHZIwJAKsZhLFat/LSu5/wl+gXcaeOJGLsw4EOyRgTIFYzCGPzlm/m7uKniI2MIOLKl2yoaWPCmNUMwlRZZQ0HPnicERGb0EtfhqT0QIdkjAkgqxmEqQ8WvMEPa9+hoO9UZNDlgQ7HGBNglgzCUMGObXxv7S/YFdODlCt/H+hwjDGtgCWDcON2s2/WzbTjAFzxMkTHBToiY0wr4GgyEJEJIrJBRDaJyAMNvN9DRBaLyEoRWS0iFzoZj4FdHzxNv9Kv+LjnPXTtNzLQ4RhjWgnHkoGIuIDngAuAU4CpInJKvc0eBuao6nDgauCvTsVjgLwsOn75FB9yGmOm/CzQ0RhjWhEnawajgE2qullVq4DZwKR62yjQ3vs6AdjuYDzhraKY8jeuZ6cmseOs35IYHxPoiIwxrYiTySAVyPVZzvOu8zUduFZE8oCFwJ0OxhO+VHG/ew/RZfk82eY+rj5rSKAjMsa0MoHuQJ4K/FNV04ALgVdF5IiYROQ2EckUkcyCgoIWDzLorZ5DxDdv80z1lVx80aVERwb6ZzfGtDZOlgr5QHef5TTvOl83A3MAVHUpEAsk19+Rqs5Q1QxVzUhJSXEo3BBVXY570XS+kd581e06LhjUJdARGWNaISeTwXKgj4j0EpFoPB3EC+ptsw0YByAiA/AkAzv1b05fzSBi/3Yer7yaBy8ejNgkNcaYBjiWDFS1BpgGfACsw3PV0Dci8piITPRu9n/ArSLyNfAGcIOqqlMxhZ3yQvTzZ1jmGgHpZzKyZ1KgIzLGtFKOjk2kqgvxdAz7rnvE53U2cIaTMYS1JX+EihJ+VTmZ207tfvTtjTFhy3oSQ1VxPnz5NzLbf5+8mJO5YJBNWGOMaZwlg1D16VOouvn5vku4dFgqsVGuQEdkjGnFLBmEooINsPI11qVOZnNNMlOsicgYcxSWDELRR49BVDyPl1zIwG7tGZSaEOiIjDGtnCWDUJP7Fax/j12Db2PpTrFagTHGL5YMQokqfPgoxHdiRtUEoiMjmDS0/gggxhhzJEsGoeTb/8K2L6g+82fMWVPIhYO6kBBn8xobY47OkkGocNfCol9Bh5P4d/R49lfUcJU1ERlj/OToTWemBa2eA7u/gStfYvYXO+nZMY7RvToGOipjTJCwmkEoqK6Axb+GrsPY2vk8lm3ex1UZ3YmIsHGIjDH+sZpBKMj8BxTnwsQ/MycrnwiBK0akBToqY0wQsZpBsKsohs+ehpPOpSb9bOZm5XFuv050SYgNdGTGmCBiySDY/e9ZKN8H35/OJxsK2L2/0jqOjTHHzJJBMNu/E5b9FQZdAd2G8WZmLsltYxjbv1OgIzPGBBlLBsHs0/8HtVUw9mF276/g4/W7uWJkKlEu+1mNMcfGSo1gdWAfrJgJI66HDicxLyufWrdyVYY1ERljjp0lg2CV/Q64ayDjRlSVOZm5jErvwMkpbQMdmTEmCFkyCFZr5kJyP+g8iK+27GPLnjLrODbGHDdLBsGoOA9y/geDJ4MIb2bm0i4mkgsHdwl0ZMaYIGXJIBitfdvzPPgKSiqqWbhmB5cM60ZctN1DaIw5Po4mAxGZICIbRGSTiDzQwPt/EJFV3sdGESlyMp6QseYtSB0JHU5iwartVFS7mWIdx8aYE+DYqaSIuIDngPFAHrBcRBaoavbBbVT1pz7b3wkMdyqekFGwEXauhglPATAnM5f+XdoxJM1mMzPGHD8nawajgE2qullVq4DZwKQmtp8KvOFgPKFh7VyQCBh4GdnbS1idV8yUU7sjYoPSGWOOn5PJIBXI9VnO8647goj0BHoBHzsYT/BT9TQRpX8P2nVhTmYu0ZERXDbcZjMzxpwYv5KBiLwtIheJiFPJ42pgrqrWNvL9t4lIpohkFhQUOBRCENi+AvZthsGTqaiuZf7KfM4f2IXEuOhAR2aMCXL+Fu5/BX4AfCsiT4lIPz8+kw/49mqmedc15GqaaCJS1RmqmqGqGSkpKX6GHILWzANXNAy4hH+tyqe4vJprTusR6KiMMSHAr2SgqotU9RpgBLAVWCQiX4jIjSLS2CS7y4E+ItJLRKLxFPgL6m8kIv2BJGDp8RxA2HDXwtp50Oc8NDaBV77IoX+XdpzWq0OgIzPGhAC/m31EpCNwA3ALsBL4E57k8GFD26tqDTAN+ABYB8xR1W9E5DERmeiz6dXAbFXV4zqCcLF1CZTuhMFXkplTSPaOEn54erp1HBtjmoVfl5aKyHygH/AqcImq7vC+9aaIZDb2OVVdCCyst+6ResvTjyXgsLXmLYhuC30n8Mpb62gfG8mlw7sFOipjTIjw9z6DZ1V1cUNvqGpGM8ZjGlJTCesWQP+L2VUuvL92JzeMSbc7jo0xzcbfZqJTRCTx4IKIJInITxyKydS3aZFnesvBk5m1LIdaVa47vWegozLGhBB/k8Gtqlo3VISqFgK3OhOSOcKatyAumcoeZ/L6V9s4t18nenaMD3RUxpgQ4m8ycIlPT6V3qAm7uL0lVO6HDe/DwEv5T/Ze9pRWcf2Y9EBHZYwJMf42Or+Pp7P4Be/yj7zrjNPWL4Sachg8mVfe28pJyfF8r3dyoKMyxoQYf2sGPwcWAz/2Pj4C7ncqKONjzVuQ0IPV0peV24q47vSeRETY5aTGmOblV81AVd3A896HaSlle+C7j2HMnbyyNJf4aBdXjkwLdFTGmBDk79hEfURkrohki8jmgw+ngwt72e+A1lLUexLvrt7O5SPSaBfb2A3fxhhz/PxtJnoZT62gBjgXmAm85lRQxmvNXEgZwKwt7aiqcXP9GLuc1BjjDH+TQRtV/QgQVc3x3jV8kXNhGYpyYdtSagdezqwvt3FG74707tQu0FEZY0KUv8mg0jt89bciMk1ELgPaOhiXWTsPgCWx57C9uILrT08PbDzGmJDmbzK4G4gD7gJGAtcC1zsVlMHTRJR2Ks+vriU1sQ3jBnQOdETGmBB21GTgvcFsiqqWqmqeqt6oqleo6rIWiC887V4Pu9awq+fFLNu8j+tO74nLLic1xjjoqMnAO/vYmS0QiznIO8/xPwqHERMZwZSM7kf/jDHGnAB/70BeKSILgLeAsoMrVfVtR6IKZ955jqt7nsWrayqYNKwbSfE28ocxxln+JoNYYC8w1medApYMmlv+CijcyhddbqC8upYfWsexMaYF+HsH8o1OB2K8Vs1CI2P5bU5vMnomMSg1IdARGWPCgL8znb2MpyZwGFW9qdkjCmcVJbD6TXamXcA36yP48/npgY7IGBMm/G0mes/ndSxwGbC9+cMJc6vfhKpSZpSPpVO7GCYM6hLoiIwxYcLfZqJ5vssi8gawxJGIwpUqZL5EZcoQXs7pwE+/35Mol7+3gRhjzIk53tKmD9DpaBuJyAQR2SAim0TkgUa2uco7AN43IvL6ccYT/LYthd3ZfBh/MZEREUw9zS4nNca0HH/7DPZzeJ/BTjxzHDT1GRfwHDAeyAOWi8gCVc322aYP8CBwhqoWishRE0zIWv4iGpvAb3IHMbZ/Jzq1iw10RMaYMOJvM9HxjJA2CtikqpsBRGQ2MAnI9tnmVuA575zKqOru4/ie4Ld/F2QvILf3NeSvhkdtzgJjTAvzdz6Dy0QkwWc5UUQuPcrHUoFcn+U87zpffYG+IvI/EVkmIhP8iSfkrJwJ7mpeqjyHjvHRnNs/fCtIxpjA8LfP4FFVLT64oKpFwKPN8P2RePofzgGmAn8XkcT6G4nIbSKSKSKZBQUFzfC1rYi7FjL/SXXPs3l9UyyThqVax7ExpsX5W+o0tN3RmpjyAd9e0DTvOl95wAJVrVbVLcBGPMnhMKo6Q1UzVDUjJSXFz5CDxMYPoCSPJUmTqKp127SWxpiA8DcZZIrI70XkZO/j90DWUT6zHOgjIr1EJBq4GlhQb5t38NQKEJFkPM1G4TWd5vIXoV03/pTbm1O6tueUbu0DHZExJgz5mwzuBKqAN4HZQAVwR1MfUNUaYBrwAbAOmKOq34jIYyIy0bvZB8BeEckGFgM/U9W9x34YQWrvd/DdR+zpN5VV+aVWKzDGBIy/VxOVAQ3eJ3CUzy0EFtZb94jPawXu9T7CT9bLEBHJ6zXnEBlRyqRh3QIdkTEmTPl7NdGHvh27IpIkIh84F1YYqC6Hla/h7ncxM9dWMbZ/Jzq2jQl0VMaYMOVvM1Gy9woiALz3Bdj1jyfim/lQXsiqLlewp7TSmoiMMQHlbzJwi0iPgwsikk4Do5iaY7D8RUjux4u53ezeAmNMwPk7aukvgCUi8ikgwPeA2xyLKtRtXwn5WRwY9ySL3i/g2tE2KJ0xJrD8KoFU9X0gA9gAvAH8H1DuYFyhbfk/ICqOf+lZdm+BMaZV8HeguluAu/HcOLYKGA0s5fBpMI0/ygthzVwYOoU3VhfbvQXGmFbB37aJu4FTgRxVPRcYDhQ1/RHToFVvQE05OSdNZXVesdUKjDGtgr/JoEJVKwBEJEZV1wP9nAsrRLndno7jtFG8npNAZITYvQXGmFbB3w7kPO99Bu8AH4pIIZDjXFghasunsO87ar/3M95emG/3FhhjWg1/70C+zPtyuogsBhKA9x2LKlRl/gPiOrIk+kwK9q+2JiJjTKvhb82gjqp+6kQgIa84H9YvhDF3MmdVgd1bYIxpVezi9pay4hVQNyUDr+XD7F1MHNbN7i0wxrQaVhq1hMr9kPky9BnPv3Ki7N4CY0yrY8mgJSz6FZQVwFn3MzcrjwFd2zOwW8LRP2eMMS3EkoHTti3zXE562o/YGN2fr+3eAmNMK2TJwEnVFbDgTkjoDmN/ybysPLu3wBjTKh3z1UTmGHz+NOzZCNfOoyYyjrdX5nNu/04k270FxphWxmoGTtm5Fpb8AYZcDb2/z+ff7qFgv81bYIxpnSwZOMFd62keik2ECb8BYG5WHh3iozm3n91bYIxpfSwZOGHZ87B9BVzw/yCuA9nbS/jgm51cOiyV6Ej7kxtjWh9HSyYRmSAiG0Rkk4g80MD7N4hIgYis8j5ucTKeFrFvC3z8BPSdAIOuoKrGzb1zVpEUH82dY3sHOjpjjGmQYx3IIuICngPGA3nAchFZoKrZ9TZ9U1WnORVHi1KF9+6BiEi46Pcgwp8/3sj6nft58YcZJMVHBzpCY4xpkJM1g1HAJlXdrKpVwGxgkoPfF3irZsHmT2D8dEhIZVVuEX/95DuuHJnG90/pHOjojDGmUU4mg1Qg12c5z7uuvitEZLWIzBWR7g7G46z9O+GDh6DHGBh5ExXVtfzfnFV0bhfDI5ecEujojDGmSYHuzXwXSFfVIcCHwCsNbSQit4lIpohkFhQUtGiAflv4M89NZhOfhYgInvnvBr4rKOO3Vw6lfWxUoKMzxpgmOZkM8gHfM/0077o6qrpXVSu9iy8CIxvakarOUNUMVc1ISUlxJNgTsu5dWLcAzr4fkvvw1ZZ9vLhkC9eO7sGZfZIDHZ0xxhyVk8lgOdBHRHqJSDRwNbDAdwMR6eqzOBFY52A8zigvgn/fB50Hwxl3U1ZZw31vfU33pDgevGBAoKMzxhi/OHY1karWiMg04APABbykqt+IyGNApqouAO4SkYlADbAPuMGpeBzz4S+hbDdMfQNcUfzm3TXkFh7gzdtOJz7GRvswxgQHR0srVV0ILKy37hGf1w8CDzoZg6PWzIUVM2HMnZA6gs+/LeC1Zdu45cxejOrVIdDRGWOM3+zU9XiUF8H7D8DXb0C34XDOQ5RUVHP/3NWcnBLPfef3C3SExhhzTCwZHKvvPoZ/TfNcSnrW/XDWzyAymsfe+prd+yuZ9+MxxEa5Ah2lMcYcE0sG/qoqgw8f8UxUk9wXbv4Q0jwXPy3K3sXcrDymndubYd0TAxyoMcYcO0sG/tj2JbxzO+zbDKN/AuMegag2ABSWVfHA22vo36Udd43rE+BAjTHm+FgyaEpNJSz+NXzxZ0hIg+vfg17fO2yTX/5rLcXlVcy8aZSNSGqMCVqWDBqz42uYfzvszoYRP4Tzn4SYdgC43UrWtkLeyszlvdU7uO+8vpzSrX2AAzbGmONnycBXTRVsXwkbFsLSv0BcR/jBHOh7PgDf7trPO6vyeWfldvKLymkT5WLqqB7cfvbJAQ7cGGNOTHgng5oqyM+CnCWwdQnkfgXVBzzvDboSLvwdO6vjePezzcxfmU/2jhIiBL7XJ4X7zu/Lead0sRvLjDEhIbxKsppKT+G/1afwryn3vNd5EAy/DtLPpLTLKBZuruadWRtZunkvqjA0LYFHLzmFi4d0I6WdTWhvjAkt4ZMMvnzBc2loTQUgnsJ/5A2Qfib0HANxnjuG3W7lkt9/ypY9ZfTsGMddY/swaVg3TkppG9DwjTHGSeGTDDoNgIybPIV/j9PrCv/6NhWUsmVPGQ9fNICbz+yFiLRwoMYY0/LCJxn0OsvzOIqsnEIAvj+gsyUCY0zYsAvj68nKKaRjfDQ9O8YFOhRjjGkxlgzqycopZETPJKsVGGPCiiUDH952Du8AABGPSURBVHtLK9myp4yRPZMCHYoxxrQoSwY+VmwrArBkYIwJO5YMfGTlFBLlEganJgQ6FGOMaVGWDHysyClkYLcEm4/AGBN2LBl4VdW4+TqvyJqIjDFhyZKBV/aOEipr3JYMjDFhydFkICITRGSDiGwSkQea2O4KEVERyXAynqYcvNnMkoExJhw5lgxExAU8B1wAnAJMFZFTGtiuHXA38KVTsfgjK2cfaUlt6Nw+NpBhGGNMQDhZMxgFbFLVzapaBcwGJjWw3ePA/wMqHIylSapKVk6h1QqMMWHLyWSQCuT6LOd519URkRFAd1X9t4NxHFV+UTm7SiotGRhjwlbAOpBFJAL4PfB/fmx7m4hkikhmQUFBs8dysL9gRA9LBsaY8ORkMsgHuvssp3nXHdQOGAR8IiJbgdHAgoY6kVV1hqpmqGpGSkpKswe6IqeQuGgX/bu0a/Z9G2NMMHAyGSwH+ohILxGJBq4GFhx8U1WLVTVZVdNVNR1YBkxU1UwHY2pQ1rZChnVPJNJlV9oaY8KTY/MZqGqNiEwDPgBcwEuq+o2IPAZkquqCpvfQMsoqa1i3Yz8/OccmtTfmeFRXV5OXl0dFRcCuATE+YmNjSUtLIyoq6pg+5+jkNqq6EFhYb90jjWx7jpOxNObrvCJq3coI6zw25rjk5eXRrl070tPTbej3AFNV9u7dS15eHr169Tqmz4Z9u8iKg53H3S0ZGHM8Kioq6NixoyWCVkBE6Nix43HV0sI+GWTmFNK3c1sS4o6tSmWMOcQSQetxvL9FWCcDt1tZYTebGWNMeCeD7wpKKamosfsLjDF+qampCXQIjgnrZGCD0xkTOi699FJGjhzJwIEDmTFjBgDvv/8+I0aMYOjQoYwbNw6A0tJSbrzxRgYPHsyQIUOYN28eAG3btq3b19y5c7nhhhsAuOGGG7j99ts57bTTuP/++/nqq684/fTTGT58OGPGjGHDhg0A1NbWct999zFo0CCGDBnCn//8Zz7++GMuvfTSuv1++OGHXHbZZS3x5zhmjl5N1Npl5RSSFBdFr+T4QIdiTEj41bvfkL29pFn3eUq39jx6ycCjbvfSSy/RoUMHysvLOfXUU5k0aRK33norn332Gb169WLfvn0APP744yQkJLBmzRoACgsLj7rvvLw8vvjiC1wuFyUlJXz++edERkayaNEiHnroIebNm8eMGTPYunUrq1atIjIykn379pGUlMRPfvITCgoKSElJ4eWXX+amm246sT+IQ8I7GWzz9BdY55cxwe/ZZ59l/vz5AOTm5jJjxgzOOuusukssO3ToAMCiRYuYPXt23eeSko7eMjB58mRcLs8MiMXFxVx//fV8++23iAjV1dV1+7399tuJjIw87Puuu+46XnvtNW688UaWLl3KzJkzm+mIm1fYJoN9ZVVsLijjypFpgQ7FmJDhzxm8Ez755BMWLVrE0qVLiYuL45xzzmHYsGGsX7/e7334nhTWvzQzPv5Q68Evf/lLzj33XObPn8/WrVs555xzmtzvjTfeyCWXXEJsbCyTJ0+uSxatTdj2Gazc5u0vsM5jY4JecXExSUlJxMXFsX79epYtW0ZFRQWfffYZW7ZsAahrJho/fjzPPfdc3WcPNhN17tyZdevW4Xa762oYjX1XaqpnAOZ//vOfdevHjx/PCy+8UNfJfPD7unXrRrdu3XjiiSe48cYbm++gm1nYJoPMnEIiI4QhaYmBDsUYc4ImTJhATU0NAwYM4IEHHmD06NGkpKQwY8YMLr/8coYOHcqUKVMAePjhhyksLGTQoEEMHTqUxYsXA/DUU09x8cUXM2bMGLp27drod91///08+OCDDB8+/LCri2655RZ69OjBkCFDGDp0KK+//nrde9dccw3du3dnwIABDv0FTpyoaqBjOCYZGRmamXniY9ld9cJSKqtr+de0M5shKmPC17p161p1IdcaTJs2jeHDh3PzzTe3yPc19JuISJaqNjq1cFjWDKpr3XydW8TInh0CHYoxJsSNHDmS1atXc+211wY6lCa1zp4Mh2VvL6Gyxm33FxhjHJeVlRXoEPwSljWDupnNelp/gTHGQLgmg22FpCa2oWtCm0CHYowxrUJYJoMVOYU2f4ExxvgIu2SwvaicHcUVjOxhTUTGGHNQ2CWDzLrB6exKImOMOSjsksGKnELaRLno37VdoEMxxgSA7+ik5pCwSwZZOYUM7Z5AlCvsDt0Y04q0trkRHL3PQEQmAH8CXMCLqvpUvfdvB+4AaoFS4DZVzXYqngNVNWTvKOHHZ5/s1FcYE97+8wDsXNO8++wyGC54qtG3H3jgAbp3784dd9wBwPTp04mMjGTx4sUUFhZSXV3NE088waRJk476VaWlpUyaNKnBz82cOZOnn34aEWHIkCG8+uqr7Nq1i9tvv53NmzcD8Pzzz9OtWzcuvvhi1q5dC8DTTz9NaWkp06dPrxtAb8mSJUydOpW+ffvyxBNPUFVVRceOHZk1axadO3emtLSUO++8k8zMTESERx99lOLiYlavXs0f//hHAP7+97+TnZ3NH/7whxP68x7kWDIQERfwHDAeyAOWi8iCeoX966r6N+/2E4HfAxOciunr3GJq3Wo3mxkTQqZMmcI999xTlwzmzJnDBx98wF133UX79u3Zs2cPo0ePZuLEiUcdrj42Npb58+cf8bns7GyeeOIJvvjiC5KTk+sGobvrrrs4++yzmT9/PrW1tZSWlh51foSqqioODqlTWFjIsmXLEBFefPFFfvvb3/LMM880OOdCVFQUv/71r/nd735HVFQUL7/8Mi+88MKJ/vnqOFkzGAVsUtXNACIyG5gE1CUDVfWdBSMecHSgpBXekUqH25VExjijiTN4pwwfPpzdu3ezfft2CgoKSEpKokuXLvz0pz/ls88+IyIigvz8fHbt2kWXLl2a3Jeq8tBDDx3xuY8//pjJkyeTnJwMHJqr4OOPP66bn8DlcpGQkHDUZHBwwDzwTJozZcoUduzYQVVVVd3cC43NuTB27Fjee+89BgwYQHV1NYMHDz7Gv1bjnEwGqUCuz3IecFr9jUTkDuBeIBoY29COROQ24DaAHj16HHdAWTmF9O7UlsS46OPehzGm9Zk8eTJz585l586dTJkyhVmzZlFQUEBWVhZRUVGkp6cfMUdBQ473c74iIyNxu911y03NjXDnnXdy7733MnHiRD755BOmT5/e5L5vueUWnnzySfr379/sw2EHvBdVVZ9T1ZOBnwMPN7LNDFXNUNWMlJSU4/oet1tZsa3Q5i8wJgRNmTKF2bNnM3fuXCZPnkxxcTGdOnUiKiqKxYsXk5OT49d+Gvvc2LFjeeutt9i7dy9waK6CcePG8fzzzwOeOZCLi4vp3Lkzu3fvZu/evVRWVvLee+81+X0H50Z45ZVX6tY3NufCaaedRm5uLq+//jpTp07198/jFyeTQT7Q3Wc5zbuuMbOBS5t4/4Rs3lNG0YFq6y8wJgQNHDiQ/fv3k5qaSteuXbnmmmvIzMxk8ODBzJw5k/79+/u1n8Y+N3DgQH7xi19w9tlnM3ToUO69914A/vSnP7F48WIGDx7MyJEjyc7OJioqikceeYRRo0Yxfvz4Jr97+vTpTJ48mZEjR9Y1QUHjcy4AXHXVVZxxxhl+Tdd5LBybz0BEIoGNwDg8SWA58ANV/cZnmz6q+q339SXAo02Ntw3HP5/Bm8u38fN5a1h079n07mTXGRvTXGw+g5Z18cUX89Of/pRx48Y1uk2rms9AVWuAacAHwDpgjqp+IyKPea8cApgmIt+IyCo8/QbXOxVPUlw040/pzEnJ8Uff2BhjWpmioiL69u1LmzZtmkwEx8vR+wxUdSGwsN66R3xe3+3k9/s6b2AXzhvY9JUExpjwsGbNGq677rrD1sXExPDll18GKKKjS0xMZOPGjY7tPywntzHGhLfBgwezatWqQIfRqgT8aiJjTPALtrnUQ9nx/haWDIwxJyQ2Npa9e/daQmgFVJW9e/cSGxt7zJ+1ZiJjzAlJS0sjLy+PgoKCQIdi8CTntLS0Y/6cJQNjzAmJioqqG0bBBC9rJjLGGGPJwBhjjCUDY4wxODgchVNEpADwb9SpIyUDe5oxnNYg1I4p1I4HQu+YQu14IPSOqaHj6amqjY70GXTJ4ESISObRxj4KNqF2TKF2PBB6xxRqxwOhd0zHczzWTGSMMcaSgTHGmPBLBjMCHYADQu2YQu14IPSOKdSOB0LvmI75eMKqz8AYY0zDwq1mYIwxpgFhkwxEZIKIbBCRTSLyQKDjOVEislVE1ojIKhE59qnfWgEReUlEdovIWp91HUTkQxH51vscNPOUNnI800Uk3/s7rRKRCwMZ47ESke4islhEsr0TUd3tXR+Uv1MTxxO0v5OIxIrIVyLytfeYfuVd30tEvvSWeW+KSHST+wmHZiIRceGZgnM8kIdnCs6pqpod0MBOgIhsBTJUNWivjRaRs4BSYKaqDvKu+y2wT1Wf8ibtJFX9eSDj9FcjxzMdKFXVpwMZ2/ESka5AV1VdISLtgCw8c5XfQBD+Tk0cz1UE6e8kIgLEq2qpiEQBS4C78cwe+baqzhaRvwFfq+rzje0nXGoGo4BNqrpZVauA2cCkAMcU9lT1M2BfvdWTgFe8r1/B8x81KDRyPEFNVXeo6grv6/14prBNJUh/pyaOJ2ipR6l3Mcr7UGAsMNe7/qi/Ubgkg1Qg12c5jyD/B4Dnx/6viGSJyG2BDqYZdVbVHd7XO4HOgQymmUwTkdXeZqSgaE5piIikA8OBLwmB36ne8UAQ/04i4vLOJb8b+BD4DijyzkUPfpR54ZIMQtGZqjoCuAC4w9tEEVLU04YZ7O2YzwMnA8OAHcAzgQ3n+IhIW2AecI+qlvi+F4y/UwPHE9S/k6rWquowIA1PS0j/Y91HuCSDfKC7z3Kad13QUtV87/NuYD6efwChYJe3Xfdg++7uAMdzQlR1l/c/qhv4O0H4O3nboecBs1T1be/qoP2dGjqeUPidAFS1CFgMnA4kisjBOWuOWuaFSzJYDvTx9q5HA1cDCwIc03ETkXhv5xciEg+cB6xt+lNBYwFwvff19cC/AhjLCTtYYHpdRpD9Tt7OyX8A61T19z5vBeXv1NjxBPPvJCIpIpLofd0Gz4Uy6/AkhSu9mx31NwqLq4kAvJeK/RFwAS+p6q8DHNJxE5GT8NQGwDNb3evBeDwi8gZwDp4RFncBjwLvAHOAHnhGp71KVYOiU7aR4zkHT9ODAluBH/m0tbd6InIm8DmwBnB7Vz+Ep5096H6nJo5nKkH6O4nIEDwdxC48J/hzVPUxbzkxG+gArASuVdXKRvcTLsnAGGNM48KlmcgYY0wTLBkYY4yxZGCMMcaSgTHGGCwZGGOMwZKBMXVEpNZn1MpVzTm6rYik+45makxrE3n0TYwJG+XeW/qNCTtWMzDmKLxzR/zWO3/EVyLS27s+XUQ+9g5u9pGI9PCu7ywi873jy38tImO8u3KJyN+9Y87/13u3KCJyl3d8/dUiMjtAh2nCnCUDYw5pU6+ZaIrPe8WqOhj4C5472QH+DLyiqkOAWcCz3vXPAp+q6lBgBPCNd30f4DlVHQgUAVd41z8ADPfu53anDs6YptgdyMZ4iUipqrZtYP1WYKyqbvYOcrZTVTuKyB48E6VUe9fvUNVkESkA0nxv/fcOl/yhqvbxLv8ciFLVJ0TkfTyT4rwDvOMzNr0xLcZqBsb4Rxt5fSx8x4Wp5VCf3UXAc3hqEct9Rpo0psVYMjDGP1N8npd6X3+BZwRcgGvwDIAG8BHwY6ibdCShsZ2KSATQXVUXAz8HEoAjaifGOM3OQIw5pI13tqiD3lfVg5eXJonIajxn91O96+4EXhaRnwEFwI3e9XcDM0TkZjw1gB/jmTClIS7gNW/CEOBZ75j0xrQo6zMw5ii8fQYZqron0LEY4xRrJjLGGGM1A2OMMVYzMMYYgyUDY4wxWDIwxhiDJQNjjDFYMjDGGIMlA2OMMcD/BwnJW5n0CXeHAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYMAAAEGCAYAAACHGfl5AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3dd3xUVfrH8c+TDmmUFAgBEkKHUCSAIE1FigVUBESQKojdXRbLWlZd96eru+7qigUVAWVRuiiIsIIinRA6SG8JgYSEUAIh7fz+uINGJCGBTG5m8rxfr3klc+/NzHMdyTfnnHvPEWMMSimlKjYPuwtQSillPw0DpZRSGgZKKaU0DJRSSqFhoJRSCvCyu4CSCgkJMVFRUXaXoZRSLmXDhg0njDGhhe13uTCIiooiPj7e7jKUUsqliMihovZrN5FSSikNA6WUUhoGSimlcMExA6VUxZSTk0NiYiJZWVl2l1Ku+fn5ERkZibe3d4l+zmlhICKTgNuBFGNM80KO6Qb8G/AGThhjujqrHqWUa0tMTCQwMJCoqChExO5yyiVjDGlpaSQmJhIdHV2in3VmN9FkoFdhO0WkCvAe0McY0wzo78RalFIuLisri+rVq2sQFEFEqF69+lW1npwWBsaY5UB6EYfcB8wxxhx2HJ/irFqUUu5Bg+DKrva/kZ0DyA2BqiLyg4hsEJGhznyzlDNZ/N/CnRw7pf2NSil1KTvDwAtoA9wG9AReEJGGlztQRMaISLyIxKempl7Vm63el8YnKw7Q5Y1lPDtnCwdPZF514UqpiikgIMDuEpzGzjBIBL4zxmQaY04Ay4GWlzvQGDPRGBNnjIkLDS30buoi9W3gS0LHNQxtFcjshCRu+ucPPDZ9IzuTT1/9GSillJuwMwy+AjqJiJeIVAbaAzud9m77lhIc/w7P7xnIxo6rebxDdZbuPE7vt39i5OT1xB8sanhDKaV+ZYxh/PjxNG/enNjYWL788ksAkpOT6dKlC61ataJ58+b89NNP5OXlMXz48F+O/de//mVz9ZfnzEtLpwPdgBARSQT+gnUJKcaYD4wxO0VkEbAFyAc+NsZsc1Y9tBgA4c1h+Rv4r3uHJ30+YWzH0XzGbby/PoN7PlhNu+hqPNwthq4NQ3WgSqly7OWvt7PjaOm26ptGBPGXO5oV69g5c+awadMmNm/ezIkTJ2jbti1dunThv//9Lz179uS5554jLy+Pc+fOsWnTJpKSkti2zfr1lpGRUap1lxanhYExZlAxjnkTeNNZNfxOeFPoPxm67IDlb+C35t+M9vmI4e1HM8OrL++uTWf4p+tpFhHEuB4NualxeJmVppRyHStWrGDQoEF4enoSHh5O165dWb9+PW3btmXkyJHk5ORw55130qpVK+rVq8f+/ft57LHHuO222+jRo4fd5V9WxbwD+ZJQ8F71bwb7fMy9bcfwtf9dvL0qjZGT47kttiZ/uaMpYUF+dleslCqguH/Bl7UuXbqwfPlyFixYwPDhw/njH//I0KFD2bx5M9999x0ffPABM2bMYNKkSXaX+jsVe26ii6Hw0Cpo0B3PlW9x5w+9+V/L5Tx/Uw2W7DzOzW/9yOdrDpGfb+yuVilVTnTu3Jkvv/ySvLw8UlNTWb58Oe3atePQoUOEh4czevRoHnjgARISEjhx4gT5+fn069ePV199lYSEBLvLv6yK2TK41CUtBc+Vb/GA/1TuuON1ntwcyfPztjF3YxKv3R1Lw/BAu6tVStnsrrvuYvXq1bRs2RIR4Y033qBGjRpMmTKFN998E29vbwICApg6dSpJSUmMGDGC/Px8AF577TWbq788Mca1/uKNi4szTl/cJnkzfPUIHNuKie3P1xFP8pclRzl7IZexXWN45Mb6+Hl7OrcGpdRv7Ny5kyZNmthdhku43H8rEdlgjIkr7GcqdjdRYWq2hNHLoNufke1z6bPiLn7sc447WkTwn6V76f32T6zel2Z3lUopVWo0DArj6Q3dnrZCISCcoHnDeMtrAtOHNCQv3zDoozWMn7mZk5nZdleqlFLXTMPgSmq2gNFLoeszsH0OHRbdxpJbM3moWwxzNybR/a0fWbXvhN1VKqXUNdEwKA4vH7jxWSsU/EPxnTWYp8+9xYLRzajq78PQT9bx2Zoi15pWSqlyTcOgJC6OJXR5CrbOotHsHszvcYbODUJ4Yd42np+3lZy8fLurVEqpEtMwKCkvH7jpOauVULk6lWcN5pM6i3mwSzSfrznM/Z+sJV3HEZRSLkbD4GpFtIIxy6D1EDx+epNnz/2Df/drTMLhDPpOWMGuY2fsrlAppYpNw+BaePlCn3eh+0uwbTZ3bh7L7PvrcyEnn7vfW8mSHcftrlApZZOi1j44ePAgzZtfdml422gYXCsR6PQHGDAVjm0h9tu7WTAolJiwAMZ8Fs+EZXtxtRv7lFIVj05HUVqa9oWgSJh+L6Ff3s6suz/lTwkRvPndLnYdO8Mb97TQu5aVKi3fPgPHtpbua9aIhd6vF7r7mWeeoXbt2jzyyCMAvPTSS3h5ebFs2TJOnjxJTk4Or776Kn379i3R22ZlZfHQQw8RHx+Pl5cXb731FjfeeCPbt29nxIgRZGdnk5+fz+zZs4mIiGDAgAEkJiaSl5fHCy+8wMCBA6/ptC/SlkFpimxjDSwH18bniwG83WATT/VqxNdbjtL/g9W6/rJSLmzgwIHMmDHjl+czZsxg2LBhzJ07l4SEBJYtW8a4ceNK3BMwYcIERIStW7cyffp0hg0bRlZWFh988AFPPPEEmzZtIj4+nsjISBYtWkRERASbN29m27Zt9OrVq9TOT1sGpa1KbRi5CGaNRL55koc7PErDwY/wxIwt9Ht/FV+MuZ7a1SrbXaVSrq2Iv+CdpXXr1qSkpHD06FFSU1OpWrUqNWrU4A9/+APLly/Hw8ODpKQkjh8/To0aNYr9uitWrOCxxx4DoHHjxtStW5fdu3fToUMH/va3v5GYmMjdd99NgwYNiI2NZdy4cTz99NPcfvvtdO7cudTOT1sGzuAXBIO+gHZjYPW7dN/6J74c0ZIzWTncO3ENR9LP2V2hUuoq9O/fn1mzZvHll18ycOBApk2bRmpqKhs2bGDTpk2Eh4eTlVU6PQD33Xcf8+fPp1KlStx6660sXbqUhg0bkpCQQGxsLM8//zyvvPJKqbwXaBg4j6cX3Pom9H4Ddn9L88UD+XJQFGcv5GogKOWiBg4cyBdffMGsWbPo378/p06dIiwsDG9vb5YtW8ahQyWfiaBz585MmzYNgN27d3P48GEaNWrE/v37qVevHo8//jh9+/Zly5YtHD16lMqVKzNkyBDGjx9fqmsjaBg4W/sHrVZC2j6afNufGQMjNBCUclHNmjXjzJkz1KpVi5o1azJ48GDi4+OJjY1l6tSpNG7cuMSv+fDDD5Ofn09sbCwDBw5k8uTJ+Pr6MmPGDJo3b06rVq3Ytm0bQ4cOZevWrbRr145WrVrx8ssv8/zzz5fauel6BmUlaQN8dhf4BrGr138ZMOMYAb5eTB99PXWq6xiCUlei6xkUX7laz0BEJolIiohsu8JxbUUkV0TucVYt5UKtNjDsa8g+S6OFA5nZP4zM7Fzunbiaw2naQlBK2cuZ3USTgSKvexIRT+DvwGIn1lF+1GwJw76BvAs0XDiQmfdU51xOHvdOXM2htEy7q1NKlbKtW7fSqlWr3zzat29vd1mX5bQwMMYsB9KvcNhjwGwgxVl1lDs1msPwBWDyabBgILPuruIIhDUaCEpdgat1a8fGxrJp06bfPNauXevU97za/0a2DSCLSC3gLuD9Yhw7RkTiRSQ+NTXV+cU5W1gTKxDEk/oL72X2XcFkOQLh4AkNBKUux8/Pj7S0NJcLhLJkjCEtLQ0/P78S/6xTB5BFJAr4xhjzuxmZRGQm8E9jzBoRmew4btaVXtNlB5AvJ20fTLkDcs6xv/c0+s3LxNfLky/GXE9UiL/d1SlVruTk5JCYmFhq1/G7Kz8/PyIjI/H29v7N9isNINsZBgcAcTwNAc4BY4wx84p6TbcKA4D0A1YgXDjNgd6f02/+BXw8PZg5toPeqayUKjW2XU10JcaYaGNMlDEmCpgFPHylIHBL1aJhxELwq0L0wsHMud2Lc9m5DP90HRnndJEcpVTZcOalpdOB1UAjEUkUkVEiMlZExjrrPV1WlTpWIPiHEPXtEKb3NBxJP8/oqfFk5eTZXZ1SqgLQm87Kk9NHYUofOH2UFTd8wpBF+dwaW4N3B12Hh4dc+eeVUqoQ5babSF1GUIR1lVFAGJ3WPsyb3SqxcOsxXl2w0+7KlFJuTsOgvAkMh/vngIcn/Xc+weNtA5i08gAf/7Tf7sqUUm5Mw6A8qlYPBs+C8+n84fgz9GsawKsLdvL15qN2V6aUclMaBuVVRCsY+DlyYg9v5r7ODXUDGDdjM2v2p9ldmVLKDWkYlGcxN8JdH+BxeBWfBn9IVDVfxkyNZ/fxM3ZXppRyMxoG5V3sPdDzNXx2L2BO1Dz8vDwYPmmdrqeslCpVGgauoMPDcMMTBGydwjetVnPqfA7DP13H6awcuytTSrkJDQNX0f1laDmIsPh/Mq/DHvamnOWhzzeQnZtvd2VKKTegYeAqRKDPf6D+LTRY9yJTbzjByr1pPDtnq87iqJS6ZhoGrsTTGwZMgYjWdNw4nr+3O8fshEQ+WXHA7sqUUi5Ow8DV+PjDfTMhOJIBu8cxssF5/m/hTn7YVXHWB1JKlT4NA1fkXx2GzEG8KvH8qRe5PiyXx6ZvZF/qWbsrU0q5KA0DV1W1Lgyegcf5DCb7/oMgj2xGT43n1Hm9wkgpVXIaBq6sZkvo/yk+qdv4JuJTEtPO8vj0jeTl64CyUqpkNAxcXcOe0PsNqiZ+z9cNFvLj7lT+vuhnu6tSSrkYL7sLUKWg3WhIP0CjNRN4LyaEh5dDo/BA+rWJtLsypZSL0JaBu+jxV2h8O72T3uHRmrt4du5WNh4+aXdVSikXoWHgLjw84e6PkIjWjDv7Jl38j/DgZxt0DiOlVLFoGLgTn8pw35dI5RA+8HiDoAvJPPiZrqOslLoyDQN3ExAGg2filZ/NvCpvsz8xmWdmb9EpK5RSRXJaGIjIJBFJEZFthewfLCJbRGSriKwSkZbOqqXCCWsMAz8j4OwBFtSYyDebDvPhcl02UylVOGe2DCYDvYrYfwDoaoyJBf4KTHRiLRVPva7Q5z/UyVjH1LDp/H3RTpbplBVKqUI4LQyMMcuB9CL2rzLGXLzcZQ2g10GWtlb3QZen6Hj6W14MXsQT0zdyKC3T7qqUUuVQeRkzGAV8W9hOERkjIvEiEp+amlqGZbmBG/8MsQMYkfUZt7KCMVM3kHkh1+6qlFLljO1hICI3YoXB04UdY4yZaIyJM8bEhYaGll1x7kAE+r4LdW/g/zzep0rqOp7SAWWl1CVsDQMRaQF8DPQ1xqTZWYtb8/KFgZ/jUTWKKZXfZufWDUzUAWWlVAG2hYGI1AHmAPcbY3bbVUeFUbkaDJ6Jr48PM/z/yceL1vLTHu1yU0pZnHlp6XRgNdBIRBJFZJSIjBWRsY5DXgSqA++JyCYRiXdWLcqhWjRy3wyqk8HUyv9m/H/XcCT9nN1VKaXKAXG1vuO4uDgTH6+5cU12fo358n6+py3/qvIcsx7uTCUfT7urUko5kYhsMMbEFbbf9gFkZYMmdyA9/4/urOOuEx/wtA4oK1XhaRhUVB0ehvZjecDrW6pu+5RPVhywuyKllI00DCqynv+HadSbv3h/xtpFn7Nq7wm7K1JK2UTDoCLz8ET6fYKp0YJ3vN9lwrSZJJ7UAWWlKiINg4rOxx/PITPxCgzjbfMaL0xeqFNeK1UBaRgoCAjDe+hsgr0Nfz75Ii/PXKUDykpVMBoGyhLaCO/7plHP8zh9dj7F5J/0PkClKhINA/Wr6C7Ine/RwXMH1Zc8yaq9OuW1UhWFhoH6DY+WA7nQ7QX6eK5i17TxOqCsVAWhYaB+x7frOE43H8YIM4/5H73M+WwdUFbK3WkYqN8TIeiutzgRcRMPZn7ItCnv6YCyUm5Ow0BdnqcXIcM/JzWoKYMTX2H+gq/srkgp5UQaBqpwPv6EPziPM94hdFr/KBsS1ttdkVLKSTQMVJEkIAz/UV/h6SGEzh9MUuJhu0tSSjmBhoG6Iv+aDTnfbxqhnOTsp/04f/a03SUppUqZhoEqlprNu7C389vUz93D3vcHYvJy7C5JKVWKNAxUscXefB8rGjxFbOYqdkx6CPQKI6XchoaBKpEug5/lu6r30ixpJge+etXucpRSpUTDQJWIiNDpwf+w1Lsr0Zv+wYkVk+wuSSlVCjQMVIn5+/kQM3oKq2hJ1f+N49wWvQdBKVfntDAQkUkikiIi2wrZLyLyjojsFZEtInKds2pRpa9uWFU87/2crfn18J4zirz9y+0uSSl1DZzZMpgM9Cpif2+ggeMxBnjfibUoJ2jfuA57b/mUA/lh5H4+EI5utLskpdRVcloYGGOWA+lFHNIXmGosa4AqIlLTWfUo57incwu+bvEeqXmVyZp8F5zYY3dJSqmrYOeYQS3gSIHniY5tvyMiY0QkXkTiU1NTy6Q4VXxP3NWFdyLeJPNCHhc+7QOnkuwuSSlVQi4xgGyMmWiMiTPGxIWGhtpdjrqEl6cHzw29g2f9XyY78yQ5k/tCZprdZSmlSsDOMEgCahd4HunYplxQcCVvnh05kCd4GnPyIHmf94MLZ+wuSylVTHaGwXxgqOOqouuBU8aYZBvrUdcoOsSfB4bcz2M5j0PyZswXQyD3gt1lKaWKwZmXlk4HVgONRCRRREaJyFgRGes4ZCGwH9gLfAQ87KxaVNnpWD+EzncMY3z2GOTADzBnNOTrSmlKlXdeznphY8ygK+w3wCPOen9lnyHX12XP8fv567qzvLDjc/jmD3DH2yBid2lKqUK4xACycj0v3N6U3fWG8l7enZAwBZa8qBPbKVWOFSsMROQJEQly9O9/IiIJItLD2cUp1+Xl6cG7g65jVtBwvqQnrHoHlv5VA0Gpcqq4LYORxpjTQA+gKnA/8LrTqlJuIbiyN5+MaMdrjOQb7x7w0z/hB/3fRqnyqLhjBhc7e28FPjPGbBfRDmB1ZdEh/nwwtC3DPsnDPwhu/PF18PCErk/ZXZpSqoDitgw2iMhirDD4TkQCgXznlaXcyfX1qvOPAa0ZdXIoawJ7wLK/Wa0EpVS5UdyWwSigFbDfGHNORKoBI5xXlnI3d7SMIPnUee5bOJT5EdD8+1fAwwtueMLu0pRSFD8MOgCbjDGZIjIEuA5423llKXc0unM9jmZk0XfV/SypK9Rb8qIVCB30CmOl7FbcbqL3gXMi0hIYB+wDpjqtKuWWRIQXbm9K92YR9Dg8hORaPeG7P8PaD+0uTakKr7hhkOu4Sawv8K4xZgIQ6LyylLvy9BDevrc1LWpX5+ZD93OyTk/49ilY95HdpSlVoRU3DM6IyLNYl5QuEBEPwNt5ZSl35uftycfD2hJeJZBbjgwjM+oWWPgniP/U7tKUqrCKGwYDgQtY9xscw5ph9E2nVaXcXjV/H6aMaAeePtx+bDQXorvDN09CgvY+KmWHYoWBIwCmAcEicjuQZYzRf7XqmtSpXplPhrXlWKZh0KlHyI2+CeY/Bmsn2l2aUhVOcaejGACsA/oDA4C1InKPMwtTFUPL2lV4977WbEo+z0O5fyS/YW/4djwse02nrlCqDBW3m+g5oK0xZpgxZijQDnjBeWWpiuTmJuH89c7mLNlzmud9nsK0HAQ/vg4Lx0O+3tuoVFko7n0GHsaYlALP09AZT1UpGty+LskZWby7bC+BnR/jmQ7VkdXvwvmTcOf74OVjd4lKubXihsEiEfkOmO54PhBrcRqlSs24Hg05dT6HD386iN9NQ/lD9xD430uQlQEDpoKPv90lKuW2ijuAPB6YCLRwPCYaY552ZmGq4hERXu7TjP5tInl76V7ez+1jLYqzbylMvRPOpdtdolJuq9grnRljZgOznViLUnh4CK/3a0FWbj5/X/Qzle7oyvD+k2H2AzD5NhgyB4Jq2l2mUm6nyJaBiJwRkdOXeZwRkdNlVaSqWDw9hLcGtKRns3Be+noHX5xtDYNnQcZhmNQD0vbZXaJSbqfIMDDGBBpjgi7zCDTGBJVVkari8fb04J1BrenWKJRn525l7qkYGPY1ZGfCpJ6QvNnuEpVyK069IkhEeonILhHZKyLPXGZ/HRFZJiIbRWSLiNzqzHqUa/H18uSDIW24Pro642ZsZmF6TRixCDx9YfLtcOAnu0tUym04LQxExBOYAPQGmgKDRKTpJYc9D8wwxrQG7gXec1Y9yjVZ8xjF0bpOVR6fvpGlacEw6jsIrAmf3anTVyhVSpzZMmgH7DXG7DfGZANfYM16WpABLnY3BQNHnViPclH+vl58OqItTSOCGPt5AitS/GDUYojuYk1fsejPkJ9nd5lKuTRnhkEt4EiB54mObQW9BAwRkUSs+xYeu9wLicgYEYkXkfjU1FRn1KrKuSA/b6aObEe9EH8emLqetcl5cN9MaD8W1kyA/w6ErFN2l6mUy7L7LuJBwGRjTCTW+sqfOabH/g1jzERjTJwxJi40NLTMi1TlQ5XKPnz+QHtqVanEyMnr2Zh0Bnr/HW7/N+xfBh/fAun77S5TKZfkzDBIAmoXeB7p2FbQKGAGgDFmNeAHhDixJuXiQgJ8mfbA9YQE+jLk47Ws2Z8GcSPg/rmQmQIf3aQDy0pdBWeGwXqggYhEi4gP1gDx/EuOOQzcDCAiTbDCQPuBVJFqBPvx5ZgO1KxSiWGT1rHs5xRr/OCB78E/zBpY3jDZ7jKVcilOCwNjTC7wKPAdsBPrqqHtIvKKiPRxHDYOGC0im7HmPRruWF5TqSLVCPZjxoMdaBAewJjP4lmwJRmqx8ADS6DejfD1E/DtM5CXa3epSrkEcbXfvXFxcSY+Pt7uMlQ5cTorh5Gfrifh8Ele79eCAXG1rSuLFr9gDSzH3Az9PwW/YLtLVcpWIrLBGBNX2H67B5CVuiZBft5MHdWOG+qH8NSsLXy68gB4eEKv/4M73oEDP8LH3SHlZ7tLVapc0zBQLq+yjxcfD4ujZ7NwXv56B+8u3YMxBtoMg6FfwfkMmNgNNn6uq6cpVQgNA+UWfL08mXDfddzduhb/WLyb1xf9bAVCVCcYuwJqt4WvHoG5Y+HCWbvLVarcKfYU1kqVd16eHvyjf0sq+3ry4Y/7OZuVy1/7NscjMBzunwfL/2Etp3k0AfpPhvBmdpesVLmhLQPlVjw8hL/2bc7YrjFMW3uYcTM3k5uXb40jdHsahs637lT+6Cbr8lPtNlIK0DBQbkhEeKZ3Y8b3bMTcjUk8NC2BrBzH3EXRnWHsSqjb0br8dPYoyNKlOZTSMFBu65Eb6/PSHU35387jDJy4hpQzWdaOgFAYPBtuegG2z4WJXXV9BFXhaRgotzb8hmjeH9yG3cfOcNeEVfx8zNEK8PCALn+C4QsgJ8u6/HTdR9ptpCosDQPl9no1r8HMsR3Izc+n33urrOkrLqrb0braKLorLPwTfDEYzhy3r1ilbKJhoCqE5rWC+eqRTkSF+DNqynomrzzw607/6nDfDOjxN9j7P3ivPWydpa0EVaFoGKgK4+J8Rjc3Ceelr3fw4lfbrCuNwOo26vio1UqoFmMNLM+4H86mFP2iSrkJDQNVofj7evHhkDY82KUeU1cfYuSUeE5n5fx6QGhDaxW17i/D7sUwoT1sm62tBOX2NAxUhePhITx7axNevzuWVXtP0O+9VRxJP1fgAE/o9CQ8uByqRcOskTBjKJzV2dWV+9IwUBXWve3qMHVkO46fzuLOCSvZcOjkbw8IawwjF8PNf4Hdi6yxhO1z7SlWKSfTMFAVWsf6Icx75AYC/bwY9NEaZm9I/O0Bnl7Q+Y9WK6FKHZg5HGYMg8wTttSrlLNoGKgKr15oAHMfvoE2daoybuZmnpq1mfPZeb89KKwJjPqfdaPazwtgQjtImAr5+fYUrVQp0zBQCqjq78Nno9rx2E31mbkhkTsnrGRvypnfHuTpZd2o9uByqN4A5j8GH90Ih9faU7RSpUjDQCkHL08PxvVoxJQR7Thx9gJ3/GclcxISf39geFMYuQj6fWJdejqpB8wZA6eTy75opUqJhoFSl+jSMJSFT3SmRWQwf5xRSLeRCMTeA4+uh85/sgaW/9MGVvwLci/YU7hS18CpYSAivURkl4jsFZFnCjlmgIjsEJHtIvJfZ9ajVHGFB/kx7YH2PO7oNuo7YQV7jp/5/YG+AXDzC/DIWqjXDf73Erx3PexapPcmKJfitDAQEU9gAtAbaAoMEpGmlxzTAHgWuMEY0wx40ln1KFVSXp4e/LFHI6aObEfa2Wz6vLvy91cbXVStHgz6LwyZAx5eMH0gTLsHTuwp26KVukrObBm0A/YaY/YbY7KBL4C+lxwzGphgjDkJYIzRe/9VudO5wa/dRuNmbmb8zMt0G11U/2Z4aBX0fA2OrLNaCd8+rZPfqXLPmWFQCzhS4HmiY1tBDYGGIrJSRNaISK/LvZCIjBGReBGJT03Vu0BV2SvYbTQrIZE+765gW9Kpyx/s6Q0dHobHEqDVYGtq7LdbwpIXITOtbAtXqpjsHkD2AhoA3YBBwEciUuXSg4wxE40xccaYuNDQ0DIuUSlLwW6jU+dz6DthJW8t3kV2biH3GgSEQp93rEHmpn1g5TvwdgtY+jc4n1G2xSt1Bc4MgySgdoHnkY5tBSUC840xOcaYA8BurHBQqtzq3CCUJX/oSt9WEbyzdG/RrQSA6jFw90R4eA3U7w7L37BCYfmbcOEyg9JK2cCZYbAeaCAi0SLiA9wLzL/kmHlYrQJEJASr22i/E2tSqlQEV/bmrQGt+HhoHGmZ2dw5YSX/WrK78FYCWHMdDZgCD/4EdTrC0let7qNV/4Hsc4X/nFJlwGlhYIzJBR4FvgN2AjOMMdtF5BUR6eM47DsgTUR2AMuA8cYY7VRVLqN703CW/KELd7SM4O3v99B3wkp2HD1d9A/VbAH3fQEPLIWaLWHx8/BOK1j7IeScL5vClbqEGBe7FjouLs7Ex8fbXYZSv0DFRv0AABPaSURBVLNkx3H+PHcrJzOzefSm+jxyY328PYvx99ahVVYr4dBKqBwC7R+Etg9A5WrOL1pVGCKywRgTV+h+DQOlSs/JzGxe+no7X206SrOIIP7RvyVNagZd+QeNsUJh5duw5zvw9ofrhkKHR6BK7Sv/vFJXoGGglA2+236M5+Zu5dT5HB7qGsND3epTycezeD98fLs1jrB1pvW8+T1ww+MQ3sx5BSu3p2GglE1OZmbz8tfbmbfpKJFVK/Hi7U25pWk4IlK8F8g4Amvegw1TICcTGvSAG56AujdYcyMpVQIaBkrZbPW+NP4yfxu7j5+lW6NQ/nJHM6JD/Iv/AufSYf0nsPYDOHcCasVZ3UeNbwcvH+cVrtyKhoFS5UBOXj5TVx/65fLT0V2ieeTG+lT28SrBi5yHTdNg1btw8gD4h1njCm2G67iCuiINA6XKkZTTWbz27c/M3ZhErSqVeP62JvRqXqP4XUdgra6273urtbB7kdVl1LAXxI2CmJvAw+6JBVR5pGGgVDm07kA6L361jZ+PnaFzgxBe6tOMmNCAkr9QxmHYMNlagjMzFapGQZsR0Pp+8K9e2mUrF6ZhoFQ5lZuXz+drDvHPxbvJys1jZKdoHu5Wn+BK3lfxYtmwcz7ET7LuV/D0hWZ3QtxIqN1eB5yVhoFS5V3qmQv8fdHPzNqQSJCfF2O7xTC8Y1TJxhMKStlpdSFt/gKyz0C1GGg5CFoMgKp1S7d45TI0DJRyEduPnuKfi3ez9OcUQgJ8eeym+tzbrja+XsW8P+FSF85ay3Fu+RIO/mRtq9sJWt4LTfuCXzFuhlNuQ8NAKRez4VA6byzaxdoD6dSqUoknujfg7ta18CrO1BaFyThshcKm6ZC+D7z8rEtTWw6yluv0vMpWiHIZGgZKuSBjDCv2nuDN73axJfEU9UL9GXdLI3o3r4GHxzX0/xsDSRtg83TYOguyMiAgHGL7Q/N+ENFaxxfclIaBUi7MGMN324/zz8W72JNylmYRQfypZyO6NQwt2eWol5N7AfYstsYWdi+C/FwIqgWNboXGt0FUJ2vVNuUWNAyUcgN5+YavNiXxr//t5kj6eVrVrsJD3WK4pUn4tbUULjqXbgXCzwtg7/eQex58g6FhT2h8q7Uoj2/gtb+Pso2GgVJuJDs3n5kbjvDhj/s5nH6OmFB/Huwaw52tauHjVUo3m2Wfg/0/WMGwayGcTwdPH2tsofFt0LA3BIaXznupMqNhoJQbys3LZ+G2Y7z/wz52Jp+mZrAfozpFM6hdHfx9S3EwOC8Xjqy1guHnbyDjECAQGQeNeltdSqGNdZzBBWgYKOXGjDH8uDuV93/Yx9oD6QRX8mZYxyiGd4yimn8pT2JnjDW99q6FsOtbOJpgba8aZYVCw15Qt6OOM5RTGgZKVRAbDp3kgx/3sWTHcfy8Pbi3bR0e6BxNZNXKznnD08nWOMOub61upbwL1jhDg1usVkP97lCpinPeW5WYhoFSFcye42f4cPl+5m1MwgDdm4Rx//VRdIypXjqDzZeTnWkFwq6FsGuRNdW2hxdEtoPoLtYjMg68fJ3z/uqKNAyUqqCOZpxnyuqDzFh/hJPncqgX4s/g6+tyz3WRBFd2YldOfp51L8PFFkPyJjD54FUJ6lwP9bpa4VCjpd7sVoZsDQMR6QW8DXgCHxtjXi/kuH7ALKCtMabI3/QaBkqVTFZOHt9uS+az1YdIOJyBn7cHfVvW4v4OdWleK9j5BZzPsNZ3PvAjHFgOKTus7b7BEHWDFQxRnSGsqU6/7US2hYGIeAK7gVuARGA9MMgYs+OS4wKBBYAP8KiGgVLOsy3pFNPWHmLexqOcz8mjVe0q3H99XW5rURM/76ucA6mkzqZYcyUdWA77f7QW6gHwq2INQNftaC3tWaOFthxKkZ1h0AF4yRjT0/H8WQBjzGuXHPdvYAkwHviThoFSznfqfA5zEhL5fM0h9qVmUrWyN3e1jqRfm1o0rRl07Xc3l0TGYTi4wpp6+9AqSN9vbfcJsKbfvhgOta7TMYdrYGcY3AP0MsY84Hh+P9DeGPNogWOuA54zxvQTkR8oJAxEZAwwBqBOnTptDh065JSalapojDGs3p/GtDWHWbLjONl5+TSuEUi/6yLp2yqCsCC/si/qdDIcXmUFw6FVv3YreflBZFuo3c5aBzoyDgLCyr4+F1Vuw0BEPIClwHBjzMGiwqAgbRko5RwZ57L5eksycxIS2Xg4Aw+Bzg1C6dcmkh5Nw8uuG+lSmWlweLUjHFbC8W3WPEoAVer8Ggy14qBmS/C2IcBcQLntJhKRYGAfcNbxIzWAdKBPUYGgYaCU8+1LPcvchCTmbkwiKeM8gb5e3NaiJndfF0nbqKpl2410qexzkLwZkuIhMd66cunUEWufhxeEN7daELXaQM0WENJQb4TD3jDwwhpAvhlIwhpAvs8Ys72Q439AWwZKlSv5+YY1B9KYvSGJb7clcy47j9rVKnFbbAS3xdakea0yHl8ozJljVigkrrcC4uhGyHb8nenpC2FNoEas1XKoEQvhzSrcxHt2X1p6K/BvrEtLJxlj/iYirwDxxpj5lxz7AxoGSpVb57JzWbTtGPM2HWXV3hPk5hvqVKvMrbE1ub1FTZpFlJNgAOtehxN74NhWOLbl16/n0hwHCFSrZwVDjVirNRHeFIJru+08S3rTmVKq1GWcy2bx9uN8szX5N8FwW4ua3BZbzoLhImPgTLIVDMlbfg2Ji5e2AvgGWa2IsCYQ1swKiLCmULmafXWXEg0DpZRTnczMZvGOY3yzJZlV+9LIyzfUrW61GHo1q0FsrWDnTYNRGrJOQ8pOSNkOx3dYVy8d326tAndRQI1fgyG0sRUWoY1cqqtJw0ApVWbSM7NZvP0YC7b+Ggyhgb7c3DiMm5uEc0P96lT2cYEbyS62IlJ2/DYgUndZE/JdFBQJYY2tgLgYEiENwS/IvtoLoWGglLLFycxslu1K4fudKfy4O5WzF3Lx9fKgY0x1bm4Szs1NwqgZXMnuMksmPw9OHoTUn63WROrPjsfu34dESH2oFgPVY6zxiWox1nTfXqU8tXgxaRgopWyXnZvPugPpfP/zcb7fmcLh9HMANIsIsoKhcRjNawXjWZ67k4ryu5DYBWl7IX0fZJ369TjxsAapq8dY4VCtnvV91SioUtep90hoGCilyhVjDHtTzvK/nSl8v/M4CYdPkm+gSmVvOsZUp1P9UDrVD6FOdSetw1CWjLHWl07fB2n7rKk2Cn5/4fRvjw+MgGrRVjhUjYKqBb73D7mmK500DJRS5Vp6ZjbLd6eyYu8JVuw5wbHTWQDUqVaZTg1C6FQ/hI4x1alS2Z7uFacxBjJPWKFw8uAljwPWmEVBPgFww5PQdfxVvZ2GgVLKZRhj2JeayYo9Vjis2Z/O2Qu5iECLWsHcUD+EjjEhtKlblUo+Nk2PUVZyzluT+J08COkHrK9RnaDJ7Vf1choGSimXlZOXz+YjGb+0GjYeySAv3+DtKbSIrML19arRPro6bepWxd/XBa5SspGGgVLKbZzJyiH+0EnW7k9n7YE0tiSeIi/f4OUhxEYG0z66OtfXq0ZcVDUCNBx+Q8NAKeW2Mi/ksuHQSdbsT2PtgXQ2H8kgN9/g6SE0jwiiTd1qtKlblTZ1q1IjuGLPZqphoJSqMM5l55JwKIM1+9NYdyCdzYkZXMjNB6BWlUpcV7cqbepUoU3dajSuGYi3Z8VZZvNKYaDtKKWU26js42VdgdQgBLDub9iZfJoNh06y4fBJ4g+m8/XmowBU8vakZe1g2tStSqvaVYmtFUx4kG/5m1OpjGjLQClVoRzNOE/C4ZNsOHSShEMn2X70NLn51u/B0EBfWtQKpnmtYFpEBhMbGUxYoHt0L2nLQCmlCoioUomIKpW4vUUEAOez89iRfJqtiRlsSTrFtqRTLNuVgiMfCA/yJbZWFSscagXTNCKIsED3a0FoGCilKrRKPp6/DDJflHkhlx3Jp9mSaIXDlsQMvv/5OBc7UkICfGhSM4imEUE0iwimac0gokP8XXc6DTQMlFLqd/x9vWgbVY22Ub+uY3AmK4cdR0+zI/n0L18nrThATp6VEH7eHjSuEUSzCCskGtcIomF4AIF+rrHkpo4ZKKXUVcrOzWdvytlfAmL70VPsSD7NmazcX46pVaUSjWoE0qhGII1rBNIwPJCY0AB8vMr2SiYdM1BKKSfx8fKgqaMlQBtrmzGGxJPn2XXsDLuOn7G+HjvD8t2pvwxUe3kI0SH+VkiEB1I/LICYsACiqvuXeUhcpGGglFKlSESoXa0ytatVpnvT8F+2Z+fmc+BEJj8fO81uR0hsTszgmy2/Tkjn6SHUrVaZmLAAYkIDqB9mPWJC/Z3e3aRhoJRSZcDHy+OX7qKCMi/kcuBEJntTzv7y2Jd6lh92pfwyHgFQI8iPUZ2iGd2lnlPqc2oYiEgv4G3AE/jYGPP6Jfv/CDwA5AKpwEhjzCFn1qSUUuWJv68XzR33NhSUk5fP4fRzv4TD3pSzhAX5Oq0Op4WBiHgCE4BbgERgvYjMN8bsKHDYRiDOGHNORB4C3gAGOqsmpZRyFd6eHsSEWt1FZcGZIxXtgL3GmP3GmGzgC6BvwQOMMcuMMeccT9cAkU6sRymlVCGcGQa1gCMFnic6thVmFPDt5XaIyBgRiReR+NTU1FIsUSmlFDg3DIpNRIYAccCbl9tvjJlojIkzxsSFhoaWbXFKKVUBOHMAOQmoXeB5pGPbb4hId+A5oKsx5oIT61FKKVUIZ7YM1gMNRCRaRHyAe4H5BQ8QkdbAh0AfY0yKE2tRSilVBKeFgTEmF3gU+A7YCcwwxmwXkVdEpI/jsDeBAGCmiGwSkfmFvJxSSikncup9BsaYhcDCS7a9WOD77s58f6WUUsVTLgaQlVJK2cvlZi0VkVTgau9SDgFOlGI55YG7nZO7nQ+43zm52/mA+53T5c6nrjGm0MsxXS4MroWIxBc1hasrcrdzcrfzAfc7J3c7H3C/c7qa89FuIqWUUhoGSimlKl4YTLS7ACdwt3Nyt/MB9zsndzsfcL9zKvH5VKgxA6WUUpdX0VoGSimlLkPDQCmlVMUJAxHpJSK7RGSviDxjdz2lQUQOishWx1Qe8XbXU1IiMklEUkRkW4Ft1URkiYjscXytameNJVXIOb0kIkmOz2mTiNxqZ40lISK1RWSZiOwQke0i8oRju0t+TkWcjyt/Rn4isk5ENjvO6WXH9mgRWev4nfelY464wl+nIowZOFZd202BVdeAQZesuuZyROQg1kpxLnmzjIh0Ac4CU40xzR3b3gDSjTGvO0K7qjHmaTvrLIlCzukl4Kwx5h921nY1RKQmUNMYkyAigcAG4E5gOC74ORVxPgNw3c9IAH9jzFkR8QZWAE8AfwTmGGO+EJEPgM3GmPcLe52K0jK44qprquwZY5YD6Zds7gtMcXw/Besfqsso5JxcljEm2RiT4Pj+DNakk7Vw0c+piPNxWcZy1vHU2/EwwE3ALMf2K35GFSUMSrrqmqswwGIR2SAiY+wuppSEG2OSHd8fA8LtLKYUPSoiWxzdSC7RpXIpEYkCWgNrcYPP6ZLzARf+jETEU0Q2ASnAEmAfkOGYPRqK8TuvooSBu+pkjLkO6A084uiicBvG6sN0h37M94EYoBWQDPzT3nJKTkQCgNnAk8aY0wX3ueLndJnzcenPyBiTZ4xphbWIWDugcUlfo6KEQbFWXXM1xpgkx9cUYC7W/wSu7rijX/di/67LL3pkjDnu+MeaD3yEi31Ojn7o2cA0Y8wcx2aX/Zwudz6u/hldZIzJAJYBHYAqInJxmYIr/s6rKGFwxVXXXI2I+DsGwBARf6AHsK3on3IJ84Fhju+HAV/ZWEupuPhL0+EuXOhzcgxOfgLsNMa8VWCXS35OhZ2Pi39GoSJSxfF9JawLZXZihcI9jsOu+BlViKuJAByXiv0b8AQmGWP+ZnNJ10RE6mG1BsBapOi/rnZOIjId6IY13e5x4C/APGAGUAdrqvIBxhiXGZAt5Jy6YXU/GOAg8GCB/vZyTUQ6AT8BW4F8x+Y/Y/Wzu9znVMT5DMJ1P6MWWAPEnlh/4M8wxrzi+B3xBVAN2AgMKWqd+QoTBkoppQpXUbqJlFJKFUHDQCmllIaBUkopDQOllFJoGCillELDQKlfiEhegVkrN5Xm7LYiElVwJlOlyhuvKx+iVIVx3nFLv1IVjrYMlLoCx7oRbzjWjlgnIvUd26NEZKljcrPvRaSOY3u4iMx1zC+/WUQ6Ol7KU0Q+csw5v9hxtygi8rhjfv0tIvKFTaepKjgNA6V+VemSbqKBBfadMsbEAu9i3ckO8B9gijGmBTANeMex/R3gR2NMS+A6YLtjewNggjGmGZAB9HNsfwZo7Xidsc46OaWKoncgK+UgImeNMQGX2X4QuMkYs98xydkxY0x1ETmBtVBKjmN7sjEmRERSgciCt/47pkteYoxp4Hj+NOBtjHlVRBZhLYgzD5hXYG56pcqMtgyUKh5TyPclUXBemDx+HbO7DZiA1YpYX2CmSaXKjIaBUsUzsMDX1Y7vV2HNgAswGGsCNIDvgYfgl0VHggt7URHxAGobY5YBTwPBwO9aJ0o5m/4FotSvKjlWi7pokTHm4uWlVUVkC9Zf94Mc2x4DPhWR8UAqMMKx/QlgooiMwmoBPIS1YMrleAKfOwJDgHccc9IrVaZ0zECpK3CMGcQZY07YXYtSzqLdREoppbRloJRSSlsGSiml0DBQSimFhoFSSik0DJRSSqFhoJRSCvh/H+ANuCuNUfIAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "def plot_graphs(history, metric):\n",
    "    plt.plot(history.history[metric])\n",
    "    plt.plot(history.history[f'val_{metric}'])\n",
    "    plt.xlabel(\"Epochs\")\n",
    "    plt.ylabel(metric)\n",
    "    plt.legend([metric, f'val_{metric}'])\n",
    "    plt.show()\n",
    "    \n",
    "plot_graphs(history, \"accuracy\")\n",
    "plot_graphs(history, \"loss\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9dc6bd9e",
   "metadata": {},
   "source": [
    "**If your model passes the previously mentioned thresholds be sure to save your notebook and submit for grading before jumping onto the next section.**"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "58d6cd3b",
   "metadata": {},
   "source": [
    "## Optional Exercise - Visualizing 3D Vectors\n",
    "\n",
    "As you saw on the lecture you can visualize the vectors associated with each word in the training set in a 3D space.\n",
    "\n",
    "For this run the following cells and visit [Tensorflow's Embedding Projector](https://projector.tensorflow.org/)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "id": "0ad672c8",
   "metadata": {
    "id": "OhnFA_TDXrih",
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Weights of embedding layer have shape: (1000, 16)\n"
     ]
    }
   ],
   "source": [
    "# Reverse word index\n",
    "reverse_word_index = dict([(value, key) for (key, value) in word_index.items()])\n",
    "\n",
    "# Save the embedding layer\n",
    "e = model.layers[0]\n",
    "\n",
    "# Save the weights of the embedding layer\n",
    "weights = e.get_weights()[0]\n",
    "print(f\"Weights of embedding layer have shape: {weights.shape}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "078d92f4",
   "metadata": {},
   "source": [
    "***Expected Output:***\n",
    "\n",
    "```\n",
    "Weights of embedding layer have shape: (1000, 16) \n",
    "\n",
    "The above is true if global variables are not modified. \n",
    "Notice that this shape will always be (NUM_WORDS, EMBEDDING_DIM).\n",
    "\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f51ba083",
   "metadata": {},
   "source": [
    "Now run the following cell to generate the `vecs.tsv` and `meta.tsv` files that you will upload to the embedding projector."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "id": "77e96202",
   "metadata": {
    "id": "_POzcWWAXudL",
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Generate files for embedding visualization\n",
    "out_v = io.open('vecs.tsv', 'w', encoding='utf-8')\n",
    "out_m = io.open('meta.tsv', 'w', encoding='utf-8')\n",
    "for word_num in range(1, NUM_WORDS):\n",
    "    word = reverse_word_index[word_num]\n",
    "    embeddings = weights[word_num]\n",
    "    out_m.write(word + \"\\n\")\n",
    "    out_v.write('\\t'.join([str(x) for x in embeddings]) + \"\\n\")\n",
    "out_v.close()\n",
    "out_m.close()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dbe5abed",
   "metadata": {},
   "source": [
    "Before downloading these files be sure to having already submitted your assignment.\n",
    "\n",
    "\n",
    "By running the previous cell, these files are placed within your filesystem. To download them, click on the `Jupyter` logo on the upper-left side of the screen. This will take you to the Jupyter filesystem and there you can download the files by selecting them and clicking on the `Download` button."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a47c5777",
   "metadata": {},
   "source": [
    "**Congratulations on finishing this week's assignment!**\n",
    "\n",
    "You have successfully implemented a neural network capable of classifying text and also learned about embeddings and tokenization along the way!\n",
    "\n",
    "**Keep it up!**"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3d771c9c",
   "metadata": {},
   "source": [
    "<details>\n",
    "  <summary><font size=\"2\" color=\"darkgreen\"><b>Please click here if you want to experiment with any of the non-graded code.</b></font></summary>\n",
    "    <p><i><b>Important Note: Please only do this when you've already passed the assignment to avoid problems with the autograder.</b></i>\n",
    "    <ol>\n",
    "        <li> On the notebook’s menu, click “View” > “Cell Toolbar” > “Edit Metadata”</li>\n",
    "        <li> Hit the “Edit Metadata” button next to the code cell which you want to lock/unlock</li>\n",
    "        <li> Set the attribute value for “editable” to:\n",
    "            <ul>\n",
    "                <li> “true” if you want to unlock it </li>\n",
    "                <li> “false” if you want to lock it </li>\n",
    "            </ul>\n",
    "        </li>\n",
    "        <li> On the notebook’s menu, click “View” > “Cell Toolbar” > “None” </li>\n",
    "    </ol>\n",
    "    <p> Here's a short demo of how to do the steps above: \n",
    "        <br>\n",
    "        <img src=\"https://drive.google.com/uc?export=view&id=14Xy_Mb17CZVgzVAgq7NCjMVBvSae3xO1\" align=\"center\">\n",
    "</details>"
   ]
  }
 ],
 "metadata": {
  "dlai_version": "1.2.0",
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}