NLassignment2022 (1) (5).ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "f2S8I2ny-ovS"
      },
      "source": [
        "# NLE Assignment: Sentiment Classification\n",
        "\n",
        "In this assignment, you will be investigating NLP methods for distinguishing positive and negative reviews written about movies.\n",
        "\n",
        "For assessment, you are expected to complete and submit this notebook file.  When answers require code, you may import and use library functions (unless explicitly told otherwise).  All of your own code should be included in the notebook rather than imported from elsewhere.  Written answers should also be included in the notebook.  You should insert as many extra cells as you want and change the type between code and markdown as appropriate.\n",
        "\n",
        "In order to avoid misconduct, you should not talk about the assignment questions with your peers.  If you are not sure what a question is asking you to do or have any other questions, please ask me or one of the Teaching Assistants.\n",
        "\n",
        "Marking guidelines are provided as a separate document.\n",
        "\n",
        "The first few cells contain code to set-up the assignment and bring in some data.   In order to provide unique datasets for analysis by different students, you must enter your candidate number in the following cell.  Otherwise do not change the code in these cells."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "1gXQAZas-l9c"
      },
      "outputs": [],
      "source": [
        "candidateno=249763 #this MUST be updated to your candidate number so that you get a unique data sample\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "nk8JTP88A8vs",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "16773285-681f-43e5-cbfc-b44a1c8c323d"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "[nltk_data] Downloading package punkt to /root/nltk_data...\n",
            "[nltk_data]   Package punkt is already up-to-date!\n",
            "[nltk_data] Downloading package stopwords to /root/nltk_data...\n",
            "[nltk_data]   Package stopwords is already up-to-date!\n",
            "[nltk_data] Downloading package movie_reviews to /root/nltk_data...\n",
            "[nltk_data]   Package movie_reviews is already up-to-date!\n"
          ]
        }
      ],
      "source": [
        "#do not change the code in this cell\n",
        "#preliminary imports\n",
        "\n",
        "#set up nltk\n",
        "import nltk\n",
        "nltk.download('punkt')\n",
        "nltk.download('stopwords')\n",
        "nltk.download('movie_reviews')\n",
        "from nltk.corpus import stopwords\n",
        "from nltk.tokenize import word_tokenize\n",
        "from nltk.corpus import movie_reviews\n",
        "\n",
        "#for setting up training and testing data\n",
        "import random\n",
        "\n",
        "#useful other tools\n",
        "import re\n",
        "import pandas as pd\n",
        "import matplotlib.pyplot as plt\n",
        "%matplotlib inline\n",
        "from itertools import zip_longest\n",
        "from nltk.probability import FreqDist\n",
        "from nltk.classify.api import ClassifierI\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "BHBkzAccCVaZ"
      },
      "outputs": [],
      "source": [
        "#do not change the code in this cell\n",
        "def split_data(data, ratio=0.7): # when the second argument is not given, it defaults to 0.7\n",
        "    \"\"\"\n",
        "    Given corpus generator and ratio:\n",
        "     - partitions the corpus into training data and test data, where the proportion in train is ratio,\n",
        "\n",
        "    :param data: A corpus generator.\n",
        "    :param ratio: The proportion of training documents (default 0.7)\n",
        "    :return: a pair (tuple) of lists where the first element of the \n",
        "            pair is a list of the training data and the second is a list of the test data.\n",
        "    \"\"\"\n",
        "    \n",
        "    data = list(data)  \n",
        "    n = len(data)  \n",
        "    train_indices = random.sample(range(n), int(n * ratio))          \n",
        "    test_indices = list(set(range(n)) - set(train_indices))    \n",
        "    train = [data[i] for i in train_indices]           \n",
        "    test = [data[i] for i in test_indices]             \n",
        "    return (train, test)                       \n",
        " \n",
        "\n",
        "def get_train_test_data():\n",
        "    \n",
        "    #get ids of positive and negative movie reviews\n",
        "    pos_review_ids=movie_reviews.fileids('pos')\n",
        "    neg_review_ids=movie_reviews.fileids('neg')\n",
        "   \n",
        "    #split positive and negative data into training and testing sets\n",
        "    pos_train_ids, pos_test_ids = split_data(pos_review_ids)\n",
        "    neg_train_ids, neg_test_ids = split_data(neg_review_ids)\n",
        "    #add labels to the data and concatenate\n",
        "    training = [(movie_reviews.words(f),'pos') for f in pos_train_ids]+[(movie_reviews.words(f),'neg') for f in neg_train_ids]\n",
        "    testing = [(movie_reviews.words(f),'pos') for f in pos_test_ids]+[(movie_reviews.words(f),'neg') for f in neg_test_ids]\n",
        "   \n",
        "    return training, testing"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "1N3LWwBYICPP"
      },
      "source": [
        "When you have run the cell below, your unique training and testing samples will be stored in `training_data` and `testing_data`"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "HJLegkdPFUJA",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "8cbbd069-30eb-4272-d948-ac33d90da82b"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "The amount of training data is 1400\n",
            "The amount of testing data is 600\n",
            "The representation of a single data item is below\n",
            "(['perhaps', 'the', 'most', 'dramatic', 'changes', 'in', ...], 'pos')\n"
          ]
        }
      ],
      "source": [
        "#do not change the code in this cell\n",
        "random.seed(249763)\n",
        "training_data,testing_data=get_train_test_data()\n",
        "print(\"The amount of training data is {}\".format(len(training_data)))\n",
        "print(\"The amount of testing data is {}\".format(len(testing_data)))\n",
        "print(\"The representation of a single data item is below\")\n",
        "print(training_data[0])"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "VE3bKQbB50Rq"
      },
      "source": [
        "1)  \n",
        "a) **Generate** a list of 10 content words which are representative of the positive reviews in your training data.\n",
        "\n",
        "b) **Generate** a list of 10 content words which are representative of the negative reviews in your training data.\n",
        "\n",
        "c) **Explain** what you have done and why\n",
        "\n",
        "[20\\%]"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "QkT9CWv250Rq",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "4f9abb8f-ec71-4182-ea54-76ca49f7c055"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "[nltk_data] Downloading package stopwords to /root/nltk_data...\n",
            "[nltk_data]   Package stopwords is already up-to-date!\n"
          ]
        }
      ],
      "source": [
        "from nltk.corpus import stopwords\n",
        "nltk.download('stopwords')\n",
        "stopping = stopwords.words('english')\n",
        "\n",
        "def list_norm(data, tag):\n",
        "    if tag != '':\n",
        "        words_normalisation = [([words for words in i[0] if words.isalpha() and words not in stopping], i[1]) for i in data if i[1] == tag]\n",
        "        bag_of_words = [(FreqDist(i[0]), i[1]) for i in words_normalisation]\n",
        "    else:\n",
        "        words_normalisation = [([words for words in i[0] if words.isalpha() and words not in stopping], i[1]) for i in data]\n",
        "        bag_of_words = [(FreqDist(i[0]), i[1]) for i in words_normalisation]\n",
        "    return bag_of_words\n",
        "\n",
        "def most_frequent_words(freq_dist_one, freq_dist_two, top_of_pos_or_neg):\n",
        "    difference = freq_dist_one - freq_dist_two\n",
        "    return list(dict(difference.most_common(top_of_pos_or_neg)).keys())\n",
        "\n",
        "def freq_dist(data):\n",
        "    my_freq_dist = FreqDist()\n",
        "    for review, label in data:\n",
        "        my_freq_dist += review\n",
        "    return my_freq_dist\n",
        "\n",
        "training_pos_freq = list_norm(training_data, \"pos\")\n",
        "training_neg_freq = list_norm(training_data, \"neg\")\n",
        "\n",
        "pos_freq_dist = freq_dist(training_pos_freq)\n",
        "neg_freq_dist = freq_dist(training_neg_freq)\n",
        "\n",
        "bag_of_training_pos = most_frequent_words(pos_freq_dist, neg_freq_dist, 10)\n",
        "bag_of_training_neg = most_frequent_words(neg_freq_dist, pos_freq_dist, 10)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "GQBi7JVR50Rr",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "4503545a-8ee6-4e2e-b648-58bff41da48c"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "['film', 'life', 'also', 'well', 'best', 'great', 'world', 'films', 'many', 'story']\n"
          ]
        }
      ],
      "source": [
        "print(bag_of_training_pos)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "OIqSAumd50Rs",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "4fa9d8f9-a74e-4375-f295-56330d4164dd"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "['bad', 'movie', 'plot', 'even', 'worst', 'stupid', 'like', 'could', 'nothing', 'boring']\n"
          ]
        }
      ],
      "source": [
        "print(bag_of_training_neg)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Zljze_ZJ50Rt"
      },
      "source": [
        "When given a corpus, the computer sees a sequence of characters, reprensented as a sequence of binary numbers. This makes it difficult to efficiently search through a collection of documents which just characters. Therefore, for retrieval of documents we need to preprocess the documents in order for efficient search. This is why I first began my code by removing any stopwords, these are words in which are very common and don't carry a massive amount of information, they are ignored by the NLTK tokenisers. My next steps were about normalising the list given, this gets rid of punctuation marks and lowercasing all characters. All of this allows efficient search to be made on the collection of documents. I then created functions which make a frequency distribution of the data that has been split into postive and negative reviews. This is so we can use the frequency distribution to observe and calculate properties of the distribution, allowing us to make a decision. Finally, I created two different bag of words which have postive words in one and negative in the other, so I could print them."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "TApOQE6vND20"
      },
      "source": [
        "2) \n",
        "a) **Use** the lists generated in Q1 to build a **word list classifier** which will classify reviews as being positive or negative.\n",
        "\n",
        "b) **Explain** what you have done.\n",
        "\n",
        "[12.5\\%]\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "BThDMrcmODJy",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 35
        },
        "outputId": "bf1c03f8-acff-4d0c-d09d-3f0c7edf4337"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "'neg'"
            ],
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            }
          },
          "metadata": {},
          "execution_count": 59
        }
      ],
      "source": [
        "from nltk.classify.api import ClassifierI\n",
        "\n",
        "class WordsListClassifier(ClassifierI): \n",
        "\n",
        "    def __init__(self, pos, neg): \n",
        "        self._pos = pos \n",
        "        self._neg = neg \n",
        "\n",
        "    def classify(self, words): \n",
        "        score = 0\n",
        "        for word, value in words.items():\n",
        "            if word in self._pos:\n",
        "                score +=value\n",
        "            if word in self._neg:\n",
        "                score -=value\n",
        "        if score > 0:\n",
        "            return 'pos'\n",
        "        elif score == 0 or score < 0: \n",
        "          return 'neg'\n",
        "\n",
        "    def labels(self): \n",
        "        return (\"pos\", \"neg\")\n",
        "\n",
        "classifier = WordsListClassifier(bag_of_training_pos, bag_of_training_neg)\n",
        "classifier.classify(FreqDist(\"terrible movie\".split()))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "B6xxvcyr50Rv"
      },
      "source": [
        "For this question I made my very own classifier, which takes words splits them and tries to work out whether they are positive words or negatives, the end result is a statement neg or pos for negative and positive. This is done by using the variable score, it takes away one everytime a negative word is detected or adds one if a positive word is detected. If the final score is positive the whole review is positive and if the final score is negative the whole review is negative. At the end I tried using a string to check if my classifier is working correctly."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "dL1iL9jg50Rv"
      },
      "source": [
        "3)\n",
        "a) **Calculate** the accuracy, precision, recall and F1 score of your classifier.\n",
        "\n",
        "b) Is it reasonable to evaluate the classifier in terms of its accuracy?  **Explain** your answer and give a counter-example (a scenario where it would / would not be reasonable to evaluate the classifier in terms of its accuracy).\n",
        "\n",
        "[20\\%]"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "r1PrZnTe50Rw"
      },
      "outputs": [],
      "source": [
        "class ConfusionMatrix:\n",
        "    def __init__(self,predictions,goldstandard,classes=(\"pos\",\"neg\"), display = True):\n",
        "    \n",
        "        (self.c1,self.c2)=classes\n",
        "        self.TP=0\n",
        "        self.FP=0\n",
        "        self.FN=0\n",
        "        self.TN=0\n",
        "        for p,g in zip(predictions,goldstandard):\n",
        "            if g==self.c1:\n",
        "                if p==self.c1:\n",
        "                    self.TP+=1\n",
        "                else:\n",
        "                    self.FN+=1\n",
        "        \n",
        "            elif p==self.c1:\n",
        "                self.FP+=1\n",
        "            else:\n",
        "                self.TN+=1\n",
        "        if display == True:\n",
        "            print(f'True positive (tp) : {self.TP}\\nFalse positive (fp) : {self.FP}\\nTrue Negative (tn) : {self.TN}\\nFalse negative (fn) : {self.FN}')\n",
        "    \n",
        "    def accuracy(self):\n",
        "        self.acc = 0\n",
        "        self.acc = (self.TP+self.TN)/(self.TP+self.FP+self.FN+self.TN)\n",
        "        return self.acc\n",
        "    \n",
        "    def precision(self):\n",
        "        self.p=0\n",
        "        self.p = self.TP/(self.TP+self.FP)\n",
        "        return self.p\n",
        "  \n",
        "    def recall(self):\n",
        "        self.r=0\n",
        "        self.r = self.TP/(self.TP+self.FN)\n",
        "        return self.r\n",
        "  \n",
        "    def f1_score(self):\n",
        "        self.f1=0\n",
        "        self.f1 = 2*self.p*self.r/(self.p+self.r)\n",
        "        return self.f1 "
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "Uq4Fmr8S50Rx",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "0bfe4716-e72c-4a15-8d53-17f2bc9d9736"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "True positive (tp) : 223\n",
            "False positive (fp) : 131\n",
            "True Negative (tn) : 169\n",
            "False negative (fn) : 77\n"
          ]
        }
      ],
      "source": [
        "testing_norm = list_norm(testing_data, '')\n",
        "test_cases, labels = zip(*testing_norm)\n",
        "myClassifer = WordsListClassifier(bag_of_training_pos, bag_of_training_neg)\n",
        "prediction = myClassifer.classify_many(test_cases)\n",
        "myClassifer = ConfusionMatrix(prediction, labels)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "NJhRGovu50Ry",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 81
        },
        "outputId": "4d4e536c-75e0-41ea-932c-94922cf11b05"
      },
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "   Accuracy  Precision    Recall   F1Score\n",
              "0  0.653333   0.629944  0.743333  0.681957"
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-6418cc39-0e6a-406d-85f0-3d932f765f10\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Accuracy</th>\n",
              "      <th>Precision</th>\n",
              "      <th>Recall</th>\n",
              "      <th>F1Score</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>0.653333</td>\n",
              "      <td>0.629944</td>\n",
              "      <td>0.743333</td>\n",
              "      <td>0.681957</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-6418cc39-0e6a-406d-85f0-3d932f765f10')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "      \n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-6418cc39-0e6a-406d-85f0-3d932f765f10 button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-6418cc39-0e6a-406d-85f0-3d932f765f10');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n",
              "  "
            ]
          },
          "metadata": {}
        }
      ],
      "source": [
        "metrics_wordlist_classifier = {\"Accuracy\" : myClassifer.accuracy(), \"Precision\" :myClassifer.precision(), \"Recall\" : myClassifer.recall(), \"F1Score\" : myClassifer.f1_score()}\n",
        "table = pd.DataFrame(metrics_wordlist_classifier, index = [0])\n",
        "display(table)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "0gHYwj0i50Ry"
      },
      "source": [
        "Accuracy is a measurement for the proportion classified items in the data. However, it is not the best metric to fully evaluate a classifier.This is because accuracy does not work well with imbalanced data, this is data that has higher number of items on one argument and fewer in the other. For instance, if the number of positive reviews were to be 70% of the data and negative reviews 30% and the classifier fails to predict the negative reviews, it's accuracy would still be 70%, but that's not representing the whole data. In bigger scenarios, accuracy will not be a good choice of evaluating a classifier as there is a very low likelihood of a data set being split in perfect half. But in the case that somehow your data is split 50%, accuracy would be a great measure of evaluation, as it will show a true value for your data, allowing you to make better decisions based on the value."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "LIS9UpmJNEAp"
      },
      "source": [
        "4) \n",
        "a)  **Construct** a Naive Bayes classifier (e.g., from NLTK).\n",
        "\n",
        "b)  **Compare** the performance of your word list classifier with the Naive Bayes classifier.  **Discuss** your results. \n",
        "\n",
        "[12.5\\%]"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "hG4DSeqD50Rz",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "b811b93c-b1b9-4985-aeb4-90f3c9cae209"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "True positive (tp) : 294\n",
            "False positive (fp) : 158\n",
            "True Negative (tn) : 142\n",
            "False negative (fn) : 6\n"
          ]
        }
      ],
      "source": [
        "from nltk.classify import NaiveBayesClassifier\n",
        "naive_training_data = list_norm(training_data, '')\n",
        "naive_class_from_nltk = NaiveBayesClassifier.train(naive_training_data)\n",
        "matrix_for_NB = ConfusionMatrix(naive_class_from_nltk.classify_many(test_cases), labels ,classes=('pos','neg'))\n",
        "metrics_NB_classifier = {\"Accuracy\" : matrix_for_NB.accuracy(), \"Precision\" : matrix_for_NB.precision(), \"Recall\" : matrix_for_NB.recall(), \"F1 score\" : matrix_for_NB.f1_score()}\n",
        "table2 = table.append(metrics_NB_classifier, True)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "56_oD8ET50Rz",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 112
        },
        "outputId": "e1ea4788-db65-4215-cbd5-8b539764c451"
      },
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "   Accuracy  Precision    Recall   F1Score  F1 score\n",
              "0  0.653333   0.629944  0.743333  0.681957       NaN\n",
              "1  0.726667   0.650442  0.980000       NaN  0.781915"
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-6a53c808-a26f-4103-a027-e37084bb5a9f\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Accuracy</th>\n",
              "      <th>Precision</th>\n",
              "      <th>Recall</th>\n",
              "      <th>F1Score</th>\n",
              "      <th>F1 score</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>0.653333</td>\n",
              "      <td>0.629944</td>\n",
              "      <td>0.743333</td>\n",
              "      <td>0.681957</td>\n",
              "      <td>NaN</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>0.726667</td>\n",
              "      <td>0.650442</td>\n",
              "      <td>0.980000</td>\n",
              "      <td>NaN</td>\n",
              "      <td>0.781915</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-6a53c808-a26f-4103-a027-e37084bb5a9f')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "      \n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-6a53c808-a26f-4103-a027-e37084bb5a9f button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-6a53c808-a26f-4103-a027-e37084bb5a9f');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n",
              "  "
            ]
          },
          "metadata": {}
        }
      ],
      "source": [
        "display(table2)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "xfuYer9U50Rz"
      },
      "source": [
        "There is a difference in the values of all atributes for both classifiers. The Naive Bayes Classifer seems to be the better classifier than my word classifier on unseen documents as it produced higher precision and recall values. The higher recall means that the Naive Bayes Classifier predicted correctly a greater number of +ve documents in proportion than my classifier or in other words less false negaties were detected by the NB Classifier. The precision value shows the proportion of +ve predictions that are correct, meaning less false positives were detected by the NB classifier, making it better than my classifier. The Accuracy and F1 score of NB classifier is also higher, these are clear values that NB's Classifier is better as the accuracy shows proportion of items classified correctly. The F1 score shows the harmonic mean between precision and recall, Since NB's classifier has higher precision and recall than my classifier we can automatically infer that the F1 score for NB's classifier is higher, making it the better performing classifier. "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "LGDXaVDqOSfY"
      },
      "source": [
        "5) \n",
        "a) Design and **carry out an experiment** into the impact of the **length of the wordlists** on the wordlist classifier.  Make sure you **describe** design decisions in your experiment, include a **graph** of your results and **discuss** your conclusions. \n",
        "\n",
        "b) Would you **recommend** a wordlist classifier or a Naive Bayes classifier for future work in this area?  **Justify** your answer.\n",
        "\n",
        "[25\\%]\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "5PVBBPhh50R0"
      },
      "source": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "9tBhqZtw50R1"
      },
      "outputs": [],
      "source": [
        "metrics = []\n",
        "for i in range(1, 10, 100):\n",
        "    bag_of_training_pos_ = most_frequent_words(pos_freq_dist, neg_freq_dist, i)\n",
        "    bag_of_training_neg_ = most_frequent_words(neg_freq_dist, pos_freq_dist, i)\n",
        "    bags_classified = WordsListClassifier(bag_of_training_pos_, bag_of_training_neg_)\n",
        "    predic = bags_classified.classify_many(test_cases)\n",
        "    graph = ConfusionMatrix(predic, labels, display = False)\n",
        "    metrics.append([i, (graph.accuracy(), graph.precision(), graph.recall(), graph.f1_score())])"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "J1ozKYMa50R1",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 796
        },
        "outputId": "6e48084f-060a-424a-c11b-fc00b43544db"
      },
      "outputs": [
        {
          "output_type": "error",
          "ename": "TypeError",
          "evalue": "ignored",
          "traceback": [
            "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
            "\u001b[0;31mTypeError\u001b[0m                                 Traceback (most recent call last)",
            "\u001b[0;32m<ipython-input-73-48416312f61f>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[1;32m      1\u001b[0m \u001b[0mfig\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0max\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mplt\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0msubplots\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;36m1\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;36m1\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mfigsize\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m(\u001b[0m\u001b[0;36m14\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;36m10\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 2\u001b[0;31m \u001b[0max\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m1\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;36m0\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mplot\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mi\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mi\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mmetrics\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m[\u001b[0m\u001b[0mi\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m1\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mi\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mmetrics\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mc\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m'blue'\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m      3\u001b[0m \u001b[0max\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m1\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;36m0\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mset_title\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'Impact of the length of the wordlists on the wordlist classifier'\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      4\u001b[0m \u001b[0max\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m1\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;36m0\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mset\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mylabel\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m'Accuracy'\u001b[0m \u001b[0;34m,\u001b[0m\u001b[0mxlabel\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m'Length of Wordlist'\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;31mTypeError\u001b[0m: 'AxesSubplot' object is not subscriptable"
          ]
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 1008x720 with 1 Axes>"
            ],
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAzoAAAJDCAYAAADD62EJAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAXP0lEQVR4nO3dX6jn913n8de7GaNQawVnFiQzMQGnW2er0O4hVnphod1lkouZC11JoGgldG424q5FiChV4lUVFYT4ZxZLVbAx9kIOOJIFjRTElEypG0xK5BDdZqKQWGNuShuz+/bi91NOj5M53znzO2eadx4PGDjf7+9zfr/3xYcz85zv7/c91d0BAACY5C03ewAAAIBNEzoAAMA4QgcAABhH6AAAAOMIHQAAYByhAwAAjLNv6FTVJ6rqxar6q9d5vKrqV6tqp6qeqqr3bH5MAACA5ZZc0flkkrPXePzuJKfXfy4k+fUbHwsAAODg9g2d7v5Mkn+8xpLzSX6nV55I8q1V9e2bGhAAAOB6beIzOrcleX7X8ZX1OQAAgJvi2FG+WFVdyOrtbXnrW9/6n9/5znce5csDAABvIJ/73Of+obtPHOR7NxE6LyQ5tev45Prcv9PdF5NcTJKtra2+fPnyBl4eAACYqKr+70G/dxNvXdtO8sPru6+9N8kr3f33G3heAACAA9n3ik5VfSrJ+5Mcr6orSX42yTckSXf/RpJLSe5JspPky0l+9LCGBQAAWGLf0Onu+/Z5vJP8941NBAAAcIM28dY1AACArytCBwAAGEfoAAAA4wgdAABgHKEDAACMI3QAAIBxhA4AADCO0AEAAMYROgAAwDhCBwAAGEfoAAAA4wgdAABgHKEDAACMI3QAAIBxhA4AADCO0AEAAMYROgAAwDhCBwAAGEfoAAAA4wgdAABgHKEDAACMI3QAAIBxhA4AADCO0AEAAMYROgAAwDhCBwAAGEfoAAAA4wgdAABgHKEDAACMI3QAAIBxhA4AADCO0AEAAMYROgAAwDhCBwAAGEfoAAAA4wgdAABgHKEDAACMI3QAAIBxhA4AADCO0AEAAMYROgAAwDhCBwAAGEfoAAAA4wgdAABgHKEDAACMI3QAAIBxhA4AADCO0AEAAMYROgAAwDhCBwAAGEfoAAAA4wgdAABgHKEDAACMI3QAAIBxhA4AADCO0AEAAMYROgAAwDhCBwAAGEfoAAAA4wgdAABgHKEDAACMI3QAAIBxhA4AADCO0AEAAMYROgAAwDhCBwAAGEfoAAAA4wgdAABgHKEDAACMI3QAAIBxhA4AADCO0AEAAMYROgAAwDhCBwAAGEfoAAAA4wgdAABgHKEDAACMI3QAAIBxhA4AADCO0AEAAMYROgAAwDhCBwAAGEfoAAAA4wgdAABgHKEDAACMI3QAAIBxhA4AADCO0AEAAMYROgAAwDhCBwAAGEfoAAAA4wgdAABgHKEDAACMI3QAAIBxhA4AADCO0AEAAMYROgAAwDhCBwAAGEfoAAAA4wgdAABgnEWhU1Vnq+rZqtqpqgev8vjtVfV4VX2+qp6qqns2PyoAAMAy+4ZOVd2S5OEkdyc5k+S+qjqzZ9nPJHm0u9+d5N4kv7bpQQEAAJZackXnriQ73f1cd7+a5JEk5/es6STfsv767Un+bnMjAgAAXJ9jC9bcluT5XcdXknzvnjU/l+R/V9WPJXlrkg9uZDoAAIAD2NTNCO5L8snuPpnkniS/W1X/7rmr6kJVXa6qyy+99NKGXhoAAOBrLQmdF5Kc2nV8cn1ut/uTPJok3f0XSb4pyfG9T9TdF7t7q7u3Tpw4cbCJAQAA9rEkdJ5Mcrqq7qyqW7O62cD2njVfTPKBJKmq78oqdFyyAQAAbop9Q6e7X0vyQJLHknwhq7urPV1VD1XVufWyjyb5SFX9nySfSvLh7u7DGhoAAOBaltyMIN19KcmlPec+tuvrZ5K8b7OjAQAAHMymbkYAAADwdUPoAAAA4wgdAABgHKEDAACMI3QAAIBxhA4AADCO0AEAAMYROgAAwDhCBwAAGEfoAAAA4wgdAABgHKEDAACMI3QAAIBxhA4AADCO0AEAAMYROgAAwDhCBwAAGEfoAAAA4wgdAABgHKEDAACMI3QAAIBxhA4AADCO0AEAAMYROgAAwDhCBwAAGEfoAAAA4wgdAABgHKEDAACMI3QAAIBxhA4AADCO0AEAAMYROgAAwDhCBwAAGEfoAAAA4wgdAABgHKEDAACMI3QAAIBxhA4AADCO0AEAAMYROgAAwDhCBwAAGEfoAAAA4wgdAABgHKEDAACMI3QAAIBxhA4AADCO0AEAAMYROgAAwDhCBwAAGEfoAAAA4wgdAABgHKEDAACMI3QAAIBxhA4AADCO0AEAAMYROgAAwDhCBwAAGEfoAAAA4wgdAABgHKEDAACMI3QAAIBxhA4AADCO0AEAAMYROgAAwDhCBwAAGEfoAAAA4wgdAABgHKEDAACMI3QAAIBxhA4AADCO0AEAAMYROgAAwDhCBwAAGEfoAAAA4wgdAABgHKEDAACMI3QAAIBxhA4AADCO0AEAAMYROgAAwDhCBwAAGEfoAAAA4wgdAABgHKEDAACMI3QAAIBxhA4AADCO0AEAAMYROgAAwDhCBwAAGEfoAAAA4wgdAABgHKEDAACMI3QAAIBxhA4AADCO0AEAAMYROgAAwDhCBwAAGEfoAAAA4wgdAABgHKEDAACMsyh0qupsVT1bVTtV9eDrrPmhqnqmqp6uqt/b7JgAAADLHdtvQVXdkuThJP8lyZUkT1bVdnc/s2vN6SQ/leR93f1yVf2HwxoYAABgP0uu6NyVZKe7n+vuV5M8kuT8njUfSfJwd7+cJN394mbHBAAAWG5J6NyW5Pldx1fW53Z7R5J3VNWfV9UTVXV2UwMCAABcr33funYdz3M6yfuTnEzymar67u7+p92LqupCkgtJcvvtt2/opQEAAL7Wkis6LyQ5tev45PrcbleSbHf3P3f33yT566zC52t098Xu3ururRMnThx0ZgAAgGtaEjpPJjldVXdW1a1J7k2yvWfNH2Z1NSdVdTyrt7I9t8E5AQAAFts3dLr7tSQPJHksyReSPNrdT1fVQ1V1br3ssSRfqqpnkjye5Ce7+0uHNTQAAMC1VHfflBfe2trqy5cv35TXBgAAvv5V1ee6e+sg37voF4YCAAC8kQgdAABgHKEDAACMI3QAAIBxhA4AADCO0AEAAMYROgAAwDhCBwAAGEfoAAAA4wgdAABgHKEDAACMI3QAAIBxhA4AADCO0AEAAMYROgAAwDhCBwAAGEfoAAAA4wgdAABgHKEDAACMI3QAAIBxhA4AADCO0AEAAMYROgAAwDhCBwAAGEfoAAAA4wgdAABgHKEDAACMI3QAAIBxhA4AADCO0AEAAMYROgAAwDhCBwAAGEfoAAAA4wgdAABgHKEDAACMI3QAAIBxhA4AADCO0AEAAMYROgAAwDhCBwAAGEfoAAAA4wgdAABgHKEDAACMI3QAAIBxhA4AADCO0AEAAMYROgAAwDhCBwAAGEfoAAAA4wgdAABgHKEDAACMI3QAAIBxhA4AADCO0AEAAMYROgAAwDhCBwAAGEfoAAAA4wgdAABgHKEDAACMI3QAAIBxhA4AADCO0AEAAMYROgAAwDhCBwAAGEfoAAAA4wgdAABgHKEDAACMI3QAAIBxhA4AADCO0AEAAMYROgAAwDhCBwAAGEfoAAAA4wgdAABgHKEDAACMI3QAAIBxhA4AADCO0AEAAMYROgAAwDhCBwAAGEfoAAAA4wgdAABgHKEDAACMI3QAAIBxhA4AADCO0AEAAMYROgAAwDhCBwAAGEfoAAAA4wgdAABgHKEDAACMI3QAAIBxhA4AADCO0AEAAMYROgAAwDhCBwAAGEfoAAAA4wgdAABgHKEDAACMsyh0qupsVT1bVTtV9eA11v1AVXVVbW1uRAAAgOuzb+hU1S1JHk5yd5IzSe6rqjNXWfe2JD+e5LObHhIAAOB6LLmic1eSne5+rrtfTfJIkvNXWffzST6e5CsbnA8AAOC6LQmd25I8v+v4yvrcv6mq9yQ51d1/tMHZAAAADuSGb0ZQVW9J8stJPrpg7YWqulxVl1966aUbfWkAAICrWhI6LyQ5tev45Prcv3pbkncl+bOq+tsk702yfbUbEnT3xe7e6u6tEydOHHxqAACAa1gSOk8mOV1Vd1bVrUnuTbL9rw929yvdfby77+juO5I8keRcd18+lIkBAAD2sW/odPdrSR5I8liSLyR5tLufrqqHqurcYQ8IAABwvY4tWdTdl5Jc2nPuY6+z9v03PhYAAMDB3fDNCAAAAL7eCB0AAGAcoQMAAIwjdAAAgHGEDgAAMI7QAQAAxhE6AADAOEIHAAAYR+gAAADjCB0AAGAcoQMAAIwjdAAAgHGEDgAAMI7QAQAAxhE6AADAOEIHAAAYR+gAAADjCB0AAGAcoQMAAIwjdAAAgHGEDgAAMI7QAQAAxhE6AADAOEIHAAAYR+gAAADjCB0AAGAcoQMAAIwjdAAAgHGEDgAAMI7QAQAAxhE6AADAOEIHAAAYR+gAAADjCB0AAGAcoQMAAIwjdAAAgHGEDgAAMI7QAQAAxhE6AADAOEIHAAAYR+gAAADjCB0AAGAcoQMAAIwjdAAAgHGEDgAAMI7QAQAAxhE6AADAOEIHAAAYR+gAAADjCB0AAGAcoQMAAIwjdAAAgHGEDgAAMI7QAQAAxhE6AADAOEIHAAAYR+gAAADjCB0AAGAcoQMAAIwjdAAAgHGEDgAAMI7QAQAAxhE6AADAOEIHAAAYR+gAAADjCB0AAGAcoQMAAIwjdAAAgHGEDgAAMI7QAQAAxhE6AADAOEIHAAAYR+gAAADjCB0AAGAcoQMAAIwjdAAAgHGEDgAAMI7QAQAAxhE6AADAOEIHAAAYR+gAAADjCB0AAGAcoQMAAIwjdAAAgHGEDgAAMI7QAQAAxhE6AADAOEIHAAAYR+gAAADjCB0AAGAcoQMAAIwjdAAAgHGEDgAAMI7QAQAAxhE6AADAOEIHAAAYR+gAAADjCB0AAGAcoQMAAIwjdAAAgHEWhU5Vna2qZ6tqp6oevMrjP1FVz1TVU1X1J1X1HZsfFQAAYJl9Q6eqbknycJK7k5xJcl9Vndmz7PNJtrr7e5J8OskvbHpQAACApZZc0bkryU53P9fdryZ5JMn53Qu6+/Hu/vL68IkkJzc7JgAAwHJLQue2JM/vOr6yPvd67k/yxzcyFAAAwI04tsknq6oPJdlK8v2v8/iFJBeS5Pbbb9/kSwMAAPybJVd0XkhyatfxyfW5r1FVH0zy00nOdfdXr/ZE3X2xu7e6e+vEiRMHmRcAAGBfS0LnySSnq+rOqro1yb1JtncvqKp3J/nNrCLnxc2PCQAAsNy+odPdryV5IMljSb6Q5NHufrqqHqqqc+tlv5jkm5P8QVX9ZVVtv87TAQAAHLpFn9Hp7ktJLu0597FdX39ww3MBAAAc2KJfGAoAAPBGInQAAIBxhA4AADCO0AEAAMYROgAAwDhCBwAAGEfoAAAA4wgdAABgHKEDAACMI3QAAIBxhA4AADCO0AEAAMYROgAAwDhCBwAAGEfoAAAA4wgdAABgHKEDAACMI3QAAIBxhA4AADCO0AEAAMYROgAAwDhCBwAAGEfoAAAA4wgdAABgHKEDAACMI3QAAIBxhA4AADCO0AEAAMYROgAAwDhCBwAAGEfoAAAA4wgdAABgHKEDAACMI3QAAIBxhA4AADCO0AEAAMYROgAAwDhCBwAAGEfoAAAA4wgdAABgHKEDAACMI3QAAIBxhA4AADCO0AEAAMYROgAAwDhCBwAAGEfoAAAA4wgdAABgHKEDAACMI3QAAIBxhA4AADCO0AEAAMYROgAAwDhCBwAAGEfoAAAA4wgdAABgHKEDAACMI3QAAIBxhA4AADCO0AEAAMYROgAAwDhCBwAAGEfoAAAA4wgdAABgHKEDAACMI3QAAIBxhA4AADCO0AEAAMYROgAAwDhCBwAAGEfoAAAA4wgdAABgHKEDAACMI3QAAIBxhA4AADCO0AEAAMYROgAAwDhCBwAAGEfoAAAA4wgdAABgHKEDAACMI3QAAIBxhA4AADCO0AEAAMYROgAAwDhCBwAAGEfoAAAA4wgdAABgHKEDAACMI3QAAIBxhA4AADCO0AEAAMYROgAAwDhCBwAAGEfoAAAA4wgdAABgHKEDAACMI3QAAIBxhA4AADCO0AEAAMZZFDpVdbaqnq2qnap68CqPf2NV/f768c9W1R2bHhQAAGCpfUOnqm5J8nCSu5OcSXJfVZ3Zs+z+JC9393cm+ZUkH9/0oAAAAEstuaJzV5Kd7n6uu19N8kiS83vWnE/y2+uvP53kA1VVmxsTAABguSWhc1uS53cdX1mfu+qa7n4tyStJvm0TAwIAAFyvY0f5YlV1IcmF9eFXq+qvjvL1eVM7nuQfbvYQvKnYcxwl+42jZL9xlP7jQb9xSei8kOTUruOT63NXW3Olqo4leXuSL+19ou6+mORiklTV5e7eOsjQcL3sN46aPcdRst84SvYbR6mqLh/0e5e8de3JJKer6s6qujXJvUm296zZTvIj669/MMmfdncfdCgAAIAbse8Vne5+raoeSPJYkluSfKK7n66qh5Jc7u7tJL+V5HeraifJP2YVQwAAADfFos/odPelJJf2nPvYrq+/kuS/XedrX7zO9XAj7DeOmj3HUbLfOEr2G0fpwPutvMMMAACYZslndAAAAN5QDj10qupsVT1bVTtV9eBVHv/Gqvr99eOfrao7Dnsm5lqw336iqp6pqqeq6k+q6jtuxpzMsN9+27XuB6qqq8pdijiwJfutqn5o/TPu6ar6vaOekVkW/J16e1U9XlWfX/+9es/NmJM3vqr6RFW9+Hq/eqZWfnW9F5+qqvcsed5DDZ2quiXJw0nuTnImyX1VdWbPsvuTvNzd35nkV5J8/DBnYq6F++3zSba6+3uSfDrJLxztlEyxcL+lqt6W5MeTfPZoJ2SSJfutqk4n+akk7+vu/5Tkfxz5oIyx8GfczyR5tLvfndWNqH7taKdkkE8mOXuNx+9Ocnr950KSX1/ypId9ReeuJDvd/Vx3v5rkkSTn96w5n+S3119/OskHqqoOeS5m2ne/dffj3f3l9eETWf1eKDiIJT/fkuTns/oPnK8c5XCMs2S/fSTJw939cpJ094tHPCOzLNlzneRb1l+/PcnfHeF8DNLdn8nqzs2v53yS3+mVJ5J8a1V9+37Pe9ihc1uS53cdX1mfu+qa7n4tyStJvu2Q52KmJfttt/uT/PGhTsRk++639aX1U939R0c5GCMt+fn2jiTvqKo/r6onqupa/zsK+1my534uyYeq6kpWd+f9saMZjTeh6/03XpKFt5eGaarqQ0m2knz/zZ6FmarqLUl+OcmHb/IovHkcy+ptHe/P6mr1Z6rqu7v7n27qVEx2X5JPdvcvVdX3ZfU7Fd/V3f//Zg8GyeFf0XkhyaldxyfX5666pqqOZXXp80uHPBczLdlvqaoPJvnpJOe6+6tHNBvz7Lff3pbkXUn+rKr+Nsl7k2y7IQEHtOTn25Uk2939z939N0n+OqvwgYNYsufuT/JoknT3XyT5piTHj2Q63mwW/Rtvr8MOnSeTnK6qO6vq1qw+qLa9Z812kh9Zf/2DSf60/XIfDmbf/VZV707ym1lFjvevcyOuud+6+5XuPt7dd3T3HVl9Juxcd1++OePyBrfk79M/zOpqTqrqeFZvZXvuKIdklCV77otJPpAkVfVdWYXOS0c6JW8W20l+eH33tfcmeaW7/36/bzrUt65192tV9UCSx5LckuQT3f10VT2U5HJ3byf5rawude5k9SGkew9zJuZauN9+Mck3J/mD9T0vvtjd527a0LxhLdxvsBEL99tjSf5rVT2T5P8l+cnu9g4JDmThnvtokv9VVf8zqxsTfNh/VnMQVfWprP6j5vj6M18/m+QbkqS7fyOrz4Ddk2QnyZeT/Oii57UfAQCAaQ79F4YCAAAcNaEDAACMI3QAAIBxhA4AADCO0AEAAMYROgAAwDhCBwAAGEfoAAAA4/wL6IOuQsf567sAAAAASUVORK5CYII=\n"
          },
          "metadata": {
            "needs_background": "light"
          }
        }
      ],
      "source": [
        "fig, ax = plt.subplots(1, 1, figsize = (14, 10))\n",
        "ax[1, 0].plot([i[0] for i in metrics], [i[1][0] for i in metrics], c = 'blue')\n",
        "ax[1, 0].set_title('Impact of the length of the wordlists on the wordlist classifier')\n",
        "ax[1, 0].set(ylabel = 'Accuracy' ,xlabel = 'Length of Wordlist')\n",
        "ax[1, 0].set(ylabel = 'Precision' ,xlabel = 'Length of Wordlist')\n",
        "ax[1, 0].set(ylabel = 'Recall' ,xlabel = 'Length of Wordlist')\n",
        "ax[1, 0].set(ylabel = 'F1Score' ,xlabel = 'Length of Wordlist')"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "vzN_HIuC50R1"
      },
      "outputs": [],
      "source": [
        "plt.show()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "nYZVd15L50R2"
      },
      "source": [
        "Both Classifiers have their disadvantages and advantages. The wordlist classifier, does its job of classifying the data in a document, but it does not do it as accurately as Naive Bayes Classifier. The better the accuracy the better the decision making, however, as previously explored it is not a good metric to evaluate on a classifier due to imbalances in the data. A disadvantage of the NB classifier is that it assumes that all features are independent, which is not always the case in the real world, this makes it harder to use if there are some set of independent events. This can be disregarded as the other metric values that NB's classifier produces come with a lot of importance. It provides higher precision and recall values, making the overall F1 score higher, This allows me to make better decisions. Although I do not have a valid experiment to explain the impact of length of word lists for the word list classifier, I have learnt that, the accuracy, precision, recall and F1 score of the classifier would get lower over the length of wordlists increases, as data set increases, there is a higher chance of getting false postives and negatives not predicted. On the other hand, NB's classifier would perform better as there is less of an impact if the data set increases. In conclusion, I would be using a NB Classifier when working in this area in the future, as this classifier performs better."
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "from google.colab import drive\n",
        "drive.mount('/content/drive')"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "WMN_tUrBOqUq",
        "outputId": "e6db77ae-2820-4221-f370-b4ecde0bc6e8"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Drive already mounted at /content/drive; to attempt to forcibly remount, call drive.mount(\"/content/drive\", force_remount=True).\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "34rdlS_iPov6",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "8e5d2674-d968-497e-a2d4-22c6f22e3909"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Submission length is 863\n"
          ]
        }
      ],
      "source": [
        "##This code will word count all of the markdown cells in the notebook saved at filepath\n",
        "##Running it before providing any answers shows that the questions have a word count of 437\n",
        "\n",
        "import io\n",
        "from nbformat import current\n",
        "\n",
        "filepath=\"/content/drive/My Drive/NLassignment2022.ipynb\"\n",
        "#filepath=\"NLassignment2021.ipynb\"\n",
        "question_count=437\n",
        "\n",
        "with io.open(filepath, 'r', encoding='utf-8') as f:\n",
        "    nb = current.read(f, 'json')\n",
        "\n",
        "word_count = 0\n",
        "for cell in nb.worksheets[0].cells:\n",
        "    if cell.cell_type == \"markdown\":\n",
        "        word_count += len(cell['source'].replace('#', '').lstrip().split(' '))\n",
        "print(\"Submission length is {}\".format(word_count-question_count))"
      ]
    }
  ],
  "metadata": {
    "colab": {
      "provenance": [],
      "collapsed_sections": []
    },
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.7.4"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}