Summary.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This note book summarizes the results of using logistic regression to determine the whether a passenger on board of the Titanic survives or not, given their social economical class, age, sex, etc. Analysis show a passengers gender, age, and social economical class are the three most important factors (in decending order). While Cabin is well correlated to Survivial outcome, it is missing for over 70% of the data thus making it unsuitable in a generalized model. The decision boundary between survived and not survived is potentially highly nonlinear due to the nature of the data. A simple linear function (with the inputs as is) gives 79% accuracy on a validation set that consists 40% of the total training data. A 2nd order polynormial function can improve the accuracy to around 84%. In the model, missing age was filled by mapping the title (extracted from name) to the median age in the title group. The qudratic logistic regression model is limited in its accuracy due to discrete nature of most of the features."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 81,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Survived</th>\n",
       "      <th>Pclass</th>\n",
       "      <th>Age</th>\n",
       "      <th>Sex</th>\n",
       "      <th>SibSp</th>\n",
       "      <th>Parch</th>\n",
       "      <th>Fare</th>\n",
       "      <th>Embarked</th>\n",
       "      <th>Name</th>\n",
       "      <th>Cabin</th>\n",
       "      <th>Ticket</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>22.0</td>\n",
       "      <td>-1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>7.2500</td>\n",
       "      <td>1</td>\n",
       "      <td>Braund, Mr. Owen Harris</td>\n",
       "      <td>NaN</td>\n",
       "      <td>A/5 21171</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>38.0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>71.2833</td>\n",
       "      <td>2</td>\n",
       "      <td>Cumings, Mrs. John Bradley (Florence Briggs Th...</td>\n",
       "      <td>C85</td>\n",
       "      <td>PC 17599</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>26.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>7.9250</td>\n",
       "      <td>1</td>\n",
       "      <td>Heikkinen, Miss. Laina</td>\n",
       "      <td>NaN</td>\n",
       "      <td>STON/O2. 3101282</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>35.0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>53.1000</td>\n",
       "      <td>1</td>\n",
       "      <td>Futrelle, Mrs. Jacques Heath (Lily May Peel)</td>\n",
       "      <td>C123</td>\n",
       "      <td>113803</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>35.0</td>\n",
       "      <td>-1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>8.0500</td>\n",
       "      <td>1</td>\n",
       "      <td>Allen, Mr. William Henry</td>\n",
       "      <td>NaN</td>\n",
       "      <td>373450</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Survived  Pclass   Age  Sex  SibSp  Parch     Fare  Embarked  \\\n",
       "0         0       3  22.0   -1      1      0   7.2500         1   \n",
       "1         1       1  38.0    1      1      0  71.2833         2   \n",
       "2         1       3  26.0    1      0      0   7.9250         1   \n",
       "3         1       1  35.0    1      1      0  53.1000         1   \n",
       "4         0       3  35.0   -1      0      0   8.0500         1   \n",
       "\n",
       "                                                Name Cabin            Ticket  \n",
       "0                            Braund, Mr. Owen Harris   NaN         A/5 21171  \n",
       "1  Cumings, Mrs. John Bradley (Florence Briggs Th...   C85          PC 17599  \n",
       "2                             Heikkinen, Miss. Laina   NaN  STON/O2. 3101282  \n",
       "3       Futrelle, Mrs. Jacques Heath (Lily May Peel)  C123            113803  \n",
       "4                           Allen, Mr. William Henry   NaN            373450  "
      ]
     },
     "execution_count": 81,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import scipy.optimize as scop\n",
    "import titanic_project as titan\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "features = ['Survived', 'Pclass', 'Age', 'Sex', 'SibSp', 'Parch', 'Fare','Embarked','Name', 'Cabin', 'Ticket']\n",
    "datain = titan.readdata('train.csv', 0.6, 0.6, features)\n",
    "trainset = datain['train']\n",
    "trainset.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 82,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Survived    0.000000\n",
       "Pclass      0.000000\n",
       "Age         0.207477\n",
       "Sex         0.000000\n",
       "SibSp       0.000000\n",
       "Parch       0.000000\n",
       "Fare        0.000000\n",
       "Embarked    0.000000\n",
       "Name        0.000000\n",
       "Cabin       0.773832\n",
       "Ticket      0.000000\n",
       "dtype: float64"
      ]
     },
     "execution_count": 82,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "trainset.isnull().sum() / trainset.shape[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Majority of the Cabin information is missing and about 1/5 of the passengers are missing Age information. The ticket number seems to have many different formats and difficult to generalize so I leave it out for now. I extract the Cabin class (first letter) from the Cabin information first and I will discuss about extracting information from Name later. \n",
    "\n",
    "The code below extracts the first letter of the Cabin number and turns it into an integer. All NaNs are assigned with a same number."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 83,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Survived</th>\n",
       "      <th>Pclass</th>\n",
       "      <th>Age</th>\n",
       "      <th>Sex</th>\n",
       "      <th>SibSp</th>\n",
       "      <th>Parch</th>\n",
       "      <th>Fare</th>\n",
       "      <th>Embarked</th>\n",
       "      <th>cabclass</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Survived</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>-0.270099</td>\n",
       "      <td>-0.094232</td>\n",
       "      <td>0.571729</td>\n",
       "      <td>-0.046940</td>\n",
       "      <td>0.087018</td>\n",
       "      <td>0.205579</td>\n",
       "      <td>0.108864</td>\n",
       "      <td>0.242205</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Pclass</th>\n",
       "      <td>-0.270099</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>-0.359838</td>\n",
       "      <td>-0.115367</td>\n",
       "      <td>0.101060</td>\n",
       "      <td>0.001775</td>\n",
       "      <td>-0.586986</td>\n",
       "      <td>0.054265</td>\n",
       "      <td>-0.529018</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Age</th>\n",
       "      <td>-0.094232</td>\n",
       "      <td>-0.359838</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>-0.091064</td>\n",
       "      <td>-0.353861</td>\n",
       "      <td>-0.238634</td>\n",
       "      <td>0.083645</td>\n",
       "      <td>0.018236</td>\n",
       "      <td>0.137889</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Sex</th>\n",
       "      <td>0.571729</td>\n",
       "      <td>-0.115367</td>\n",
       "      <td>-0.091064</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.070997</td>\n",
       "      <td>0.168163</td>\n",
       "      <td>0.180971</td>\n",
       "      <td>0.115029</td>\n",
       "      <td>0.111086</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>SibSp</th>\n",
       "      <td>-0.046940</td>\n",
       "      <td>0.101060</td>\n",
       "      <td>-0.353861</td>\n",
       "      <td>0.070997</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.440383</td>\n",
       "      <td>0.181967</td>\n",
       "      <td>-0.041111</td>\n",
       "      <td>-0.065250</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Parch</th>\n",
       "      <td>0.087018</td>\n",
       "      <td>0.001775</td>\n",
       "      <td>-0.238634</td>\n",
       "      <td>0.168163</td>\n",
       "      <td>0.440383</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.260505</td>\n",
       "      <td>-0.124586</td>\n",
       "      <td>0.054288</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Fare</th>\n",
       "      <td>0.205579</td>\n",
       "      <td>-0.586986</td>\n",
       "      <td>0.083645</td>\n",
       "      <td>0.180971</td>\n",
       "      <td>0.181967</td>\n",
       "      <td>0.260505</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.038452</td>\n",
       "      <td>0.325645</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Embarked</th>\n",
       "      <td>0.108864</td>\n",
       "      <td>0.054265</td>\n",
       "      <td>0.018236</td>\n",
       "      <td>0.115029</td>\n",
       "      <td>-0.041111</td>\n",
       "      <td>-0.124586</td>\n",
       "      <td>0.038452</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>-0.005776</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>cabclass</th>\n",
       "      <td>0.242205</td>\n",
       "      <td>-0.529018</td>\n",
       "      <td>0.137889</td>\n",
       "      <td>0.111086</td>\n",
       "      <td>-0.065250</td>\n",
       "      <td>0.054288</td>\n",
       "      <td>0.325645</td>\n",
       "      <td>-0.005776</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "          Survived    Pclass       Age       Sex     SibSp     Parch  \\\n",
       "Survived  1.000000 -0.270099 -0.094232  0.571729 -0.046940  0.087018   \n",
       "Pclass   -0.270099  1.000000 -0.359838 -0.115367  0.101060  0.001775   \n",
       "Age      -0.094232 -0.359838  1.000000 -0.091064 -0.353861 -0.238634   \n",
       "Sex       0.571729 -0.115367 -0.091064  1.000000  0.070997  0.168163   \n",
       "SibSp    -0.046940  0.101060 -0.353861  0.070997  1.000000  0.440383   \n",
       "Parch     0.087018  0.001775 -0.238634  0.168163  0.440383  1.000000   \n",
       "Fare      0.205579 -0.586986  0.083645  0.180971  0.181967  0.260505   \n",
       "Embarked  0.108864  0.054265  0.018236  0.115029 -0.041111 -0.124586   \n",
       "cabclass  0.242205 -0.529018  0.137889  0.111086 -0.065250  0.054288   \n",
       "\n",
       "              Fare  Embarked  cabclass  \n",
       "Survived  0.205579  0.108864  0.242205  \n",
       "Pclass   -0.586986  0.054265 -0.529018  \n",
       "Age       0.083645  0.018236  0.137889  \n",
       "Sex       0.180971  0.115029  0.111086  \n",
       "SibSp     0.181967 -0.041111 -0.065250  \n",
       "Parch     0.260505 -0.124586  0.054288  \n",
       "Fare      1.000000  0.038452  0.325645  \n",
       "Embarked  0.038452  1.000000 -0.005776  \n",
       "cabclass  0.325645 -0.005776  1.000000  "
      ]
     },
     "execution_count": 83,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cabinclass = trainset.Cabin.apply(lambda x: str(x)[0])\n",
    "trainset = trainset.assign(cabclass = cabinclass)\n",
    "ckey = cabinclass.unique()\n",
    "cnum = np.arange(0, ckey.size)\n",
    "ckey2num = dict(zip(ckey, cnum))\n",
    "trainset.cabclass = trainset.cabclass.map(ckey2num)\n",
    "trainset.corr()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The Cabin class seems to have non-trivial correlation to Survival outcome. The cabin class also seems to be closely correlated to Pclass (social economical class) and Fare. We look more closely in the summary below. Here cabclass = 0 is where there is no Cabin number on record. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 97,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th>count</th>\n",
       "      <th>mean</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Pclass</th>\n",
       "      <th>cabclass</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th rowspan=\"7\" valign=\"top\">1</th>\n",
       "      <th>0</th>\n",
       "      <td>24</td>\n",
       "      <td>0.500000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>42</td>\n",
       "      <td>0.523810</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>14</td>\n",
       "      <td>0.571429</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>14</td>\n",
       "      <td>0.785714</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>8</td>\n",
       "      <td>0.375000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>22</td>\n",
       "      <td>0.727273</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>1</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"4\" valign=\"top\">2</th>\n",
       "      <th>0</th>\n",
       "      <td>98</td>\n",
       "      <td>0.438776</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>4</td>\n",
       "      <td>0.750000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>7</td>\n",
       "      <td>0.857143</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"4\" valign=\"top\">3</th>\n",
       "      <th>0</th>\n",
       "      <td>292</td>\n",
       "      <td>0.263699</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>0.500000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>2</td>\n",
       "      <td>0.500000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                 count      mean\n",
       "Pclass cabclass                 \n",
       "1      0            24  0.500000\n",
       "       1            42  0.523810\n",
       "       2            14  0.571429\n",
       "       4            14  0.785714\n",
       "       5             8  0.375000\n",
       "       6            22  0.727273\n",
       "       8             1  0.000000\n",
       "2      0            98  0.438776\n",
       "       2             2  1.000000\n",
       "       4             4  0.750000\n",
       "       7             7  0.857143\n",
       "3      0           292  0.263699\n",
       "       2             1  1.000000\n",
       "       3             4  0.500000\n",
       "       7             2  0.500000"
      ]
     },
     "execution_count": 97,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "trainset.groupby(['Pclass', 'cabclass']).Survived.agg({'count', 'mean'})"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A few Cabin classes stands out, like class 4, or Cabin number with 'D', has a high survival rate. Most of the survival rate could be insignificant due to the small number of samples in each category. As a result, for the general model I am building below I leave Cabin out. In the future Cabin can be used to modify a  subset of the data, where there is statistically significant correlation between a certain cabin class and survival rate."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "features = ['Survived', 'Pclass', 'Age', 'Sex', 'SibSp', 'Parch', 'Fare','Embarked','Name']\n",
    "datain = titan.readdata('train.csv', 0.6, 0.6, features)\n",
    "datain['train'].describe()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Survived</th>\n",
       "      <th>Pclass</th>\n",
       "      <th>Age</th>\n",
       "      <th>Sex</th>\n",
       "      <th>SibSp</th>\n",
       "      <th>Parch</th>\n",
       "      <th>Fare</th>\n",
       "      <th>Embarked</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Survived</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>-0.285660</td>\n",
       "      <td>-0.094232</td>\n",
       "      <td>0.559736</td>\n",
       "      <td>-0.029968</td>\n",
       "      <td>0.101782</td>\n",
       "      <td>0.210668</td>\n",
       "      <td>0.074648</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Pclass</th>\n",
       "      <td>-0.285660</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>-0.359838</td>\n",
       "      <td>-0.140481</td>\n",
       "      <td>0.085795</td>\n",
       "      <td>0.008385</td>\n",
       "      <td>-0.587926</td>\n",
       "      <td>-0.094167</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Age</th>\n",
       "      <td>-0.094232</td>\n",
       "      <td>-0.359838</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>-0.091064</td>\n",
       "      <td>-0.353861</td>\n",
       "      <td>-0.238634</td>\n",
       "      <td>0.083645</td>\n",
       "      <td>0.018236</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Sex</th>\n",
       "      <td>0.559736</td>\n",
       "      <td>-0.140481</td>\n",
       "      <td>-0.091064</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.069462</td>\n",
       "      <td>0.167201</td>\n",
       "      <td>0.183197</td>\n",
       "      <td>0.061273</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>SibSp</th>\n",
       "      <td>-0.029968</td>\n",
       "      <td>0.085795</td>\n",
       "      <td>-0.353861</td>\n",
       "      <td>0.069462</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.414155</td>\n",
       "      <td>0.167069</td>\n",
       "      <td>0.021752</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Parch</th>\n",
       "      <td>0.101782</td>\n",
       "      <td>0.008385</td>\n",
       "      <td>-0.238634</td>\n",
       "      <td>0.167201</td>\n",
       "      <td>0.414155</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.252656</td>\n",
       "      <td>-0.072881</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Fare</th>\n",
       "      <td>0.210668</td>\n",
       "      <td>-0.587926</td>\n",
       "      <td>0.083645</td>\n",
       "      <td>0.183197</td>\n",
       "      <td>0.167069</td>\n",
       "      <td>0.252656</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.142913</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Embarked</th>\n",
       "      <td>0.074648</td>\n",
       "      <td>-0.094167</td>\n",
       "      <td>0.018236</td>\n",
       "      <td>0.061273</td>\n",
       "      <td>0.021752</td>\n",
       "      <td>-0.072881</td>\n",
       "      <td>0.142913</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "          Survived    Pclass       Age       Sex     SibSp     Parch  \\\n",
       "Survived  1.000000 -0.285660 -0.094232  0.559736 -0.029968  0.101782   \n",
       "Pclass   -0.285660  1.000000 -0.359838 -0.140481  0.085795  0.008385   \n",
       "Age      -0.094232 -0.359838  1.000000 -0.091064 -0.353861 -0.238634   \n",
       "Sex       0.559736 -0.140481 -0.091064  1.000000  0.069462  0.167201   \n",
       "SibSp    -0.029968  0.085795 -0.353861  0.069462  1.000000  0.414155   \n",
       "Parch     0.101782  0.008385 -0.238634  0.167201  0.414155  1.000000   \n",
       "Fare      0.210668 -0.587926  0.083645  0.183197  0.167069  0.252656   \n",
       "Embarked  0.074648 -0.094167  0.018236  0.061273  0.021752 -0.072881   \n",
       "\n",
       "              Fare  Embarked  \n",
       "Survived  0.210668  0.074648  \n",
       "Pclass   -0.587926 -0.094167  \n",
       "Age       0.083645  0.018236  \n",
       "Sex       0.183197  0.061273  \n",
       "SibSp     0.167069  0.021752  \n",
       "Parch     0.252656 -0.072881  \n",
       "Fare      1.000000  0.142913  \n",
       "Embarked  0.142913  1.000000  "
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "datain['train'].dropna(how='any').corr()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The correlation table shows that the most important features are the gender (Sex), social economical class (Pclass and to some extent Fare, those two features are closely correlated). Where the passenger embarked and the sibling spouse count doesn't matter too much. Age is not insignificant, however it is missing in 26% of the training records as shown here. All other features contains no NaN values in the training set."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Survived    0.000000\n",
       "Pclass      0.000000\n",
       "Age         0.261792\n",
       "Sex         0.000000\n",
       "SibSp       0.000000\n",
       "Parch       0.000000\n",
       "Fare        0.000000\n",
       "Embarked    0.000000\n",
       "Name        0.000000\n",
       "dtype: float64"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "datain['train'].isnull().sum() * 1.0 / datain['train'].count()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Given the potential important of Age I attempt to correlate it to other parameters to improve the final model. The correlation table shows that Age correlates well with Pclass, SibSp, and Parch. After examining the names I find the title in the names, e.g., Mr, could also be used to correlate to age because each title group seem to have a distinct median age."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "tmap = {' Mr':0, ' Mrs':1, ' Miss':2, ' Mme':1, ' Master':3}\n",
    "title = datain['train'].Name.apply(titan.name2title).map(tmap).fillna(4).astype(int)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Survived</th>\n",
       "      <th>Pclass</th>\n",
       "      <th>Age</th>\n",
       "      <th>Sex</th>\n",
       "      <th>SibSp</th>\n",
       "      <th>Parch</th>\n",
       "      <th>Fare</th>\n",
       "      <th>Embarked</th>\n",
       "      <th>title</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Survived</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>-0.270099</td>\n",
       "      <td>-0.094232</td>\n",
       "      <td>0.571729</td>\n",
       "      <td>-0.046940</td>\n",
       "      <td>0.087018</td>\n",
       "      <td>0.205579</td>\n",
       "      <td>0.108864</td>\n",
       "      <td>0.411510</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Pclass</th>\n",
       "      <td>-0.270099</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>-0.359838</td>\n",
       "      <td>-0.115367</td>\n",
       "      <td>0.101060</td>\n",
       "      <td>0.001775</td>\n",
       "      <td>-0.586986</td>\n",
       "      <td>0.054265</td>\n",
       "      <td>-0.051237</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Age</th>\n",
       "      <td>-0.094232</td>\n",
       "      <td>-0.359838</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>-0.091064</td>\n",
       "      <td>-0.353861</td>\n",
       "      <td>-0.238634</td>\n",
       "      <td>0.083645</td>\n",
       "      <td>0.018236</td>\n",
       "      <td>-0.314928</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Sex</th>\n",
       "      <td>0.571729</td>\n",
       "      <td>-0.115367</td>\n",
       "      <td>-0.091064</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.070997</td>\n",
       "      <td>0.168163</td>\n",
       "      <td>0.180971</td>\n",
       "      <td>0.115029</td>\n",
       "      <td>0.597094</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>SibSp</th>\n",
       "      <td>-0.046940</td>\n",
       "      <td>0.101060</td>\n",
       "      <td>-0.353861</td>\n",
       "      <td>0.070997</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.440383</td>\n",
       "      <td>0.181967</td>\n",
       "      <td>-0.041111</td>\n",
       "      <td>0.309132</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Parch</th>\n",
       "      <td>0.087018</td>\n",
       "      <td>0.001775</td>\n",
       "      <td>-0.238634</td>\n",
       "      <td>0.168163</td>\n",
       "      <td>0.440383</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.260505</td>\n",
       "      <td>-0.124586</td>\n",
       "      <td>0.261974</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Fare</th>\n",
       "      <td>0.205579</td>\n",
       "      <td>-0.586986</td>\n",
       "      <td>0.083645</td>\n",
       "      <td>0.180971</td>\n",
       "      <td>0.181967</td>\n",
       "      <td>0.260505</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.038452</td>\n",
       "      <td>0.147810</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Embarked</th>\n",
       "      <td>0.108864</td>\n",
       "      <td>0.054265</td>\n",
       "      <td>0.018236</td>\n",
       "      <td>0.115029</td>\n",
       "      <td>-0.041111</td>\n",
       "      <td>-0.124586</td>\n",
       "      <td>0.038452</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.103845</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>title</th>\n",
       "      <td>0.411510</td>\n",
       "      <td>-0.051237</td>\n",
       "      <td>-0.314928</td>\n",
       "      <td>0.597094</td>\n",
       "      <td>0.309132</td>\n",
       "      <td>0.261974</td>\n",
       "      <td>0.147810</td>\n",
       "      <td>0.103845</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "          Survived    Pclass       Age       Sex     SibSp     Parch  \\\n",
       "Survived  1.000000 -0.270099 -0.094232  0.571729 -0.046940  0.087018   \n",
       "Pclass   -0.270099  1.000000 -0.359838 -0.115367  0.101060  0.001775   \n",
       "Age      -0.094232 -0.359838  1.000000 -0.091064 -0.353861 -0.238634   \n",
       "Sex       0.571729 -0.115367 -0.091064  1.000000  0.070997  0.168163   \n",
       "SibSp    -0.046940  0.101060 -0.353861  0.070997  1.000000  0.440383   \n",
       "Parch     0.087018  0.001775 -0.238634  0.168163  0.440383  1.000000   \n",
       "Fare      0.205579 -0.586986  0.083645  0.180971  0.181967  0.260505   \n",
       "Embarked  0.108864  0.054265  0.018236  0.115029 -0.041111 -0.124586   \n",
       "title     0.411510 -0.051237 -0.314928  0.597094  0.309132  0.261974   \n",
       "\n",
       "              Fare  Embarked     title  \n",
       "Survived  0.205579  0.108864  0.411510  \n",
       "Pclass   -0.586986  0.054265 -0.051237  \n",
       "Age       0.083645  0.018236 -0.314928  \n",
       "Sex       0.180971  0.115029  0.597094  \n",
       "SibSp     0.181967 -0.041111  0.309132  \n",
       "Parch     0.260505 -0.124586  0.261974  \n",
       "Fare      1.000000  0.038452  0.147810  \n",
       "Embarked  0.038452  1.000000  0.103845  \n",
       "title     0.147810  0.103845  1.000000  "
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "datain['train'] = datain['train'].assign(title = title.values)\n",
    "title = datain['validation'].Name.apply(titan.name2title).map(tmap).fillna(4).astype(int)\n",
    "datain['validation'] = datain['validation'].assign(title = title.values)\n",
    "\n",
    "datain['train'].corr()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The new correlation matrix show that title is second to Sex in correlation to survived. The reason is title contains Sex information, e.g., Mr is male and Miss is female. It also contains Age information, e.g., Masters are children and have younger ages. I treat it as a new class of feature. The SibSp, while not very correlatable to Survived, does correlate well to Age. However, it does have a very large spread at a given SibSp value (figure below)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEKCAYAAAAfGVI8AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3X+YXVV97/H3Jwk/KvaKhJhGgRsvpFja1IDzoFNFRwMG\nKpWUtFZbJba0Q/vQXr29reD1aq+l9xFsb1tvS713CmootFqbGHhqKaEpc9E+MTiJ4ZcIQYVWDSSA\nqYDU/PreP9Y+zcx4Zs+PM/uczFqf1/PMs893zz6z15ozc75nrb3XWooIzMysXPN6XQAzM+stJwIz\ns8I5EZiZFc6JwMyscE4EZmaFcyIwMyucE4GZWeGcCMzMCudEYGZWuAW9LsBUnHjiibF06dJeF8PM\nbE7Ztm3bExGxaLLj5kQiWLp0KSMjI70uhpnZnCLp0akc564hM7PCORGYmRXOicDMrHBOBGZmhXMi\nMDMrnBOBmVnhsk4EQ0OwalXamplZe42NI5B0OvCpUbv+E/AB4IZq/1LgEeAtEfHt2T7/0BBcdll6\nvGlT2g4OzvZZzMzmvsZaBBHxYESsiIgVwCuA7wKfAa4ENkfEMmBzFc+69evrYzMzS7rVNbQS+GpE\nPApcBKyr9q8DVjdxwhUr6mMzM0u6lQjeCvxV9XhxROyqHj8GLG7ihMcfD1J6LKXYzMy+X+OJQNLR\nwJuBT4//XkQEEBM8b1DSiKSRPXv2TPu8AwNw7LEwf37aDgxM+0eYmRWhG5POXQBsj4jHq/hxSUsi\nYpekJcDudk+KiCFgCKCvr69tsqjT3w+bN8PwcEoC/f0zK7yZWe66kQjexuFuIYBbgLXA1dX25qZO\n3N/vBGBmNplGu4YkHQecB2wYtftq4DxJO4Fzq9jMzHqk0RZBRDwLLBy370nSXURmZnYEyHpksZmZ\nTc6JwMyscE4EZmaFcyIwMytc1olgyxb40IfS1szM2uvGOIKe2LIFVq6Effvg6KPT4DKPKTAz+37Z\ntgiGh1MSOHgwbYeHe10iM7MjU7aJYPzcQp5ryMysvWwTwcaNqTUAabtxY2/LY2Z2pMo2EWzYUB+b\nmVmSbSK4+OL6OFe+U8rMpivbu4auuSZtN2xISaAV58x3SpnZTGTbIoD05r9zZxlJAHynlJnNTNaJ\noLRukoEBWLAgLc25YIHvlDKzqcm2a6jUbpKIsVszs8lk2yIosZtkeDjVNyJtS6izmXUu20RQYjfJ\nwEBq/cyfn7Yl1NnMOpdt1xCU103S35+6wIaHUxIooSvMzDqXbSIYHob9+1MS2L8/xSW8Mfb3l1FP\nM5s92XYN7d07tkWwd29vy2NmdqRqNBFIOl7S30j6iqQHJPVLOkHS7ZJ2VtsXNnHuHTvqYzMzS5pu\nEXwE+PuIeBnwcuAB4Epgc0QsAzZX8axbs6Y+zlVpYyfMrHONXSOQ9ALgtcA7ASJiH7BP0kXAQHXY\nOmAYuGK2zz84mLbr16ck0IpzVurYCTPrTJMtgpcCe4CPS/qSpOskHQcsjohd1TGPAYvbPVnSoKQR\nSSN79uyZUQEGB+G228pIAlDm2Akz61yTiWABcBbw0Yg4E3iWcd1AERFA25s7I2IoIvoiom/RokUz\nKsDQEKxalbYl8DgCM5uJJm8f/QbwjYjYWsV/Q0oEj0taEhG7JC0Bdjdx8qEhuOyy9HjTprTNvWXg\ncQRmNhONtQgi4jHgXySdXu1aCXwZuAVYW+1bC9zcxPnXr6+Pc9XfD+99r5OAmU1d0wPKfgO4SdLR\nwNeAXyQln7+WdCnwKPCWJk68Zs3hlkArNjOz79doIoiIHUBfm2+tbPK8AHfe+f1x7l1DZmYzke3I\n4ltvrY/NzCzJNhFccEF9bGZmSbaTzt14Y9reemtKAq3YzMzGyjYRALz2tbBnT9qamVl72SaCEscR\nmJnNRLbXCEodR2BmNl3ZJoIVK+pjMzNLsk0EDz1UH5uZWZJtIvjWt+pjMzNLsk0El15aH+fqiitg\n2bK0NTObimzvGvrqV+vjHF1xBXz4w+lxa3vNNb0rj5nNDdm2CDZsqI9zVGKdzaxz2SaCiy+uj3NU\nYp3NrHPZdg21ukQ2bEhviCV0kZRYZzPrnNJqkUe2vr6+GBkZ6XUxzMzmFEnbIqLdUgBjZNs1BLBl\nC3zoQ2lbihLrbGadybZraMsWWLkS9u1LC7lv3pz/8o0l1tnMOpdti2B4OL0hHjyYtsPDvS5R80qs\ns5l1LttEMDBQH+doYCC1BObPT9sS6mxmnWu0a0jSI8DTwEHgQET0SToB+BSwFHgEeEtEfHu2z33t\ntemTMaTttdfm303S35+6g4aHUxLIvb5mNju60SJ4fUSsGHXl+kpgc0QsAzZX8azzmsVmZlPTi4vF\nFwED1eN1wDDgmXFmgS8Wm9lMNN0iCOAfJG2T1FofbHFE7KoePwYsbvdESYOSRiSN7NmzZ9on/s53\n6uMc+WKxmc1E04ngNRGxArgAuFzSmNWDI41mazuiLSKGIqIvIvoWLVo07ROfdVZ9nKOBAZhXvaLz\n5vlisZlNTaOJICK+WW13A58BzgYel7QEoNrubuLcW7fC2WfDggVpu3VrE2c5stx7L+zfnx7v359i\nM7PJNJYIJB0n6Qdbj4E3AvcBtwBrq8PWAjc3VYatW9MbYglJALxOs5nNTJMtgsXA5yXdDdwFfDYi\n/h64GjhP0k7g3CpuxNAQrFqVtiVYs6Y+NjNrp7G7hiLia8DL2+x/EljZ1HlbhobgssvS402b0nZw\ncOLjc9Cq3/r1KQnkXl8zmx3ZjiwutZtkcBBuu81JwMymLttEUGo3iWcfNbPpynb20RLXLPaAMjOb\niWxbBCWu3+sBZWY2E9kmghLX7x0YSOMmpLT1gDIzm4psE8Hq1WNH2a5e3dvydEtr5dE5sAKpmR0h\nsk0EH/4wHDqUHh86lOLctbqGItw1ZGZTl20iaI0dmCjO0f3318dmZu1kmwj+7d/q4xyNn0qjlKk1\nzKwz2SaCvr76OEclXiAHj50w61S2iWDZsvo4R6tXp/WKIW1LuEDeGjvx/venrZOB2fRlmwhKXKpy\n/MXhEi4We+yEWeeyTQQXXFAf52hgILUEpLQtYRzBwEAaRT1/ftqWUGez2ZbtFBM33pi2t96akkAr\nzp00dpu7/v40lcbwcEoCnlLDbPqyTQRQzpt/y/AwHDiQxhEcOJDiEt4Y+/vLqKdZU7LtGoK0KM3z\nnpe2JRjfLeJuEjObimwTwapVaRDZc8+lbQnJYOPGdNEU0nbjxt6Wx8zmhmwTwec+Vx/nqMQZV82s\nc9kmgnPOqY9zVOqAMjPrTOOJQNJ8SV+S9LdVfIKk2yXtrLYvbOK8K1bUxzm65hp4z3vgtNPS9ppr\nel0iM5sLutEieBfwwKj4SmBzRCwDNlfxrPv4x+vjXH3zm/DUU2lrZjYVjSYCSScBbwKuG7X7ImBd\n9Xgd0MhECE8/XR/n6O1vh5tuSongpptSbGY2maZbBH8MvAc4NGrf4ojYVT1+DFjc7omSBiWNSBrZ\ns2fPtE/cuntmojhHJU6rYWadaywRSLoQ2B0R2yY6JiICaLuWVkQMRURfRPQtWrRo2ud//evr4xyV\nOK2GmXWuyRbBq4E3S3oE+CTwBkk3Ao9LWgJQbXc3cfI1a+rjHF1++djlOS+/vLflMbO5obFEEBHv\njYiTImIp8FbgHyPi7cAtwNrqsLXAzU2cf/36+jhHw8Nj5xryTJxmNhW9GEdwNXCepJ3AuVU860ps\nEXiKCTObiUkTgaTFkq6XdGsVnyHp0umcJCKGI+LC6vGTEbEyIpZFxLkR8dTMil7v+uvr4xx5igkz\nm4mptAg+AdwGvLiKHwLe3VSBZsv27fVxjjzFhJnNxFQSwYkR8ddUt4BGxAHgiL8Z86yz6uMclTrF\nhNcsNuvMVNYjeFbSQqrbPCW9CvjXRks1Cy69FO66a2ycu1NPrY9z1FqzeN++tELZ5s1em8BsuqbS\nIvhN0p0+p0r6J+AG4DcaLdUsKPEaQal3SnnNYrPOTNoiiIjtkl4HnA4IeDAi9jdesg69+MX1cY7W\nrElrL4yOc9das7jVIvCdUmbTN2kikDS+p/mHJf0rcG9ENDIYbDbceWd9nKOvfrU+zpHXLDbrnNIs\nDzUHSJ8F+oE7ql0DwDbgpcDvRsRfNFlAgL6+vhgZGZnWc9ot3j5JVee8Zcvg4YcPx6edBjt39q48\nZtZbkrZFRN9kx03lGsEC4EciYk1ErAHOIF04fiVwRWfFbM4P/VB9nKNS7xoys85MJRGcHBGPj4p3\nV/ueAo7YawWnnFIf52jHjvrYzKydqdw+OlytLvbpKl5T7TsO2NtYyTpU4oCyEtdpNrPOTSURXA5c\nDLymikdIawo8CxyxkzsffTQcODA2zt0LXgDPPTc2NjObzKRdQ9WaAV8DDgA/TXrzf6D2SUeAffvq\n4xyVWGcz69yEiUDSD0v6HUlfAf4E+GfSXUavj4g/7VoJZ6jEKSa8ME05PK2Gzaa6rqGvAJ8DLoyI\nhwEk/ZeulGoWHH98fZyj1742rVU8Orb8eFoNm211XUMXA7uAOyT9uaSVpJHFc0KJF05LnGKiRJ5W\nw2bbhIkgIjZGxFuBl5EGk70beJGkj0p6Y7cKOFPLl9fHOVqxoj62PLSm1Zg/39Nq2OyYylxDzwJ/\nCfylpBcCP0saSLap9ok99vWv18c5+s536mPLg6fVsNk2ldtH/11EfBsYqr6OaE88UR+bzWX9/U4A\nNnt6sWZxV7zsZfVxji65BOZVr+i8eSk2M5tMY4lA0rGS7pJ0t6T7JX2w2n+CpNsl7ay2L2zi/D/1\nU/VxjjZuhEOH0uNDh7xmsZlNTZMtgu8Bb4iIlwMrgPOr1c2uBDZHxDJgcxXPuhLX7y2xzmbWucYS\nQSTPVOFR1VcAFwHrqv3rgNVNnL/EmThLrLOZda7RawSS5kvaQZqx9PaI2Eqap2hXdchjwOImzv0H\nf1Af56jENYvNrHONJoKIOBgRK4CTgLMl/di47weplfB9JA1KGpE0smfPnmmfu9VXPlGcIw8oM7OZ\n6MpdQxGxlzQo7XzgcUlLAKpt2+UuI2IoIvoiom/RokXdKOacd8899bGZWTtN3jW0SNLx1eMfAM4j\nzV90C7C2OmwtcHNTZSjN7t31sZlZO022CJaQ5im6B/gi6RrB3wJXA+dJ2gmcW8Wz7oQT6uMcnX56\nfZwrz8Rp1plpjSyejoi4Bzizzf4ngZVNnbdl7976OEdnnQUPPDA2zp1n4jTrXLYji0u8WHzrrfVx\njjwTp1nnsk0ERx1VH+eoxIVpPBOnWeeyTQRnnlkf5+jyy8fONXT55b0tTze0ZuK86ip3C5nNVGPX\nCHpt+/b6OEfDw6Bq6SApxSW8MXomTrPOZNsiKHHN4vHdIqV0k/iuIbPOZNsiePrp+jhHGzemi6aQ\nths35v9J2XcNmXUu2xbBgw/WxzkqcfZR3zVk1rlsE0GJg6tKnH3Udw2ZdS7brqGTTx47uOrkk3tX\nlm4Z/2m4hE/HXr/XrHPZtgg+97n6OEcl3ilVKl8gt9mUbYvguefq4xy95CXw6KNj49yVeLG4xDpb\ns7JtEZSoxOsiJV4sLrHO1qxsE0FrYNVEcY7WrKmPczQwkC4US2lbwsXigQFYsCDVecGCMupszco2\nEZx2Wn2co+XLD8+pdNRRKS7B6NHUpYgYuzXrRLaJ4J//uT7O0fDw4VlWDx0qo8tgeBgOHEhviAcO\nlFPngwdTnQ8eLKPO1qxsE8HrXlcf52hgYOyn4xK6DEocR1Bina1Z2d41dPvt9XGONm5Mn4ohbUuY\nYqLEcQQl1tmapZgDnYx9fX0xMjIyree06y+eA1XtyLJl8PDDh+PTToOdO3tXHjPrLUnbIqJvsuOy\n7Roq8a6hEqeYABgaglWr0tbMpq+xRCDpZEl3SPqypPslvavaf4Kk2yXtrLYvbOL8CxbUxzlavfpw\nPRcsSHHuhobgsstg06a0dTIwm74mWwQHgP8aEWcArwIul3QGcCWwOSKWAZureNbt318f52h4eOxt\nhSXcTbJ+fX1sZpNrLBFExK6I2F49fhp4AHgJcBGwrjpsHdDI59Z58+rjHJU40KjEQXRms60rHSaS\nlgJnAluBxRGxq/rWY8DiJs55+uljZx8tYboFKG+g0eBg2q5fn5JAKzazqWs8EUh6PrAeeHdEfEej\nrtpGREhq+5YlaRAYBDjllFOmfd7xd8uUcPdMu4FGJdxaODjoBGDWiUY7TCQdRUoCN0VEa72sxyUt\nqb6/BNjd7rkRMRQRfRHRt2jRommf22sWl9E1VCpPQ22zqcm7hgRcDzwQEX846lu3AGurx2uBm5s4\n/3331cc5ardmseWnNQ31+9+ftk4G1qkmWwSvBt4BvEHSjurrJ4GrgfMk7QTOreJZ993v1sc5KnHN\nYihvHIGnobbZ1tg1goj4PDDRMK6VTZ23RRp7wbSEAWUl3jLbGkcAaSwB5H+9oDXXUGthGncBWqey\nvany2GPr4xzt3l0f56jEcQStuYauusqrk9nsyDYRnHNOfZyjEutc6jiC/n5473vLSgK+QN6cbCde\n2Lu3Ps7RmjWHu0dace48jqAMXqe5Wdm2CLZvr49zVGI3CaQ3/9tucxLImS+QNyvbRFDiOIJSu0nc\nZZC/Etemhu79bWfbNfT00/VxjpYvT3MMHTiQtiWsWewug3KUtjZ1N/+2s20RPPhgfZyjEmcfdZdB\nGUpdm7pbf9vZJoLxk8yVMOlcic1nr99bhhJf527WOduuoeuvh5/4ibFxCSIOf5XA6/eWocTXuZt1\nzjYRDA+nTHrwYNqWMBPnDTccHk28f3+Kc68zpDqWUM/Slfg6d6vO2XYNlbhIi5nZTGSbCKC8RVpK\nvFPKzDqXbSIo8S6DrVvrYzOzdrJNBAsXwqFD6fGhQynO3cUX18dmZu1ke7H4ySfTgvWHDqXtk0/2\nukTNu+aatN2wISWBVmxmVifbRDAwAMccU96c7atXw/HHl1NfM+tctomgxPuOPd2Cmc1EtokAyrvv\nuN2Q9JLqb2Yzk+3F4hKVOAzfLGdzfvZRSR8DLgR2R8SPVftOAD4FLAUeAd4SEd9uqgylKbE7zCxX\nucw++gng/HH7rgQ2R8QyYHMV2ywqcQlDsxxlMftoRNwJPDVu90XAuurxOmB1U+cHL1hiZnNXzrOP\nLo6IXdXjx4DFTZ3Id9CY2VxWxOyjERGSJpwFSNIgMAhwyimnTPvn+w4aM5vrcp199HFJSwCq7e6J\nDoyIoYjoi4i+RYsWTftEpc4+WmJ3WIl1LlGJr/Ocv2toArcAa4Grq+3NTZ6stNlHS+wOK7HOJSrx\ndc7iriFJfwVsAU6X9A1Jl5ISwHmSdgLnVnEjhodTt1BE2pYw+2iJ6/eWWOcSlfg6d7POjbUIIuJt\nE3xrZVPnHK21fu+hQ+Wt31vS/Eol1hnSp8WSxouU+Dp3s85ZTzEhjd3mrsQBZSXWucRukhJf5yLu\nGmpau4VpSvnjKaGeo5VW51LviLv33lTXhQvLqC90728720RQYlPSylDi3/bQEFx2WXq8aVPaDg72\nrjy5yTYRlNiUtDKU+Le9fv33x04EsyfbRADldRlYOUr7216z5nBLoBXb7Mk6EZhZHlqf/tevT0nA\nrYHZ5URgZnPC4KATQFO8ME1mPAzfcuXXuTluEWSkxPvLS6xzifw6N8stgox4GH4ZdS6RX+dmZZ0I\nhoZg1aq0LUGJM66Wuk5zad0kJf5td1O2XUOlDkApbcbVEu+pL7WbpLS/7W7KtkXQbgBK7kqccRXK\nW6e5xG6SUv+2uyXbRDB+wEkJA1AGBsZ+anLzOU+tmXWlsmbWLa3O3ZRt11CJA1CuvTZNuw1pe+21\n5XxKLk1pM+tCmXXulmxbBCW69db62PIwPAz796dW3/79ZXSTtLrDSqpzN2XbIijxYvHixfDUU2Nj\ny8/ChWNbfgsX9rY83bB37+Fuz0OHUmyzJ9sWQYkXi/fvr48tD08+CfOq/9x581Kcux076mPrTLaJ\noMSLxRdfXB/nqsR76o85Jl00PeaYMi6clvj/DN372+5J15Ck84GPAPOB6yJi1hexX748/aMcPJi2\ny5fP9hmOPKeeWh/naMuW9Ea4fz8cdVQZq3WVOHZi+fL0+rZe5xL+n7s5XqTrLQJJ84FrgQuAM4C3\nSTpjts9zww0pCUDa3nDDbJ/hyFNid9gNNxy+iLhvXxmvM5Q5dmL0dZESLhZ3c7xIL7qGzgYejoiv\nRcQ+4JPART0oR3ZKbT5b/kqcSqSbde5F19BLgH8ZFX8DeOVsn+SSS+BjHzvclLzkktk+w5GnxLET\nl1wCH//44eZzCa9ziUrsDutmnRVdnrhD0s8A50fEL1fxO4BXRsSvjztuEBgEOOWUU17x6KOPTvtc\nW7aU9YdTKr/OZu1J2hYRfZMd14sWwTeBk0fFJ1X7xoiIIWAIoK+vb0bZqrR1XUvl19msM724RvBF\nYJmkl0o6GngrcEsPymFmZvSgRRARByT9OnAb6fbRj0XE/d0uh5mZJT0ZRxARfwf8XS/ObWZmY2U7\nstjMzKbGicDMrHBOBGZmhev6OIKZkLQHmP5AguRE4IlZLM5c4DqXwXUuQyd1/o8RsWiyg+ZEIuiE\npJGpDKjIietcBte5DN2os7uGzMwK50RgZla4EhLBUK8L0AOucxlc5zI0XufsrxGYmVm9EloEZmZW\nI+tEIOl8SQ9KeljSlb0uT9MkfUzSbkn39bos3SDpZEl3SPqypPslvavXZWqapGMl3SXp7qrOH+x1\nmbpF0nxJX5L0t70uSzdIekTSvZJ2SBpp9Fy5dg1VS2I+BJxHWvzmi8DbIuLLPS1YgyS9FngGuCEi\nfqzX5WmapCXAkojYLukHgW3A6sxfYwHHRcQzko4CPg+8KyK+0OOiNU7SbwJ9wH+IiAt7XZ6mSXoE\n6IuIxsdN5NwiKG5JzIi4E3iq1+XolojYFRHbq8dPAw+QVsDLViTPVOFR1Veen+ZGkXQS8Cbgul6X\nJUc5J4J2S2Jm/SZRMklLgTOBrb0tSfOqLpIdwG7g9ojIvs7AHwPvAQ71uiBdFMA/SNpWrdjYmJwT\ngRVC0vOB9cC7I+I7vS5P0yLiYESsIK3ud7akrLsBJV0I7I6Ibb0uS5e9pnqdLwAur7p+G5FzIpjS\nkpg2t1X95OuBmyJiQ6/L000RsRe4Azi/12Vp2KuBN1d95p8E3iDpxt4WqXkR8c1quxv4DKm7uxE5\nJwIviZm56sLp9cADEfGHvS5PN0haJOn46vEPkG6G+EpvS9WsiHhvRJwUEUtJ/8f/GBFv73GxGiXp\nuOoGCCQdB7wRaOxuwGwTQUQcAFpLYj4A/HXuS2JK+itgC3C6pG9IurTXZWrYq4F3kD4h7qi+frLX\nhWrYEuAOSfeQPuzcHhFF3E5ZmMXA5yXdDdwFfDYi/r6pk2V7+6iZmU1Nti0CMzObGicCM7PCORGY\nmRXOicDMrHBOBGZmhXMiKIik91UzVt5T3Wr5ymr/dZLOqB4/M8FzXyVpa/W8ByT9jy4WfUok/Q9J\nv9WF85xT/R53VPfyT3b8VH6/n5D0M+OPN+uGBb0ugHWHpH7gQuCsiPiepBOBowEi4pen8CPWAW+J\niLurmV1Pb6603VcNTlNETGUum18APhQRUxrdOsXf74yPnwpJA8A7I+Kds/2zbe5zi6AcS4AnIuJ7\nABHxRER8C0DSsKS+1oGS/qj6xLtZ0qJq94uAXdVzD7ameq4+hf+FpC2Sdkr6lckKUs2z/kFJ26v5\n1l826mf91qjj7pO0tPr6SvWp+SFJN0k6V9I/VeccPfT+5e3KIum3JX2xag19sNq3VGm9ihtIozZH\nT0mCpJXV/Pf3Kq31cIykXwbeAlwl6aZxxx8n6bNKawXcJ+nnpvH7Hf1z/v14Sc9I+p/Vz/yCpMXV\n/lOr+F5JvzdRS2M6JP1sVe67Jd1Z7XunpJurMu2U9Dujjv/N6vj7JL171O/0vlHH/Far9SjpPyut\nHXGPpE+O+p19TGmNhS9JynqG4COVE0E5NgEnV2+kfybpdRMcdxwwEhE/Cvw/oPWP/0fAg5I+I+ky\nSceOes6PA28A+oEPSHoxgNIMmRN5IiLOAj4KTKU75zTgfwEvq75+HnhN9dz/VlcWSW8ElpHmalkB\nvEKHJ/BaBvxZRPxoRDza+iFV/T4B/FxELCe1nn8tIq4jTVXy2xHxC+PKeD7wrYh4ebUeRLuRoBP9\nfidyHPCFiHg5cCfQSm4fAT5Sle0bk/yMqfoAsKo615tH7T8bWEP63f6spD5JrwB+EXgl8CrgVySd\nOcnPvxI4MyJ+HPjVat/7SFNGnA28Hvh9pSkVrIucCApRzWH/CmAQ2AN8StI72xx6CPhU9fhG0pst\nEfG7pEVBNpHehEe/yd0cEc9VC2jcQTU5VjVz4kRaE8RtA5ZOoQpfj4h7q66b+4HNkYbF3zvu+e3K\n8sbq60vAdlIiWVYd/+gEi7qcXp3zoSpeB0w2++O9wHmSrpF0TkT8a5tj2v5+a+wDWlNIjP5d9QOf\nrh7/5URPVnVdhzSP/5t1eCqOVW0O/yfgE1VLav6o/bdHxJMR8RzpdXtN9fWZiHi2+tvaAJwzSV3u\nAW6S9HbgQLXvjcCVVRmHgWOBUyb5OTbLfI2gIBFxkPTPNizpXmAt6VNv7dNGPf+rwEcl/TmwR9LC\n8cdMELfzvWp7kMN/hwcY++Hk2DbHQ3oz/d6ox6P/jtuVRaQ+/f87+htKaxg8O4WyTklEPCTpLOAn\ngd+TtLlKoLVPm+T7++PwPDCjf1dTLVPrhoABJrlGEBG/qnQDwZuAbdWn/nZlrCtz3Wv4JlIy/Sng\nfZKWk16bNRHx4OS1saa4RVAISadLWjZq1wrg0TaHzgN+pnr886SlEJH0Jkmq9i8jvSntreKLlNbS\nXQgMkCZDm4lHgLOq850FvHQGP6NdWW4Dfklp3QIkvUTSiyb5OQ8CSyWdVsXvIHXlTKjqEvtudRH5\n91t1Gaft73cGvkDqroE0I2fHJJ0aEVsj4gOkVmPrmsl5kk5QukNqNanl8DlgtaTnVV05P13texx4\nkaSFko7Sa1BUAAABWklEQVQh3aCApHnAyRFxB3AF8ALg+aTX5jdaf1tT6F6yBrhFUI7nA3+iNIXx\nAeBhUjfReM+SFjv576QVsH6u2v8O4I8kfbd6/i9ExMHq//ceUjfMicBVoy5C75ike2i89cAlku4n\nrTT20CTHt9OuLN+S9CPAlqq8zwBvJyWztiLi3yT9IvBpSQtICeX/THLu5aQ+7kPAfuDX2hwz0e93\nut4N3CjpfaRuunbdUNP1+9WHBQGbgbtJHxjuIr02JwE3RsQIpFteq+8BXBcRX6r2/261/5scniJ7\nflXeF1Q//39HxF5JV5FWH7unShZfp0oe1j2efdQ6Ut0R8kxE/EGvy1ISSc8DnouIkPRW4G0RMet3\n3FTXkfoi4tdn+2fbkcMtArO56RXAn1ZdKnuBX+pxeWwOc4vAzKxwvlhsZlY4JwIzs8I5EZiZFc6J\nwMyscE4EZmaFcyIwMyvc/wcVbU7spN7gtQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0xacabf60>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(datain['train'].SibSp, datain['train'].Age, '.b')\n",
    "plt.xlabel('SibSp: number of sibling + spouse')\n",
    "plt.ylabel('Age')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEKCAYAAAAfGVI8AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAH1dJREFUeJzt3X+QXWWd5/H3J0kLos6EhDYb1EwGjCiYNWIv2ipOIzCI\nv8gEF3VFwpZVzczOurqzWwS2SpdZtkbAnam4M+NY7c8g6OqSGJAaBtiM11+bZewAEgSZAAMrbEJi\nEGRAQxK++8dz7tC30336nk6fe+499/Oq6jrne+/pPo+Pl/vN8+M8jyICMzPrX/OqLoCZmVXLicDM\nrM85EZiZ9TknAjOzPudEYGbW55wIzMz6nBOBmVmfcyIwM+tzTgRmZn1uQdUFaMcxxxwTy5cvr7oY\nZmY9Zdu2bT+PiMGZruuJRLB8+XLGx8erLoaZWU+R9HA717lryMyszzkRmJn1OScCM7M+50RgZtbn\nnAjMzPqcE4GZWZ+rdSIYG4OzzkpHMzObWmnPEUg6AfjGhJeOAz4JXJ29vhx4CDgvIn4x1/cfG4OL\nLkrnt9ySjqOjc30XM7PeV1qLICLui4hVEbEKeAPwDPAt4BJgS0SsALZk8ZzbuDE/NjOzpFNdQ6cD\nD0TEw8A5wIbs9Q3A6jJuuGpVfmxmZkmnEsEHgK9n50siYmd2vgtYUsYNFy4EKZ1LKTYzs0OVnggk\nvQB4L/A/J78XEQHENL83Kmlc0viePXsK33dkBI48EubPT8eRkcJ/wsysL3Ri0bmzgdsj4rEsfkzS\n0ojYKWkpsHuqX4qIMWAMYGhoaMpkkWd4GLZsgUYjJYHh4dkV3sys7jqRCD7I891CADcAa4ErsuP1\nZd14eNgJwMxsJqV2DUl6EXAmsGnCy1cAZ0raAZyRxWZmVpFSWwQR8TSweNJre0mziMzMrAvU+sli\nMzObmROBmVmfcyIwM+tzTgRmZn2u1olg61b41KfS0czMptaJ5wgqsXUrnH46PPssvOAF6eEyP1Ng\nZnao2rYIGo2UBA4eTMdGo+oSmZl1p9omgslrC3mtITOzqdU2EWzenFoDkI6bN1dbHjOzblXbRLBp\nU35sZmZJbRPBmjX5sZmZJbVNBKtXp70IIB1Xl7IPmplZ76ttIpg8S8izhszMplbbRDAykp4fmD8/\nHT1raGZ+AK8Y15fVRW0fKPMOZcX4AbxiXF9WJ7VNBOAdyoqY6gE81930XF9WJ7XtGgI33YtwV1ox\nri+rk9q2CNx0L8ZdacW4vqxOapsIGg349a8hIh3ddJ+Zu9KKcX1ZXdS2a+iJJ1ISgHR84olqy2Nm\n1q1KTQSSFkq6TtJPJd0raVjSIkm3StqRHY8u49533pkfm5lZUnaL4DPA30TEq4HXAfcClwBbImIF\nsCWL59y55+bHdigPrpv1p9LGCCT9JvA24EKAiHgWeFbSOcBIdtkGoAGsm+v7j46m48aNKQk0Y5ua\nB9fN+leZLYLfBvYAX5Z0h6QvSHoRsCQidmbX7AKWTPXLkkYljUsa37Nnz6wKMDoKN9/sJNAOb+Rj\n1r/KTAQLgJOBv4qI1wNPM6kbKCICiKl+OSLGImIoIoYGBwdnVYCxMTjrrHS0fJ4Xb9a/ypw++gjw\nSETclsXXkRLBY5KWRsROSUuB3WXcfGwMLroond9ySzq6ZTA9z4s361+ltQgiYhfwM0knZC+dDtwD\n3ACszV5bC1xfxv03bsyP7VDDw3DppU4CZv2m7FlDHwWulXQXsAr4E+AK4ExJO4AzsnjOedZQcZ41\nVIy7Hq0uSn2yOCLuBIameOv0Mu8LsHIlDAzA/v3puHJl2XfsbZ41VIy7Hq1OavtkcaMBzz2Xzp97\nzrNgZuJZQ8W469HqpLaJwLNginF9FeOuR6uT2i4651kwxbi+ivEDi1YntU0EANu3py+2xYv9xdYO\nr6ZZzOioE4DVQ20TgQfzzMzaU9sxAg/mmZm1p7aJYNWq/NgO5ecIzPpTbbuGfvnL/Nha+TkCs/5V\n2xaBFePnCMz6V20TwQUXwIKsvbNgQYpteiMj6RkCKR39HMHM3JVmZevUMia17Rravh0OHEjnBw6k\n2F0d+aTWo03PXWlWtk7OfKxti8CzhoppNFLCjEhHdw3lc1eala2T32G1TQReAqAYLzFRjOvLytbJ\n77Dadg15CYBivMREMa4vK1snv8OUdovsbkNDQzE+Pl51MczMeoqkbREx1VYALWrbNQSe1WFmva1T\n32G17RryrA4z62Wd/A6rbYvAszrMrJd18justolg8iwOz+ows14yMpIehpXSsczvsFITgaSHJG2X\ndKek8ey1RZJulbQjOx5dxr03b06ZFNJx8+Yy7mJmVp7mXJ6y5/R0okVwWkSsmjByfQmwJSJWAFuy\neM5t2pQf26E69Th7XaxbBytWpKPZXGs00j9iI9KxzK6hKgaLzwFGsvMNQAOY8/+UjjsO7r+/Nbbp\neSOfYtatg6uuSufN45VXVlceq5/mQ4vNweKe7RoCAvhfkrZJan6tLImIndn5LmDJVL8oaVTSuKTx\nPXv2lFxM85IcxbjFaWVrPrR4+eXlz3osOxG8NSJWAWcDfyjpbRPfjPQ025S9XxExFhFDETE0ODhY\n+MZeYqIY11cxa9bkx2ZzYXgYLr20/KnvpXYNRcSj2XG3pG8BpwCPSVoaETslLQV2l3HvlSthYAD2\n70/HlSvLuEt9rFyZZiYcOJCOrq98zW6gTZtSEnC3kPWy0loEkl4k6SXNc+B3gbuBG4C12WVrgevL\nuH+jAc89l86fe87PEcyk0WidoeD6mtmVV8KOHU4C1vvKbBEsAb6ltLj9AuBrEfE3kn4EfFPSR4CH\ngfPKuPnISOv6+n6OIF8nB6bMrLuUlggi4kHgdVO8vhc4vaz7NnljmmK8mqZZ/6rtWkNTzYLxdMh8\nw8NOAGb9qLZLTHgWTHFerdWsP9W2RfDAA/mxtfJqrWb9q7YtAj/wU4xXazXrX7VNBH7gpxjvwWvW\nv2rbNXT88fmxtfKsIbP+VdsWwWc+kx/boTZvhi99yUt2t8uD61YXtW0RPPRQfmytvJpmMR5ctzqp\nbYtg//782Fp5cL0YD65bndQ2EZx2Wn5srTy4XowH160TOtX9WNuuoVWrnt9gpRnb9Dy4XowH161s\nnex+rG2LwF0dxXhjmuI6tVa89adOdj/WNhG4q6MYL8lh1l062f1Y266h1avhT/80ZdP581Ns02su\nyLdxY0oCXqDPrFqd7H6sbSKY3IxqNNyEn8noqBOAWTfp1IrAte0aGhlp3XHLszpm5gekinF9Wdk8\na+gwXXZZ61aVl10GN99cZYm6mx+QKsb1ZWXzrKE58P3v58fWyg9IFeP6srJ51tAcOPXU/NhajYzA\nvOzTMG+eu9JmMjICCxak/bAXLHB92dzr5Kyh0hOBpPmS7pB0YxYvknSrpB3Z8egy7nvzzfCa16Qv\ntde8xt1CM9m+/fllOPbvT7HlmzgGZTbXmrOGLr+8/K7HTrQIPgbcOyG+BNgSESuALVk858bG4N57\n0/jAvfem2KbnB8qKaTTgwIGUBA4ccNeQ9bZSE4GklwPvAr4w4eVzgA3Z+QaglBn+l12WH5sdjsWL\nWycjLF5cbXmsfpqDxZ/4RDqWOXOo7BbBeuBi4LkJry2JiJ3Z+S5gyVS/KGlU0rik8T179hS+8ZNP\n5sfW6sEH82NrtXdv65jK3r3VlsfqpxaDxZLeDeyOiG3TXRMRAUzZwxoRYxExFBFDg4ODhe//e7+X\nH1srL8lRzMgIHHFEGsg74ggPFtvcq8sSE28B3ivpncCRwG9IugZ4TNLSiNgpaSmwu4ybv+xl+bG1\nam5Cs2lTSgLelCafVx+1snXyM6bowJQHSSPAf4yId0v6NLA3Iq6QdAmwKCIuzvv9oaGhGB8fL3TP\nFSvg/vufj1/5Stixo3DRzcx6lqRtETE003VVPEdwBXCmpB3AGVk859zVUdzYGJx1lmdYtcv1ZXUx\nY9eQpCXAnwDHRsTZkk4EhiPii+3eJCIaQCM73wucPqvSFnDnnfmxtRobg4suSufNDX28AN30XF9W\nJ+20CL4C3Awcm8V/D3y8rALNFS8xUYyfIyjG9WV10k4iOCYivkk2BTQiDgAHSy3VHPASE8V4Y5pi\nXF9WJ+3MGnpa0mKyaZ6S3gR0/az8yTNOZzEDta94Y5piVq6EgYG0HMfAQIrNelU7ieCPgBuA4yX9\nEBgE3ldqqebAt7+dH9uhvDFN+xqN1ieLvfGR9bIZu4Yi4nbgd4A3AxcBJ0XEXWUX7HAdd1x+bIfy\nLJj2ebVWq5N2Zg1Nnnj5KklPAtsjopSHwebCrl35sbXyLJhiplqt1S0C61XtDBZ/hLRo3Ieyn88D\n64AfSvpwiWU7LI89lh9bK8+CKcb1ZXXSTiJYALwmIs6NiHOBE0kDx28kJYSutGxZfmytPAumGNeX\n1Uk7ieAVETHx39O7s9ceB/aXU6zD9/7358fWyv/CLeaBB/Jjs17STiJoSLpR0lpJa4Hrs9deBDxR\nbvFmb9Om/Nha+QG8Yvz5sjppJxH8IfBlYFX2M05aQfrpiDitzMIdjscfz4+t1Utfmh9bK69lZXUy\n46yhiAhJDwJvAv4l8A9A13ccPPFEfmytBgbyY2vlZbutTqZNBJJeBXww+/k58A3SstVd2wqY6IQT\n0l7FE2Ob3po1cNVVrbHlW70aFi70MwTW+/JaBD8Fvg+8OyLuB5D07ztSqjlw8smtieDkk6sri9VP\ncz/ZZ59Nu0dt2eLnCKx35Y0RrAF2At+R9HlJpwPqTLEO30035cfWyoOfxTQasG9f2k92375y95M1\nK9u0iSAiNkfEB4BXA98hLT39Ukl/Jel3O1XA2Roayo+tlQc/i1m8uHWtocWLqy2P2eFoZ62hpyPi\naxHxHuDlwB108YNkTffckx9bq9Wr0ybZkI6rV1dbnm53xx35sVkvKbRVZUT8IiLGIqL0HcYO186d\n+bG1mty14a4Os/5RxZ7FHfGGN+TH1sqraRZzwQWt9XXBBdWWx+xwlJYIJB0p6e8k/VjSTyT9cfb6\nIkm3StqRHY8u4/7r14PULEuKbXpTraZp09u8uXWMYPPmastjdjjKbBHsA94eEa8jPZH8jmx3s0uA\nLRGxAtiSxXOu0Wj9F5u7OvJ5raFiPMvK6qS0RBDJP2bhQPYTwDnAhuz1DUApw5IjI2lqH6Sjuzry\neTXNYjzLyuqkna0qZ03SfGAb8ErgLyPiNklLIqI5dLsLWFLGvc8449D46afLuFM9eM/iYjy4bnVS\n6mBxRByMiFWkaaenSHrtpPeD1Eo4hKRRSeOSxvfs2VP43s88kx/boTZuTKuOultoZrffnh+b9ZKO\nzBqKiCdID6W9A3hM0lKA7DjldpfZNNWhiBgaHBzsRDH72llnpS0qf/WrdDzrrKpL1N0mL1niJUys\nl5U5a2hQ0sLs/IXAmaT1i24A1maXNfc3sIp5P4JibrsNTjkFFixIx9tuq7pEZrNX5hjBUmBDNk4w\nD/hmRNwoaSvwTUkfAR4Gzivj5kcd1doddNRRZdylPk499flN65ux5Vu/Po0NeCKClWXr1uc/Y2Uu\nalhaIoiIu4DXT/H6XqD0J5N//ev82FqtWtWaCFatqq4svcCrj1rZOvkZq+2Txc2HfaaLrZXnxRfT\naKT/QA8eTEfPGrK51snPWG0TgXfcKsbz4osZGUmL80np6O4hm2sjI6klMH9+Opb5GattIjjvvPzY\nWh1/fH5sh5q4hInZXBseTt1Bl19eftdjbROBN6YpxktMFNNowIEDEJGO7hqyMgwPw6WXlj/+VNtE\ncPbZ+bG18hITxUxuprtryHpZbROBn/wsZuXK58dRBgZSbNPbvLl1LSuvPmq9rLaJ4L778mNr1Wi0\nLqvsro58nmVldVLbRHDCCfmxtfIsmGI8y8rqpLaJ4Be/yI/tUJ4F0z7v8Wx1UttEsGtXfmytGg3Y\nty/Ngtm3z11DM2k0WscIXF/Wy2qbCKyYr341P7ZWkweHPVhsvcyJwADYsSM/tlaT93T2Hs/Wy2qb\nCBYtyo+tldfXL2by6qxerdXKsHUrfOpT6Vim2iaCZcvyY2u1fj3Myz4N8+al2KbnB/CsbM3VRz/x\niXQsMxnUNhE8+GB+bK0ajdZZQx78zOclOaxsXn10DrznPfmxtRoZaW0R+DmCfG4RWNm8+ugcuO66\n/Nhabd8O+/en8/37Pfg5E7cIrGydXH20zK0qK7VvX35srab6YhsdraYsvcB7PFsnDA93Zue72rYI\njjgiP7ZW7uooxrOGrE5KaxFIegVwNbAECGAsIj4jaRHwDWA58BBwXkTM+QIQzcf/p4utVfNf/xs3\npiTg1oBZ/1BElPOHpaXA0oi4XdJLgG3AauBC4PGIuELSJcDREbEu728NDQ3F+Ph4wfsf+lpJ/1Ot\nDx11FPzqV8/HL3whPPNMdeUxm4qkbRExNNN1pXUNRcTOiLg9O38KuBd4GXAOsCG7bAMpOcy5BQvy\nY7PD4a4hq5OOjBFIWg68HrgNWBIRO7O3dpG6juacn5S1MnlMxeqk9EQg6cXARuDjEfHLie9F6pea\nssNG0qikcUnje/bsKXxfrwVjZfL0UauTUhOBpAFSErg2Ipp7OD2WjR80xxF2T/W7ETEWEUMRMTQ4\nOFj43m66F9epdU3qwC0Cq5PSEoEkAV8E7o2IP5vw1g3A2ux8LXB9Gfef/GXmL7d8nVzXpA6+9738\n2KyXlNkieAvwYeDtku7Mft4JXAGcKWkHcEYWz7mnnsqPrVVzY5qDB70xTTtuuik/Nuslpc2liYgf\nANNtenh6Wfe12Vm8uHXz+sWLqy1Pt1uyBB5/vDU261W1fbJ48nME3oc33969rYvO7d1bbXm6XXNd\npulis15S20Tg/QiKGRlJz1pI6ejVR/OtWZMfm/WS2iaCt741P7ZDTdyPwPI9+mh+bNZLapsIPJhX\nTKMBBw6kZTgOHPBg8Uz8+bI6qW0iOPvs/NhadXITjDrw58vqpLYr8OzYkR9bq+YmGI1GSgKdWAO9\nl01+2H0WD7+bdY3SVh+dS7NZfXRgIHVxNC1Y4JkdNne8+qj1gspXH62aF52zMnkJE6uT2iaC9etb\n58WvX19teaxeLrssPzbrJbVNBI1G63RIz4KxuXT11fmxWS+pbSLwLJjivPqoWX+q7awhz4Ipprn6\n6LPPpsS5ZYvrLI8XNbQ6qW0igPRF5i+z9jQaKQkcPJiOjYbrLs9tt+XHZr2ktl1DVoy70op54xvz\nY7NeUusWgbXPXWnFnHRSmoQQkY4nnVR1icxmz4nA/om70to3MgJHHvn8mIpbUDPbutX/0OhWTgRm\ns+AWVDGejNDdnAjMZsktqPZ5MkJ382CxmZXOGx91t9ISgaQvSdot6e4Jry2SdKukHdnx6LLub2bd\npbm+ZQ+sc9l3ymwRfAV4x6TXLgG2RMQKYEsWm1nNNRqpWygiHb3kS3cpLRFExPeAxye9fA6wITvf\nAKwu6/5mZfOSHO3zcyrdrdODxUsiYmd2vgtY0uH7m80Jz4IpxrOsultls4YiIiRN21soaRQYBVi2\nbFnHymXWDs+CKc6zrLpXp2cNPSZpKUB23D3dhRExFhFDETE0ODjYsQKatcOzYIo7/3xYvDgdrbt0\nOhHcAKzNztcC13f4/mZzxrNg2nf++XDttfD44+noZNBdypw++nVgK3CCpEckfQS4AjhT0g7gjCw2\n6zmeBVPMTTflx1at0sYIIuKD07x1eln3NOuUkZG0BerBg+norqF8Z5+dWgITY+sefrLYbBa2b4f9\n+9P5/v0ptuldcw186EOwaFE6XnNN1SWyiZwIzGZh48b82A71trfB0FA6WnfxonNms3DuuXDLLa2x\nTW9sDC66KJ036210tLryWCsnArNZaH6JbdyYkoC/1PJN1YJynXUPJwKzWRod9ZdZu9yC6m5OBGZW\nOregupsTgZl1hFtQ3cuzhsxmad06WLEiHc16mVsEZrOwbh1cdVU6bx6vvLK68pgdDrcIzGZh06b8\n2KyXOBGYzcKaNfmxHcpdad3LXUNms3D88fmxtXJXWndzi8BsFrzERDHuSutuTgRmszD5gSg/IJXP\nXWndzYnAzEr36KP5sVXLicBsFtw1VIw3puluTgRms/Czn+XH1mrJkvzYquVEYDYLDz2UH1ur5iY+\n08VWLScCs1k49dT82Fp5sLi7VZIIJL1D0n2S7pd0SRVlMDscnjVUjAeLu1vHE4Gk+cBfAmcDJwIf\nlHRip8thdji++MX82Fp9+9v5sVWrihbBKcD9EfFgRDwL/A/gnArKYTZrxx6bH1ur447Lj61aVSSC\nlwET51g8kr1m1jMuvhgGBtL5wECKbXqf/SzMy75t5s1LsXWPrl1rSNIoMAqwbNmyiktj1mp4GL77\nXWg0YGQkxTa94WH4wQ9cX92qikTwKPCKCfHLs9daRMQYMAYwNDQUnSmaWfuGh/2FVoTrq3tV0TX0\nI2CFpN+W9ALgA8ANFZTDzMyooEUQEQck/VvgZmA+8KWI+Emny2FmZkklYwQR8dfAX1dxbzMza+Un\ni83M+pwTgZlZn3MiMDPrc4ro/pmZkvYAD8/y148Bfj6HxZkrLlcxLlcxLlcx3VouOLyy/VZEDM50\nUU8kgsMhaTwihqoux2QuVzEuVzEuVzHdWi7oTNncNWRm1uecCMzM+lw/JIKxqgswDZerGJerGJer\nmG4tF3SgbLUfIzAzs3z90CIwM7MctUkEM21/qeS/Z+/fJenkLinXiKQnJd2Z/XyyA2X6kqTdku6e\n5v2q6mqmcnW8rrL7vkLSdyTdI+knkj42xTUdr7M2y1XF5+tISX8n6cdZuf54imuqqK92ylXJZyy7\n93xJd0i6cYr3yq2viOj5H9LidQ8AxwEvAH4MnDjpmncCNwEC3gTc1iXlGgFu7HB9vQ04Gbh7mvc7\nXldtlqvjdZXddylwcnb+EuDvu+Tz1U65qvh8CXhxdj4A3Aa8qQvqq51yVfIZy+79R8DXprp/2fVV\nlxZBO9tfngNcHcn/ARZKWtoF5eq4iPge8HjOJVXUVTvlqkRE7IyI27Pzp4B7OXRXvY7XWZvl6ris\nDv4xCweyn8mDkVXUVzvlqoSklwPvAr4wzSWl1lddEkE7219WsUVmu/d8c9bcu0nSSSWXqR3dvJ1o\npXUlaTnwetK/JieqtM5yygUV1FnWzXEnsBu4NSK6or7aKBdU8xlbD1wMPDfN+6XWV10SQS+7HVgW\nEf8c+HNgc8Xl6WaV1pWkFwMbgY9HxC87ee88M5SrkjqLiIMRsYq0A+Epkl7bifvOpI1ydby+JL0b\n2B0R28q+13Tqkgja2f6yrS0yO12uiPhls7kaaZ+GAUnHlFyumVRRVzOqsq4kDZC+bK+NiE1TXFJJ\nnc1Urqo/XxHxBPAd4B2T3qr0MzZduSqqr7cA75X0EKn7+O2Srpl0Tan1VZdE0M72lzcAF2Sj728C\nnoyInVWXS9I/k6Ts/BTS/yd7Sy7XTKqoqxlVVVfZPb8I3BsRfzbNZR2vs3bKVUWdSRqUtDA7fyFw\nJvDTSZdVUV8zlquK+oqISyPi5RGxnPQd8bcRcf6ky0qtr0p2KJtrMc32l5J+P3v/c6Qd0d4J3A88\nA/zrLinX+4A/kHQA+BXwgcimCZRF0tdJsyOOkfQI8J9JA2eV1VWb5ep4XWXeAnwY2J71LwP8J2DZ\nhLJVUWftlKuKOlsKbJA0n/RF+s2IuLHq/x7bLFdVn7FDdLK+/GSxmVmfq0vXkJmZzZITgZlZn3Mi\nMDPrc04EZmZ9zonAzKzPORFYz5K0UNK/mRAfK+m67HyVpHdOeO9CSX9RYlkulHRsWX/frExOBNbL\nFgL/lAgi4v9FxPuycBVp3nWnXAiUmggk1eK5H+s+TgTWy64AjldaN/7TkpZLujt7ivu/AO/P3nv/\nxF/KnjDdKOlH2c9bJv/hbHGy/5b9vbskfTR7/ZPZ79wtaSx70vN9wBBwbXa/F0p6g6TvStom6WZl\nK0VK+hfZ32uW+e7s9SMlfVnSdqU16U/LXr9Q0g2S/hbYIulqSasnlPNaSZWvaGs97nDWsPaPf6r8\nAZYzYe+CiTHpX+h/MeG9f4pJa76/NTtfRlqiYfLf/gPgOmBBFi+aeMzOvwq8JztvAEPZ+QDwv4HB\nLH4/6alygLuB4ez8ignl/Q8Trnk18H+BI7NyPzLh/r8DbM7OfxP4h2YZ/eOf2f64qWn96AzgxGxJ\nGYDfkPTieH6t+uY1n4uIAwAR0dwn4TRJFwNHAYuAnwDfnvT3TwBeC9ya3WM+sDNb5+YlEbE1u+5r\nwLuz87eSVrskIn4q6WHgVdl7tzbvHxHflfRZSYPAucDGZhnNZsuJwPrRPNLOVL8u8kuSjgQ+S/qX\n/88kXUb6V/shlwI/iYjhSb+/cJblfXpSfDVwPmmBso6sA2X15jEC62VPkbZoLPreLcBHm4GkVVNc\ncytwUXOAVtIinv/S/7nSHgDvm3D9xPvdBwxKGs5+d0DSSZGWPn5K0huz6z4w4fe/D3wou/5VpC6r\n+6Yp/1eAjwNExD3TXGPWNicC61kRsRf4YTZw++lJb3+H1P1zyGAx8O+AoWzQ9h7g96f4818g9dPf\nJenHwL/Kvsg/T+rnv5m0zHjTV4DPZauAzicliSuz370TeHN23UeAz2fXvQh4Mnv9s8A8SduBbwAX\nRsS+af53P0balvLL09WNWRFefdSsgyaORUi6BFgaER8r+DeOAraTNq5/cqbrzWbiFoFZZ70ra6Xc\nDZwK/NcivyzpDFJr4M+dBGyuuEVgZtbn3CIwM+tzTgRmZn3OicDMrM85EZiZ9TknAjOzPudEYGbW\n5/4/cRpbtxdVdm8AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0xac44550>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(datain['train'].title, datain['train'].Age, '.b')\n",
    "plt.xlabel('title category')\n",
    "plt.ylabel('Age')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Compared to SibSp the title feature has less a spread at a given category therefore I map age to each title category. Furthermore, title is better at separating younger and older ages. For those missing Age in the database I use the median age for respective titles (extracted from the figure above) to fill in the blank."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "def readdata(fname, tp, vstart, features):\n",
    "    train = pd.read_csv(fname)\n",
    "    \"\"\" only use a small subset of features for now\"\"\"\n",
    "    trainlean = train.loc[:, features]\n",
    "    \"\"\" digitize string values \"\"\"\n",
    "    trainlean.loc[:, 'Sex'] = trainlean.loc[:, 'Sex'].apply(titan.gendertonum)\n",
    "    trainlean.loc[:, 'Embarked'] = trainlean.loc[:, 'Embarked'].apply(titan.embarknum)\n",
    "    \n",
    "    \"\"\" extract titles and map titles to age for those missing age in db \"\"\"\n",
    "    tmap = {' Mr':0, ' Mrs':1, ' Miss':2, ' Mme':1, ' Master':3}\n",
    "    title = trainlean.Name.apply(titan.name2title).map(tmap).fillna(4).astype(int)\n",
    "    medage = trainlean.groupby(title)['Age'].median()\n",
    "    trainlean.loc[trainlean.Age.isnull(), 'Age'] = \\\n",
    "                  title[trainlean['Age'].isnull()].map(medage)\n",
    "    trainlean = trainlean.assign(title = title.values)\n",
    "    del trainlean['Name']   \n",
    "  \n",
    "    m = trainlean.shape[0]\n",
    "    trainset = trainlean.loc[:int(tp * m), :]\n",
    "    vadset = trainlean.loc[int(vstart * m):, :]\n",
    "    \n",
    "    return {'train':trainset, 'validation':vadset}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Survived</th>\n",
       "      <th>Pclass</th>\n",
       "      <th>Age</th>\n",
       "      <th>Sex</th>\n",
       "      <th>SibSp</th>\n",
       "      <th>Parch</th>\n",
       "      <th>Fare</th>\n",
       "      <th>Embarked</th>\n",
       "      <th>title</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>count</th>\n",
       "      <td>535.000000</td>\n",
       "      <td>535.000000</td>\n",
       "      <td>535.000000</td>\n",
       "      <td>535.000000</td>\n",
       "      <td>535.000000</td>\n",
       "      <td>535.000000</td>\n",
       "      <td>535.000000</td>\n",
       "      <td>535.000000</td>\n",
       "      <td>535.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>0.386916</td>\n",
       "      <td>2.325234</td>\n",
       "      <td>28.997196</td>\n",
       "      <td>-0.256075</td>\n",
       "      <td>0.549533</td>\n",
       "      <td>0.370093</td>\n",
       "      <td>31.716035</td>\n",
       "      <td>1.366355</td>\n",
       "      <td>0.809346</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>0.487500</td>\n",
       "      <td>0.829477</td>\n",
       "      <td>13.141558</td>\n",
       "      <td>0.967562</td>\n",
       "      <td>1.125494</td>\n",
       "      <td>0.767702</td>\n",
       "      <td>47.259528</td>\n",
       "      <td>0.676234</td>\n",
       "      <td>1.049838</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.750000</td>\n",
       "      <td>-1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>-3.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>2.000000</td>\n",
       "      <td>21.000000</td>\n",
       "      <td>-1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>7.925000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>3.000000</td>\n",
       "      <td>30.000000</td>\n",
       "      <td>-1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>14.454200</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>3.000000</td>\n",
       "      <td>35.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>29.885400</td>\n",
       "      <td>2.000000</td>\n",
       "      <td>2.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>3.000000</td>\n",
       "      <td>71.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>8.000000</td>\n",
       "      <td>5.000000</td>\n",
       "      <td>512.329200</td>\n",
       "      <td>3.000000</td>\n",
       "      <td>4.000000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "         Survived      Pclass         Age         Sex       SibSp       Parch  \\\n",
       "count  535.000000  535.000000  535.000000  535.000000  535.000000  535.000000   \n",
       "mean     0.386916    2.325234   28.997196   -0.256075    0.549533    0.370093   \n",
       "std      0.487500    0.829477   13.141558    0.967562    1.125494    0.767702   \n",
       "min      0.000000    1.000000    0.750000   -1.000000    0.000000    0.000000   \n",
       "25%      0.000000    2.000000   21.000000   -1.000000    0.000000    0.000000   \n",
       "50%      0.000000    3.000000   30.000000   -1.000000    0.000000    0.000000   \n",
       "75%      1.000000    3.000000   35.000000    1.000000    1.000000    0.000000   \n",
       "max      1.000000    3.000000   71.000000    1.000000    8.000000    5.000000   \n",
       "\n",
       "             Fare    Embarked       title  \n",
       "count  535.000000  535.000000  535.000000  \n",
       "mean    31.716035    1.366355    0.809346  \n",
       "std     47.259528    0.676234    1.049838  \n",
       "min      0.000000   -3.000000    0.000000  \n",
       "25%      7.925000    1.000000    0.000000  \n",
       "50%     14.454200    1.000000    0.000000  \n",
       "75%     29.885400    2.000000    2.000000  \n",
       "max    512.329200    3.000000    4.000000  "
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "features = ['Survived', 'Pclass', 'Age', 'Sex', 'SibSp', 'Parch', 'Fare','Embarked','Name']\n",
    "datain = readdata('train.csv', 0.6, 0.6, features)\n",
    "datain['train'].describe()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "First, I use just linear terms (features as is) to train a logistic regression model with the training set data then I find the accuracy on the validation set. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYwAAAEKCAYAAAAB0GKPAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3X10VfWd7/H3xwgFgYKFtEWegh10yWOqAbF0FFwOAnW0\nTFVgIbZza1O03NI7Ha5OW6vV1TWtentvdbxaplrUUh4UrS7rQ/VKK/VpCDSggK0UsIKsBfIkQlED\n3/vH2eAhOUl2IDsnkM9rrbNyzm//9tnf/Aj5ZD+c/VNEYGZm1pgTil2AmZkdGxwYZmaWigPDzMxS\ncWCYmVkqDgwzM0vFgWFmZqk4MMzMLBUHhpmZpeLAMDOzVE4sdgHNqUePHlFWVlbsMszMjhnLli17\nJyJK0/Q9rgKjrKyMqqqqYpdhZnbMkPRm2r4+JGVmZqk4MMzMLBUHhpmZpeLAMDOzVBwYZmaWSmaB\nIamDpP+StELSKkk/KNBHkm6XtFbSSkln5i0bJ+lPybLrsqkRDki5J8njgISUxdaOfR6vpvF4NY0E\n7+jkw8brHZ3s8WpFstzDeB84PyKGAeXAOEkja/UZDwxIHpXAXQCSSoA7k+UDgSmSBjZ3gfsRtX8W\nlbRbXR6vpvF4Nc1WTqY7Ow9r685OtnJykSqy2jILjMh5L3nZLnnUng/2EuD+pO/LQDdJPYERwNqI\nWBcRHwDzk77N6gQo+B/ax+kK83g1jceraXqws+B49agVIlY8mf7sSiqRVA1sAZ6JiFdqdekFvJX3\nemPSVl97oW1USqqSVLV169bmK97MzA6TaWBExP6IKAd6AyMkDc5gG7MjoiIiKkpLU3263czMjkCL\n7B1HxE5gMTCu1qJNQJ+8172Ttvram9UB6h4ji6Td6vJ4NY3Hq2neoVvB8XqHbsUoxwrI8iqpUknd\nkucdgX8AXq/V7THgyuRqqZHArojYDCwFBkjqL6k9MDnp26xKiII/oCV1Wg08Xk3l8WqaUnawrVY4\nbKMbpewoUkVWW5Y3H+wJ3Jdc8XQCsDAiHpc0HSAi7gaeACYAa4G9wD8ny2okzQCeBkqAeyNiVXMX\nGAG1/wY8oU6LHeTxahqPV9PkxuvwcOiBx6s1UcTx889RUVERvlutmVl6kpZFREWavr7Cz8zMUnFg\nmJlZKg4MMzNLxYFhZmapODDMzCwVB4aZmaXiwDAzs1QcGGZmlooDw8zMUnFgmJlZKg4MMzNLxYFh\nZmapODDMzCwVB4aZmaXiwDAzs1Qym0BJUh/gfuBT5OZAmR0RP63VZxYwNa+WM4DSiNguaQOwG9gP\n1KS9X7uZmWUjyxn3aoBvR8RySV2AZZKeiYjVBztExK3ArQCS/hH4HxGxPe89xkTEOxnWaGZmKWV2\nSCoiNkfE8uT5bmAN0KuBVaYA87Kqx8zMjk6LnMOQVAZ8FnilnuUnAeOARXnNATwraZmkyqxrNDOz\nhmV5SAoASZ3JBcG3IuLderr9I/BCrcNRn4+ITZI+CTwj6fWIeL7A+1cClQB9+/Zt5urNzOygTPcw\nJLUjFxZzI+LhBrpOptbhqIjYlHzdAjwCjCi0YkTMjoiKiKgoLS1tnsLNzKyOzAJDkoB7gDUR8ZMG\n+nUFzgMezWvrlJwoR1InYCzwWla1mplZ47I8JDUKmAa8Kqk6afsO0BcgIu5O2iYCv42IPXnrfgp4\nJJc5nAj8KiKeyrBWMzNrRGaBERF/AJSi3xxgTq22dcCwTAozM7Mj4k96m5lZKg4MMzNLxYFhZmap\nODDMzCwVB4aZmaXiwDAzs1QcGGZmlooDw8zMUnFgmJlZKg4MMzNLxYFhZmapODDMzCwVB4aZmaXi\nwDAzs1QcGGZmlooDw8zMUslyitY+khZLWi1plaSZBfqMlrRLUnXy+H7esnGS/iRpraTrsqrTzMzS\nyXKK1hrg2xGxPJmfe5mkZyJida1+SyLiovwGSSXAncA/ABuBpZIeK7CumZm1kMz2MCJic0QsT57v\nBtYAvVKuPgJYGxHrIuIDYD5wSTaVmplZGi1yDkNSGfBZ4JUCiz8naaWkJyUNStp6AW/l9dlI+rAx\nM7MMZHlICgBJnYFFwLci4t1ai5cDfSPiPUkTgF8DA5r4/pVAJUDfvn2boWIzMysk0z0MSe3IhcXc\niHi49vKIeDci3kuePwG0k9QD2AT0yevaO2mrIyJmR0RFRFSUlpY2+/dgZmY5WV4lJeAeYE1E/KSe\nPp9O+iFpRFLPNmApMEBSf0ntgcnAY1nVamZmjcvykNQoYBrwqqTqpO07QF+AiLgbuBS4WlIN8Ddg\nckQEUCNpBvA0UALcGxGrMqzVzMwaodzv5+NDRUVFVFVVFbsMM7NjhqRlEVGRpq8/6W1mZqk4MMzM\nLBUHhpmZpeLAMDOzVBwYZmaWigPDzMxScWCYmVkqDgwzM0vFgWFmZqk4MMzMLBUHhpmZpeLAMDOz\nVBwYZmaWigPDzMxScWCYmVkqDgwzM0slyyla+0haLGm1pFWSZhboM1XSSkmvSnpR0rC8ZRuS9mpJ\nnhXJzKzIspyitQb4dkQsl9QFWCbpmYhYnddnPXBeROyQNB6YDZydt3xMRLyTYY1mZpZSZoEREZuB\nzcnz3ZLWAL2A1Xl9Xsxb5WWgd1b1mJnZ0WmRcxiSyoDPAq800O2rwJN5rwN4VtIySZUNvHelpCpJ\nVVu3bm2Ocs3MrIAsD0kBIKkzsAj4VkS8W0+fMeQC4/N5zZ+PiE2SPgk8I+n1iHi+9roRMZvcoSwq\nKiqi2b8BMzMDMt7DkNSOXFjMjYiH6+kzFPg5cElEbDvYHhGbkq9bgEeAEVnWamZmDcvyKikB9wBr\nIuIn9fTpCzwMTIuIP+e1d0pOlCOpEzAWeC2rWs3MrHFZHpIaBUwDXpVUnbR9B+gLEBF3A98HugP/\nN5cv1EREBfAp4JGk7UTgVxHxVIa1mplZI7K8SuoPgBrpcxVwVYH2dcCwumuYmVmx+JPeZmaWigPD\nzMxScWCYmVkqDgwzM0vFgWFmZqk4MMzMLJVUgSHpM5I+ljwfLembkrplW5qZmbUmafcwFgH7Jf0d\nufs29QF+lVlVZmbW6qQNjAMRUQNMBO6IiFlAz+zKMjOz1iZtYHwoaQrwZeDxpK1dNiWZmVlrlDYw\n/hk4B/hhRKyX1B94ILuyzMystUl1L6lkWtVvAkg6GegSET/OsjAzM2td0l4l9TtJH5f0CWA58J+S\nCt6y3MzMjk9pD0l1TWbL+yfg/og4G7ggu7LMzKy1SRsYJ0rqCVzORye9zcysDUkbGDcBTwNrI2Kp\npFOBNxpaQVIfSYslrZa0StLMAn0k6XZJayWtlHRm3rJxkv6ULLuuKd+UmZk1v1SBEREPRsTQiLgm\neb0uIr7UyGo1wLcjYiAwEviGpIG1+owHBiSPSuAuAEklwJ3J8oHAlALrHjUJVmhQ7knyWKFBqMFp\nn9quCRPgQN5YIXFAYsKEYlfWOnm8jkCvXoeNF716FbuiVqtTJ7jxtLlQVgYnnABlZdx42lw6dcpu\nm6mukpLUAfgqMAjocLA9Iv5bfetExGZgc/J8t6Q1QC9gdV63S8idEwngZUndkkNfZeT2ZtYl25+f\n9M1f96hVM4ihtd5yKKupZhCwqjk3dVx4/EnVmUJRSTtEESpq3TxeTdSrF7z99uFtb7+da9+0qTg1\ntWKzes1l1huVwN5cw5tvMovK3J/fTM1km2kPST0AfBq4EPg90BvYnXYjksqAzwKv1FrUC3gr7/XG\npK2+9mY1jNUF/0MPa95cOm6cQN05d4XvYFkfj1cT1Q6LxtrbuBs/+C6dDoZFohN7ufGD72a2zbQ/\nu38XEdcDeyLiPuALwNlpVpTUmdy9qL6VXGnVrCRVSqqSVLV169bmfnszs9bpr39tWnszSH1rkOTr\nTkmDga7AJxtbSVI7cmExNyIeLtBlE7kbGR7UO2mrr72OiJgdERURUVFaWtroN2Jmdlzo27dp7c0g\nbWDMTj7hfT3wGLlzCbc0tIIkAfcAayKivg/5PQZcmVwtNRLYlZz7WAoMkNRfUntgctK3Wa1gYJ0j\nyZG0W10HqHvkPZJ2q8vj1USnnNK09jbuxvY/ZA8nHda2h5O4sf0PM9tm2qukfh4ROyLi9xFxakR8\nMiLubmS1UcA04HxJ1cljgqTpkqYnfZ4A1gFrgf8EDl6FVQPMIHcp7xpgYUQ0+1noclaxslY4rGQg\n5T7hXdBF46PgL8CLxvsEbiEerybatKluOJxyik941+PWTVO5dcBs6Ncvd0VZv37cOmA2t27K5oQ3\ngHIXKNWzUPqXhlZuYM+hKCoqKqKqqqrYZZiZHTMkLYuIijR9G7ustksz1GNmZseBBgMjIn7QUoWY\nmVnrlvZutfflz+Et6WRJ92ZXlpmZtTZpr5IaGhE7D76IiB3kPohnZmZtRNrAOCG5rBaAZF6MVLcV\nMTOz40PaX/r/i9y9nhYmry8DsrvY18zMWp20U7TeL6kKOD9p+qdk2lYzM2sjGgyM5C6104G/A14F\n7k4+VGdmZm1MY+cw7gMqyIXFeOC2zCsyM7NWqbFDUgMjYgiApHuA/8q+JDMza40a28M4eJdafCjK\nzKxta2wPY5ikg3NYCOiYvBYQEfHxTKszM7NWo7Fbg5S0VCFmZta6ebZIMzNLxYFhZmapODDMzCyV\nzO4HldzN9iJgS0QMLrB8FnBwaqgTgTOA0ojYLmkDsBvYD9SkndzDzMyyk+UexhxgXH0LI+LWiCiP\niHLg34DfR8T2vC5jkuUOCzOzViCzwIiI54HtjXbMmQLMy6oWMzM7ekU/hyHpJHJ7IovymgN4VtIy\nSZXFqczMzPK1hjkt/hF4odbhqM9HxCZJnwSekfR6ssdSRxIolQB9+/bNvlozszaq6HsYwGRqHY6K\niE3J1y3AI8CI+laOiNkRURERFaWlpZkWambWlhU1MCR1Bc4DHs1r6ySpy8HnwFjgteJUaGZmB2V5\nWe08YDTQQ9JG4AagHUBE3J10mwj8NiL25K36KeARSQfr+1VEPJVVnWZmlk5mgRERU1L0mUPu8tv8\ntnXAsGyqMjOzI9UazmGYmdkxwIFhZmapODDMzCwVB4aZmaXiwDAzs1QcGGZmlooDw8zMUnFgmJlZ\nKg4MMzNLxYFhZmapODDMzCwVB4aZmaXiwDAzs1QcGGZmlooDw8zMUnFgmJlZKpkFhqR7JW2RVHB6\nVUmjJe2SVJ08vp+3bJykP0laK+m6rGo0M7P0stzDmAOMa6TPkogoTx43AUgqAe4ExgMDgSmSBmZY\np5mZpZBZYETE88D2I1h1BLA2ItZFxAfAfOCSZi3OzMyarNjnMD4naaWkJyUNStp6AW/l9dmYtBUk\nqVJSlaSqrVu3ZlmrmVmbVszAWA70jYihwB3Ar4/kTSJidkRURERFaWlpsxZoZmYfKVpgRMS7EfFe\n8vwJoJ2kHsAmoE9e195Jm5mZFVHRAkPSpyUpeT4iqWUbsBQYIKm/pPbAZOCxYtVpZmY5J2b1xpLm\nAaOBHpI2AjcA7QAi4m7gUuBqSTXA34DJERFAjaQZwNNACXBvRKzKqk4zM0tHud/Rx4eKioqoqqoq\ndhlmZscMScsioiJN32JfJWVmZscIB4aZmaXiwDAzs1QcGGZmlooDw8zMUnFgmJlZKg4MMzNLxYFh\nZmapODDMzCwVB4aZmaXiwDAzs1QcGGZmlooDw8zMUnFgmJlZKg4MMzNLxYFhZmapZDnj3r3ARcCW\niBhcYPlU4FpAwG7g6ohYkSzbkLTtB2rSTu5hZq3Thx9+yMaNG9m3b1+xS2mzOnToQO/evWnXrt0R\nv0dmgQHMAf4DuL+e5euB8yJih6TxwGzg7LzlYyLinQzrM7MWsnHjRrp06UJZWRmSil1OmxMRbNu2\njY0bN9K/f/8jfp/MDklFxPPA9gaWvxgRO5KXLwO9s6rFzIpr3759dO/e3WFRJJLo3r37Ue/htZZz\nGF8Fnsx7HcCzkpZJqmxoRUmVkqokVW3dujXTIs3syDksiqs5xr/ogSFpDLnAuDav+fMRUQ6MB74h\n6dz61o+I2RFREREVpaWlGVdrZm3FnDlzePvtt1t0m5/73Oea5X2+8pWv8NBDDzXLe+UramBIGgr8\nHLgkIrYdbI+ITcnXLcAjwIjiVGhmLe2WW2Dx4sPbFi/OtbekLAIjIjhw4EC9y1988cVm3V5zK1pg\nSOoLPAxMi4g/57V3ktTl4HNgLPBacao0s5Y2fDhcfvlHobF4ce718OFH/p4bNmzgjDPO4Gtf+xqD\nBg1i7Nix/O1vfwOgurqakSNHMnToUCZOnMiOHTt46KGHqKqqYurUqZSXlx/qe9Dtt9/OwIEDGTp0\nKJMnTwbgxhtv5LbbbjvUZ/DgwWzYsIENGzZw+umnc+WVVzJ48GBuvvlmZs2adajfnDlzmDFjBgCd\nO3cGYPLkyfzmN7851OfgHsP+/fuZNWsWw4cPZ+jQofzsZz8DckE0Y8YMTj/9dC644AK2bNly5IPV\nkIjI5AHMAzYDHwIbyR12mg5MT5b/HNgBVCePqqT9VGBF8lgFfDftNs8666wws9Zn9erVTer/3HMR\nPXpEXH997utzzx3d9tevXx8lJSXxxz/+MSIiLrvssnjggQciImLIkCHxu9/9LiIirr/++pg5c2ZE\nRJx33nmxdOnSgu/Xs2fP2LdvX0RE7NixIyIibrjhhrj11lsP9Rk0aFCsX78+1q9fH5LipZdeioiI\nLVu2xGc+85lD/caNGxdLliyJiIhOnTpFRMTDDz8cV155ZUREvP/++9G7d+/Yu3dv/OxnP4ubb745\nIiL27dsXZ511Vqxbty4WLVoUF1xwQdTU1MSmTZuia9eu8eCDD9apu9C/w8HfvWkemV1WGxFTGll+\nFXBVgfZ1wLCs6jKz1m/MGLj6arj5Zrj++tzro9W/f3/Ky8sBOOuss9iwYQO7du1i586dnHfeeQB8\n+ctf5rLLLmv0vYYOHcrUqVP54he/yBe/+MVG+/fr14+RI0cCUFpayqmnnsrLL7/MgAEDeP311xk1\natRh/cePH8/MmTN5//33eeqppzj33HPp2LEjv/3tb1m5cuWh8xO7du3ijTfe4Pnnn2fKlCmUlJRw\nyimncP755zdpbNIq+klvM7PaFi+Gu+7KhcVdd9U9p3EkPvaxjx16XlJSQk1NzRG/129+8xu+8Y1v\nsHz5coYPH05NTQ0nnnjiYecn8i9h7dSp02HrT548mYULF7Jo0SImTpxY5wqmDh06MHr0aJ5++mkW\nLFjApEmTgNwRoTvuuIPq6mqqq6tZv349Y8eOPeLvo6kcGGbWqhw8Z7FwIdx0U+5r/jmN5tS1a1dO\nPvlklixZAsADDzxwaG+jS5cu7N69u846Bw4c4K233mLMmDH8+Mc/ZteuXbz33nuUlZWxfPlyAJYv\nX8769evr3e7EiRN59NFHmTdv3qFzILVNmjSJX/ziFyxZsoRx48YBcOGFF3LXXXfx4YcfAvDnP/+Z\nPXv2cO6557JgwQL279/P5s2bWZzFYJHtJ73NzJps6dJcSBw8DDVmTO710qXNc2iqtvvuu4/p06ez\nd+9eTj31VH7xi18AuRPN06dPp2PHjrz00kt07NgRgP3793PFFVewa9cuIoJvfvObdOvWjS996Uvc\nf//9DBo0iLPPPpvTTjut3m2efPLJnHHGGaxevZoRIwpfBDp27FimTZvGJZdcQvv27QG46qqr2LBh\nA2eeeSYRQWlpKb/+9a+ZOHEizz33HAMHDqRv376cc845zTxKOcqd8zg+VFRURFVVVbHLMLNa1qxZ\nwxlnnFHsMtq8Qv8OkpZFyvv1+ZCUmZml4sAwM7NUHBhmZpaKA8PMzFJxYJiZWSoODDMzS8WBYWZ2\nhB577DF+9KMfNct7HbzxYGvmwDCz1mfuXCgrgxNOyH2dO7dopTR0C5GLL76Y6667rgWrKS4Hhpm1\nLnPnQmUlvPkmROS+VlYedWjs2bOHL3zhCwwbNozBgwezYMECysrKeOeddwCoqqpi9OjRQO5W5dOm\nTWPUqFFMmzaNkSNHsmrVqkPvNXr0aKqqqg7dmnzXrl3069fv0L2k9uzZQ58+ffjwww/5y1/+wrhx\n4zjrrLP4+7//e15//XUA1q9fzznnnMOQIUP43ve+d1TfW0txYJhZ6/Ld78LevYe37d2baz8KTz31\nFKeccgorVqzgtddeO3R/pvqsXr2aZ599lnnz5jFp0iQWLlwIwObNm9m8eTMVFR99OLpr166Ul5fz\n+9//HoDHH3+cCy+8kHbt2lFZWckdd9zBsmXLuO2227jmmmsAmDlzJldffTWvvvoqPXv2PKrvraU4\nMMysdfnrX5vWntKQIUN45plnuPbaa1myZAldu3ZtsP/FF1986P5Rl19++aFbii9cuJBLL720Tv9J\nkyaxYMECAObPn8+kSZN47733ePHFF7nssssoLy/n61//Ops3bwbghRdeYMqU3CwQ06ZNO6rvraVk\nFhiS7pW0RVLB2fKUc7uktZJWSjozb9k4SX9KlrWdA4RmBn37Nq09pdNOO43ly5cfOgR00003HXZL\n8vzbkcPhtyTv1asX3bt3Z+XKlYfdbjzfxRdfzFNPPcX27dtZtmwZ559/PgcOHKBbt26HbkdeXV3N\nmjVrDq1T+7bmrV2WexhzgIb2+cYDA5JHJXAXgKQS4M5k+UBgiqSBWRQowQEp9yR5HJA4xv4NW8wt\ntxQer5aea/lYMWFC4fGaMKHYlbVib74JX/0qdOhwePtJJ8EPf3hUb/32229z0kknccUVVzBr1iyW\nL19OWVkZy5YtA2DRokUNrj9p0iRuueUWdu3axdChQ+ss79y5M8OHD2fmzJlcdNFFlJSU8PGPf5z+\n/fvz4IMPArn5LFasWAHAqFGjmD9/PgBzj+D8zBtvwM5122DlSqiqgpUr2bluG2+80eS3Si2zwIiI\n54HtDXS5BLg/mSXwZaCbpJ7ACGBtRKyLiA+A+UnfZrcfUTsblLRbXf96beHx+tdrPV6FPP5k4fF6\n/EmPV0Fvvglbt8L48fCd78CnP50L2l69YPZsmDr1qN7+1VdfZcSIEZSXl/ODH/yA733ve9xwww3M\nnDmTiooKSkpKGlz/0ksvZf78+Vx++eX19pk0aRK//OUvD9sDmTt3Lvfccw/Dhg1j0KBBPProowD8\n9Kc/5c4772TIkCFs2rSpyd9Pack2umx/Ez74INfwwQd02f4mpSXbmvxeaWV6e3NJZcDjETG4wLLH\ngR9FxB+S1/8PuBYoA8YlU7giaRpwdkTMaGx7Tb69eUO7EsfRbd+bjceraTxeh6S6vXlD/3crUt19\nu21ZufKjsMjXvj0U2AMC394cSZWSqiRVbd26tdjlmJm1jEJh0VB7MyhmYGwC+uS97p201ddeUETM\njoiKiKgoLS3NpFAzs1YnmYUvdXszKGZgPAZcmVwtNRLYFRGbgaXAAEn9JbUHJid9m90BoPaBgUja\nrS6PV9N4vJqovj/4/IdgQTs792J/rV/h+zmBnZ17ZbbNLC+rnQe8BJwuaaOkr0qaLml60uUJYB2w\nFvhP4BqAiKgBZgBPA2uAhRGxqs4GmkEJUfA/dEmdVgO47ceFx+u2H3u8CrlofOHxumh82xyvRs+X\n9utXNxxKS3PtVsfW/d3Z/Yl+H+1RtG/P7k/0Y+v+7gX7N8f5as/pbWaZW79+PV26dKF79+7H3GcP\njgcRwbZt29i9ezf9+/c/bFlTTnqfmEl1ZmZ5evfuzcaNG/GFKcXToUMHevfufVTv4cAws8y1a9eu\nzl+2duw55i+rNTOzluHAMDOzVBwYZmaWynF1lZSkrcCbR7h6D+CdZiynubiupnFdTeO6muZ4rKtf\nRKT6sMtxFRhHQ1JV2kvLWpLrahrX1TSuq2nael0+JGVmZqk4MMzMLBUHxkdmF7uAeriupnFdTeO6\nmqZN1+VzGGZmlor3MMzMLJU2FRiSxkn6k6S1kq4rsFySbk+Wr5R0Ziupa7SkXZKqk8f3W6iueyVt\nkfRaPcuLNV6N1VWs8eojabGk1ZJWSZpZoE+Lj1nKulp8zCR1kPRfklYkdf2gQJ9ijFeauoryM5Zs\nu0TSH5NZS2svy3a8IqJNPIAS4C/AqUB7YAUwsFafCcCT5KZeHgm80krqGk1uqtuWHrNzgTOB1+pZ\n3uLjlbKuYo1XT+DM5HkX4M+t5GcsTV0tPmbJGHROnrcDXgFGtoLxSlNXUX7Gkm3/C/CrQtvPerza\n0h7GCGBtRKyLiA+A+cAltfpcAtwfOS8D3ST1bAV1FUVEPA9sb6BLMcYrTV1FERGbI2J58nw3uflc\nas9m0+JjlrKuFpeMwXvJy3bJo/ZJ1WKMV5q6ikJSb+ALwM/r6ZLpeLWlwOgFvJX3eiN1/9Ok6VOM\nugA+l+xiPilpUMY1pVWM8UqrqOMlqQz4LLm/TvMVdcwaqAuKMGbJ4ZVqYAvwTES0ivFKURcU52fs\n/wD/k/onbsx0vNpSYBzLlgN9I2IocAfw6yLX09oVdbwkdQYWAd+KiHdbctsNaaSuooxZROyPiHKg\nNzBC0uCW2G5jUtTV4uMl6SJgS0Qsy3pb9WlLgbEJ6JP3unfS1tQ+LV5XRLx7cBc5Ip4A2knqkXFd\naRRjvBpVzPGS1I7cL+W5EfFwgS5FGbPG6ir2z1hE7AQWA+NqLSrqz1h9dRVpvEYBF0vaQO7Q9fmS\nflmrT6bj1ZYCYykwQFJ/Se2BycBjtfo8BlyZXGkwEtgVEZuLXZekT0u5eS0ljSD377Yt47rSKMZ4\nNapY45Vs8x5gTUT8pJ5uLT5maeoqxphJKpXULXneEfgH4PVa3YoxXo3WVYzxioh/i4jeEVFG7vfE\ncxFxRa1umY5Xm5lxLyJqJM0AniZ3ZdK9EbFK0vRk+d3AE+SuMlgL7AX+uZXUdSlwtaQa4G/A5Egu\niciSpHnkrgbpIWkjcAO5E4BFG6+UdRVlvMj9BTgNeDU5/g3wHaBvXm3FGLM0dRVjzHoC90kqIfcL\nd2FEPF7s/5Mp6yrWz1gdLTle/qS3mZml0pYOSZmZ2VFwYJiZWSoODDMzS8WBYWZmqTgwzMwsFQeG\n2VGQ1E2uOzKBAAABK0lEQVTSNcnzUyQ9lDwvlzQhr99XJP1Hseo0aw4ODLOj0w24BiAi3o6IS5P2\ncnLXw5sdN9rMB/fMMvIj4DPJB+LeAM4gd+v1m4COkj4P/Hv+CpJKgbtJPjhH7t5OL7RcyWZHxnsY\nZkfnOuAvyY3qZgEkt6n/PrAgIsojYkGtdX4K/O+IGA58ifpvVW3WqngPw6zlXQAMTG5FBPBxSZ3z\n5mAwa5UcGGYt7wRyM7jtK3YhZk3hQ1JmR2c3uWlP07YD/Bb47wdfSCrPoC6zZufAMDsKEbENeEHS\na8CteYsWkzvsVC1pUq3VvglUJLO1rQamt1C5ZkfFd6s1M7NUvIdhZmapODDMzCwVB4aZmaXiwDAz\ns1QcGGZmlooDw8zMUnFgmJlZKg4MMzNL5f8DkwCl3fFy11EAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0xacab7b8>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "X = np.array(datain['train'].ix[:,1:])\n",
    "y = np.array(datain['train'].ix[:,0])\n",
    "\n",
    "Xvad = np.array(datain['validation'].ix[:,1:])\n",
    "yvad = np.array(datain['validation'].ix[:,0])\n",
    "\n",
    "m = X.shape[0]\n",
    "Xfull = titan.normdata(np.concatenate((X, Xvad)))\n",
    "\n",
    "X = np.concatenate((np.ones(shape=(X.shape[0], 1)), Xfull[:m, :]), axis = 1)\n",
    "Xvad = np.concatenate((np.ones(shape=(Xvad.shape[0], 1)), Xfull[m:, :]), axis = 1)\n",
    "\n",
    "plt.plot(datain['train'].title[y==0], datain['train'].Pclass[y==0], 'xb')\n",
    "plt.plot(datain['train'].title[y==1], datain['train'].Pclass[y==1], 'or')\n",
    "plt.xlabel('title')\n",
    "plt.ylabel('Pclass')\n",
    "plt.legend(['not survived', 'survived'])\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The figure above shows the potential difficulty of a linear logistic regression. Most of the important features, like Sex, title, and Pclass are discrete sets of small numbers. As a result, there is heavy overlap of the outcome (survived or not) that makes drawing of decision boundary very difficult. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYwAAAEKCAYAAAAB0GKPAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3X14FPW99/H3l00QAgpIolUhCbaWmihQieATVUxqSbR4\ne2oFSml7X/VQQz219+nxPq1KbevV69ygeJ3Wtj6caq1N6rPWHq1aPaT1qbaGyDNqq6CGYnlQUUCU\nkO/9x8wmu5tNdvKw7CKf13XNtZnf/H6/+e7M7HwzM7sz5u6IiIhkMijXAYiIyP5BCUNERCJRwhAR\nkUiUMEREJBIlDBERiUQJQ0REIlHCEBGRSJQwREQkEiUMERGJpCDXAQyk4uJiLy8vz3UYIiL7jWXL\nlm1195IodT9UCaO8vJzm5uZchyEist8ws1ej1tUpKRERiUQJQ0REIlHCEBGRSJQwREQkEiUMERGJ\nJGsJw8yGmNlfzGyFma0xs++nqWNm9mMz+5uZrTSzExKmzTCzF8Np385WnIsnNvLBQQeDGZix1wbx\ndFEN/xhaDoMGQXk5lx7VyE2FC9hrBbhZl2GvFbDl8Er2WkHQT0EBLFiQNJ+6Orj22oT5Lg6q1NV1\nlpWWwj2HLwjam9E+qICHyhfwiU90trn22uA1rqkJ/nt2I5R3xktjY9J81lzeyPZR5Xg4/ednNvLz\nMxt57/By3AbxqpWzrqASD5eBm7HKKnnVOvuca43Mtc75vDaonFWW3Ob3VsPgwUFMECyK9pRlFR9P\nbGPW2aagAJoKapLavGZH0RYu2zYrYJuN6ugDM9YVVPKY1XSMx/utqwv6HTYMVoSxxocVVskWG5XU\n5h0rYkP4njcWlvPnETVJ6/S/Bi/gwqLOZfCPoeV87+ONHeunpAR2WFFSbO9aEcOHJ28H557bud4/\n8Qn43scbk7a3b4xu5KJDOuezsTCYT0lJ5/o/99yg7bXXBn1NnQqf/Wzyth2flrgdJG4nFxY1culR\nydvGksnBfJqa6Fh+iZqakre/VIsXd20zdSqcc073sfVGuv4zxdSbtpH6b+z+89bfGKPIdv89cves\nDIABw8O/C4E/Ayel1KkDHg7rngT8OSyPAS8DRwODgRVARaZ5Tp482XulocHbBhW4Q9LQnjL+AQVd\nyjK1cXCvr++Y1ZIl7mbBq3swKbHK0qXuP7P6Lv20gz9YVp+2j6VL3ecPb/C2g4qS51tU5N7Q4O7u\nqy9r8J0kT9/NYH+Pwh7jTx1/j0LfzeCMbR6l2ouL3Q8/3H1vhH4T2yxZ4v4o1ZHapI731O9yKiK3\nyTT+AbGksh0U+ZXHBMv6XYambbOdoR3rbObMYNLMmcH4lcc0+I6U9ZNuWe+gyL9R3OBm7pWVQXFl\nZbA91Ne7DxuWvG2kbitRt4P4+4mvj+LiYDuLb2+J4+mk1lm6NIgNuo+tN9L1nymm3rTNWKehIfh8\ndfN562+MUQx0/0CzR92vR63YnwEoAlqAqSnlNwJzEsZfBI4ATgYeTSj/DvCdTPPpdcIoK0te8QM9\nxGJJs4t/UKZN6/ygFxe7L1wYvO4dFEvbzx5iHXXiH+L4+K7DunkPZWX75j2m2UHGRzMl2cQ28+YF\nyyRqm97EMpB9phvWU+bTpnU/n3ZIWu8zZ3ZuB+uJvn7WU+YjRwajhx4avH760507i9TtK2mH3Ivt\n4LVBZV12RvHtrTc75sQ2PcbWS32JqTdte6zT3XKMf94GIMaBeh9R5U3CCI8UlgM7gEVppj8InJYw\n/j9AFXA+8POE8nnAT7qZx3ygGWguLS3t3ZIyy+qOJFi8yaZNC4qnTQvGFy4Mxhcu9G77iO/4Fi5M\n06a792C2795jmlihdwkjvkz2x4SxF8s4n9T1Hh/fS/T1E5/P4PDgY+zYhO2gm+2rL9t6O5bUNGl7\niyhdm25j64O+xNSbtt3WyfR5G6AYoxio/vMmYXTMBEYCTcBxKeX9ThiJg44wEgYdYegIQ0cYfWqr\nI4wcJ4wgJr4L/FtKWW5PSekaRrfx6xqGrmHoGoauYaQOA5oUkjqGEmBk+PdQ4EngnJQ6Z6dc9P5L\nWF4AvAKMS7joXZlpnr1OGO6+aEKDvz94eMfKb8P8qaHV/saQsmCrLivzfzuywW8sqPc2Yh07msSh\njZhvPqzC2+I7k1gsKVm4u9fWJn9AFi0KqtTWdpaNHet+92H1QXvwvRbzB8vqffz4zjZLlgSvcUuX\nuv92VkPwH04Yb+LGu2hRsLN4e2SZt4fT/2t6g//X9AbfdViZt2O+gTJfG+vcqbaDr6TCN9DZ5xdo\n8C/QOZ9XrcxXktzmUaq9sLBzw4XOpBEfEpNIvA10tonF3JfGqpPavMqRvidctnuI+VZGJu2Y18Yq\n/PdUd0kWtbVBv0VFnUkjPiynwjczssuOfX34nlsLyvzZQ6qT1ulNhfX+1aGdy+CNIWV+5TENHeun\nuLhr0niHoT5sWPJ2MHNm53ofPz5IGonb278c2uBfO7hzPq0FZUk78UWLgj7Gjw/Ga2vdp0xxP+ec\n5G07Pi1xO0jcTr46tMH/7cjkbeOaExo6dj7x5Zdo6dLk7S/VokVd20yZ4n722d3H1hvp+s8UU2/a\nRuq/ofvPW39jjGKg++9NwrCg/sAzswnALwmuYwwC7nL3H5jZRQDufoOZGfATYAawC/jf7t4ctq8D\n/jNsf4u7/zDTPKuqqlw3HxQRic7Mlrl7VaS62UoYuaCEISLSO71JGPqlt4iIRKKEISIikShhiIhI\nJEoYIiISiRKGiIhEooQhIiKRKGGIiEgkShgiIhKJEoaIiESihCEiIpEoYYiISCRKGCIiEokShoiI\nRKKEISIikShhiIhIJAXZ6tjMxgK3AYcDDtzk7j9KqXMpMDchlmOBEnd/08w2AO8Ce4G2qPdrFxGR\n7MhawgDagG+5e4uZHQwsM7PH3H1tvIK7Xw1cDWBmnwX+j7u/mdDHdHffmsUYRUQkoqydknL3Te7e\nEv79LrAOOKqHJnOA27MVj4iI9M8+uYZhZuXAJ4E/dzO9iOC53vcmFDvwuJktM7P52Y5RRER6ls1T\nUgCY2XCCRPBNd3+nm2qfBZ5OOR11mrtvNLPDgMfM7AV3fyJN//OB+QClpaUDHL2IiMRl9QjDzAoJ\nkkWju9/XQ9XZpJyOcveN4etm4H5gSrqG7n6Tu1e5e1VJScnABC4iIl1kLWGYmQE3A+vc/doe6o0A\nTgceSCgbFl4ox8yGAWcBq7MVq4iIZJbNU1KnAvOAVWa2PCy7DCgFcPcbwrLzgN+7+86EtocD9wc5\nhwLg1+7+SBZjFRGRDLKWMNz9KcAi1LsVuDWl7BVgYlYCExGRPtEvvUVEJBIlDBERiUQJQ0REIlHC\nEBGRSJQwREQkEiUMERGJRAlDREQiUcIQEZFIlDBERCQSJQwREYlECUNERCJRwhARkUiUMEREJBIl\nDBERiUQJQ0REIlHCEBGRSLL5iNaxZtZkZmvNbI2ZXZKmzhlmtt3MlofDdxOmzTCzF83sb2b27WzF\nKSIi0WTzEa1twLfcvSV8PvcyM3vM3dem1HvS3c9JLDCzGPBT4NNAK/Ccmf02TVsREdlHsnaE4e6b\n3L0l/PtdYB1wVMTmU4C/ufsr7v4BcAdwbnYiFRGRKPbJNQwzKwc+Cfw5zeRTzGylmT1sZpVh2VHA\n6wl1WomebEREJAuyeUoKADMbDtwLfNPd30mZ3AKUuvsOM6sDfgMc08v+5wPzAUpLSwcgYhERSSer\nRxhmVkiQLBrd/b7U6e7+jrvvCP/+HVBoZsXARmBsQtUxYVkX7n6Tu1e5e1VJScmAvwcREQlk81tS\nBtwMrHP3a7up85GwHmY2JYxnG/AccIyZjTOzwcBs4LfZilVERDLL5impU4F5wCozWx6WXQaUArj7\nDcD5QL2ZtQHvAbPd3YE2M7sYeBSIAbe4+5osxioiIhlYsH/+cKiqqvLm5uZchyEist8ws2XuXhWl\nrn7pLSIikShhiIhIJEoYIiISiRKGiIhEooQhIiKRKGGIiEgkShgiIhKJEoaIiESihCEiIpEoYYiI\nSCRKGCIiEokShoiIRKKEISIikShhiIhIJEoYIiISiRKGiIhEks1HtI41syYzW2tma8zskjR15prZ\nSjNbZWbPmNnEhGkbwvLlZqanIomI5Fg2H9HaBnzL3VvM7GBgmZk95u5rE+qsB05397fMrBa4CZia\nMH26u2/NYowiIhJR1hKGu28CNoV/v2tm64CjgLUJdZ5JaPIsMCZb8YiISP/sk2sYZlYOfBL4cw/V\nvgo8nDDuwONmtszM5vfQ93wzazaz5i1btgxEuCIikkY2T0kBYGbDgXuBb7r7O93UmU6QME5LKD7N\n3Tea2WHAY2b2grs/kdrW3W8iOJVFVVWVD/gbEBERIMtHGGZWSJAsGt39vm7qTAB+Dpzr7tvi5e6+\nMXzdDNwPTMlmrCIi0rNsfkvKgJuBde5+bTd1SoH7gHnu/lJC+bDwQjlmNgw4C1idrVhFRCSzbJ6S\nOhWYB6wys+Vh2WVAKYC73wB8FxgN/CzIL7S5exVwOHB/WFYA/NrdH8lirCIikkE2vyX1FGAZ6lwI\nXJim/BVgYtcWIiKSK/qlt4iIRKKEISIikShhiIhIJFn/HYaIyJ49e2htbWX37t25DuWANWTIEMaM\nGUNhYWGf+1DCEJGsa21t5eCDD6a8vJzw24+yD7k727Zto7W1lXHjxvW5H52SEpGs2717N6NHj1ay\nyBEzY/To0f0+wouUMMzso2Z2UPj3GWb2DTMb2a85i8gBRckitwZi+Uc9wrgX2GtmHyO4b9NY4Nf9\nnruISJ669dZb+fvf/75P53nKKacMSD9f+cpXuOeeewakr0RRE0a7u7cB5wHXufulwBEDHo2IHPAW\nL4ampuSypqagfF/KRsJwd9rb27ud/swzz3Q7LR9ETRh7zGwO8GXgwbCs75faRUS6ceKJcMEFnUmj\nqSkYP/HEvve5YcMGjj32WP75n/+ZyspKzjrrLN577z0Ali9fzkknncSECRM477zzeOutt7jnnnto\nbm5m7ty5TJo0qaNu3I9//GMqKiqYMGECs2fPBuB73/se11xzTUed4447jg0bNrBhwwbGjx/Pl770\nJY477jiuuuoqLr300o56t956KxdffDEAw4cPB2D27Nk89NBDHXXiRwx79+7l0ksv5cQTT2TChAnc\neOONQJCILr74YsaPH09NTQ2bN2/u+8LqibtnHIAK4MfAnHB8HPDvUdruy2Hy5MkuIvln7dq1vaq/\ndKl7cbH7woXB69Kl/Zv/+vXrPRaL+fPPP+/u7p///Of9V7/6lbu7H3/88f6HP/zB3d0XLlzol1xy\nibu7n3766f7cc8+l7e+II47w3bt3u7v7W2+95e7uV155pV999dUddSorK339+vW+fv16NzP/05/+\n5O7umzdv9o9+9KMd9WbMmOFPPvmku7sPGzbM3d3vu+8+/9KXvuTu7u+//76PGTPGd+3a5TfeeKNf\nddVV7u6+e/dunzx5sr/yyit+7733ek1Njbe1tfnGjRt9xIgRfvfdd3eJO916AJo94j420hGGu691\n92+4++1mNgo42N0XZSeFiciBbvp0qK+Hq64KXqdP73+f48aNY9KkSQBMnjyZDRs2sH37dt5++21O\nP/10AL785S/zxBNdHrvTxYQJE5g7dy4NDQ0UFGT+dUJZWRknnXQSACUlJRx99NE8++yzbNu2jRde\neIFTTz01qX5tbS1NTU28//77PPzww3zqU59i6NCh/P73v+e2225j0qRJTJ06lW3btvHXv/6VJ554\ngjlz5hCLxTjyyCM588wze7t4Ion6Lak/mNkhZnYo0AL8l5mlvWW5iEh/NTXB9dfDwoXBa+o1jb44\n6KCDOv6OxWK0tbX1ua+HHnqIr3/967S0tHDiiSfS1tZGQUFB0vWJxK+wDhs2LKn97Nmzueuuu7j3\n3ns577zzunyDaciQIZxxxhk8+uij3HnnncyaNQsIzghdd911LF++nOXLl7N+/XrOOuusPr+P3op6\nDWOEB0/L+yfgNnefCtRkLywROVDFr1ncdRf84AfBa+I1jYE0YsQIRo0axZNPPgnAr371q46jjYMP\nPph33323S5v29nZef/11pk+fzqJFi9i+fTs7duygvLyclpYWAFpaWli/fn238z3vvPN44IEHuP32\n2zuugaSaNWsWv/jFL3jyySeZMWMGAJ/5zGe4/vrr2bNnDwAvvfQSO3fu5FOf+hR33nkne/fuZdOm\nTTRlY2ER/ZfeBWZ2BHABcHlWIhERAZ57LkgS8dNQ06cH4889NzCnplL98pe/5KKLLmLXrl0cffTR\n/OIXvwCCC80XXXQRQ4cO5U9/+hNDhw4FYO/evXzxi19k+/btuDvf+MY3GDlyJJ/73Oe47bbbqKys\nZOrUqXz84x/vdp6jRo3i2GOPZe3atUyZkv5homeddRbz5s3j3HPPZfDgwQBceOGFbNiwgRNOOAF3\np6SkhN/85jecd955LF26lIqKCkpLSzn55JMHeCkFLLjmkaGS2eeBhcBT7r7AzI4Grnb3z/XQZixw\nG8HDkBy4yd1/lFLHgB8BdcAu4Cvu3hJOmxFOiwE/d/f/lynOqqoqb25uzvh+RGTfWrduHccee2yu\nwzjgpVsPZrbMgwfXZRT1ovfd7j7B3ReE46/0lCxCbcC33L0COAn4uplVpNSpBY4Jh/nA9eEbiAE/\nDadXAHPStB0YjY1QXAxmwVBcDDU1UFAQjBcUwIIFQVm8jlkwntpPeTkMGhS8NjZGm3dv20R9T4n9\nLljQ+/ksWJC0DBpGLOCWms5+NxaW83RRDW3WWecPlQv4xCc6u6irg+dPSe5nwzE1/GNoZx9/qEyO\n7V9GN7JmUGXHcnYzNgyvpK4u6HPx4iC0+DjA1Klwzjmd44sXw9qjavCEPt4pGNUxjhmPWg0/sc7Y\n2gcVcP2gBcRPJS9eDA/NbeQfQ8uDdgUFuBnvFQzHB8VwM9qsIKlfN2PLQUexsbDz/VxY1MjTRUEs\n8XrPF9dQUhKcYqmri/ibg5T1QU1N0nJbPLExaVm3DwrWR+Jyu/ba5H6bmuC/Z3eu0+2jyllzefK2\nkRpLpN9IpHxWXvlYDal3pHjnHXjjDbr11792nf72K9vYs2wlNDfDypWwbVv3HaTxxhvBfHsTx0DI\n1XyzIspXqYAhwNeBnwG3xIeoX8UK+3gA+HRK2Y2EX9UNx18k+EHgycCjCeXfAb6TaR69/lptQ4N7\nYaE79G2oru7sp6goeVpRUVDe07x72ybqe0rtN3XINJ/6+i5t2sE/oKBLWep4U0V9RzctJ9enrdPT\n+F4sbZ31wyqSQqsPZ7N0qfuwYUHZkiVB2ZojqyPNN13Zzyzo+MEvNPgOMizHCO9nTzd1nj2k2ouL\ng5gTvzYa/zpp0tdI06yP1KGNQT2ujyVL3M06l9HSpe7zhzd420HJ73EnRb76soZuY0kt61Knujrt\nMmp5uMm3bw+qbN/u/vzz3jGezqZN7s89F7y6u7/18lZve25ZUBgfli1z37q1+05SpM43ShwDIVfz\nTae/X6uNurO/G7gKeJngx3u/B34UeSZQDrwGHJJS/iBwWsL4/wBVwPkEp6Hi5fOAn2SaT68TRllZ\nxg9ixqGnfsrKej/vntoM5HvqaT6xWN+XRyw2MP2k2elMmxbs9Orru35HP75DnDbNu+w4ezPsIeYL\nF7q/NmgAto0M7yd1p9vtbw76sRzj7ycxOcXHdx2W/j2+Nqisx98/9BhvN3Gsffhhf/5599bW6DvL\neNJYt85993MrkpNFfFixInNHCeI7697EMRByNd9U+yphPB++rgxfC4FnI7YdDiwD/inNtH4nDIJT\nWc1Ac2lpae+Wnln/P/w99WPW+3n31GYg31NP8xmIZTIQ/aTsYCFICO7BzgqC17hp04Ky/iSM+Hza\nGYBtI8N8EqV7PwOxHOPvJ95v0ny62Vbase5jyRRvN3Gsffhhb20N9vGtrd33m2rduqBNe7pkER96\nqS9xDIRczTfRPvnhHrAnfH3bzI4DRgCHZWpkZoUENy5sdPf70lTZSHAjw7gxYVl35V24+03uXuXu\nVSUlJRnfSJLS0t7V720/PfXflzb9iaU39WKxvs8/sW1/+klj2jR46qngdH7qd/SvvTaYNm1a/+ax\nlxgLF0LroAHaNnqQeOuLHn9z0I/lGH8/118fLKPE+bxXkv49tg4q7fH3D339jcSWLXDEEcFr6jn9\ndN54A3bsgOHD4QMGp680uJvybrzzTu/jGAi5mu+Ai5JVgAuBUcDpwCvAZuCiDG2M4FtS/9lDnbOB\nh8O6JwF/CcsLwvmMAwYDK4DKTHHqGkY3/aYOuoahaxi6hqFrGCEG+pRUXwbgNMCBlcDycKgDLoon\nmzBR/JTg2sgqoCqhfR3wUjjt8ijz7NO9pBoa3EeP7tzAR48ONvr4eeNYLPjApn4Q4skisZ+ysuBT\nWVYWbcfflzZR31Niv/X1vZ9PfX3SMvjVIfV+c3Vnv60FZf7U0GrfQ2edpop6Hz++s4va2iBpJPaz\n/mPV/saQzj6aKpJju/jQBl9tFV2SRW1t0OeiRUFo8XF39ylT3M8+u3N80aLkpNEOvj02MmmH+gjV\nfh2dse21mP/M6oNPRNjHg19o8DeGlAXtYjFvB98VG+btFuyc9xBL6rcdfPPgI721oPP9fHVogz81\ntDopQbWMru7YydbWdr1OsHRpMP+e1odXVyctt0UTGpKW9V4L1kficluyJLnfpUvdfzurc52+PbKs\nI1l0F8uiRRHiTfmsvPzRam9pSd5Rbd/emQzSeemlrtPfenmrf9C8ovPaRS+ShXvQX+pOOlMcmTzw\nwAP+H//xHwMy3/h9pLKpvwmjx99hmNm/Zjg6yavbg+h3GCL5qde/w2hshMsvh9deC06f/vCHMHdu\n9gLsQfy2H9k2fPhwduzYkdV5ZPt3GAdnGEREBlZjI8yfD6++GhyjvPpqMN7P3ynt3LmTs88+m4kT\nJ3Lcccdx5513Ul5eztatWwFobm7mjDPOAIJblc+bN49TTz2VefPmcdJJJ7FmzZqOvs444wyam5s7\nbk2+fft2ysrKOu4ltXPnTsaOHcuePXt4+eWXmTFjBpMnT2batGm88MILAKxfv56TTz6Z448/niuu\nuKJf721f6TFhuPv3exr2VZAicgC5/HLYtSu5bNeuoLwfHnnkEY488khWrFjB6tWrO+7P1J21a9fy\n+OOPc/vttzNr1izuuusuADZt2sSmTZuoqur8p3zEiBFMmjSJP/7xjwA8+OCDfOYzn6GwsJD58+dz\n3XXXsWzZMq655hoWLFgAwCWXXEJ9fT2rVq3iiCP2j+fRRb1b7S8Tn+FtZqPM7JbshSUiB6zXXutd\neUTHH388jz32GP/+7//Ok08+yYgRI3qsP3PmzI77R11wwQUdjzy96667OP/887vUnzVrFnfeeScA\nd9xxB7NmzWLHjh0888wzfP7zn2fSpEl87WtfY9OmTQA8/fTTzJkzB4B58+b1673tK1FPzE1w97fj\nI+7+lpl9MksxiciBrLQ0OA2VrrwfPv7xj9PS0sLvfvc7rrjiCqqrq5NuSb475f4libckP+qooxg9\nejQrV67kzjvv5IYbbujS/8yZM7nssst48803WbZsGWeeeSY7d+5k5MiRLF++PG1Mqbc1z3dRf4cx\nKHxwEgDhczGyfxVIRA48P/whFBUllxUVBeX98Pe//52ioiK++MUvcumll9LS0kJ5eTnLli0D4N57\n7+2x/axZs1i8eDHbt29nwoQJXaYPHz6cE088kUsuuYRzzjmHWCzGIYccwrhx47j77ruB4FupK1as\nAODUU0/ljjvuAKBxoO4jl2VRE8YS4Fkzu8rMrgKeAfbxI9lF5IAwdy7cdBOUlQU3MCwrC8b7+S2p\nVatWMWXKFCZNmsT3v/99rrjiCq688kouueQSqqqqiGX4geT555/PHXfcwQUXXNBtnVmzZtHQ0NDx\nwCMIksHNN9/MxIkTqays5IEHHgDgRz/6ET/96U85/vjj2bgx7e+S806k25sDhHeLjT/3b6m7r81a\nVH2kr9WK5Cfd3jw/9PdrtT2eVjKzIQQ/tPsYwQ/rbnD3vj/XUERE9luZTkn9kuBmgKsInk1xTdYj\nEhGRvJTpwnWFux8PYGY3A3/JfkgiIpKPMh1hxO9Si05FiUh/RL1eKtkxEMs/0xHGRDOL34jXgKHh\nuAXz90P6HYGIfOgNGTKEbdu2MXr06P3utwcfBu7Otm3bGDJkSL/66TFhuPvAPtBARA5IY8aMobW1\nlS1btuQ6lAPWkCFDGDNmTL/60I/vRCTrCgsLGTduXK7DkH6K+sM9ERE5wClhiIhIJFk7JRXezfYc\nYLO7H5dm+qVA/Lf+BcCxQIm7v2lmG4B3gb1AW9RfIYqISPZk8wjjVqDbG867+9XuPsndJwHfAf7o\n7m8mVJkeTleyEBHJA1lLGO7+BPBmxoqBOcDt2YpFRET6L+fXMMysiOBIJPHewg48bmbLzGx+biIT\nEZFE+fC12s8CT6ecjjrN3Tea2WHAY2b2QnjE0kWYUOYDlPbzASsiItK9nB9hALNJOR3l7hvD183A\n/cCU7hq7+03uXuXuVSUlJVkNVETkQJbThGFmI4DTgQcSyoaZ2cHxv4GzgNW5iVBEROKy+bXa24Ez\ngGIzawWuBAoB3D3+QNzzgN+7+86EpocD94f3mykAfu3uj2QrThERiSZrCcPd50SocyvB128Ty14B\nJmYnKhER6at8uIYhIiL7ASUMERGJRAlDREQiUcIQEZFIlDBERCQSJQwREYlECUNERCJRwhARkUiU\nMEREJBIlDBERiUQJQ0REIlHCEBGRSJQwREQkEiUMERGJRAlDREQiUcIQEZFIspYwzOwWM9tsZmkf\nr2pmZ5jZdjNbHg7fTZg2w8xeNLO/mdm3sxWjiIhEl80jjFuBGRnqPOnuk8LhBwBmFgN+CtQCFcAc\nM6vIYpwiIhJB1hKGuz8BvNmHplOAv7n7K+7+AXAHcO6ABiciIr2W62sYp5jZSjN72Mwqw7KjgNcT\n6rSGZWlOF7vnAAANxElEQVSZ2Xwzazaz5i1btmQzVhGRA1ouE0YLUOruE4DrgN/0pRN3v8ndq9y9\nqqSkZEADFBGRTjlLGO7+jrvvCP/+HVBoZsXARmBsQtUxYZmIiORQzhKGmX3EzCz8e0oYyzbgOeAY\nMxtnZoOB2cBvcxWniIgECrLVsZndDpwBFJtZK3AlUAjg7jcA5wP1ZtYGvAfMdncH2szsYuBRIAbc\n4u5rshWniIhEY8E++sOhqqrKm5ubcx2GiMh+w8yWuXtVlLq5/paUiIjsJ5QwREQkEiUMERGJRAlD\nREQiUcIQEZFIlDBERCQSJQwREYlECUNERCJRwhARkUiUMEREJBIlDBERiUQJQ0REIlHCEBGRSJQw\nREQkEiUMERGJRAlDREQiyVrCMLNbzGyzma3uZvpcM1tpZqvM7Bkzm5gwbUNYvtzM9EQkEZE8kM0j\njFuBGT1MXw+c7u7HA1cBN6VMn+7uk6I+CUpERLIra8/0dvcnzKy8h+nPJIw+C4zJViwiItJ/+XIN\n46vAwwnjDjxuZsvMbH5PDc1svpk1m1nzli1bshqkiMiBLGtHGFGZ2XSChHFaQvFp7r7RzA4DHjOz\nF9z9iXTt3f0mwtNZVVVVnvWARUQOUDk9wjCzCcDPgXPdfVu83N03hq+bgfuBKbmJUERE4nKWMMys\nFLgPmOfuLyWUDzOzg+N/A2cBab9pJSIi+07WTkmZ2e3AGUCxmbUCVwKFAO5+A/BdYDTwMzMDaAu/\nEXU4cH9YVgD82t0fyVacIiISTTa/JTUnw/QLgQvTlL8CTOzaQkREcilfviUlIiJ5TglDREQiUcIQ\nEZFIlDBERCQSJQwREYlECUNERCJRwhARkUiUMEREJBIlDBERiUQJQ0REIlHCEBGRSJQwREQkEiUM\nERGJRAlDREQiUcIQEZFIspYwzOwWM9tsZmmflmeBH5vZ38xspZmdkDBthpm9GE77drZiFBGR6LJ5\nhHErMKOH6bXAMeEwH7gewMxiwE/D6RXAHDOryGKc0NgI5eUwaBDbR5Wz5vLGpMlNTbB4cfdtKC8P\nxvNVlFgXLICCAjALXmtqktr89+zGYLkkLKcNx9TQPqizzevnLkhaTosXk7HNe8VH4WbBuBnLRtew\n4ZiajnE3Y1vRUey1zjbbJtewfVRybE1NnfOtq4M/VC7omE/7oAL+ULmAurrOOmaw1UZ1zAczdtlg\n2qz7ZUBjY9dluWBB8nhCfx1DTfL72XJ4ZdIyaR1VmbQMtk2uSVqOdXXw/CnJ62fDMTX8Y2j364fy\nYDuO97N4Mbx+bnIfdx+2gGs+2cj2UeV4GMstNY3U1QXbfEkJnHtu8ra/YAF864iUZVBT09HvXgti\nS5x+zScbmTo1ebt4aG5j0jpMjDXde05dh3V1QSyJba69lqR1vHgxbJtck7QuUpdt6vpM3ZYWLw76\nTWyz5vLGjNtfptgGQl1d0G+ibMwnibtnbQDKgdXdTLsRmJMw/iJwBHAy8GhC+XeA70SZ3+TJk73X\nGhrci4rcoWPYSZGvvqzB3d2XLnUvLg5ee2rjRUVBeb6JEmt9ffL0NMPeWKHvZnBSWXtKnXbw12bW\nd3S7+rIG30lRxjap41HqJI63HVTk84c3dKyjpor6tG2aKoLYli5138zIjP12GQoL3QcP7rlOhCHK\n+9t6QnXHcmw5Of37ybR+Erfj12am7+N9BiWV7aDIrzmhwYuL3U85JSieObNzM5lDg+8uKOrV+9tB\nkX+lsHP9PPiFBt9B95+5nt5zfB3GN9n6cHNbssTdLHiN23pCdc/LNs1nI3VbSu033Tad2iZKbAMh\ntd++zgdo9gz71vgQqVJfhwwJ40HgtITx/wGqgPOBnyeUzwN+EmV+fUoYZWVpN/rXBpX5woVpkkUP\nbbysrPfzz7YoscZivd7pdTvEYpnnnYVh12FlXlzsvnCh+x7Sv589xDrWacbkkA/DQKyf+HruRR/r\nKfN584LlNHNmUDx2bPC6dXjf1mni+nltUN+3ycR1WF8f7CCnTetmR5lp2XazfSbGWlwc9Jsp9tQ2\nGWMbIPEk0Z/5HFAJg+B0VjPQXFpa2vulZZZ2A2jHHIINIGobN+v9/LMtSqzZ2tF1N+9sDGa+cGHw\nZ3fJoB061+m+imsglmM/l0tv+9ibsu3Hk8XYsf1Yp0nrp+/bZNI69GBHCcFrF5mWbQ+fjXis8flE\niT21TY+xDaD+zmd/SRj5cUpKRxg6wsjXYSDWj44wul+2OsJwd/f9JWGcDTwMGHAS8JewvAB4BRgH\nDAZWAJVR5qdrGGnoGkZHG13D6NqHrmHoGkZeJAzgdmATsAdoBb4KXARcFE43gm9DvQysAqoS2tYB\nL4XTLo86zz4lDPdgwykrczfzt0eWJW247sEOZtGi7tt4WVl+Jou4KLHW13f+VxeLuVdXJ7X57ayG\nYLkkLKf1H6v2vdbZ5rWZ9UnLadEiz9hm1+gjkz7UzYdW+/qPVSd/wIce6W10ttl6QrW/PTI5tsSE\nXlsbJI34fPZazJsq6r22trMOuG9hZMpOq7Dz6CTNMvCGhq7Lsr4+eTzdTrQ6+f1sPqwiaZm8PrIi\naRlsPaE6aTnW1gY70MT1s/5j1f7GkO7Xj5cF23G8n0WLgqSR2MddJfV+9aQGf3tkmbeHsdxc3eC1\ntZ3/KM2cmbzt19e7/+tHUpZBdXVHv20EsSVOv3pSg0+ZkrxdPPiFhqR1mBhruvecug5ra4NYEtss\nWeJJ63jRoiBpeA/LNnV9pm5LixYF/Sa2WX1ZQ8btL1NsA6G2tmty6Mt8epMwLKj/4VBVVeXNzc25\nDkNEZL9hZsvcvSpKXf3SW0REIlHCEBGRSJQwREQkEiUMERGJRAlDREQi+VB9S8rMtgCv9qJJMbA1\nS+EMNMWaPftTvIo1O/anWGFg4y1z95IoFT9UCaO3zKw56tfJck2xZs/+FK9izY79KVbIXbw6JSUi\nIpEoYYiISCQHesK4KdcB9IJizZ79KV7Fmh37U6yQo3gP6GsYIiIS3YF+hCEiIhEdkAnDzGaY2Ytm\n9jcz+3au40llZreY2WYzW51QdqiZPWZmfw1fR+UyxjgzG2tmTWa21szWmNklYXnexWtmQ8zsL2a2\nIoz1+/kaa5yZxczseTN7MBzP51g3mNkqM1tuZs1hWV7Ga2YjzeweM3vBzNaZ2cn5GKuZjQ+XZ3x4\nx8y+matYD7iEYWYxgtuq1wIVwBwzq8htVF3cCsxIKfs28D/ufgzB0wnzJdG1Ad9y9wqC55p8PVye\n+Rjv+8CZ7j4RmATMMLOTyM9Y4y4B1iWM53OsANPdfVLCVz7zNd4fAY+4+yeAiQTLOO9idfcXw+U5\nCZgM7ALuJ1exRr0P+odloB9P9NvHcZaT8PApwicShn8fAbyY6xi7ifsB4NP5Hi9QBLQAU/M1VmAM\nwc7gTODBfN8OgA1AcUpZ3sULjADWE17DzedYU+I7C3g6l7EecEcYwFHA6wnjrWFZvjvc3TeFf78B\nHJ7LYNIxs3Lgk8CfydN4w1M8y4HNwGPunrexAv8J/F+gPaEsX2MFcOBxM1tmZvPDsnyMdxywBfhF\neLrv52Y2jPyMNdFsggfTQY5iPRATxn7Pg38r8urrbWY2HLgX+Ka7v5M4LZ/idfe9HhzejwGmmNlx\nKdPzIlYzOwfY7O7LuquTL7EmOC1ctrUEpyY/lTgxj+ItAE4Arnf3TwI7STmlk0exAmBmg4GZwN2p\n0/ZlrAdiwtgIjE0YHxOW5bt/mNkRAOHr5hzH08HMCgmSRaO73xcW5228AO7+NtBEcK0oH2M9FZhp\nZhuAO4AzzayB/IwVAHffGL5uJjjPPoX8jLcVaA2PLgHuIUgg+RhrXC3Q4u7/CMdzEuuBmDCeA44x\ns3Fh1p4N/DbHMUXxW+DL4d9fJrhWkHNmZsDNwDp3vzZhUt7Fa2YlZjYy/HsowbWWF8jDWN39O+4+\nxt3LCbbRpe7+RfIwVgAzG2ZmB8f/Jjjfvpo8jNfd3wBeN7PxYVE1sJY8jDXBHDpPR0GuYs31hZwc\nXTyqA14CXgYuz3U8aeK7HdgE7CH4b+irwGiCC6B/BR4HDs11nGGspxEcDq8ElodDXT7GC0wAng9j\nXQ18NyzPu1hT4j6DzoveeRkrcDSwIhzWxD9XeRzvJKA53BZ+A4zK41iHAduAEQllOYlVv/QWEZFI\nDsRTUiIi0gdKGCIiEokShoiIRKKEISIikShhiIhIJEoYIgPAzP6XmbmZfSLXsYhkixKGyMCYAzwV\nvop8KClhiPRTeB+t0wh+YDk7LBtkZj8Ln7fwmJn9zszOD6dNNrM/hjfpezR+iweRfKeEIdJ/5xI8\nW+ElYJuZTQb+ieAW9RXAPILb6sfvu3UdcL67TwZuAX6Yi6BFeqsg1wGIfAjMIXggDwQ3CpxD8Nm6\n293bgTfMrCmcPh44DngsuA0XMYLbwIjkPSUMkX4ws0MJHnB0vJk5QQJwgru1pm0CrHH3k/dRiCID\nRqekRPrnfOBX7l7m7uXuPpbgaW5vAp8Lr2UcTnADQQielFZiZh2nqMysMheBi/SWEoZI/8yh69HE\nvcBHCO40vBZoIHgc7HZ3/4AgySwysxUEd/c9Zd+FK9J3ulutSJaY2XB332Fmo4G/AKd68CwGkf2S\nrmGIZM+D4QObBgNXKVnI/k5HGCIiEomuYYiISCRKGCIiEokShoiIRKKEISIikShhiIhIJEoYIiIS\nyf8Hn9QRrn7ehkgAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0xb7eab00>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(datain['train'].Age[y==0], datain['train'].Pclass[y==0], 'xb')\n",
    "plt.plot(datain['train'].Age[y==1], datain['train'].Pclass[y==1], 'or')\n",
    "plt.xlabel('Age')\n",
    "plt.ylabel('Pclass')\n",
    "plt.legend(['not survived', 'survived'])\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Continous features like Age helps separatong the outcomes (Figure above) but still suffers from heavy overlap"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.81121495327102799"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "theta0 = np.zeros(shape = (X.shape[1], 1))\n",
    "regpara = 1.0\n",
    "w = scop.fmin_l_bfgs_b(titan.lfCost, theta0, titan.lfGradient, (X, y, regpara))\n",
    "titan.getacu(w[0], X, y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The linear model scores a 81% accuracy on the training set."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.79271708683473385"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "titan.getacu(w[0], Xvad, yvad)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And 79% on the validation set. Below I calculate the learning curve for the model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYwAAAEKCAYAAAAB0GKPAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xd4VOXywPHvEKL0IlioguVKEymhKBYQUdSfBRtgxS72\nesV27QVFLzZA9NoLKipXEbugqJEmRRAURLyAinRQasj8/pgNLCFlk+zZs7uZz/Psky1nz87JJjt7\n3jKvqCrOOedccSqEHYBzzrnU4AnDOedcTDxhOOeci4knDOecczHxhOGccy4mnjCcc87FxBOGc865\nmHjCcM45FxNPGM4552JSMewA4qlu3brapEmTsMNwzrmUMWXKlGWqumss26ZVwmjSpAmTJ08OOwzn\nnEsZIvJrrNt6k5RzzrmYeMJwzjkXk0AThoj0FJEfRWSeiAwoYrsOIpIjIqdE3bdARL4XkWki4u1M\nzjkXssD6MEQkA3gS6AEsAiaJyLuq+kMB2w0EPi5gN91UdVlQMTrnnItdkGcYHYF5qjpfVTcBI4AT\nCtjuCuAt4M8AY3HOOVdGQSaMBsDCqNuLIvdtJSINgF7A0AKer8CnIjJFRC4KLErnnHMxCXtY7WDg\nRlXNFZH8jx2sqotFZDfgExGZo6pf5t8okkwuAmjcuHHgATvnXHkV5BnGYqBR1O2GkfuiZQEjRGQB\ncAowREROBFDVxZGffwLvYE1cO1DV4aqapapZu+4a09wTl2Tefx+mTQs7CudccYJMGJOAfUWkqYjs\nBPQB3o3eQFWbqmoTVW0CjAQuVdVRIlJVRKoDiEhV4EhgZoCxuhCowp13wv/9H5x4ImzYEHZEzrmi\nBJYwVDUHuBz4CJgNvKGqs0TkEhG5pJin7w58JSLTgYnA+6r6YVCxusTbsgX694c77oBu3eDXX+HJ\nJ8OOyrmSU4W//go7isQQVQ07hrjJyspSLw2S/DZsgNNPh3fegZtugnvvhWOOgQkT4OefoXbtsCN0\nLjZLlsDxx8OkSdCqFRx8MBxyiF0aNgw7utiIyBRVzYplW5/p7RJq1So48kgYNQoefRTuuw9EYOBA\ne+z++8OO0LnY/PQTHHQQfP89XHcd1K8PL79sX4YaNYKmTeHss2H4cJg9285EUl3Yo6RcObJ4MRx9\nNMyZA6+9Br17b3usdWv753rsMbjsMthzz/DidK442dlw3HFQoQKMGwcdI0NycnJgxgwYP94uH30E\nL71kj9Wtu+0M5OCDoW1byMwM7RBKxZukXELMmQNHHQUrVlhT1BFH7LjNwoWw775w2mnw4ouJj9G5\nWIwaBX37WpPTBx/APvsUvq0qzJu3LYGMH2/NrgBVq0LnzpZAjjzSru84uyB4JWmS8oThAvfttzYS\nKiPD/sHatSt82wED4MEH4bvvoE2bxMXoXCyefBKuuMLOKN57D0ozkv/33+Grr7YlkOnTLbG0bQtX\nXQV9+sDOO8c/9sJ4H4ZLGmPGwOGHQ61a8M03RScLsIRRuzbceGNi4nMuFrm59jd5+eXWFPX556VL\nFgD16sGpp1rz69SpsHIlDBsGGzdCv37QuDHcfjv88UdcDyEuPGG4wLzwgo0gad4cvv4a9t67+OfU\nqgW33goff2wX58K2cSOceaad+fbvD2+/DVWqxG//NWvCxRfDzJn2N9+hA9x1lyWOs8+GKVPi91pl\n5QnDxZ2qjXrq18/mWIwbB7vvHvvzL70UmjSBf/7Tvtk5F5ZVq6BnTxuk8cAD1iSVkRHMa4lAjx4w\nerSNwLrkEuvvy8qyTvI337RO9TB5wnBxlZsL11xjTUt9+ljZj+rVS7aPnXe24bbTp8MrrwQTp9vm\nvffsfXLbW7jQPqi//tpGOt14Y+I6pffd15qsFi2Cf//b+j1OOw322su+jK1YkZg4dqCqaXNp3769\nuvBs3Kjat68qqF59teqWLaXf15Ytqu3bqzZurLp+ffxidNts2aJ6yy32flWqpPrzz2FHlDymT1et\nX1+1Rg3VTz8NOxrVnBzV//5XtVs3e78qV1a96CLVWbPKvm9gssb4GetnGC4u1q6FY4+1U/eBA+GR\nR2yMemlVqGBtxv/7HzzxRPzidGb9ehsaeu+9cMYZULGijf5Jo0GTpfbZZ3ZmIWKjmLp3DzsiawY7\n/njrbJ8+3SYHvvgitGxpQ3Lffz9BzbexZpZUuPgZRjj++EO1XTvVjAzV55+P776PPlq1Vi3V5cvj\nu9/y7PffVTt2VBVRfegh1dxc1UcesW+u77wTdnThevFF1YoVVVu1Ul24MOxoirZ0qeq999qZUL16\ndoZfGpTgDCP0D/l4XjxhJN6WLapt2qhWqaL6/vvx3/+MGfbBdt118d93eTRjhjXzVamyfXLYvFm1\ndWvVRo1U//orvPjCkpuret999onYrZvqypVhRxS7TZvK1jRVkoThTVKuTEaPtrUshg2zAoLxtv/+\nNtrq8cdhwYL47788+eAD6NLFRtqMH28l5fNUrAhDhlhH7913hxdjGHJybGTezTdbM90HH9jw7lSR\nmQktWiTmtTxhuDIZONCGwPbtG9xr3HWX9Wncdltwr5HuHn/cZtvvsw9MnFjwBMouXeC88+Dhh+GH\nHxIfYxi+/hoOPdS+8Nx4oxUPTOQs61TjCcOV2ldf2ezt666zb6hBadgQrr7a/pmnTg3uddJRTo7N\nTr7ySpuh/OWX0KBB4dsPHAg1atg37nTuAJ81C044wTq3f/nFJpk+8EDZBmqUB/7rcaX24INQpw6c\ne27wrzVggL3WDTek9wdZPK1ZY0niySfh+uvhrbegWrWin1O3rn1wfvFFes6BWbjQzqJat7YJpffe\na8UBzz477MhSgycMVyqzZtmEryuusKqbQatZ05qkPvvMS4bE4tdfrYnp00/h6afhoYdin6F8/vnQ\nqZOdOa5aFWycibJihX3Z2HdfS4TXXAPz51u/RSL+ftOFJwxXKg89ZPV0Lr88ca/Zv7/NdL3xRlvi\n1RXs22+tmurChfDhh3DBBSV7foUKMHQoLFtmdb1S2bp1dsa0117WN9O3L8ydC4MG2RmrKxlPGK7E\nFi60b2nnn5/Yf7qddrImBC8ZUrjXX4euXa3p6dtvSz/prG1bW8hqyJDgi9899RScdRYMHmyd0OvW\nlX2fOTm20t2++9oywIceagsbPfecFfVzpRTr+NtUuPg8jMS49lqbpPfLL4l/7S1bVLOybL7AunWJ\nf/1klZuretddNo/g4INtUldZrVqluvvuqh06WGmKeMvNVb31Vou5Vi37Cfa3dcABqhdcoPrUU6pT\np9pcg1j3OXKk6n772b4OOkh1/Pj4x55O8Il7LigrVqhWq6Z6xhnhxfD55/aXO3BgeDEkkw0bVM88\n034nZ51lt+PllVdsv0OHxm+fqvbBft11tu/zz7eE9NtvVi/plltUjzxStXbtbUmkUiX78L/qKtWX\nX1b96SfbR7SxY20GO6i2aKE6atSO27gdecJwgbnnHvurmT493DiOPVa1Zk3VZcvCjSMZnHaavSf3\n3BP/D8jcXJv5XKuW6pIl8dnnli2ql15qMV92WeFFKnNzVefOVX31VdVrrlHt0sWK7uUlkVq1VI84\nQvWmm6yEDKg2bKj67LPBnBGlK08YLhDr1qnuuqtqz55hR6L6/feqFSpY81h5NmOG/Rffdltwr/HD\nD6qZmarnnFP2feXkqJ53nsV8/fUlT3CbN9uXlWeesWqtbdta7afata0uljdTlpwnjJLatKlstbjL\niaFD7S9m7NiwIzHnnae6006q8+eHHUl4zjxTtWrV4Isz3nSTvfdffFH6fWzatK38/b/+Fb+zoXXr\n4tsMV96UJGH4KKmVK23A+pAhYUeS1LZssaGIHTvCYYeFHY25806bW5DqQz9L69dfrZz8xRfDLrsE\n+1q33gp77mkzwDdvLvnzN22C3r0t3vvvt/cuXosRVa7s5TwSxRNGrVq2mvsNN8CcOWFHk7Teegt+\n/jmxq44Vp2FDm4D16qvw3XdhR5N4Dz9scyauuSb416pSxVaAmzULHn20ZM/dsAF69bLlRgcPtln7\nLjWJnZGkh6ysLJ08eXLJn/j771YWtUkTyM628o9uK1VbV3jtWpg9O7g1jUtj9WorqNeuHXz0UdjR\nJM7SpfaNv08fePbZxL1u3iI+s2dDo0bFb//331az6fPPrcDfRRcFH6MrGRGZoqpZsWzrZxgA9erZ\nLJ8pU+Cee8KOJul8/rl9g7/hhuRKFmAlQ264wcqFTJgQdjSJ8/jj9s39hhsS+7qPPrpt3fbirFkD\nPXvC2LHw/POeLNJCrJ0dpbkAPYEfgXnAgCK26wDkAKfkuz8DmAqMjuX1yjxK6pxzbNZQdnbZ9pNm\nevRQ3WOP5F1be+1a1Tp1bKhtebB2rY0KOvHEcF7/3nut43rMmMK3WbHC5kRUrKj6+uuJi82VHMnQ\n6S0iGcCTwNFAC6CviOywzEdku4FAQSXlrgJmBxXjDh591BrGzzrLzqUd330Hn3wCV10FlSqFHU3B\nqlWDa6+1dY2DLmORDJ5+2sZq3HhjOK9/3XWw335WR2z9+h0fX7oUDj/cFtYaORJOOy3xMbpgBNkk\n1RGYp6rzVXUTMAI4oYDtrgDeAv6MvlNEGgLHAs8EGOP2ata0wvg//2z1oB0PPgjVq8Mll4QdSdEu\nv9zGL6T7anGbNllnd9eu0LlzODHsvLOVTJ8/39bPiPb77xbbnDnw7rvWf+HSR5AJowGwMOr2osh9\nW4lIA6AXMLSA5w8G/gnkBhVggQ47zJLFsGEwZkxCXzrZzJ8Pb75pySLZl6ysUcPa1f/7X/tmm65e\nfRUWLw7v7CJP9+5W+fWBB6z6K1hRysMOs+G+H3wARx0Vbowu/sLu9B4M3Kiq2yUFEfk/4E9VLbaB\nQUQuEpHJIjJ56dKl8Ynq7rtt1NR551mN53Lq4Yetk/vqq8OOJDZXXmmJI13HLeTm2jf6Aw5Ijg/j\nhx+2CsJXXGFfLg49FJYssQEIXbuGHZ0LQpAJYzEQPfCuYeS+aFnACBFZAJwCDBGRE4EuwPGR+0cA\nh4vIywW9iKoOV9UsVc3adddd4xP5zjvbeqArV9qsqDQaehyrP/+04ZpnnQX164cdTWxq1bK+lrfe\ngu+/Dzua+Hv3XWvqGTAgOebC1Ktnyfmjj2xY85o1NqLuoIPCjswFJtbe8ZJegIrAfKApsBMwHWhZ\nxPbPk2+UVOT+riRqlFR+Dz5ow0FeeCG++00Bt96qKqI6e3bYkZTM8uWq1atbQb50kpur2qmTatOm\nVk8pWWzerNq+vdUYC7sgpSsdkmGUlKrmAJcDH2Ejnd5Q1VkicomIJHkXasS119p59uWXw4IFYUeT\nMH/9ZZ2aJ5wAzZqFHU3J7LKLNZG8+Sb88EPY0cTPl1/aPJMbboCKFcOOZpuKFW1t7LlzbZ1sl958\npndxFiyw/4S2be18O9lmrgVg8GDrQM7ODm8kTlksW2aT9k84IX1W5jv6aBvivGCB1U5yLl58pnc8\nNWli02q//BL+/e+wownc5s3wyCNwyCGpmSwA6ta15UVHjIAffww7mrKbNs3W5r7qKk8WLlyeMGJx\n9tlw0klwyy22MHAae+01Gx4Z9rDNsrruOhu7cN99YUdSdnlzYS69NOxIXHnnCSMWIrZSfe3acOaZ\nsHFj2BEFQtU+nFq1gmOOCTuastltN+jf35qk5s0LO5rSmz8fXn89NebCuPTnCSNWdevCf/5j4zVv\nuy3saAIxZoyVr77hhuQYtllW119vhYfvvz/sSEpv0CDrWE6VuTAuvXnCKIljj7V5GYMGWZ9Gmhk4\n0EpW9+0bdiTxUa+eVUh98UX45Zewoym5JUvgueesRTRV5sK49OYJo6QGDYK997b/4jVrwo4mbrKz\nYfx4G0mcTsuB/POftsjQAw+EHUnJPfaYtX4muoS5c4XxhFFS1arBSy9Zz/BVV4UdTdw8+KB10Vxw\nQdiRxFeDBnZMzz0H//tf2NHEbs0amwtz8snwj3+EHY1zxhNGaXTubCOmnn8e3n477GjKbM4cK9p3\n2WWWD9NN3oiv/JVVk9lTT9lqgqk+Ws2lF5+4V1qbN8OBB9pMqpkzYY89EvO6ATj/fKuC+uuvNroo\nHV18seX3+fPtrCOZbdwITZtCixbw6adhR+PSnU/cS4TMTCtQ+Pff9ombool3wwab4HbWWembLABu\nusmqvT74YNiRFO+ll2xdiQEDwo7Eue15wiiLZs3sE2jMGPv6moI+/xzWrbN5iemsSRMbpzB8uH0Y\nJ6stW+xPql07W3PCuWTiCaOsLrvM/rsffTTsSEpl9GioWrV8rF9w883WkjhoUNiRFG7UKCvklywl\nzJ2L5gmjrCpUsCap6dNTbqk3VUsYPXok73rd8bT33nDGGTB0qK33kWxUbfjvPvuk/xmfS02eMOKh\nTx9beizFmqVmzLDRwccdF3YkiXPzzdap/PDDYUeyo88/h8mTbd5FOSiK7FKQJ4x42GWXbbW0N20K\nO5qYvfee/Tz22HDjSKT99rP8/uSTybf67gMP2GC7s88OOxLnCuYJI1769bNPoDFjwo4kZu+9Bx07\nwu67hx1JYt1yi3X0J1O1+ilTbAjtNdeUj+ZBl5o8YcTLkUfa18MUaZZasgQmTixfzVF5WrSAU0+1\nZU5WrAg7GjNwINSsaVVpnUtWnjDipWJFm8zw/vvJ2aOaz/vv28//+79w4wjLrbfC2rXJMbht7lwY\nOdLKsdeoEXY0zhXOE0Y8nXMO5OTYtOkkN3q0VaY94ICwIwnH/vvbSKRHH4VVq8KNZdAgGzORRqXJ\nXJryhBFPLVtChw5J3yy1YQN8/LGdXZTnsf633Wb1mh5/PLwYfv/d/lz69Uvp6jKunPCEEW/nnmtz\nMqZODTuSQo0bZxVNymtzVJ42beD4463zO6xK9YMH20mplzB3qcATRrylwJyM0aOhShU4/PCwIwnf\nbbfBypXwxBOJf+1Vq2wS4amn2qRC55KdJ4x4q10bTjwxaedkqNpw2iOO8OGbAFlZNg/l4YcTf5Yx\nbJh1vHsJc5cqPGEEoV8/WL5821CkJDJzpi0kVB6H0xbmzjtteO3gwYl7zfXr7fWOOgratk3c6zpX\nFp4wgtCjhy0onYTNUuVxdndx2reHXr3sLCNR8zJeeMHmwngJc5dKPGEEIXpOxpIlYUeznffes2aY\nevXCjiS53HmnNQ8lopJtTg489JDNsj/ssOBfz7l48YQRlHPOscUNkmhOxp9/woQJ3hxVkP33t/EK\njz4a/LzLt96ylf+8hLlLNZ4wgtKihX2FfO65pFmNb8wYC6W8D6ctzO232xyVINf+zithvt9+Vq/S\nuVTiCSNI/frB998nzToZo0dD/freyVqY/fazSrFDhsDixcG8xscf25/DjTfaUirOpZJA/2RFpKeI\n/Cgi80Sk0O49EekgIjkickrkdiURmSgi00VklojcGWScgUmiORkbN8JHH/ns7uL861/Wx3DffcHs\n/4EHoEEDW8jJuVQTWMIQkQzgSeBooAXQV0RaFLLdQODjqLs3Aoer6gFAG6CniHQOKtbAJNGcjC++\ngL/+8v6L4jRtagsoPv00/PprfPc9YYLNsr/2Wvse4VyqCfIMoyMwT1Xnq+omYARQUKvtFcBbwNau\nRjV/RW5mRi7J0RFQUkkyJ2P0aJuo57O7i3frrdZcdPfd8d3vwIH2HeLCC+O7X+cSJciE0QBYGHV7\nUeS+rUSkAdALGJr/ySKSISLTsETyiapOKOhFROQiEZksIpOXLl0at+DjJgnmZETP7q5SJbQwUkbD\nhrYuxfPPw7x58dnn7Nnwzjtw+eVQvXp89ulcooXd7TYYuFFVc/M/oKpbVLUN0BDoKCKtCtqBqg5X\n1SxVzdp1110DDrcUkmBOxg8/wIIF3hxVEgMGWLPRnXHqPXvoIahcGa64Ij77cy4MQSaMxUCjqNsN\nI/dFywJGiMgC4BRgiIicGL2Bqq4CxgI9gws1YCHPyfDZ3SW3xx724f7KK5Zwy2LRInj5ZesbScbv\nNM7FKsiEMQnYV0SaishOQB/g3egNVLWpqjZR1SbASOBSVR0lIruKSC0AEakM9ADmBBhrsEKek/He\ne9CunY3OcbH75z+hWjWbn1EW//435ObCddfFJy7nwhJYwlDVHOBy4CNgNvCGqs4SkUtEpLiVi+sB\nY0VkBpZ4PlHV0UHFmhAhzclYtgyys32yXmnUqQNXX23Lp5b2bVuxAp56Cvr2hSZN4hqecwknmiSz\nkOMhKytLJ0+eHHYYBVu50to5LrkkoQtJv/iitYhNmmQ1pFzJrFplQ20POQTefbf47fO7+26b2zFj\nhpUfcS7ZiMgUVY3p0yHsTu/yI6Q5GaNH2yCtdu0S9pJppVYtWw3vvfdsHkVJrFsHjz1mfUeeLFw6\n8ISRSHlzMkYnpnVt0yb48EP7wPIyFKV35ZVQt66tzlcSzz5rTYJewtylC/8YSaQEz8n48ksr2e3D\nacumWjX70P/kE/udxmLzZiuV3qULHHxwsPE5lyieMBIpb07GmDEJmZMxejTsvDN07x74S6W9/v2t\nC+rWW2Mb6Pb661ZaxM8uXDrxhJFo/frZnIxXXgn0ZfJmd3fvDlWrBvpS5UKVKnDLLTB+PHz6adHb\n5uZakcFWreCYYxITn3OJ4Akj0Zo3h06dAp+TMWeOLdLjw2nj58ILoVEj68so6q0bMwZmzbJ5HN53\n5NKJ/zmHoV8/mDkTpk4N7CXyZnd7woifnXe2ZDFhQtG1JAcOhMaNrbq9c+nEE0YYeve2T58AO79H\nj4Y2bewbsYuffv1gr70sceTuUAENvvrKLtdfD5mZCQ/PuUB5wghD9JyMjRvjvvvly+Hrr/3sIgiZ\nmXDHHTbz+513dnx84EAbgnv++QkPzbnAecIIS79+VjcigHUyPvjAvv36cNpgnH46NGtmM7i3bNl2\n/8yZdmZ3xRVeRt6lJ08YYenRwxbYDqBZavRo2H13LwUSlIwMK3v+ww8wYsS2+x980EakXXZZeLE5\nF6RiE0ZkIaNrEhFMuZKREcicjM2bfXZ3IpxyCrRubc1TOTk25+LVV+Gii6xooXPpqNiPFFXdAvRN\nQCzlT946GXGck/HVV7B6tfdfBK1CBbjrLluR78UX4eGH7b5rrw07MueCE1O1WhH5N7au9uvA33n3\nq+p3wYVWckldrbYwnTvD339bOVORMu/u2mvhySet47tatTjE5wqlasuc/PGH/b779LH6Uc6lkpJU\nq60Y4z7bRH7eFXWfAoeXJDBXgH79rO7E11/bp89OO5V6V3mzuw8/3JNFIojAPfdAz8hakDfcEG48\nzgUtpoShqt2CDqTc6t3bVuk55BC7XbGi9ZwWd6lWbfvrxx/PT383ZN48251LjCOPtPIfdevaJH7n\n0llMCUNEagK3A4dG7voCuEtVVwcVWLlRu7b1Uk+bZk1ThV1Wr4bFi7e/b926bft58EE+O2cCsLv3\nXySQSCAjo51LSrE2ST0LzAROi9w+C3gOOCmIoMqdrl3tUlK5ubB+vZUYOfJIuv77BLJajmXPPSvH\nO0LnnIt5Hsbeqnq7qs6PXO4E9goyMBeDChWsSergg/nrqVdotnYi/8k9t+CaFc45V0axJoz1IrJ1\nGRgR6QKsDyYkVxrvZvRiAA/QevbrNjnAOefiLNYmqUuAFyN9GQArgXOCCcmVxujR8GndG3jguB+p\ncPfdsO++NjHQOefipNiEISIVgP1U9QARqQGgqmsCj8zFbPNmqx914olChWFDYcEvcMEF0KTJttFX\nzjlXRrHM9M4F/hm5vsaTRfL55htYtSoyu3unneCttyxZ9OplU5Fdcvr5Z3joIVv4+8ILve/JJb1Y\n+zA+FZHrRaSRiOySdwk0Mhez996zsttHHhm5o3ZtG+upallk5cpQ43MRqlbS9q674IADYJ99bFm+\npUvhmWdswXDnklisCaM3cBnwJTAlckmxGhzpacMGO6Ho2hWqV496YJ99bMGG+fOtUt7mzWGFWL6p\nwqRJcNNNsN9+sP/+NiihRg145BH45Rf48UerWnj//V5bxCW1WPswzlTVrxMQjyuh226DBQtg6NAC\nHjz0UPvmes45cOmlMHx4XOpVuWJs2WKlXt5+2y4LF1p14sMPt2JfJ5wA9ept/5wnnrDkcfHFsOee\n0L17OLE7V4RiE4aq5orIE0DbBMTjSuDLL61K6sUXb6tntIOzz4affoJ777VvuNdfn9AYy41Nm2Ds\nWEsQo0bBn3/aMrxHHQV3322rWe1SRCtuZia8+ab1Z5x8snVMtWiRuPhdyaxYAXPnbrvssYedJWZk\nhB1ZoGKtVjsIyAbe1lieEJKUrFZbSmvWWDN4RoZVFSmy2GBurpVSHTnSPtBOPDFhcQZuwwZr8lm8\n2Ja73bjRPrzzrsd6X3EdzkWdmanC9Ok28qBaNVuM5KST4Oij87UTxuDXX6FTJ6hcGb791lbCcuFY\ns2b7pDB3rn35mjvXEkYeEfsbOPRQePllaNQovJhLoSTVamNNGGuBKsAWYAMggKpqjWKe1xN4FMgA\nnlHVBwrZrgOWkPqo6kgRaQS8COyOVcUdrqqPFhdneUoYF1wAzz1nZxldusTwhPXrraNj5kwYPx7a\ntQs6xGCsWwfZ2fDFF3aZMKH4ddEzM+3bfvRlp522v13UN8Pi/kdUYe+9LUn06AGVKpX8uKJNnGjv\nVevWdtZS2Uu9BCYnx/qQZs/ePiHMnWtnidEaNbL5Tfkve+0Fr79uSy1WrAhPP239hikiiIRRATgD\naKqqd4lIY6Ceqk4o4jkZwE9AD2ARMAnoq6o/FLDdJ1giejaSMOpF9v+diFTHOtlPzP/c/MpLwnjv\nPTj+eBgwwPpJY/bHH/btNSfHPmgbNgwsxrhZu9aaZ/ISxKRJ1oFfoQK0bQuHHWaXffbZMSnsvLMl\ni1RcevDtt+1D55RTbB3YVDyGZLNihZ0J5l1mzIBZs7b/wlGvXsFJYe+9i1+o/eefbcH3iRPh/PPh\n0UetdE+SCyJhDAVygcNVtbmI1AY+VtUORTznQOAOVT0qcvsmAFW9P992VwObgQ7AaFUdWcC+/gs8\noaqfFBXYmgRFAAAdL0lEQVRneUgYS5dCq1bWZDpxon0mlsj339spyd5725lGsi2csWqVLRuYlyC+\n+846kStWtEXK8xLEQQdBzZrF7y+VDRpki2zcdBPcd1/Y0aSOnBw7Q8hLCnkJYvHibdvstpu16eZd\nWra0xFDW/4fNm20U3P332/5eey3pz+aDWECpk6q2E5GpAKq6UkSKW+mnAbAw6vYioFO+QBsAvYBu\nWMLYgYg0wTrcCzybEZGLgIsAGjduXNxxpDRVuOQS+0z95JNSJAuwYZ2vv27zM844w77JhtVRt369\n/WPPnm3t9V98YR0yqtZk1LGjnUYddhgceGDyJbegXXed/X7uv9/OoM47L+yIktPs2fYPkZcgZs60\nvi2wLxrNmlkTX3SCCKpvKDPTBpj06AFnnmkrat53n42OS4ezRFUt9oJ9WGcA30Vu7wpMLeY5p2D9\nFnm3z8LOEqK3eRPoHLn+PHBKvserYc1RJ8USZ/v27TWdvfiiKqgOHBiHnT3+uO3suuvisLMi5Oaq\n/vGH6rhxqsOGqV59tWrPnqpNm6qKWAygWqmSarduqnfcoTp2rOq6dcHGlSo2bVI98kjVihVVP/00\n7GiSx+bNqiNH2t9M3t9QnTqqhx+ues01qs8/rzp1quqGDeHFuHy56kknWWw9eqj+9lt4sRQBmKwx\nfL6qHUlMCeMM4F3sLOFe4Efg1GKecyDwUdTtm4Cb8m3zC7AgcvkL+BPrqwBbQ/wj4NpYDyadE8b/\n/qdao4bqwQer5uTEaaeXX25/Ak89VfZ9bdqkOnu26qhRqg88oNqvn2rnzqq1am37hwbVKlVU27ZV\n7dtX9c47VUeMUJ02Ldx/7GS3apVqy5aqNWuqzpoVdjTh+uMP1bvvVm3QwP6eGjdWve8++wfJzQ07\nuh3l5qoOH65aubJq3bqq774bdkQ7KEnCiKkPA0BEmgHdsRFSn6nq7GK2r4h1encHFmOd3qer6qxC\ntn+eSB+GiAjwArBCVWNecDRd+zByc63sx7ff2hn3XvFaiSQnx3rPP/7Y2rpEbKjp5s32M//1wh7b\nuBGWLLH95alf35oCmjWz+R951xs2TI9T80RL5uG2eV8HgnpfVW3gw5NP2tDwzZutyeeyy6xpNRXm\nPsyZA337WpPrZZdZDbEkGf0WRB8GqjoHmFOC7XNE5HLsLCEDGwE1S0QuiTw+rIind8GasL4XkWmR\n+25W1TGxvn46eeIJ+Owzm6gdt2QB1r47YoT9073wgvUbZGbaz8KuV6u24/2ZmTa6pHnzbQmiRpEj\nrl1J7bknvPuutcWfcEJyDLddsMDGdj/3HCxfDu3bW1Lr1Mn6nxo1Kltlgb//hldftUQxfboNcrj0\nUujf3/7GUkmzZpbob7nFZtuOG2cd4vvvH3ZkJRPrqUgqXNKxSWr2bGveP/bY5Dzjdgn21lvW93Pq\nqapbtiT+9TdsUH3jDetXEbHLUUepXnGFaqdOqjvttK35cY89VI8/XvXee1U/+cSa1mLx00/WD1Gz\npu2ndWtrNv3rr2CPLVE+/FB1991Vd97Z+hJD/scm3n0YqXJJt4SxaZNqVpb15f3+e9jRuKTx0EP2\nr3vTTYl7zVmzVK+91trhQbVRI9Xbb1ddsGD77TZsUJ040T4IzzpL9R//2JZARFSbN1c95xzVIUNU\nJ0+2P3JV65h7911LPmCd/H36qI4fH/oHaiCWLLFvgWA/f/1V9e+/rTM/wUqSMGLuw0gF6daHcddd\ncPvtVmIohSaOuqBpZHz18OHwn/8EN9z2r7/sj++ZZ6wPoWJFaw674ALrQ4i172DlSptwOXGiTRid\nMMEmFIHNim/XDn77zZq46te3Y7vwQptslM5Urbnt+uu3nzxYoULB1Qiib+d/rG5dGFZUK3/h4j5x\nL1WkU8KYPNmGcPfpY+VpnNvO5s3W9/T55/Dhh/GrbquRcuzPPGP9W2vXWvv7BRfYkr+77Raf1/j1\nV0sceUmkShUr3nfCCdYnVp7Mnm0DTwqqcZa/Blphj9WqZRNeS8ETRopbv96+dK1da3OQatUKOyKX\nlFavtln7ixbZmUC9eoV/C83IKLoDesUK+2byzDNWDaByZejd2xLFQQd5Wfw0FsgoKZc4N99so/A+\n/tiThStCzZq2smKnTlHLLRZCpPBmjZ12sj+4jRut/MqwYXZqm+6lV1yJecJIMp9/DoMHw+WXWzOx\nc0Xac0+YOtWakYpqssi7Xth9hx0G554LbdqEfUQuiXmTVBJZvdqGZVeubJ8BxRXHdM65svImqRR1\n1VU2WOSbbzxZOOeSj9doSBLvvGOTrW++2SbJOudcsvGEkQSWLLERhe3awW23hR2Nc84VzBNGErj0\nUhtC+9JL5W8IunMudXjCCNnatTBqlPVftGgRdjTOOVc4TxghmzjRypd36xZ2JM45VzRPGCHLzraf\nnTuHG4dzzhXHE0bIsrOtKcpndDvnkp0njBCp2poqBx4YdiTOOVc8Txgh+uknq/nmCcM5lwo8YYQo\nr//CE4ZzLhV4wgjRN99Y30WzZmFH4pxzxfOEEaLsbKtMXcHfBedcCvCPqpCsXg2zZnlzlHMudXjC\nCMnEiTZKyhOGcy5VeMIISXa2LYLWqVPYkTjnXGw8YYQkb8Ker4LpnEsVnjBCkJvrE/acc6nHE0YI\nfvwRVq2Cgw4KOxLnnIudJ4wQ+IQ951wq8oQRguxsqF0b/vGPsCNxzrnYBZowRKSniPwoIvNEZEAR\n23UQkRwROSXqvmdF5E8RmRlkjGHIzrZy5j5hzzmXSgL7yBKRDOBJ4GigBdBXRHZYUy6y3UDg43wP\nPQ/0DCq+sKxeDT/84M1RzrnUE+R33I7APFWdr6qbgBHACQVsdwXwFvBn9J2q+iWwIsD4QjFhgk/Y\nc86lpiATRgNgYdTtRZH7thKRBkAvYGhpX0RELhKRySIyeenSpaXdTcLkTdjr2DHsSJxzrmTCbkUf\nDNyoqrml3YGqDlfVLFXN2nXXXeMYWjCys6FVK6hRI+xInHOuZCoGuO/FQKOo2w0j90XLAkaICEBd\n4BgRyVHVUQHGFZq8CXu9e4cdiXPOlVyQCWMSsK+INMUSRR/g9OgNVLVp3nUReR4Yna7JAmD2bOv0\n9v4L51wqCqxJSlVzgMuBj4DZwBuqOktELhGRS4p7voi8BmQD+4nIIhE5P6hYE8Un7DnnUlmQZxio\n6hhgTL77hhWybb98t/sGF1k4srNhl118wp5zLjWF3eldruRN2LMuG+ecSy2eMBJk5Urrw/CCg865\nVOUJI0EmTLCf3n/hnEtVnjASJDvbakf5hD3nXKryhJEg2dmw//5QrVrYkTjnXOl4wkiA3FxrkvLm\nKOdcKvOEkQA//ABr1njCcM6lNk8YCeAT9pxz6cATRgJkZ0PdurDPPmFH4pxzpecJIwF8wp5zLh14\nwgjYihUwZ443RznnUp8njID5hD3nXLrwhBGwvAl7HTqEHYlzzpWNJ4yAffMNtG7tE/acc6nPE0aA\ntmyxJikvOOicSweeMAI0axb89Zf3Xzjn0oMnjAD5hD3nXDrxhBGg7GzYdVfYa6+wI3HOubILdInW\n8i47284ufMKecyW3efNmFi1axIYNG8IOJS1UqlSJhg0bkpmZWep9eMIIyPLl8NNPcO65YUfiXGpa\ntGgR1atXp0mTJoh/6yoTVWX58uUsWrSIpk2blno/3iQVkG+/tZ/ef+Fc6WzYsIE6dep4sogDEaFO\nnTplPlvzhBGQ7GzIyICsrLAjcS51ebKIn3j8Lj1hBCQ7Gw44AKpWDTsS51xprFq1iiFDhpT4eccc\ncwyrVq0qcpt//etffPrpp6UNLTSeMAKwZQtMnOjNUc6lssISRk5OTpHPGzNmDLVq1Spym7vuuosj\njjiiTPGFwRNGAGbO9Al7zqW6AQMG8PPPP9OmTRs6dOjAIYccwvHHH0+LFi0AOPHEE2nfvj0tW7Zk\n+PDhW5/XpEkTli1bxoIFC2jevDkXXnghLVu25Mgjj2T9+vUA9OvXj5EjR27d/vbbb6ddu3bsv//+\nzJkzB4ClS5fSo0cPWrZsyQUXXMCee+7JsmXLEvxb2J6PkgqAT9hzLr6uvhqmTYvvPtu0gcGDC3/8\ngQceYObMmUybNo1x48Zx7LHHMnPmzK2jjJ599ll22WUX1q9fT4cOHTj55JOpU6fOdvuYO3cur732\nGk8//TSnnXYab731FmeeeeYOr1W3bl2+++47hgwZwqBBg3jmmWe48847Ofzww7npppv48MMP+c9/\n/hPX4y8NP8MIQHY27L47lGH0mnMuyXTs2HG7IamPPfYYBxxwAJ07d2bhwoXMnTt3h+c0bdqUNm3a\nANC+fXsWLFhQ4L5POumkHbb56quv6NOnDwA9e/akdu3acTya0vEzjAD4hD3n4quoM4FEqRo1gmXc\nuHF8+umnZGdnU6VKFbp27VrgkNWdd9556/WMjIytTVKFbZeRkVFsH0mYAj3DEJGeIvKjiMwTkQFF\nbNdBRHJE5JSSPjfZLFsGc+d6c5Rzqa569eqsXbu2wMdWr15N7dq1qVKlCnPmzOHbvIlXcdSlSxfe\neOMNAD7++GNWrlwZ99coqcDOMEQkA3gS6AEsAiaJyLuq+kMB2w0EPi7pc5ORT9hzLj3UqVOHLl26\n0KpVKypXrszuu+++9bGePXsybNgwmjdvzn777Ufnzp3j/vq33347ffv25aWXXuLAAw9kjz32oHr1\n6nF/nZIQVQ1mxyIHAneo6lGR2zcBqOr9+ba7GtgMdABGq+rIWJ+bX1ZWlk6ePDnux1ISN98MDz0E\na9ZA5cqhhuJcSps9ezbNmzcPO4zQbNy4kYyMDCpWrEh2djb9+/dnWhl7/gv6nYrIFFWNaYpxkH0Y\nDYCFUbcXAZ2iNxCRBkAvoBuWMGJ+btQ+LgIuAmjcuHGZgy6r7GwbfeHJwjlXFv/73/847bTTyM3N\nZaedduLpp58OO6TQO70HAzeqam5pp62r6nBgONgZRhxjK7GcHJuwd/75YUbhnEsH++67L1OnTg07\njO0EmTAWA42ibjeM3BctCxgRSRZ1gWNEJCfG5yad77+Hdeu8/8I5l56CTBiTgH1FpCn2Yd8HOD16\nA1XdOqhZRJ7H+jBGiUjF4p6bjHzCnnMunQWWMFQ1R0QuBz4CMoBnVXWWiFwSeXxYSZ8bVKzxkp0N\ne+wBe+4ZdiTOORd/gfZhqOoYYEy++wpMFKrar7jnJjufsOecS2deGiRO/vwTfv7Zm6OcK6+qVasG\nwG+//cYpp5xS4DZdu3aluKH/gwcPZt26dVtvx1IuPVE8YcSJT9hzzgHUr19/ayXa0sifMGIpl54o\nnjDiJDsbMjOhffuwI3HOxcOAAQN48sknt96+4447uOeee+jevfvWUuT//e9/d3jeggULaNWqFQDr\n16+nT58+NG/enF69em1XS6p///5kZWXRsmVLbr/9dsAKGv72229069aNbt26AdvKpQM88sgjtGrV\nilatWjE4UmCrqDLq8Rb2PIy0kZ0Nbdv6hD3nAhFCffPevXtz9dVXc9lllwHwxhtv8NFHH3HllVdS\no0YNli1bRufOnTn++OMLXf506NChVKlShdmzZzNjxgzatWu39bF7772XXXbZhS1bttC9e3dmzJjB\nlVdeySOPPMLYsWOpW7fudvuaMmUKzz33HBMmTEBV6dSpE4cddhi1a9eOuYx6WfkZBjB/vk26K62c\nHJg0yZujnEsnbdu25c8//+S3335j+vTp1K5dmz322IObb76Z1q1bc8QRR7B48WKWLFlS6D6+/PLL\nrR/crVu3pnXr1lsfe+ONN2jXrh1t27Zl1qxZ/PBD0aXyvvrqK3r16kXVqlWpVq0aJ510EuPHjwdi\nL6NeVuX+DCM3184MNm+2LxwdOkBWll3+8Q/IyCh+HzNm+IQ95wIVUn3zU089lZEjR/LHH3/Qu3dv\nXnnlFZYuXcqUKVPIzMykSZMmBZY1L84vv/zCoEGDmDRpErVr16Zfv36l2k+eWMuol1W5P8PYsgWG\nDoWLL7bk8MwzcPbZ0KIF1KoFXbvC9dfDiBEwbx4UVKvRJ+w5l5569+7NiBEjGDlyJKeeeiqrV69m\nt912IzMzk7Fjx/Lrr78W+fxDDz2UV199FYCZM2cyY8YMANasWUPVqlWpWbMmS5Ys4YMPPtj6nMLK\nqh9yyCGMGjWKdevW8ffff/POO+9wyCGHxPFoi1fuzzAyM+H00+0ClkDmzLEmpsmT7fLEE7Bxoz1e\nq9a2M5C8s5HsbKhfHxo1Kvx1nHOpp2XLlqxdu5YGDRpQr149zjjjDI477jj2339/srKyaNasWZHP\n79+/P+eeey7NmzenefPmtI+MijnggANo27YtzZo1o1GjRnTp0mXrcy666CJ69uxJ/fr1GTt27Nb7\n27VrR79+/ejYsSMAF1xwAW3btg2s+akggZU3D0NQ5c03b4aZM7clkMmTrRkqut/j5JOhDCPpnHP5\nlPfy5kFI5vLmaSMz0/o52raFCy+0+zZssKQxeTJMnw7nnBNujM45FzRPGKVUqRJ07GgX55wrD8p9\np7dzzrnYeMJwziWtdOpjDVs8fpeeMJxzSalSpUosX77ck0YcqCrLly+nUqVKZdqP92E455JSw4YN\nWbRoEUuXLg07lLRQqVIlGjZsWKZ9eMJwziWlzMxMmjZtWvyGLmG8Sco551xMPGE455yLiScM55xz\nMUmr0iAishQouhpY4eoCy+IYTirwY05/5e14wY+5pPZU1V1j2TCtEkZZiMjkWOuppAs/5vRX3o4X\n/JiD5E1SzjnnYuIJwznnXEw8YWwzPOwAQuDHnP7K2/GCH3NgvA/DOedcTPwMwznnXEzKVcIQkZ4i\n8qOIzBORAQU8foaIzBCR70XkGxE5IIw44ymGYz4hcszTRGSyiBwcRpzxVNwxR23XQURyROSURMYX\nhBje564isjryPk8TkX+FEWc8xfI+R457mojMEpEvEh1jvMXwPt8Q9R7PFJEtIrJL3AJQ1XJxATKA\nn4G9gJ2A6UCLfNscBNSOXD8amBB23Ak45mpsa5psDcwJO+6gjzlqu8+BMcApYcedgPe5KzA67FgT\nfMy1gB+AxpHbu4Udd9DHnG/744DP4xlDeTrD6AjMU9X5qroJGAGcEL2Bqn6jqisjN78FylbaMXyx\nHPNfGvnrAqoCqd6pVewxR1wBvAX8mcjgAhLrMaeTWI75dOBtVf0fgKqm+ntd0ve5L/BaPAMoTwmj\nAbAw6vaiyH2FOR/4INCIghfTMYtILxGZA7wPnJeg2IJS7DGLSAOgFzA0gXEFKda/7YMizY8fiEjL\nxIQWmFiO+R9AbREZJyJTROTshEUXjJg/w0SkCtAT+1IUN17evAAi0g1LGCnfnh8LVX0HeEdEDgXu\nBo4IOaSgDQZuVNVcEQk7lkT5Dmua+UtEjgFGAfuGHFPQKgLtge5AZSBbRL5V1Z/CDSshjgO+VtUV\n8dxpeUoYi4FGUbcbRu7bjoi0Bp4BjlbV5QmKLSgxHXMeVf1SRPYSkbqqmqq1eGI55ixgRCRZ1AWO\nEZEcVR2VmBDjrthjVtU1UdfHiMiQcvA+LwKWq+rfwN8i8iVwAJCqCaMk/899iHNzFFCuOr0rAvOB\npmzrMGqZb5vGwDzgoLDjTeAx78O2Tu92kT9ACTv2II853/bPk/qd3rG8z3tEvc8dgf+l+/sMNAc+\ni2xbBZgJtAo79iCPObJdTWAFUDXeMZSbMwxVzRGRy4GPsNEGz6rqLBG5JPL4MOBfQB1gSOTbZ46m\ncBGzGI/5ZOBsEdkMrAd6a+SvLhXFeMxpJcZjPgXoLyI52PvcJ93fZ1WdLSIfAjOAXOAZVZ0ZXtRl\nU4K/7V7Ax2pnVnHlM72dc87FpDyNknLOOVcGnjCcc87FxBOGc865mHjCcM45FxNPGM4552LiCcMl\nDRFpE5mFnHf7+KKqzcbpNa8Ukdki8kox240TkSKHWIvI1ZGSDKESkX4i8kQB9+8sIp9GKpn2LuW+\nTxSRFqV4XrHvpYjUF5GRpYnLJUa5mYfh4kNEMlR1SwD7rQi0wWZhjwFQ1XeBd+P9WvlcChyhqovi\nsK+rgZeBdXHYVxDaAqhqm1ifUMD7fSIwGqsCm3/biqqaU9B+YnkvVfU3bL6IS1J+huEAEJEmIjJH\nRF6JfOMemfdtWUQWiMhAEfkOODVyJvBtpJDdOyJSO7LdOBF5NKoWf8fI/buIyKjI9t9Gyq8gIneI\nyEsi8jXwEnAX0DvvG3D0N+VIfJ9H9vGZiDSO3P+8iDwmtn7JfClkbQsRuTYS00wRuTpy3zCsVPQH\nInJNvu0ri8iIyO/iHawWUd5jQ8XWDpklIndG7rsSqA+MFZGxhW1XQFxbz1xEpK6ILIhcbykiEyO/\nixkism/k/jOj7n9KRDIi958rIj+JyESgSwGvsxuWzDpEnru3iHQXkali6788KyI7F/R+R+3jIOB4\n4KGofYwTkcEiMhm4SkSOE5EJkf1+KiK7R54b/V4W+J5F3uOZUdu/LSIfishcEXkwKo7z845VRJ6W\nAs6mXEDCnu7ul+S4AE2w0uZdIrefBa6PXF8A/DNq2xnAYZHrdwGDI9fHAU9Hrh8KzIxcfxy4PXL9\ncGBa5PodwBSgcuR2P+CJqNfZeht4Dzgncv08YFTk+vPAm9iXnxZY+ef8x9Ye+B4r314NmAW0jTq2\nugU851psJi3YOiE5QFbk9i6RnxmRY25d0L4K2y7f64yL2m9dYEHU7+yMyPWdsITVPPJ7yIzcPwQ4\nG6iHlfrYNbLt19G/x6jX6kpkTQygElb59B+R2y8CVxf0fufbx/NElVKJxD8k6nZttk0IvgB4uID3\nssD3DPsbnBm1/XyszEUl4FesjlL9SHy7AJnA+IKO1S/BXPwMw0VbqKpfR66/zPbVel8HEJGaQC1V\nzVu97AUsOeR5DayQIVBDRGpF9vNS5P7PgToiUiOy/buquj6G2A4EXo1cfylfbKNUNVdVfwB2L+C5\nBwPvqOrfqvoX8DZwSDGvdyj2O0BVZ2BJMs9pkW/fU4GW2IdeQWLdriDZwM0iciOwZ+R31B1LfpNE\nZFrk9l5AJ2Ccqi5VWyfh9Rj2vx/wi26r3Jr/fYxlHwVt2xD4SES+B27Ajrsgxb1nAJ+p6mpV3YA1\nge2J1cH6QlVXqOpmLPG4BPGE4aLlrxMTfTvWujRF7aMg8ah3szHqeqD1ykWkKXA90F1VW2NriFQq\n7XbYmUve/+HWx1X1Vaz5Zz0wRkQOx47tBVVtE7nsp6p3xO3gtleS9yV628exb/z7AxdT8DFDbO9Z\n9DZb8D7X0HnCcNEai8iBkeunA1/l30BVVwMrRSTvG/pZQPRayb0BxNYGXx3ZfjxwRuT+rsAyjSq3\nHWUtUL2Q2L7BSjYT2df4GI+JyLYnikgVEamKFWcr7vlfYr8DRKQV1iwFUAP7gFwdaZ8/upD4i9ou\n2gLsrAGiOnxFZC9gvqo+Bvw38vqfAadE+iPy+ob2BCYAh4lIHRHJJKrfoQg/Ak1EZJ/I7fzvY2GK\neo/AmpDySm6fE8P+SmoSdqy1xQZKnBzAa7hCeMZ20X4ELhORZ7EmgMJWpDsHGCbWKT4fODfqsQ0i\nMhVrX85bve8O4FkRmYGNICrsg2QsMCDS3HJ/vseuAJ4TkRuApfles0iq+p2IPA9MjNz1jKpOLeZp\nQyOvNxuYjfW1oKrTI8c3B+sD+DrqOcOBD0XkN1XtVsR20QYBb4jIRdhZSJ7TgLPEqgj/AdynqitE\n5FbgYxGpAGwGLlPVb0XkDqwZaxUwrZhjQ1U3iMi5wJuRD95JQCyVfEcAT0c6+QsaYHBHZJ8rsTXT\nm8awz5ip6mIRuQ97L1dgv9/V8XwNVzivVusAG6GCdYi2KsM+xmEd5ZPjFJZzOxCRamorB1YE3sEG\nJ7wTdlzlgTdJOedSzR2Rs9CZwC/YcrMuAfwMwznnXEz8DMM551xMPGE455yLiScM55xzMfGE4Zxz\nLiaeMJxzzsXEE4ZzzrmY/D/DMarQkGfbLwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0xad66ef0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "tp = np.linspace(0.2, 0.7, 20)\n",
    "trainerror = np.zeros(shape = (tp.size, 1))\n",
    "vaderror = np.zeros(shape = (tp.size, 1))\n",
    "for i in range(0, tp.size):\n",
    "    datain = readdata('train.csv', tp[i], 0.7, features)\n",
    "    X = np.array(datain['train'].ix[:,1:])\n",
    "    y = np.array(datain['train'].ix[:,0])\n",
    "    Xvad = np.array(datain['validation'].ix[:,1:])\n",
    "    yvad = np.array(datain['validation'].ix[:,0])\n",
    "    m = X.shape[0]\n",
    "    Xfull = titan.normdata(np.concatenate((X, Xvad)))\n",
    "    X = np.concatenate((np.ones(shape=(X.shape[0], 1)), Xfull[:m, :]), axis = 1)\n",
    "    Xvad = np.concatenate((np.ones(shape=(Xvad.shape[0], 1)), Xfull[m:, :]), axis = 1)\n",
    "\n",
    "    theta0 = np.zeros(shape = (X.shape[1], 1))\n",
    "    regpara = 1.0\n",
    "    theta = scop.fmin_l_bfgs_b(titan.lfCost, theta0, titan.lfGradient, (X, y, regpara))[0]\n",
    "    trainerror[i] = titan.lfCost(theta, X, y, 0.0)\n",
    "    vaderror[i] = titan.lfCost(theta, Xvad, yvad, 0.0)\n",
    "plt.plot(tp, trainerror, '-b', tp, vaderror,'-r')\n",
    "plt.xlabel('proportion of data used for training')\n",
    "plt.ylabel('error')\n",
    "plt.legend(['training','validation'])\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The plot above shows that the errors are quite large in the model. Suggesting there is room for more features to improve the model.\n",
    "\n",
    "As mentioned above, important features like Sex are discrete numbers that makes drawing decision boundary difficult. Combining those features with other continous ones could potentially improve the results. For example, if we multiple Age with Sex (f=1, m=-1) and plot the result we can see much better separation of the outcomes (figure below)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "age_gender = pd.Series.copy(datain['train'].loc[:,'Age'])\n",
    "age_gender[datain['train'].Sex == -1] = -age_gender[datain['train'].Sex == -1]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYwAAAEKCAYAAAAB0GKPAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xt8VNXd7/HPL5NwCXchKgWSYItyF0uKCKJSqAVE1Kda\nQYq9o6FWe7SeenmwT9vD69QLPE+r9oLWapt4oWqrxXotWFFrS0BE8dKjgIhFuago3iDkd/5YezKT\nZJLskEwS8Pt+vfZrZtZee63f3rOzf5m9Z/Yyd0dERKQxOW0dgIiI7B+UMEREJBYlDBERiUUJQ0RE\nYlHCEBGRWJQwREQkFiUMERGJRQlDRERiUcIQEZFYcts6gJbUp08fLy4ubuswRET2G6tWrdru7gVx\n6h5QCaO4uJiKioq2DkNEZL9hZq/GratTUiIiEosShoiIxKKEISIisShhiIhILEoYIiISS9YShpl1\nMrN/mtkzZrbOzH6UoY6Z2c/N7GUzW2tmn02bN8XMXormXZKtOLnqKli+vGbZ8uWhvDn1a9e76ipY\ntKhmvfTl4rY7bVqqnWT96dPh6KNT9Wv3ddVVcPLJlH/+Roo7v4mZk5uowswp7rWT8k99H4YMSbV/\n8MEwfjwUFFA++z6KD3qXHKuiOGcT5UddE/oqKICBA2HwYOjZE4YOha5doXPnMO/QQyGRYFiHlzDz\n6mmYPUM/e7VGWXKanLuc8ryvUmwbMauqMz9he+lqu0IstpF5dl1U18nN2YtZFX1sG31sG2ZV5Fpl\nmGeVmFVRbK9SbrMpT8yhOO/10E4xlPf/39ChQ43tXV4Oxb12kpPjoU55tP2PPpp5RUvJzQUzyLW9\nzCu+L2y3pEWLwutp02DIEMpn30fXrqF+cj3mdbox1EurP3nQxhrr2zlvT3X/kwdtJNf2ps2H3FyY\nNy/D/pbcR845J0xXXRUqTpuWWse0faX88zdSfMiHoc2cmts9keN0zdlVva0mj96Rtl33Ms+uq/Ve\nkXqve73GvKHLqtvMzali2CHb6vSRm7OXeYV/Dvtar5013rPkOhf32kn57PsoP/IqioshJ4fwvvS9\niPzEh1G/qb6L815n3rxQJ8ec4m47KJ/559R6p/8NpT8mDR6ctnEjxxwDhYWN/422ovJyam6P8saW\naCZ3z8oEGNA1ep4H/AMYW6vONOD+qO5Y4B9ReQJ4BTgM6AA8AwxtrM/Ro0d7ky1b5t6nT3jM9Hpf\n69cuX7jQ3Sw8xmmnvnaT7ZSWhvmlpe7g3qmTe/fuoX7tvhYu9DLO8nx2OXidKZ9dXsasUH/hwuoZ\nZX3Or7NMPru8rMPX3A85JBQccoh7Xl7NBjt1cgcfyjMOVbX6q8pQlppn7KlnXub68euGKY8PvQMf\n1V2nPudXb++yMvf8/FrbKN+97LLnvDTnVxnXqbRoad33eeFCL2OW5/BxxthLuc59xgx3M5/Ue3Uj\n61P/vNLSWvtLMoaOHd179Ah9QHjs0yfMT67rZc95Pu83Y5s3/H5mrp+5bi67G+y7Ax95Xk7t/WNv\n7P0gP2+3l3WdGzZY+t9Q2vaolvy7Ki2t+bpjx/jHiiyrdz8ta1o7QIU3cmxNTrEqNXcC8oHVwNG1\nyn8NzEp7/RLQFzgGeDCt/FLg0sb62aeE4Z564+fPj7cDxK1fu15yx6xvubjtJg8II0aEt/ALXwjJ\nokePevsq6rWzwT+mol47Q5sTJlQXFrEhc92cTeFJ9+6pvbT6iBwlD7PYf8jtYSrK2VS9vYuK6q+T\nyNmbcV6CPWHbpSdq9wa3e4I94cnAgc3aVolEhv0nP7/mPjJ6dHg9Z06Nfau+dT1Qp6Iu28J2+MIX\nMm6PGpJJYsCA8Fha2vRjRRbVu58WNa2ddpMwok8Ka4BdwJUZ5i8Fjk17/VegBDgduDGtfA5wXT19\nzAUqgIrCwsKmbal08+eHzTF/fsvWr12vseXitps8sCd35vnzG+zLrOE/JLO0NidMcJ8wwY3MB0dj\nbypZdOgQHrt3T5VF0/6UMIyq6k1b37ayRj4BVG+7NA1v96rqbdbcbZVx/6m9jyRfp+1bje0XB9pk\n7K25n9faHnUkt92AAU3/G82yevdTa1o77SZhVHcCPYHlwPBa5c1OGOmTPmHoE8a+TvqE8cmY9Amj\nrnaXMEJMXAF8v1ZZ25+S0jUMXcPQNYxmbHNdw9A1jBaYgAKgZ/S8M7ACmF6rzkm1Lnr/MyrPBdYD\nA9Mueg9rrM99ShhXXpn5YH/llc2rX7velVeGHTO9XvpycdudOjXVTrL+SSe5jxmTql+7ryuvdJ8+\n3csm3uBFnd5wqIr+U67yop7veFnfi9wHD061X1DgPm6ce58+XnbWUi/qtdONvV5kr3rZqKtDX336\nuBcXux9xRDgoDRni3qVLSBZ9+oREkpPjQ/NeTDtwVPlQ1vin2FijLDlNSizzstyzo082e+vMz6HS\nu/BeiIUNXsq11XUTVumw13uz1XuzNZSxJ6wrexz2ehEbvYyzvCznK16Uuzm0U+Re1u/ikPTStndZ\nmXtRz3fcrCrUSf4RjhnjpYV/9kQi/PUkqAzJYurU1Hu0cGF4PXWq++DBXnbWUu/SJXVgzKHSSzve\nUCOh+9SpPukzG2qsb6fc3dX9T/rMBk9QWeMgnEikjmc19ofkPjJ3bpiuvDJUnDo1tY5p+0rZxBu8\n6OAPQptWc7vnWJV3sfeqt9Wkz25P266VXsq1td6rVKIY2nOTlw75a3WbCdvrQw/eWqePhFV66YB7\nw77W850a71lynYt6vuNlZy31spFXelFRON4XFbmXHXqhd875oE6SKsrd7KWloY5R5UVdt3vZmfem\n1jv9byj9MemII9I2bmTs2Jqnpur7G21FZWVec3s0MVm4Ny1hWKjf8sxsJHAL4TpGDrDE3X9sZucC\nuPuvzMyA64ApwAfA1929Ilp+GvA/0fI3ufuCxvosKSlx3XxQRCQ+M1vl7iWx6mYrYbQFJQwRkaZp\nSsLQL71FRCQWJQwREYlFCUNERGJRwhARkViUMEREJBYlDBERiUUJQ0REYlHCEBGRWJQwREQkFiUM\nERGJRQlDRERiUcIQEZFYlDBERCQWJQwREYlFCUNERGLJzVbDZjYA+B1wCODAYnf/Wa06FwOz02IZ\nAhS4+1tmthF4D9gLVMa9X7uIiGRH1hIGUAlc5O6rzawbsMrMHnb355MV3P1q4GoAMzsZ+F/u/lZa\nGxPdfXsWYxQRkZiydkrK3be4++ro+XvAC0C/BhaZBdyWrXhERKR5WuUahpkVA0cB/6hnfj5hXO+7\n0oodeMTMVpnZ3GzHKCIiDcvmKSkAzKwrIRF8z93frafaycATtU5HHevur5vZwcDDZvaiuz+Wof25\nwFyAwsLCFo5eRESSsvoJw8zyCMmi3N3vbqDqTGqdjnL316PHrcAfgTGZFnT3xe5e4u4lBQUFLRO4\niIjUkbWEYWYG/AZ4wd0XNVCvB3A8cE9aWZfoQjlm1gU4EXguW7GKiEjjsnlKajwwB3jWzNZEZZcB\nhQDu/quo7DTgIXd/P23ZQ4A/hpxDLnCruz+QxVhFRKQRWUsY7v44YDHq3QzcXKtsPXBkVgITEZF9\nol96i4hILEoYIiISixKGiIjEooQhIiKxKGGIiEgsShgiIhKLEoaIiMSihCEiIrEoYYiISCxKGCIi\nEosShoiIxKKEISIisShhiIhILEoYIiISixKGiIjEooQhIiKxZHOI1gFmttzMnjezdWZ2QYY6J5jZ\nTjNbE01XpM2bYmYvmdnLZnZJtuIUEZF4sjlEayVwkbuvjsbnXmVmD7v787XqrXD36ekFZpYArge+\nAGwGVprZvRmWFRGRVpK1TxjuvsXdV0fP3wNeAPrFXHwM8LK7r3f33cDtwCnZiVREROJolWsYZlYM\nHAX8I8PscWa21szuN7NhUVk/4LW0OpuJn2xERCQLsnlKCgAz6wrcBXzP3d+tNXs1UOjuu8xsGvAn\nYFAT258LzAUoLCxsgYhFRCSTrH7CMLM8QrIod/e7a89393fdfVf0/C9Anpn1AV4HBqRV7R+V1eHu\ni929xN1LCgoKWnwdREQkyOa3pAz4DfCCuy+qp86hUT3MbEwUzw5gJTDIzAaaWQdgJnBvtmIVEZHG\nZfOU1HhgDvCsma2Jyi4DCgHc/VfA6UCpmVUCHwIz3d2BSjM7D3gQSAA3ufu6LMYqIiKNsHB8PjCU\nlJR4RUVFW4chIrLfMLNV7l4Sp65+6S0iIrEoYYiISCxKGCIiEosShoiIxKKEISIisShhiIhILEoY\nIiISixKGiIjEooQhIiKxKGGIiEgsShgiIhKLEoaIiMSihCEiIrEoYYiISCxKGCIiEosShoiIxJLN\nIVoHmNlyM3vezNaZ2QUZ6sw2s7Vm9qyZPWlmR6bN2xiVrzEzjYokItLGsjlEayVwkbuvNrNuwCoz\ne9jdn0+rswE43t3fNrOpwGLg6LT5E919exZjFBGRmLKWMNx9C7Alev6emb0A9AOeT6vzZNoiTwH9\nsxWPiIg0T6tcwzCzYuAo4B8NVPsmcH/aawceMbNVZja3gbbnmlmFmVVs27atJcIVEZEMsnlKCgAz\n6wrcBXzP3d+tp85EQsI4Nq34WHd/3cwOBh42sxfd/bHay7r7YsKpLEpKSrzFV0BERIAsf8IwszxC\nsih397vrqTMSuBE4xd13JMvd/fXocSvwR2BMNmMVEZGGZfNbUgb8BnjB3RfVU6cQuBuY4+7/Sivv\nEl0ox8y6ACcCz2UrVhERaVw2T0mNB+YAz5rZmqjsMqAQwN1/BVwB9AZ+EfILle5eAhwC/DEqywVu\ndfcHshiriIg0IpvfknocsEbqfAv4Voby9cCRdZcQEZG2ol96i4hILEoYIiISixKGiIjEooQhIiKx\nKGGIiEgsShgiIhJLrIRhZp82s47R8xPM7Hwz65nd0EREpD2J+wnjLmCvmX2GcN+mAcCtWYtKRETa\nnbgJo8rdK4HTgGvd/WKgb/bCEhGR9iZuwthjZrOArwJLo7K87IQkIiLtUdyE8XXgGGCBu28ws4HA\n77MXloiItDex7iUVDat6PoCZ9QK6ufuV2QxMRETal7jfknrUzLqb2UHAauAGM8t4y3IRETkwxT0l\n1SMaLe8/gN+5+9HA5OyFJSIi7U3chJFrZn2BL5O66C0iIp8gcRPGj4EHgZfdfaWZHQb8v4YWMLMB\nZrbczJ43s3VmdkGGOmZmPzezl81srZl9Nm3eFDN7KZp3SVNWSkREWl6shOHuf3D3ke4+L3q93t2/\n1MhilcBF7j4UGAt8x8yG1qozFRgUTXOBXwKYWQK4Ppo/FJiVYdnWddVVsHx5zbLly0N5mvJyKC6G\nnJzwWF7etOUBGDwY5s2rWTZvXihvqJ3kvPQ655wTpvQ6ixbBtGmpOkcfTflRV1N8yIfkmIe4Z98X\n6mTaDuecAz17wimnpPqYPh06dKA876sU99pJjlVR3HEL84rvozjv9dBu/lbK7Szm2S/ItUrMnFyr\nZJ79AsygqAgOPji0dfTRof9TTgmPyfU4+GAoLEyt9+DBMH58eEw6+GDKe3831W8xlE++KbVd0+s2\nQ73vdQbz5kFubljNRAK6dg3Pk1Nubs23PNl2cl6mx2Sfmeo2Fk99fdReLj3u2lMy5smTM883c8yq\nar12iju/Sfnl60IHy5cz+TPrq+fVbiN9Hfv0qa+fTDE6k0fvqLnCy5cz75hVderWt+5NeX+h7naY\n3MBJ+6a23W64e6MT0An4DvAL4KbkFGfZtDbuAb5Qq+zXwKy01y8RfhB4DPBgWvmlwKWN9TF69GjP\nmmXL3Pv0CY+ZXrt7WZl7fr47pKb8/FAeZ/lqpaVh4dLSuq8baif5fOHC1GOPHiGIHj3C/IUL3c1C\nW1Gdsg5f83x21YybXV521tLM26FHD/dEIlQcN869Sxd38DJm1WkHqmq8TrC7ThlUeSnX1dxwXbqE\ntpN99Ojh3qlTav6MGSH+YcNSddzD+mSII59dXtbv4prbtRkafK/reTvjTKWlmduub8rLc+/QIfO8\n+uKpL/7ayzUl7qZO+bzvZWct9UmJv2bYH2pOHTqkdremTVU+6bPbq/fb0o43NNpXct2b8v66u0+a\nlLm9SZOat++0BqDCGzm2Jqd4leAPwE+AVwg/3nsI+FnsTqAY2AR0r1W+FDg27fVfgRLgdODGtPI5\nwHWN9ZPVhOGeOiDPn5/xYF9UlHmnKSqKt3wNyb/WAQO8zkGuoXaS8+bMCYlhzpxwsO3ePVU/mVCi\nOkVdtjUcd6bt0KOHe05OjQWK2LjPB5AEe9z79k0VjBgRHg8/PPXaLCSS0aND2cEHh8dx48K8CRPC\n+vTamXl92NAiycI9xnudpikHu0Si/rb3ZarvPWysj6KifT1INyE2NnhjB/DmT1XV+30iZ2/sbdaU\n99e94faas++0hmwkjKejx7XRYx7wVMxluwKrgP/IMK/ZCYNwKqsCqCgsLMzSJk0zf37YbPPn15ll\nlnlHMIu3fB3JZDFgQJPiqJ43YUKqTu36aXWMzH9INeKur4/k1L27W7P++KtSMSfjTl//5Lxkv927\n19w2yWUmTKj/fWBv49s8pljvdaSp26K+tvdlqu89bKyPloyh3j7Y662SMKL9vinbrCnvb2PvcXP2\nndaQjYTxz+jxMWA40AdYH2O5PMLF8gvrmb//nJJy1yeM9D70CSP2NtMnjHr60CeMWG1nWzYSxreA\nXsDxwHpgK3BuI8sY8DvgfxqocxJwf1R3bFpiyo36GQh0AJ4BhjUWp65h6BqGrmE0Hk998ddeTtcw\n4m9PXcNo5gQcCziwFlgTTdOAc5PJJkoU10fXRp4FStKWnwb8K5p3eZw+s5owrryy7sF92bJQnqas\nLPynYBYeq3eCmMu7u/sRR9Q9qJWWhvKG2knOS68zd26Y0ussXOg+dWqqzpgxXjbqKi86+AM3qkLc\nZy0NdTJth7lzwwF8xoxUHyed5J6X52W5Z3tRz3fc2OtFHf7tpUVLvSh3c2i385texiwv5XpPsMeh\nyhPs8VKuD7tiYaF7QUFoa8yY0P+MGeExuR4FBanTUMuWhW0yblx4TCoo8LKDzkv1W+ReNuk3qe2a\nXrcZ6n2vMygtTR30cnKqc2wqkSZqvuXJtpPzMj0m+8xUt7F46uuj9nLpcdeekjHXd7AMB+e9tV5X\neVGnN7zssudCB8uW+aRPv1I9r3Yb6evYu3f9B/q6MaYli6Rly7x0bEWduvWte1PeX/e62yFTstjX\ntrOpKQnDQv3MzOzCemcC7t6ubg9SUlLiFRUVbR2GiMh+w8xWuXtJnLqN3XywWwvEIyIiB4AGE4a7\n/6i1AhERkfYt7t1qb0kfw9vMepnZTdkLS0RE2pu495Ia6e7vJF+4+9vAUdkJSURE2qO4CSMnGjgJ\ngGhcjFiDL4mIyIEh7kF/IfCUmS2JXp8BLMhOSCIi0h7FHaL1d2ZWAXw+KvoPD8O2iojIJ0SDCcPM\nOhF+aPcZwg/rfuXula0RmIiItC+NXcO4hXAzwGcJY1Nck/WIRESkXWrslNRQdx8BYGa/Af6Z/ZBE\nRKQ9auwTxp7kE52KEhH5ZGvsE8aRZvZu9NyAztFrA9zdu2c1OhERaTcauzVIorUCERGR9i3uD/dE\nROQTTglDRERiUcIQEZFYsnY/qOhuttOBre4+PMP8i4HZaXEMAQrc/S0z2wi8B+wFKuMO7iEiItmT\nzU8YNwNT6pvp7le7+yh3HwVcCvzN3d9KqzIxmq9kISLSDmQtYbj7Y8BbjVYMZgG3ZSsWERFpvja/\nhmFm+YRPInelFTvwiJmtMrO5bROZiIikaw9jWpwMPFHrdNSx7v66mR0MPGxmL0afWOqIEspcgMLC\nwuxHKyLyCdXmnzCAmdQ6HeXur0ePW4E/AmPqW9jdF7t7ibuXFBQUZDVQEZFPsjZNGGbWAzgeuCet\nrIuZdUs+B04EnmubCEVEJCmbX6u9DTgB6GNmm4EfAnkA7v6rqNppwEPu/n7aoocAfzSzZHy3uvsD\n2YpTRETiyVrCcPdZMercTPj6bXrZeuDI7EQlIiL7qj1cwxARkf2AEoaIiMSihCEiIrEoYYiISCxK\nGCIiEosShoiIxKKEISIisShhiIhILEoYIiISixKGiIjEooQhIiKxKGGIiEgsShgiIhKLEoaIiMSi\nhCEiIrEoYYiISCxZSxhmdpOZbTWzjMOrmtkJZrbTzNZE0xVp86aY2Utm9rKZXZKtGEVEJL5sfsK4\nGZjSSJ0V7j4qmn4MYGYJ4HpgKjAUmGVmQ7MYp4iIxJC1hOHujwFv7cOiY4CX3X29u+8GbgdOadHg\nRESkydr6GsY4M1trZveb2bCorB/wWlqdzVFZRmY218wqzKxi27Zt2YxVROQTrS0Txmqg0N1HAtcC\nf9qXRtx9sbuXuHtJQUFBiwYoIiIpbZYw3P1dd98VPf8LkGdmfYDXgQFpVftHZSIi0obaLGGY2aFm\nZtHzMVEsO4CVwCAzG2hmHYCZwL1tFaeIiAS52WrYzG4DTgD6mNlm4IdAHoC7/wo4HSg1s0rgQ2Cm\nuztQaWbnAQ8CCeAmd1+XrThFRCQeC8foA0NJSYlXVFS0dRgiIvsNM1vl7iVx6rb1t6RERGQ/oYQh\nIiKxKGGIiEgsShgiIhKLEoaIiMSihCEiIrEoYYiISCxKGCIiEosShoiIxKKEISIisShhiIhILEoY\nIiISixKGiIjEooQhIiKxKGGIiEgsShgiIhJLNkfcuwmYDmx19+EZ5s8GfgAY8B5Q6u7PRPM2RmV7\ngcq4g3vIgWXPnj1s3ryZjz76qK1DkUinTp3o378/eXl5bR2KtIGsJQzgZuA64Hf1zN8AHO/ub5vZ\nVGAxcHTa/Inuvj2L8Uk7t3nzZrp160ZxcTHR8O/ShtydHTt2sHnzZgYOHNjW4UgbyNopKXd/DHir\ngflPuvvb0cungP7ZikX2Tx999BG9e/dWsmgnzIzevXvrE98nWHu5hvFN4P601w48YmarzGxuQwua\n2VwzqzCzim3btmU1SGl9Shbti96PT7Y2TxhmNpGQMH6QVnysu48CpgLfMbPj6lve3Re7e4m7lxQU\nFGQ5WpGG3Xzzzfz73/9u1T7HjRvX4PyKigrOP//8JrV5wgknUFFR0Zyw5ADUpgnDzEYCNwKnuPuO\nZLm7vx49bgX+CIxpmwhlv3HVVbB8ec2y5ctDeSvKRsJwd6qqquqd/+STTza4fElJCT//+c9bNCb5\nZGqzhGFmhcDdwBx3/1daeRcz65Z8DpwIPNc2Ucp+43Ofgy9/OZU0li8Prz/3uX1ucuPGjQwZMoRv\nf/vbDBs2jBNPPJEPP/wQgDVr1jB27FhGjhzJaaedxttvv82dd95JRUUFs2fPZtSoUdV1k37+858z\ndOhQRo4cycyZMwH4r//6L6655prqOsOHD2fjxo1s3LiRI444grPPPpvhw4fzk5/8hIsvvri63s03\n38x5550HQNeuXQGYOXMm9913X3Wdr33ta9x55508+uijTJ8+HYD333+fb3zjG4wZM4ajjjqKe+65\nB4APP/yQmTNnMmTIEE477bQ6sYsA4b+XbEzAbcAWYA+wmXDa6Vzg3Gj+jcDbwJpoqojKDwOeiaZ1\nwOVx+xw9erTLgeP5559v2gLLlrn36eM+f354XLasWf1v2LDBE4mEP/300+7ufsYZZ/jvf/97d3cf\nMWKEP/roo+7uPn/+fL/gggvc3f3444/3lStXZmyvb9++/tFHH7m7+9tvv+3u7j/84Q/96quvrq4z\nbNgw37Bhg2/YsMHNzP/+97+7u/vWrVv905/+dHW9KVOm+IoVK9zdvUuXLu7ufvfdd/vZZ5/t7u4f\nf/yx9+/f3z/44ANfvny5n3TSSe7ufumll1avw9tvv+2DBg3yXbt2+cKFC/3rX/+6u7s/88wznkgk\n6l2PJr8v0q4lj71xpqx9rdbdZzUy/1vAtzKUrweOzFZccgCbOBFKS+EnP4H588PrZho4cCCjRo0C\nYPTo0WzcuJGdO3fyzjvvcPzxxwPw1a9+lTPOOKPRtkaOHMns2bM59dRTOfXUUxutX1RUxNixYwEo\nKCjgsMMO46mnnmLQoEG8+OKLjB8/vkb9qVOncsEFF/Dxxx/zwAMPcNxxx9G5c+cadR566CHuvffe\n6k81H330EZs2beKxxx6rvs4xcuRIRo4c2Wh88smTzd9hiLSu5cvhl78MyeKXvwwJo5lJo2PHjtXP\nE4lEs07V3HfffTz22GP8+c9/ZsGCBTz77LPk5ubWuD6R/pXVLl261Fh+5syZLFmyhMGDB3PaaafV\n+cZSp06dOOGEE3jwwQe54447qk97pXN37rrrLo444oh9Xg/55Grzb0mJtIjkNYslS+DHPw6P6dc0\nWlCPHj3o1asXK1asAOD3v/999aeNbt268d5779VZpqqqitdee42JEydy5ZVXsnPnTnbt2kVxcTGr\nV68GYPXq1WzYsKHefk877TTuuecebrvttozJAODMM8/kt7/9LStWrGDKlCl15n/xi1/k2muvTZ42\n5umnnwbguOOO49ZbbwXgueeeY+3atXE3h3yCKGHIgWHlypAkkp8oJk4Mr1euzEp3t9xyCxdffDEj\nR45kzZo1XHHFFUC40HzuuefWuei9d+9evvKVrzBixAiOOuoozj//fHr27MmXvvQl3nrrLYYNG8Z1\n113H4YcfXm+fvXr1YsiQIbz66quMGZP5i4Mnnngif/vb35g8eTIdOnSoM3/+/Pns2bOHkSNHMmzY\nMObPnw9AaWkpu3btYsiQIVxxxRWMHj26OZtHDlCW/E/jQFBSUuL67viB44UXXmDIkCFtHYbUovfl\nwGJmqzzm/fr0CUNERGJRwhARkViUMEREJBYlDBERiUUJQ0REYlHCEBGRWJQwRFrZvffey09/+tMW\naSt540GR1qCEIQeM8nIoLoacnPBYXt52sVRWVtY7b8aMGVxyySWtGI1Iy1DCkANCeTnMnQuvvgru\n4XHu3OYnjffff5+TTjqJI488kuHDh3PHHXdQXFzM9u1huPmKigpOOOEEINyqfM6cOYwfP545c+Yw\nduxY1q3O7tSwAAAUtklEQVRbV91WclCi5K3Jd+7cSVFRUfW9pN5//30GDBjAnj17eOWVV5gyZQqj\nR49mwoQJvPjiiwBs2LCBY445hhEjRvCf//mfzVs5kSZSwpADwuWXwwcf1Cz74INQ3hwPPPAAn/rU\np3jmmWd47rnnMt6fKd3zzz/PI488wm233caZZ57JkiVLANiyZQtbtmyhpCT1g9oePXowatQo/va3\nvwGwdOlSvvjFL5KXl8fcuXO59tprWbVqFddccw3z5s0D4IILLqC0tJRnn32Wvn37Nm/lRJpICUMO\nCJs2Na08rhEjRvDwww/zgx/8gBUrVtCjR48G68+YMaP6luJf/vKXufPOOwFYsmQJp59+ep36Z555\nJnfccQcAt99+O2eeeSa7du3iySef5IwzzmDUqFGcc845bNmyBYAnnniCWbPCyAFz5sxp3sqJNFHW\nEoaZ3WRmW80s42h5FvzczF42s7Vm9tm0eVPM7KVonk72SqMKC5tWHtfhhx/O6tWrq08B/fjHP65x\nS/L025FDzVuS9+vXj969e7N27VruuOMOzjzzzDrtz5gxgwceeIC33nqLVatW8fnPf56qqip69uzJ\nmjVrqqcXXnihepnatzUXaS3Z/IRxM9DQ5/epwKBomgv8EsDMEsD10fyhwCwzG5qVCKdNg0WLapYt\nWhTK4yyTfJ5cZto0OOWU8JgcY3rRIsqPvCq6GOsUd36T8svX1Wzz6KNh+vTUMuntJMelnj6d8k/P\np7jXzlQ7k2+qXmbeMavITVRh5tVTcTGUz/wznHMOdO1Kef63wvLmFHfbQflnfgiDB8M551D++RtT\nF4w7v8m8YY9GfUUXkI+6OvTf+U3MnNycmn2ZOZ077A3zcsGsngvPZky2h2os1882kWuVmDk5VkW3\njrvJsSo2b4aXnq+kosKrp1dfhR07YO3ToXztqt3seP1DFiyA/M41b6SZ37mKBf8ZzlPteP3D1DJP\nV7JjB7H8+9//Jj8/n6985StcfPHFrF69muLiYlatWgXAXXfd1eDy06efyaWXXsUbb+wERtbod8cO\nWL++K5/5zOc4++wLmDx5OolEgu4ffMDAwkL+8Ic/AGEMi2eeeALeeIPx48dz++23A1DeQlf1d+yA\ntWuhoiI8vvRSeJ6cVq+u+frVV8P7m5sL8+alvmxgVt+UfK9h2DBSfwu9dma8xjRsWN02Egno2jXV\nb/r+lanP3NzQTrJuQ1Pt9ubNS5Ul17G29HUOdZ0+3XfTp0/qSxfzTnmN4l47a/TRrVuY31DbkyfX\njC+1zer+TbX6Fz3iDs23LxNQDDxXz7xfA7PSXr8E9AWOAR5MK78UuDROf00eonXhQnez8JjpdWPL\nLFzoHq6xhuczZoTnM2aE4UHz872Mszw/b3d1NXDP530vu+y50N6yZe5duoQZpaVhaNFx48LrcePC\n69JSL2OW57OrVju7vOyz13hpxxscqmrMq67TsdLLcr7S4PJlnb/p+bxfa9ma7eXxoXfgo4x9NDTl\n57uXlaU23yQezBBr5tjvv/95X7myyleu9LSp9mv3VSsrffsrb3vZgg1e1H+Pm1V5Ud+Pvez6t92f\nftq3v/K2r1pZWXOZVe7btze+izzwwAM+YsQIP/LII72kpMRXrlzpjz32mA8aNMhHjx7tF110kR9/\n/PHuXne41e3b3R966A1PJBI+d+5/Vfd77bW/9W9+8zu+alWI5ac//YMDvnjxoyGmnTt9/dKl/sVJ\nk3zkyJE+5Igj/EfnnhvK16/3sWPH+vDhw/3yyy+vHp51X23f7tVxxJ3uv//5Gu9Tbm7T9omG9o+h\nQ/e9rZaYEonM5aWlqRjLykLcjbeXeb9uqO1Jk+Jvs0xx1N6ecdCEIVqzentzMysGlrr78AzzlgI/\ndffHo9d/BX4QJZkpHoZwxczmAEe7+3mN9bdPtzdftAi+/3049lh4/HG45hq48ML4y0SD6DBhQlj+\n5JPhz3+ubq+Yjbzqdc+LFOW8xsbLbwgjwy1ZAk8/HdocPhyefRYOPxz+9S8YMQKee47inu/w6tvd\n67bDRjbTn70NDJ5YxEbAeJWijHHQpw+vbu1cZ15LKSqCjRvD86acTbn//hfo0yfebbQ78DEje/8b\ndu6Eww4LhevXQ48erN3xKXbTse4yHSCbI5GuXQu7d2eINRqmor55I0cC774b4i8ogG3bwjp1r/v+\nZyvGhmzf/gJTp7bc7c33df9oTYkEJL8pXVwcPmVlo+24618U/SlniiN9e8bRlNub7/dDtJrZXMIp\nLQr35YT1hRfCn/4UDvwTJjSeLDItA6nn99wDxx1X/XrTigEZm9hU1b/m2NMTJ6baHDAgJIsBA0Ly\nmDCBTY9nPlhsohCn4b1sE4VQT51NVf1hW3b/Spt74TmO3XQI51b69k0dWAsKYMsWdlOceZkmHiib\nHFM97TfUb/W87t2r46+xTi0s29sgjtbYP5pr797U85aON73tuBqKIZvbsy2/JfU6kH407R+V1Vee\nkbsvdvcSdy8pKChoehSLFoVPBslPCLWvaTS2zIoVqWTx+OPh2kNae4X2WsYmCnM2p8aejq518Pjj\n4RPFa6+FTxivvRZeP/44hT3fzdwOm0jQ8B5XyCYKybwXFeZsprBg38epjqO5F57j6MBu6N07/Df+\n7rth2rYNevcO8zItU3dAupaNqZ72O3RoeB6Qir9v39Q6tWKMrak19o/mSiRSz1s63vS24yoszN4X\nPRrSlgnjXuDs6NtSY4Gd7r4FWAkMMrOBZtYBmBnVbXnJU0vXXAOPPRYev//9hpNG+jKnnpoqP/XU\ncDrq3nvD449+BJ07s8AvJT9vT40m8vmABZe8mxp7+uST4aKL4Nxzw3+U48aFTxjjxoXX557LgrfP\nJZ/3a7XzPgs+exdzO94MZD61mN9xLwty5rOASzMvP2oJC977LvnU+hFDrfby+IgOfFz/dqlHfj4s\nWJB6PYmHMsTa0GnRxuvmsJd+B30YTkcdeii88kqYDj0Udu6k30EfklMrqebkQL9+TVmTpuvXL/ST\nqd+G5lWfjjrssFBw2GHhdRaSRqY4miq3Gecpau8fQ7Pz9ZbY6jt4z52ber5gQYi7cfFO96e3PWlS\n4/WT2yxTHLW3Z0vL5tdqbwP+DhxhZpvN7Jtmdq6ZnRtV+QuwHngZuAGYB+DulcB5wIPAC8ASd19X\np4OW8MgjNa9ZXHhheP3II/GWeeQRWLgwTI88Anv2wIwZ4XHlSli6lNkLR7N4yH9TVBS+SVHU6U0W\nX7aB2QuGhfYmTgxfgzjppHBydMkS6NEjtNOjR3hdXMzsk95l8WE/pajnzlQ7k+5g9pl7+cX9n6Z0\n7GoSOVWEnTRMRUWw+NS/MPtb+czuci+LO18Qlscp6rqDxZ++itnv38DsOQkWT7w1ihGKOr1J6dC/\nRX2Fc6K/HXUtNx32fyjq9CbgJKxmX+B0ytsb5kV/dEVFsHgxzJ6dtvn4IpN4uMZyn+I1ElQCjlFF\n1w67MarIyYGunStr1C0oMAYOhA6JUN7BdlPUdze9D+uZunZx0EFhAjjsMHof1pOivrtTyyQqKSoK\nH0iyqXfvsA2S/8V36EB1vw3N44MPal6z6N49vK79y8QsxditW806NROKE32jmEQCSkvh5ptT59Qz\nS75/ISFU/y303Fln/1i3LnPSyMmB5DeW0/evsrLMPSYSoZ04/72nt3fLLWGdkmXJdfzFL1L1Z88O\n+3VynUNdp3e33fTuTfXfTOmMzRT13Fmjj+Q3vepr+5FH6iaN1Dar+TeVHkftedmiMb2l3dqwYQPd\nunWjd+/e+u1BO+Du7Nixg/fee4+BAwe2dTjSQj5RF73lwNW/f382b97Mtm3b2joUiXTq1In+/fu3\ndRjSRpQwpN3Ky8vTf7Ii7YjuJSUiIrEoYYiISCxKGCIiEssB9S0pM9sGtOCP9ltcH2B7WwfRRPtj\nzLB/xq2YW8/+GHe2Yi5y91i/ej6gEkZ7Z2YVcb++1l7sjzHD/hm3Ym49+2Pc7SFmnZISEZFYlDBE\nRCQWJYzWtbitA9gH+2PMsH/GrZhbz/4Yd5vHrGsYIiISiz5hiIhILEoYrcDMvmtmL5rZOjO7Kq38\nUjN72cxeMrMvtmWM9TGzi8zMzaxPWlm7jNvMro6281oz+6OZ9Uyb1y5jBjCzKVFcL5vZJW0dT33M\nbICZLTez56N9+YKo/CAze9jM/l/02KutY63NzBJm9nQ00uf+EnNPM7sz2qdfMLNj2jpuJYwsM7OJ\nwCnAke4+DLgmKh9KGOtjGDAF+IWZ7cNQKtljZgOAEyE1+lI7j/thYLi7jwT+RRgPvl3HHMVxPTAV\nGArMiuJtjyqBi9x9KDAW+E4U6yXAX919EPDX6HV7cwFhuISk/SHmnwEPuPtg4EhC/G0atxJG9pUS\nxi7/GMDdt0blpwC3u/vH7r6BMC7ImDaKsT7/Dfxvao4E027jdveHovFUAJ4ijNYI7ThmQhwvu/t6\nd98N3E6It91x9y3uvjp6/h7hANaPEO8tUbVbgFMzt9A2zKw/cBJwY1pxe4+5B3Ac8BsAd9/t7u/Q\nxnErYWTf4cAEM/uHmf3NzD4XlfcD0sdv3RyVtQtmdgrwurs/U2tWu447zTeA+6Pn7Tnm9hxbvcys\nGDgK+AdwSDRaJsAbwCFtFFZ9/ofwj09VWll7j3kgsA34bXQq7UYz60Ibx63bm7cAM3sEODTDrMsJ\n2/ggwkf4zwFLzOywVgyvXo3EfRnhdFS70lDM7n5PVOdywumT8taM7ZPCzLoCdwHfc/d30we3cnc3\ns3bz1Uszmw5sdfdVZnZCpjrtLeZILvBZ4Lvu/g8z+xm1Tj+1RdxKGC3A3SfXN8/MSoG7PXx/+Z9m\nVkW4J8zrwIC0qv2jslZTX9xmNoLwH84z0cGgP7DazMbQxnE3tK0BzOxrwHRgkqe+M97m27oB7Tm2\nOswsj5Asyt397qj4TTPr6+5bzKwvsLX+FlrdeGCGmU0DOgHdzayM9h0zhE+am939H9HrOwkJo03j\n1imp7PsTMBHAzA4HOhBuIHYvMNPMOprZQGAQ8M82izKNuz/r7ge7e7G7FxN23s+6+xu047jNbArh\n1MMMd08fALvdxgysBAaZ2UAz60C4OH9vG8eUkYX/Hn4DvODui9Jm3Qt8NXr+VeCe1o6tPu5+qbv3\nj/bjmcAyd/8K7ThmgOhv7TUzOyIqmgQ8TxvHrU8Y2XcTcJOZPQfsBr4a/ee7zsyWEHaCSuA77r63\nDeOMxd3bc9zXAR2Bh6NPRk+5+7ntOWZ3rzSz84AHgQRwk7uva+Ow6jMemAM8a2ZrorLLgJ8STrV+\nk3C36C+3UXxNsT/E/F2gPPpHYj3wdcI/+W0Wt37pLSIiseiUlIiIxKKEISIisShhiIhILEoYIiIS\nixKGiIjEooQhrcbMTo3ufDu4rWNpCWZWHH1dujltmJktM7PuLRVXhj4Gm9nfzexjM/t+zGUmRHek\nXWNmnWPUPy+6227tOxtPN7MfNyd+aT+UMKQ1zQIejx4/ccws0++epgHPuPu7Wez6LeB8ojslxzQb\n+L/uPsrdP4xR/wlgMuG3AenuA042s/wm9C3tlBKGtIro/kPHAt8k/OI2WZ5jZr+I7vn/sJn9xcxO\nj+aNjm7YuMrMHoxuhVC73XvM7Ozo+TlmVuf+UWb2aTN7ysyeNbP/Y2a70uZdbGYrLYyh8aOorNjC\n+AM3RP9lP5T8LzuK6Rkzewb4Tlo7CQvjcSTbOicqP8HMVpjZvYQfDtY2m+jXulG/L5rZzWb2LzMr\nN7PJZvaEhfEP9ukOu+6+1d1XAnvi1DezbxF+EPaTTNuznj6edveNGcodeJRwuxbZzylhSGs5hXBv\n/38BO8xsdFT+H0AxYSyIOcAxUH3PomuB0919NOEX8wsytDsXuMLMJgAXEX4dW9vPgJ+5+wjCbU6I\n+jiRcJuQMcAoYLSZHRfNHgRcH41h8g7wpaj8t4Qbwh1Zq49vAjvd/XOEm0x+O7oNCYSbyF3g7odn\niG08sCrt9WeAhcDgaDqLkGi/T/hVNWY2MTpVVHt6MkP7TebuNxJuQXGxu882s2719LfG4o3dUQFM\naInYpG3p1iDSWmYRDtwQxnyYRThQHgv8wd2rgDfMbHlU5whgOKnbfCSALdTi7m+a2RXAcuA0d38r\nQ9/HkBo34FZSp2ZOjKano9ddCYliE7DB3ZO3v1gFFFsYwa+nuz8Wlf+eMPBRsq2RyU9HQI+ord3A\nP6NxODI5KBpbImmDuz8LYGbrCIPluJk9S0isuPtyQoJrFVF8zelvK/CpFgpH2pAShmSdmR0EfB4Y\nYeF2zAnAzezihhYD1rn7MTG6GAHsoOkHJSOcp/91rXiLgY/TivYCjV34NcInjwdrtXUC8H4Dy1Wa\nWU6UMKnVb1Xa6yqiv1cLozj+d4a2PnD3cWb2HeDbUdk0d/93I7E3yMy6ASvqmX2Wu2c61ZauExDn\nOoi0czolJa3hdOD37l4U3QF3ALCBcJriCeBL0bWMQ4ATomVeAgrMrPoUlZkNq91wdF5/KmEwn++n\nnQZK9xSpU0oz08ofBL4RXV/BzPqZ2cH1rUQ04tk7ZnZsVDS7Vlul0ak0zOxwCwPeNOYloEnjo7j7\n8uhidO1pXDT/+rSyBpOFmf3VzBocsMnd36unv1ExkgWEQcSa9W0yaR+UMKQ1zAL+WKvsrqj8LsJ1\nheeBMmA14VrAbkKiuTK6wLwGGJfegJl1BG4AvhEdGC8i3BnYqOl7wIVmtpZwjWAnhCFdCaeo/h6d\n8rkT6NbIunwduN7C3VrT+7kxWofVFr5q+2vifYK/j1SSzAozO9TMNgMXAv9pZpvNrLuZ5RC2R6bT\neE3t4/yoj/7AWjNLHw51ImE9ZT+nu9VKmzOzru6+y8x6E8apGB+NB9BS7ecDH0bXAmYCs9y9XYyb\nHX3z63fu/oU26Hs4IdlemMU+DgFudfdJ2epDWo8ShrQ5M3sU6EkYXOoqd7+5hdufQBgrwwjfePqG\nu7/ckn00h5l9mfANsmz+FqNNWBjDfk/aFwhkP6aEISIisegahoiIxKKEISIisShhiIhILEoYIiIS\nixKGiIjEooQhIiKx/H81D14Z3nvOKgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0xb0bb630>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(age_gender[y == 0], datain['train'].Pclass[y == 0], 'xr')\n",
    "plt.plot(age_gender[y == 1], datain['train'].Pclass[y == 1], 'ob')\n",
    "plt.xlabel('Age x gender (m=-1, f=1)')\n",
    "plt.ylabel('Pclass')\n",
    "plt.legend(['not survivied', 'survived'])\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As a result, I consider a full 2nd order polynomail function that contains terms like $Age^2$ and $Age*Sex$ as features as well as the linear terms."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "datain = readdata('train.csv', 0.6, 0.6, features)\n",
    "\n",
    "X = np.array(datain['train'].ix[:,1:])\n",
    "y = np.array(datain['train'].ix[:,0])\n",
    "\n",
    "Xvad = np.array(datain['validation'].ix[:,1:])\n",
    "yvad = np.array(datain['validation'].ix[:,0])\n",
    "\n",
    "Xq = titan.xquad(X)\n",
    "Xvadq = titan.xquad(Xvad)\n",
    "m = Xq.shape[0]\n",
    "Xfull = titan.normdata(np.concatenate((Xq, Xvadq)))\n",
    "Xq = np.concatenate((np.ones(shape=(Xq.shape[0], 1)), Xfull[:m, :]), axis = 1)\n",
    "Xvadq = np.concatenate((np.ones(shape=(Xvadq.shape[0], 1)), Xfull[m:, :]), axis = 1)\n",
    "\n",
    "theta0 = np.zeros(shape = (Xq.shape[1], 1))\n",
    "regpara = 1.0\n",
    "w = scop.fmin_l_bfgs_b(titan.lfCost, theta0, titan.lfGradient, (Xq, y, regpara))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Before evaluating the results I visulize the parameters for all the quadratic terms below"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAScAAAD8CAYAAAA11GIZAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAE11JREFUeJzt3X2wXVV9xvHv4w0hvERQgw4msWQ6gQ5jFWgEHRxfoGhA\natoZZhpQGalOyoz40nZGsX/Uaf3HjjOOb0gmQyM6otQi1JSJpDpqqSNIEkQwhNhLpOZGNATkVZvk\n3vv0j3PSHi659+zLPWefde5+PjN7uPvsfdb63Qn5Za21115LtomIKM0LBh1ARMSRJDlFRJGSnCKi\nSElOEVGkJKeIKFKSU0QUKckpIoqU5BQRRUpyiogiLehLoccc54WLX9yPop/jqMcP1FIPwKETjq6t\nLoDJhfXVtfDxidrq0u/q+zPzsYtqqwvAI6qlnv/53W84dPCZOVX21jcf50cfq/bnvv3eA1tsr55L\nfbPVl+S0cPGLOfWSv+pH0c9x8jd/Xks9AL9+24ra6gJ4+hX11XXKN5+srS7teLC2uiZfubK2ugAO\nnVjPP2Db7/jcnMt49LEJ7tpS7X+ykZP/a8mcK5ylviSniCifgUkmBx3GtJKcIhrKmEOurzs/W0lO\nEQ2WllNEFMeYiYKXTMpUgogGm8SVjm4kbZS0T9JPp7kuSZ+VNCrpXklndSszySmioQxM4EpHBdcD\nM001uBBY2T7WAdd2KzDJKaLBetVysn078NgMt6wBvuyWO4ETJZ08U5kZc4poKAOHqo85LZG0reN8\ng+0Ns6huKbCn43ys/dnD032hUnKStBr4DDACXGf7E7MIKiIK5OpdNoD9tlf1M56puiYnSSPANcAF\ntLLdVkmbbN/f7+Aioo8ME/U9rNsLLO84X9b+bFpVxpzOBkZt77Z9ELiRVv8xIoZYa4Z4taMHNgGX\nt5/avRZ4wva0XTqo1q07Ul/xnOcfY0SUQUzQmxeVJX0NeBOtsakx4GPAUQC21wObgYuAUeC3wBXd\nyuzZgLikdbQeEXLU8S/qVbER0SetAfHeJCfbl3a5buB9symzSnKq1Fdsj9xvADj2pcvLnXYaEcDh\neU71LPHyfFRJTluBlZJW0EpKa4HL+hpVRNRiskctp37ompxsj0u6CthCayrBRts7+h5ZRPTVfGg5\nYXszrQGtiJgnjJgo+CWRzBCPaLCh7tZFxPxkxEGPDDqMaSU5RTRUaxJmunURUaChHxCPiPnHFhNO\nyykiCjSZllNElKY1IF5uCig3sojoq0YOiC945BlOWn9HP4p+jomj69sifNs/3FZbXQB//I6/qK2u\nJ05dXFtdT76l69r2PXPMI/W+5vm7k+rpJo3/pDf1TGSeU0SUJjPEI6JYk3laFxGlab34m+QUEYUx\n4lBeX4mI0thkEmZElEiZhBkR5TFpOUVEoTIgHhHFMRruxeYkbQQuBvbZfmX/Q4qIOrS2hiq3fVKl\nTXc9sLrPcURE7VqbalY5BqHK7iu3Szql/6FERJ1MZohHRKEasRJm53bkizi2V8VGRJ/YakbLqXM7\n8hfqxdmOPKJwrQHxvL4SEcUpew3xrpFJ+hpwB3CapDFJ7+l/WBHRb60BcVU6BqHK07pL6wgkIuqX\nGeIRUZzSZ4iXmzYjou8meUGloxtJqyXtkjQq6eojXD9B0r9J+omkHZKu6FZmWk4RDWXDocm5t08k\njQDXABcAY8BWSZts399x2/uA+23/iaSTgF2SbrB9cLpyk5wiGqrVretJ5+lsYNT2bgBJNwJrgM7k\nZGCxJAHHA48B4zMVmuQU0WA9miG+FNjTcT4GnDPlns8Dm4BfAouBP7c9OVOhGXOKaKhZTiVYImlb\nx7FultW9FbgHeDlwBvB5SS+c6QtpOUU01qy6dfttr5rm2l5gecf5svZnna4APmHbwKiknwN/ANw1\nXYVpOUU02GR7HfFuRxdbgZWSVkhaCKyl1YXr9AvgfABJLwNOA3bPVOjQt5x84EBtdb1x3WxbsnPz\n+Bn1/fEc88iM3f+eWnLfjOOgPaUP7KutLoBHHzqplnomj57766utp3Vzf7fO9rikq4AtwAiw0fYO\nSVe2r68HPg5cL+k+QMBHbO+fqdyhT04R8fz0chKm7c3A5imfre/4+ZfAW2ZTZpJTRINla6iIKM7h\np3WlSnKKaLBGLDYXEcPFFuNJThFRonTrIqI4GXOKiGIlOUVEcUpfbC7JKaLBhnqek6TlwJeBl9Hq\npm6w/Zl+BxYR/WXDeA8Wm+uXKi2nceBvbN8taTGwXdK3p6xyFxFDaKi7dbYfBh5u//yUpJ20FpdK\ncooYYvNqzEnSKcCZwI+OcC3bkUcMGc+H5CTpeOAbwIdsPzn1erYjjxg+Qz0gDiDpKFqJ6QbbN/c3\npIiogz3kY07t3RL+Cdhp+1P9Dyki6iEmCn5aVyWyc4F3AedJuqd9XNTnuCKiBrYqHYNQ5WndD6Dg\njmlEPC95ty4iyuTWuFOpkpwiGmzon9ZFxPzjwgfEk5wiGizduogo0ryYIR4R84ud5BQRhcpUgogo\nUsac5olFt95Va32LLzmntrqOu+k5C030zYLly2qr6+nPvby2ugAWnVXPXykdnHuLx4jJPK2LiBIV\n3HBKcoporAyIR0SxCm46JTlFNFhaThFRHAOTk0lOEVEaAwW3nMp9jhgRfWdXO7qRtFrSLkmjkq6e\n5p43tRer3CHpP7qVmZZTRJP1YEBc0ghwDXABMAZslbSpc29LSScCXwBW2/6FpJd2Kzctp4jGqrZE\nb4VB87OBUdu7bR8EbgTWTLnnMuBm278AsL2vW6Fdk5OkRZLukvSTdnPs77t9JyKGhCseM1sK7Ok4\nH2t/1ulU4EWSvi9pu6TLuxVapVt3ADjP9tPtLaJ+IOlbtu+s8N2IKJXB1Z/WLZG0reN8Q3uvyqoW\nAH8EnA8cA9wh6U7bP5vpCzOybeDp9ulR7aPgqVsRUV3l5LTf9qppru0FlnecL2t/1mkMeNT2M8Az\nkm4HXg1Mm5wqjTlJGpF0D7AP+LbtI25HLmmbpG2HOFCl2IgYtN5067YCKyWtkLQQWAtsmnLPN4HX\nS1og6VjgHGDnTIVWelpnewI4oz3ifoukV9r+6ZR7sh15xLDpwd9U2+OSrgK2ACPARts7JF3Zvr7e\n9k5JtwH3ApPAdVNzyFSzmkpg+3FJ3wNWAzMWHBGF6+EkTNubgc1TPls/5fyTwCerllnlad1J7RYT\nko6hNZfhgaoVRES5ejUJsx+qtJxOBr7Unmj1AuDrtm/tb1gRUYthfrfO9r3AmTXEEhE1U8Gjw3l9\nJaKpqj2JG5gkp4jGUtGrEiQ5RTRZWk4RUaTJQQcwvSSniKYqfLG5JKeIBsvTuogoU8HJKYvNRUSR\n0nIqWJ1bhB9c/Zra6nrw8kO11XXqBx6srS6ACz72ZC31fP2ff9uTctKti4jymOF+fSUi5rG0nCKi\nROnWRUSZkpwiokhJThFRGjnduogoVZ7WRUSJ0nKKiDIVnJwqv77S3rvux5KyfnjEfOD/H3fqdgzC\nbN6t+yBdNsGLiCHTm001+6Lqjr/LgLcB1/U3nIiokyarHYNQteX0aeDDFL1uXkTMJ1U21bwY2Gd7\ne5f71knaJmnbIQ70LMCI6KMh79adC7xd0kPAjcB5kr4y9SbbG2yvsr3qKI7ucZgR0XPDPiBu+6O2\nl9k+BVgLfNf2O/seWUT0X8Etp8xzimiyguc5zSo52f4+8P2+RBIRtRKDexJXRVpOEU2VF38jolhJ\nThFRpIKTU7aGimiwXk0lkLRa0i5Jo5KunuG+10gal3RJtzKTnCKarAdTCSSNANcAFwKnA5dKOn2a\n+/4R+PcqoSU5RTSVe/Zu3dnAqO3dtg/Smqy95gj3vR/4BrCvSnhJThFNVr3ltOTw62ntY11HKUuB\nPR3nY+3P/o+kpcCfAddWDS0D4hENNoupBPttr5pDVZ8GPmJ7Uqq2NHCSUwCw8LattdX1+4+/qra6\nJp94qra6AEYve0Ut9Rx4aGFvCurN07q9wPKO82XtzzqtAm5sJ6YlwEWSxm3/63SFJjlFNFXv3pvb\nCqyUtIJWUloLXPasquwVh3+WdD1w60yJCZKcIhpL9GaGuO1xSVcBW4ARYKPtHZKubF9f/3zKTXKK\naLBevb5iezOwecpnR0xKtt9dpcwkp4gmK3iGeJJTRJMlOUVEcbIqQUQUK8kpIkqUxeYiokhD361r\n77zyFDABjM9xGntElGCAmxdUMZuW05tt7+9bJBFRv3mSnCJiHunVDPF+qbpkioHvSNo+ZamEiBhi\nmnSlYxCqtpxeb3uvpJcC35b0gO3bO29oJ611AIs4tsdhRkTPFT7mVKnlZHtv+7/7gFtorXw39Z5s\nRx4xZIZ6O3JJx0lafPhn4C3AT/sdWETUYMi3I38ZcEt7kagFwFdt39bXqCKiFiUPiHdNTrZ3A6+u\nIZaIqNswJ6eImKec11ciokClz3NKcopoMpebnZKcIhosLaeIKE/hkzCTnCIaLAPiEVGkJKeIKI/J\ngHjEs9x5b21VPXHZa2urC+BF39pVT0Xj4z0pJgPiEVGmJKeIKE0mYUZEmTy4heSqSHKKaLJyc1OS\nU0STpVsXEeUxkG5dRBSp3NyU5BTRZCV366puDRUR81CvtoaStFrSLkmjkq4+wvV3SLpX0n2Sfiip\n6+q6lZKTpBMl3STpAUk7Jb2uyvciomBVNzfokpskjQDXABcCpwOXSjp9ym0/B95o+w+BjwMbuoVX\ntVv3GeA225dIWgjZmC5i2LUmYfakX3c2MNrebwBJNwJrgPsP32D7hx333wks61Zo1+Qk6QTgDcC7\n25UcBA7OIvCIKFVvViVYCuzpOB8Dzpnh/vcA3+pWaJWW0wrgEeCL7X7iduCDtp+p8N2IKNgsWk5L\nJG3rON9gu2vX7Dn1SW+mlZxe3+3eKmNOC4CzgGttnwk8AxxpwGudpG2Sth3iwCxDjojazW7Maf/h\nHb3bR2di2gss7zhf1v7sWSS9CrgOWGP70W7hVUlOY8CY7R+1z2+ilayeJduRRwybak/qKjyt2wqs\nlLSiPSa9FtjUeYOkVwA3A++y/bMq0VXZVPNXkvZIOs32LuB8Oga6ImKI9WBA3Pa4pKuALcAIsNH2\nDklXtq+vB/4OeAnwhfbu4eO2V81UbtWnde8Hbmhnxd3AFc/v14iIYvRwU03bm4HNUz5b3/Hze4H3\nzqbMSsnJ9j3AjFkuIoZQlumNiCKVm5uSnCKaTJPlbr+S5BTRVKZXkzD7IskpoqGEe/X6Sl8kOUU0\nWZJTRBQpySkiipMxp4goVZ7WRUSBnG5dxKC88Kt31lqfFy+uqaIeJBX3qJw+SXKKaLJye3VJThFN\nlnlOEVGmJKeIKI4NE+X265KcIposLaeIKFKSU0QUx0CF3XwHJckporEMzphTRJTGFD0g3nVrKEmn\nSbqn43hS0ofqCC4i+syudgxAla2hdgFnAEgaobVZ3i19jisi6jCPBsTPBx60/d/9CCYi6jS/Xvxd\nC3ztSBckrQPWASzi2DmGFRF9Z6DgJVOqbEcOQHtDzbcD/3Kk69mOPGIIDfOYU4cLgbtt/7pfwURE\nnebP6yuXMk2XLiKGkMHDPs9J0nHABcBf9jeciKjVsM8Qt/0M8JI+xxIRdZtHT+siYr6wi35al+QU\n0WRpOUVEeYwnJgYdxLSSnCKaqvAlUypPwoyIeciT1Y4uJK2WtEvSqKSrj3Bdkj7bvn6vpLO6lZmW\nU0RDGXAPWk7tBQGuoTXdaAzYKmmT7fs7brsQWNk+zgGubf93Wmk5RTSV3auW09nAqO3dtg8CNwJr\nptyzBviyW+4ETpR08kyFpuUU0WA9GhBfCuzpOB/jua2iI92zFHh4ukL7kpye4jf7v+ObZrusyhJg\nfz/iKcB8/d3ye031ZG8DmcHvzbWAp/jNlu/4piUVb18kaVvH+QbbG+Yaw0z6kpxsnzTb70jaZntV\nP+IZtPn6u+X3Gm62V/eoqL3A8o7zZe3PZnvPs2TMKSLmaiuwUtKK9tJKa4FNU+7ZBFzefmr3WuAJ\n29N26SBjThExR7bHJV0FbAFGgI22d0i6sn19PbAZuAgYBX4LXNGt3JKSU1/7rwM2X3+3/F4BgO3N\ntBJQ52frO3428L7ZlCkX/G5NRDRXxpwiokhFJKduU9+HkaTlkr4n6X5JOyR9cNAx9ZKkEUk/lnTr\noGPpJUknSrpJ0gOSdkp63aBjaqqBd+vaU99/RsfUd+DSKVPfh0579uvJtu+WtBjYDvzpsP9eh0n6\na2AV8ELbFw86nl6R9CXgP21f137ydKztxwcdVxOV0HKqMvV96Nh+2Pbd7Z+fAnbSmhE79CQtA94G\nXDfoWHpJ0gnAG4B/ArB9MIlpcEpITtNNa583JJ0CnAn8aLCR9MyngQ8D5S6j+PysAB4Bvtjusl7X\nXj8/BqCE5DSvSToe+AbwIdv1vdzQJ5IuBvbZ3j7oWPpgAXAWcK3tM4FngHkxBjqMSkhOs57WPiwk\nHUUrMd1g++ZBx9Mj5wJvl/QQrS74eZK+MtiQemYMGLN9uIV7E61kFQNQQnKqMvV96EgSrbGLnbY/\nNeh4esX2R20vs30KrT+r79p+54DD6gnbvwL2SDqt/dH5wLx4gDGMBj5DfLqp7wMOqxfOBd4F3Cfp\nnvZnf9ueSRvlej9wQ/sfyt1UeM0i+mPgUwkiIo6khG5dRMRzJDlFRJGSnCKiSElOEVGkJKeIKFKS\nU0QUKckpIoqU5BQRRfpfzUpPcqqUkhgAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0xd1b3d30>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "fullw = np.zeros(shape=(X.shape[1], X.shape[1]))\n",
    "fullw[np.triu_indices(8)] = w[0][9:]\n",
    "plt.imshow(np.abs(fullw))\n",
    "plt.colorbar()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The plot above shows the relative magnitude (absolute values) of the model parameters for the quadratic terms. The item (row=1, col=2), i.e., (normalized) $Age * Sex$, has a much larger weight than other parameters. Some other unexpected parameters that has relatively harge weight are $Pclass * SibSp$ (0, 3), and $Parch * title$ (4, 7). "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.83551401869158881"
      ]
     },
     "execution_count": 37,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y_predict = titan.sigmoid(np.dot(Xqn,w[0]))\n",
    "y_predict[y_predict > 0.5] = 1\n",
    "y_predict[y_predict <= 0.5] = 0\n",
    "(y_predict == y).sum()*1.0/y.size"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.82352941176470584"
      ]
     },
     "execution_count": 38,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "yvad_predict = titan.sigmoid(np.dot(Xvadq, w[0]))\n",
    "yvad_predict[yvad_predict > 0.5] = 1\n",
    "yvad_predict[yvad_predict <= 0.5] = 0\n",
    "(yvad_predict == yvad).sum()*1.0/yvad.size"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Both the test set and the validation set accuracies are improved in this quadratic model. Next we analyze the model by investigating the learning curve for training set size and regularization parameter value.\n",
    "\n",
    "For the learning curve of dataset size I use different training set sizes and a fixed validation set."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYwAAAEKCAYAAAAB0GKPAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XmczfX+wPHX2xh7JKJCKCJkG1tUiKIklJKWm1tdaaG6\ndUvdfrR3tdItt1TSLlGuokhMWiRrsqfN0lVKxdiH9++P9xmOaZYzM+d7zpyZ9/PxOI8553u+3+95\nf+bMnPf5flZRVZxzzrnclIh3AM455xKDJwznnHMR8YThnHMuIp4wnHPORcQThnPOuYh4wnDOOReR\nQBOGiHQXkdUislZEhmbxfCcR+UNEloRuwyI91jnnXGyVDOrEIpIEPAWcAWwA5ovIFFVdkWnXj1X1\nnHwe65xzLkaCvMJoA6xV1W9VdQ8wHugVg2Odc84FILArDKAGsD7s8QagbRb7tReRpcBG4BZVXZ6H\nYxGRgcBAgLJly6bUqlUrX8Hu37+fEiWKV5OOl7noK27lBS9zXq1Zs+YXVT0ykn2DTBiRWAQcq6pp\nInI2MBmon5cTqOoYYAxAq1atdMGCBfkKJDU1lU6dOuXr2ETlZS76ilt5wcucVyLyQ6T7BpmGNwLh\nX/drhrYdoKpbVTUtdH8akCwiVSM51jnnXGwFmTDmA/VFpK6IlAIuAqaE7yAiR4mIhO63CcXzayTH\nOueci63AqqRUNV1ErgemA0nAWFVdLiKDQs8/DfQFrhGRdGAncJHa9LlZHhtUrM4553IXaBtGqJpp\nWqZtT4fdfxJ4MtJjnXPFx969e9mwYQO7du3K03GVKlVi5cqVAUVVOEVS5jJlylCzZk2Sk5Pz/Trx\nbvR2zrksbdiwgcMOO4w6deoQqrmOyLZt2zjssMMCjKzwya3Mqsqvv/7Khg0bqFu3br5fp3j1PXPO\nJYxdu3ZRpUqVPCULlzURoUqVKnm+WsvME4ZzrtDyZBE90fhdesIAuPdeKnz9dbyjcM65Qs0Txq+/\nwpgxtBg8GF57Ld7ROOcKid9//53Ro0fn+bizzz6b33//Pcd9hg0bxsyZM/MbWtx4wqhSBRYsYFuD\nBnDJJXDLLZCeHu+onHNxll3CSM/l82HatGkcfvjhOe5zzz330LVr1wLFFw+eMACqV+fLRx6B666D\nRx+Fs86yKw/nXLE1dOhQvvnmG5o3b07r1q059dRTOffcc2nUqBEAvXv3JiUlhcaNGzNmzJgDx9Wp\nU4dffvmF77//nhNPPJG//e1vNG7cmDPPPJOdO3cCMGDAACZOnHhg/+HDh9OyZUtOOukkVq1aBcDm\nzZs544wzaNy4MVdddRW1a9fml19+ifFv4VDerTZEk5PhySehRQu49lpo3RomT4amTeMdmnPF3o03\nwpIlke27b19ZkpJy3695cxg5Mvvn//Wvf7Fs2TKWLFlCamoqPXr0YNmyZQe6pY4dO5YjjjiCnTt3\n0rp1a84//3yqVKlyyDm+/vprXn/9dZ599lkuvPBCJk2axKWXXvqn16patSqLFi1i9OjRPPLIIzz3\n3HPcfffdnH766dx+++28//77PP/885H9AgLkVxiZXXklfPQR7N4NJ58Mb74Z74icc4VAmzZtDhnD\n8MQTT9CsWTPatWvH+vXr+TqLjjN169alefPmAKSkpPD9999nee7zzjvvT/t88sknXHTRRQB0796d\nypUrR7E0+eNXGFlp1w4WLIDzz4cLL4Tbb4d77yWiry3OuajL6Uogs23bdgYycK98+fIH7qempjJz\n5kzmzp1LuXLl6NSpU5ZjHEqXLn3gflJS0oEqqez2S0pKyrWNJJ78CiM7Rx8Ns2fDwIHw4IPQsyfk\n0vPBOVd0HHbYYWzbti3L5/744w8qV65MuXLlWLVqFZ9//nnUX79Dhw5MmDABgBkzZvDbb79F/TXy\nyhNGTkqXhmeegaefhpkzoU0bWOGrxDpXHFSpUoUOHTrQpEkT/vGPfxzyXPfu3UlPT+fEE09k6NCh\ntGvXLuqvP3z4cGbMmEGTJk148803Oeqoo+I+5YlXSUXi6quhSROromrbFl5+GXr3jndUzrmAvZbN\n2KzSpUvz3nvvZflcRhtE1apVWbZs2YHtt9xyy4H748aN+9P+AK1atSI1NRWwCQWnT59OyZIlmTt3\nLvPnzz+kiisePGFEqkMHWLgQzjsP+vSBYcNg+HAoZktBOudiY926dVx44YXs37+fUqVK8eyzz8Y7\nJE8YeVKjhvWguvZauOce6+f38stQsWK8I3POFTH169dn8eLF8Q7jEP71OK/KlIHnn7cxG9OmWRXV\n6tXxjso55wLnCSM/RGxU+MyZNiK8TRv45JN4R+Wcc4EKNGGISHcRWS0ia0VkaA77tRaRdBHpG7bt\nJhFZLiLLROR1ESkTZKz50rGjjdeoUAEeeije0TjnXKACSxgikgQ8BZwFNAL6i0ijbPYbAcwI21YD\nGAK0UtUm2LreFwUVa4Eceyz06mVjNvbsiXc0zjkXmCCvMNoAa1X1W1XdA4wHemWx32BgEvBzpu0l\ngbIiUhIoB/wYYKwF060bpKXBZ5/FOxLnXJxUqFABgB9//JG+fftmuU+nTp1YsGBBjucZOXIkO3bs\nOPA4kunSYyXIXlI1gPVhjzcAbcN3CF1J9AE6A60ztqvqRhF5BFgH7ARmqOoMsiAiA4GBANWrVz/Q\nhzmv0tLS8n1sUnIyHZKSWP/ss3yXrzPER0HKnKiKW5kTubyVKlXKdqR1Tvbt25ev46IhY23tF154\nIcsY9u3bx/bt23OM7/HHH6d3794HJjJ84403Dpw7O5GWedeuXQX7e1DVQG5AX+C5sMeXAU9m2udN\noF3o/jigb+h+ZWAWcCSQDEwGLs3tNVNSUjS/Zs+ene9jVVW1Y0fV5s0Ldo4YK3CZE1BxK3Mil3fF\nihX5Om7r1q1Ref3bbrtNn3zyyQOPhw8frvfee6+efvrp2qJFC23SpIlOnjz5wPPly5dXVdXvvvtO\nGzdurKqqO3bs0H79+mnDhg21d+/e2qZNG50/f76qqg4aNEhTUlK0UaNGOmzYMFVVHTVqlCYnJ2uT\nJk20U6dOqqpau3Zt3bx5s6qqPvroo9q4cWNt3LixPv744wde74QTTtCrrrpKGzVqpGeccYbu2LEj\nyzJl9TsFFmiEn+tBXmFsBGqFPa4Z2hauFTA+tNZsVeBsEUkPJYnvVHUzgIi8BbQHXgkw3oLp1g3u\nuAN++gmqV493NM4VLXmY37zsvn2RTRSay/zm/fr148Ybb+S6664DYMKECUyfPp0hQ4ZQsWJFfvnl\nF9q1a8e5556b7XrZ//nPfyhXrhwrV65k6dKltGzZ8sBz999/P0cccQT79u2jS5cuLF26lCFDhvDY\nY48xe/Zsqlatesi5Fi5cyAsvvMC8efNQVdq2bUvHjh2pXLky33zzDW+88Uau06gXVJBtGPOB+iJS\nV0RKYY3WU8J3UNW6qlpHVesAE4FrVXUyVhXVTkTKib0TXYCVAcZacN262c8ZWdacOecSTIsWLfj5\n55/58ccf+fLLL6lcuTJHHXUUd9xxB02bNqVr165s3LiRn376KdtzzJkz58AHd9OmTWkatr7OhAkT\naNmyJS1atGD58uWsyGWeuk8++YQ+ffpQvnx5KlSowHnnncfHH38MQO3atSOaRr2gArvCUNV0Ebke\nmI71chqrqstFZFDo+adzOHaeiEwEFgHpwGJgTHb7FwrNm8ORR8L06XDZZfGOxrmiJQ/zm+8MtSNE\nwwUXXMDEiRPZtGkT/fr149VXX2Xz5s0sXLiQ5ORk6tSpk+W05rn57rvveOSRR5g/fz6VK1dmwIAB\n+TpPhkinUS+oQMdhqOo0VT1BVY9X1ftD257OKlmo6gBVnRj2eLiqNlTVJqp6maruDjLWAitRAs48\n064w9u+PdzTOuSjo168f48ePZ+LEiVxwwQX88ccfVKtWjeTkZGbPns0PP/yQ4/GnnXbagQkMly1b\nxtKlSwHYunUr5cuXp1KlSvz000+HTGSY3bTqp556KpMnT2bHjh1s376dt99+m1NPPTWKpc2dj/SO\npm7dYPPmyNeSdM4Vao0bN2bbtm3UqFGDo48+mksuuYQFCxZw0kkn8dJLL9GwYcMcj7/mmmtIS0vj\nxBNPZNiwYaSkpADQrFkzWrRoQcOGDbn44ovp0KHDgWMGDhxI9+7d6dy58yHnatmyJQMGDKBNmza0\nbduWq666ihYtWkS/0DmJtHU8EW5x7SWlqrppkyqoPvBAwc8VA4ncgya/iluZE7m88e4llUgiLXNB\ne0n5FUY0Va8OLVrA++/HOxLnnIs6TxjR1q2bjfjeujXekTjnXFR5woi2bt0gPd3mlnLOFYjVmLho\niMbv0hNGtLVvb7PXTp8e70icS2hlypTh119/9aQRBarKr7/+SpkyBZv021fci7ZSpaBzZ08YzhVQ\nzZo12bBhA5s3b87Tcbt27SrwB2OiiaTMZcqUoWbNmgV6HU8YQejWDd55B9auhXr14h2NcwkpOTmZ\nunXr5vm41NTU2Hc3jbNYldmrpILQvbv99KsM51wR4gkjCMcfbzfvXuucK0I8YQSlWzdfhc85V6R4\nwghKt26wfTt8+mm8I3HOuajwhBGUzp2hZElvx3DOFRmeMIJy2GHQoYMnDOdckeEJI0jdutnMtTks\nsOKcc4nCE0aQfBU+51wR4gkjSM2bQ7Vq3r3WOVckeMIIkq/C55wrQgJNGCLSXURWi8haERmaw36t\nRSRdRPqGbTtcRCaKyCoRWSkiJwcZa2C6dYNffoHFi+MdiXPOFUhgCUNEkoCngLOARkB/EWmUzX4j\ngMwV/aOA91W1IdAMWBlUrIE680z76b2lnHMJLsgrjDbAWlX9VlX3AOOBXlnsNxiYBPycsUFEKgGn\nAc8DqOoeVf09wFiDU62arcLnCcM5l+CCnK22BrA+7PEGoG34DiJSA+gDdAZahz1VF9gMvCAizYCF\nwA2quj3zi4jIQGAgQPXq1UlNTc1XsGlpafk+Njd1TzyRWm+8wadTp7KvfPlAXiM/gixzYVXcylzc\nygte5iDFe3rzkcBtqrpfRMK3lwRaAoNVdZ6IjAKGAv+X+QSqOgYYA9CqVSvt1KlTvgJJTU0lv8dG\n5LXXODU9HYJ8jTwKvMyFUHErc3ErL3iZgxRkldRGoFbY45qhbeFaAeNF5HugLzBaRHpjVyMbVHVe\naL+JWAJJTBmr8Hn3WudcAgvyCmM+UF9E6mKJ4iLg4vAdVPXA6igiMg54V1Unhx6vF5EGqroa6AKs\nCDDWYJUqBaefbu0YqnDo1ZRzziWEwK4wVDUduB6YjvVwmqCqy0VkkIgMiuAUg4FXRWQp0Bx4IKhY\nY6JbN/juO1uFzznnElCgbRiqOg2Ylmnb09nsOyDT4yVYlVXRkDFNyPTpUL9+fGNxzrl88JHesZKx\nCp93r3XOJShPGLHkq/A55xKYJ4xY8lX4nHMJzBNGLHXuDMnJ3r3WOZeQPGHEkq/C55xLYJ4wYq1b\nN/jyS9i0Kd6ROOdcnnjCiDVfhc85l6A8YcRas2Y2g61XSznnEownjFjzVficcwnKE0Y8+Cp8zrkE\n5AkjHjJW4YtG99qvv4beveHii218h2rBz+mcc1nwhBEP1apBy5YFa8fYvx/+/W9rE/noI5g2DU45\nBVJSYOxY2LkzevE65xyeMOKnWzeYOxe2bs37sd9/D127wpAhtiDT8uWwcSM88wykp8OVV0LNmnDr\nrTZDrnMuGHv3wmOPwT33FIure08Y8dKtm324z5oV+TGq8NxzcNJJMH++3Z86FY45BsqXh4EDbYzH\nRx9Bly72h3z88XDuud7I7ly0zZ8PrVvDzTfD8OHw0kvxjihwnjDi5eSTbRW+SKulfvwRevSAv/3N\n/ki/+squJDIvxiQCp50GEybYlcg//wnz5lmCOvFEeOIJ+OOPqBfHuWJj2za48UZo1w42b4aJE6Fj\nR7juuiK/3o0njHjJvApfdlThtdegSRNITbUP/JkzoU6d3F+jZk24915Ytw5eeQWOOAJuuAFq1IBr\nr6WcV1c5lzdTp0LjxvZ/OGgQrFgB558PL79s/9MXX2zVVEWUJ4x4ym0Vvp9/hr594ZJLoGFDq24a\nPNjGcuRF6dJ2jrlz7TL6ggtg7FjaXHGFJa233oJduwpeHueKqk2boF8/OOccqFgRPvkEnnoKKlWy\n52vVgjFj7P/rrrviGmqQAk0YItJdRFaLyFoRGZrDfq1FJF1E+mbaniQii0Xk3SDjjJuMaUKy6l77\n1lt2VfHuuzBiBHz8cXRW6mvVCl54ATZs4JuBA+Gbb+wbUtmyduVx6qnwl7/YH/2LL9rrbtjg7R+u\neNq/H5591qpzJ0+2K/ZFi6B9+z/v27cvXHEFPPigtSMWQYEt0SoiScBTwBnABmC+iExR1RVZ7DcC\nyGpypRuw9cArBhVnXB1/PNSrZ9VSgwfbtt9+s/uvvmpdb2fNssQRbVWrsr5/f44fPdoS1qJF8O23\ndsXz0UdWhRVeVVaqlFWDHXcc1K3755+HHx79GJ2Lp1Wr4OqrYc4ca6N45hlo0CDnY0aNsi9Zl15q\nNQJHHBGbWGMkyDW92wBrVfVbABEZD/QCVmTabzAwCWgdvlFEagI9gPuBvwcYZ3x162bf+HfvtuRw\n1VVWFXXXXXDHHbZ+RpCSkqwxvUePQ7fv2WNtHxlJJPznF1/Ali2H7v/Xv8Lzz/+5Ed65RLN7t13V\n33+/9T58/nn7+47kb7tCBWtzPPlkSzYTJhSp/4kgE0YNYH3Y4w1A2/AdRKQG0AfoTKaEAYwEbgUO\ny+lFRGQgMBCgevXqpKam5ivYtLS0fB9bEFWOOYaTduzgt/btqbxoEdvr1GHlk0+S1qBB4CvzRVTm\nUqXsW1Wmb1ZJaWmU3bSJMv/7H5UXLKDGCy/wbcmSrLv44uACjoJ4vc/xUtzKCwUrc8WvvqLBo49S\n/ocf+On001l73XXsPeKIPFcx1briCo4fM4ZVt93GprPPzlcseRGz91lVA7kBfYHnwh5fBjyZaZ83\ngXah++OAvqH75wCjQ/c7Ae9G8popKSmaX7Nnz873sQWybZtqcrKqiOqtt6ru3Bmzl45amffvV+3f\n38rwzjvROWdA4vY+x0lxK69qPsv8+++qgwapguqxx6pOnVqwIPbtUz39dNVy5VRXry7YuSJQkPcZ\nWKARfq4HeYWxEagV9rhmaFu4VsB4sUu2qsDZIpKOXYmcKyJnA2WAiiLyiqpeGmC88VGhArz9Nhx5\nJLRpE+9o8kfEBhGuXm3dCufNs0ZC5xLBBx/A5ZfDTz/B3/8Od99t/5cFUaKEDeRr2tT+Jz77zK7W\nE1yQvaTmA/VFpK6IlAIuAqaE76CqdVW1jqrWASYC16rqZFW9XVVrhrZfBMwqkskiQ48eiZssMpQr\nZ71IypWDXr2s8d65wi41FXr2hCpV7IvOo48WPFlkqFHDvkgtXAjDhkXnnHEWWMJQ1XTgemA61tNp\ngqouF5FBIjIoqNd1cVSrFkyaZCPM+/eHffviHZFz2Vu0yKbNOf54SxytWkX/Nfr0sSl7Hnoob9MA\nFVJBVkmhqtOAaZm2PZ3NvgOy2Z4KpEY5NBeUDh3gP/+x3l633QaPPBLviIK1bZtVZ3z11cHOAQ0a\n2EDLBg1sZuIi1EumyFizBrp3t26vM2bYFUZQHnvMGs0vuwyWLg32tQIWaMJwxdSVV8KSJXZ537Sp\nDQQsirZsgbPOsiqHc86xK6sPPzx01HylSocmkIxbvXpQpkzur6FqMxr/+qstuhV+C992zjnW9dPl\nbv16OOMMuz9jhlUdBal8eXj9dWjb1uaCmzQpYb9EeMJwwXjsMZt2feBA+7BM9DaazDZtsoWwVq+2\nD4BevWz7/v32gbR6tQ38Wr3abrNmHTqbaYkSNhAyI4GUL599MkhPzzqGpCSoWtV+Tp4MjRrZh5LL\n3i+/2Pv2++9WDXXCCbF53RYtbAT4LbdYu8bf/hab140yTxguGMnJ8OabNrNu796wYIFNw14UrFtn\n65Fs3GiT0XXtevC5EiWgdm27ZaysmCEtzapCMpJIRkL56CO7KqlSxRJA1ao2DczJJ9v98O3ht4oV\n7Zvq1q02G8AVV1i9fOnSsf19JIpt2+Dss+1K8P337UM8lm66yWZ1uOEGm4KnYcPYvn4UeMJwwalS\nBf77X/vgO+88+0YXSTVMYbZmjSWIrVutO2ZWcwplp0IFm+6lZctDt2fM05XXSSUzVKxoE9+ddRbc\nd5/Nd+QOtXu3NUAvWmTztHXsGPsYSpSAceMOdrWdOzfhkrvPVuuCddJJNvXzvHk2HXQir0r25Zf2\nzXDXLkt+eUkWOSlRIv/JIkP37jBggFV7LF4clbCKjH37bLbmDz+05YvPPTd+sRxzjMWweDHceWf8\n4sgnTxgueH36HJz9duTIeEeTP59/bsvhliplk9E1bx7viP7sscdsAOgVVxTpNRnyRNXmdJo0CR5/\nvHB0wDj3XLjmGutB+MEH+T/P7t1W1fvMMxz72mvRiy8HnjBcbPzf/9k06rfcYj1TEsmsWVYNVaWK\nrYNQWOueK1eGp5+2HmoPPRTvaAqH22+3yQP/+U9bJa+weOQRmw3h8sutIT43O3faVXpGl/UWLayK\ns3VrGDSIGm+/HZOrd2/DcLGRUX+7Zo0tRPPFF9FZ3yNo77xjC07Vr2+J7uij4x1Rznr1gosugnvu\nsc4GxdnDD9uss4MGFb52nXLlrKttmzbWDX3y5INdbXfssOrPhQvttmiR9TjMGAh7xBGQkmJfvlJS\noGVL5v7wA51i0FXXE4aLnQoVrBG8dWv7YPv8c2uwLaxef90GW7VsCe+9lzgDrjKW8b3iCnjggXhH\nEx/PPw+33mpfTp58snCOe2jWzBLaTTdZtdnu3ZYgVq482BHiyCMtKfTsaX+HKSlw7LF/Ls+6dTEJ\n2ROGi626da277RlnWEPkf/9b8AbfIIwZY99MTzsNpkwp3IktsyOPhH//G/r3p+akSdClS7wjiqmq\nc+bYBIJnnmljX5KS4h1S9oYMsSvXZ5+Fo46yhHDeeQeuHKhZs1AlO08YLvY6d7aVya6/3to27r8/\nb8dv2QJff223NWtsTfRKlezD/bTTCj5y95FH4B//sD77Eyfa8rWJpl8/GD+eus8/b99gE6H6Lxpm\nzaLRffdZVc9bbxX+GWJLlLDqqN9+g+rV4x1NrjxhuPi49lqrp33gAeuX3q/foc+npR1MCOHJ4euv\nbRR0hhIl7BL911+twRdsMrmM5HHaaXZVE8m3NFUYPtzquy+4wJapLewfONkRgdGj2d+gAUlXXmnd\ngAvjlVw0zZ8PvXqxs0YNyk+daqPnE0GpUgmRLMAThosXEatbXrHC5kBatcqm1MhIDJs2Hbp/zZr2\nLblvX/tZv75N61C3rg1+Sk+3BDRnjt3++19b+jbj2FDyKFemjCWGzAlk/35bC2HUKGuEfOaZwl2V\nEYljjuGb666j4YgR1rvmuuviHVFwVq60gYtVq/Llww/TvoitpV1YeMJw8VOqlPWPP/lkG6dRrZol\ngrPOOjQp1KtnvUpyUrKk1fumpFgVzP79lowyEsisWfDaa7QBq24KvwJp1MjaK154wbpePvZYoao3\nLohN3brRcMkSmzm4Rw+bv6qoWbfO2iuSkuCDD9izYUO8Iyqyck0YIpIEDFHVx2MQjytuqle3+ZS2\nb4fDD4/eeUuUsPmVmjSx6i9VWLuWVWPG0PDnny2JTJpk+5YubT1Uhg+3WxFJFoCVZcwYaNzYJryb\nMaNolW/7dpupd+tWm5OrXj3whBGYXBOGqu4Tkf6AJwwXjOTk6CaLrIhA/fps6tGDhp062bZ16yxx\nfPqpNZIW1enBjz3WxiRcc41NS3HllfGOKDpULQkuWwbTphXO0fdFTKRVUp+KyJPAG8D2jI2quiiQ\nqJyLhWOPhUsvtVtRN3AgvPGGtdN07x78GhCx8MQTNlbmvvusTC5wkXabaA40Bu4BHg3dcl1KTUS6\ni8hqEVkrIkNz2K+1iKSLSN/Q41oiMltEVojIchG5IcI4nXNZKVHC1mHYuzfxJ4EEuzK85Rabl+n2\n2+MdTbER0RWGqnbO64lDbR9PAWcAG4D5IjJFVVdksd8IIHyCoXTgZlVdJCKHAQtF5IPMxzrn8uD4\n460b8003wWuv2cDJRPTjj3DhhdZD7qWXin534UIkot+0iFQSkcdEZEHo9qiIVMrlsDbAWlX9VlX3\nAOOBXlnsNxiYBPycsUFV/5dR3aWq24CVQBG4hnYuzgYPtl5pQ4bATz/FO5q827PHulanpdnAvEq5\nfQy5aBKN4NJURCYBy4AXQ5suA5qp6nk5HNMX6K6qV4UeXwa0VdXrw/apAbwGdAbGAu+q6sRM56kD\nzAGaqOrWLF5nIDAQoHr16injx4/PtTxZSUtLo0KFCvk6NlF5mYu+rMpbbt06Wl11Fb+0b8+Ku+6K\nT2D5VH/UKGpMnszyYcPY3Dnrio/i9h5DwcrcuXPnharaKqKdVTXXG7Akkm2Znu8LPBf2+DLgyUz7\nvAm0C90fB/TN9HwFYCFwXiRxpqSkaH7Nnj0738cmKi9z0ZdteR98UBVU33wzpvEUyIsvWsw335zj\nbsXtPVYtWJmBBRrB56uqRtzovVNETsl4ICIdgJ25HLMRqBX2uGZoW7hWwHgR+T6UYEaLSO/QayRj\nVVWvqupbEcbpnItExtTY110X2XoM8bZ4sc3o2qkT/Otf8Y6m2Iq0W+0g4KWwdovfgMtzOWY+UF9E\n6mKJ4iLg4vAdVLVuxn0RGYdVSU0WEQGeB1aq6mMRxuici1TJkjYmIyXFRre/8kr2++7ebVO1/O9/\nf/6Zcb9BAxshX61a9GPdssVmcK1SxboGl/QJKuIlkpHeJYAGqtpMRCoCaBZtCZmparqIXA9MB5KA\nsaq6XEQGhZ5/OofDO2BVWF+JyJLQtjtUdVpur+uci1DTprYS3d132yqCZcv+ORH87382k2pmIpYc\njjrKfk6caKPIn3nGluSNln374OKLrWfUnDnBJCQXsUhGeu8XkVuBCZEkikzHTgOmZdqWZaJQ1QFh\n9z8BitAMSuV/AAAgAElEQVT8Bc4VUnfcAW+/bdPMA5QpY6sKHn20JZHOnS0pHH30wZ9HH21rboR/\n01++3NbLPu88+zlqVHRG7991F0yfbjMRt21b8PO5Aon02m6miNzCn0d6bwkkKudcbJQqBR9/bN/g\njz7aForKz1xTjRvbCor33Wfrm8yaZZM5du2a/9imTLHz/fWvNlLdxV2kjd79gOuw7q0LQ7cFQQXl\nnIuhihXtaqJSpYJNTJicbNVbn31ma1GccYaN+9ixI+/n+vprWx43JQWeeqpoTZiYwHJNGKE2jEtV\ntW6m23ExiM85l2jatLFeTTfcYGuetGgB8+ZFfnxamrWDJCfbjMKJuOJhEZVrwlDV/cCTMYjFOVdU\nlC0LI0fChx/Crl3Qvj3ceaeN1M6JKlx1lS2I9PrrULt2bOJ1EYm0SupDETk/1N3VOecic/rpsHQp\nXH65tW20bQtffZX9/iNHWtfZ+++3Ki1XqESaMK4GJgC7RWSriGwTkTz1mHLOFVOVKtmYj8mTrXG9\nVSt46CHrMhsuNdVWQ+zTx1YIdIVOpAmjEjAAuE9VK2JTnXv6d85FrlcvW+zonHMsIXTsCN98Y89t\n2AD9+tmKeePGeSN3IRVpwngKaAf0Dz3ehrdrOOfy6sgjbZDfyy9b8mjWDEaPhgsusN5Ub71lvbZc\noRTpOIy2qtpSRBYDqOpvIlIqwLicc0WViK1y2LEjXHGFzWcF8Oab0KhRfGNzOYo0YewNLXSkACJy\nJLA/sKicc0VfrVo2ivuFF2D/flvnwhVqkSaMJ4C3gWoicj82s+ydgUXlnCseSpSAK6+MdxQuQpEu\n0fqqiCwEumBzPPVW1ZWBRuacc65QiXieYFVdBawKMBbnnHOFmK+e7pxzLiKeMJxzzkXEE4ZzzrmI\neMJwzjkXkUAThoh0F5HVIrJWRIbmsF9rEUkXkb55PdY551xsBJYwQgP9ngLOAhoB/UXkT8M4Q/uN\nAGbk9VjnnHOxE3G32nxoA6xV1W8BRGQ80AtYkWm/wcAkoHU+jnXOuYSmCjt32uS96en2M6/3ly+v\nSKdOwccaZMKoAawPe7wBOGQVdxGpAfQBOnNowsj12LBzDAQGAlSvXp3U1NR8BZuWlpbvYxNVYSnz\nxo1lqFBhH5Uq7Q38tQpLmWOluJUXEqPMO3YksWhRZebNO4IvvjiCn38uU6DzHX54Ixo3To1OcDkI\nMmFEYiRwm6ruz+/aTKo6BhgD0KpVK+2UzzSbmppKfo9NVPEuc1oaDB8Oo0bBMcfY4mz16wf7mvEu\nc6wVt/JC4Syzqi0iOG0avPcefPwx7N0Lhx0GXbtC69ZQqhSULAlJSQdv4Y+zu5+UBCtXroxJmYNM\nGBuBWmGPa4a2hWsFjA8li6rA2SKSHuGxLoFNngyDB9syCJddZv9Ep50GM2dC48bxjs65gktLg1mz\nDiaJdetse5MmcOONcPbZtnJtqSjM+52c/EfBTxKBIBPGfKC+iNTFPuwvAi4O30FV62bcF5FxwLuq\nOllESuZ2rEtMP/wAQ4bAlClw0km2Gmf79rBihX3T6tQJZsyAFi3iHalzeaMKq1ZZcpg2za4i9uyB\nChXsb/uf/4SzzrJJehNVYAlDVdNF5HpgOpAEjFXV5SIyKPT803k9NqhYXfD27rWqp+HD7fFDD9m3\nrORke9yoEcyZA1262DLQ779vyz87V5jt2HHwKmLaNPtCBPb3PGSIJYhTTonOVURhEGgbhqpOA6Zl\n2pZlolDVAbkd6xLT3Llw9dXw1VfQsyf8+99Qu/af96tX72DS6NoVpk61airnCpPvvrO/zalTYfZs\n2L0bype3v9vbb4fu3bP++y4K4t3o7Yqw336DoUNhzBioWRPeftuWdc6pf0Pt2pY0una1f7zJk+HM\nM2MXs3OZ7d0Ln3xyMEmsCs3ZXb8+XHONtUWcdhqULh3fOGPBE4aLOlV49VX4+99hyxb7effdVpcb\niWOOgdRUSxQ9e9rKneeeG2jIzh1i0yZri5g6FT74ALZutWqljh3tarlHj+B79BVGnjBcVK1eDdde\na/W6bdpYA3bz5nk/T7Vqdo7u3eH88y0BXXhh9ON1DmyF2AULrB1i6lS7D/bl5cILLUF07Rr5l56i\nyhOGi4pdu+DBB+Ff/4KyZWH0aBg40PqI59cRR1g323POgf79bTTs5ZdHL2ZXvK1fb2N/PvzQvtj8\n/LOtGNuuHdx3nyWJZs1yrkItbjxhuALZtcsu3W+9FdauhYsvhkcfhaOOis75K1a08/fuDQMGWNIY\nNCg653YmNdWqXHr2LNofjlu2WCP1zJmWJL7+2rYfeaRdPfToAd26QdWq8Y2zMPOE4fLsp5/g3Xfh\nnXesfnfHDuvhNGMGnHFG9F+vfHl7rQsusEbGnTvhppui/zrFze7dcMcd8Nhj9jglBR54wN7DopA4\nduywsRAZVxGLF1v7WoUK1hZxzTXWs6lJE7uycLnzhOFypWpdYt95x25ffGHbata0KqKePW3sRJC9\nRMqUgUmT4JJLrBF9xw4bCOXyZ80aq+ZbtAiuv96Sxd132zfsjh0tcbRvH+8o82bvXpg/H156qTZ3\n3QWffWbbkpPh5JPhrrsOTsORMf7H5Y0nDJel3bvho48OJomMAUmtW9sHS8+esa/fLVUKXn/d2kju\nvNOSxn33FY1vw7GiCi+9BNddZwl+8mTr6gyWQJ57Du69Fzp0sLaj++6z97mw2bPHqpRWrbI5mubN\ns7/XbdtApA7Nm9vA0C5dbOBc+fLxjrho8IThDti82XqJvPMOTJ9uc+GULWtVFHfeaXW8Rx8d3xhL\nloRx4yyuBx6A7dvh8cc9aURi61arhnntNZuC5ZVXoEaNg8+XLm2JZMAAG1w5YoT1cOvf374kxKMb\n6e+/W1LISAwZP7/91qb1zlC/vl19dukCycmf0qvXKbEPthjwhOF46SUbXPfZZ/YN9JhjrPG6Z0/7\nByxbNt4RHqpECXj6aYtr1Chr0/jPf7weOifz5tl7+sMPdtUwdGj2PdjKl7fnBw2Chx+GkSNhwgS4\n4goYNsyqIqNJ1SahzJwUVq2y8RAZSpWyxNCsGfTrBw0bwoknwgknHNrdNTU1PboBugM8YRRzL75o\n3yibNLEPg549oWXLwv+NXcSuLMqVs+68mzZZEon3FVBhs3+/XSnceaddTcyZE3nbxOGHw/3325xI\nDzxgv9+M6qzbb89fb6KdO2H5cvjyS1iyxH4uXQp/hE22evjhlgjOOutgUmjYEOrWtStMFz/+6y/G\nliypxK232lXEe+8lXkOgiH2QVa9u34gbNbIuvX/9a+FPeLHwv//Brbc2ZeFCG3z2zDP2YZxX1avb\nlVzGiP2RI+2K9OabbVvFilkft2mTJYTw5LB69cGqpAoVoGlTq0pq0sQSw4kn2qBNf/8KJ08YxdSa\nNTBsWBPq1YOJExMvWYS74Qb7Nvq3v8GVV1rD+DPPwHHHxTuy+Jk2zXqwpaVV4rnnrDqpoB/CtWvD\n2LE25ub//s+Sx5NP2tXGmWdaT7qMxPDll9b9OsOxx1p7yPnnW5VS8+Z2xeDViInFE0Yx9Msv1oCd\nlKRMnZq/b52FzQkn2KCsMWPsA+2kk6yufsiQgo02TzS7d8Ntt9kVQbNmcNNNC7n88jZRfY2GDW1+\nr4ULrarrllsOPleqlC2AdfbZBxND06ZQuXJUQ3Bx4gmjmNm9G847z6ZFePTRZdSt2zLeIUVNiRLW\nUHvOOfbz73+H8ePh+eetyqOoW70aLrrIvuUPGWJtF59/viOw10tJsarMzz6zKb+bNYMGDRL7atXl\nzC8IixFVuOoqG/364ovQuPHWeIcUiJo1rWvwa69Z98uWLW3Q1u7d8Y4sGKrwwgtWzvXrreyjRtlg\nx1ho3/5gO4Qni6LNE0Yxcu+91vf+3nutW2JRJmLjB1autAbfu++2D9QVK7JpoU1Qv/9u5bziCps0\nb+lSu8JyLgiBJgwR6S4iq0VkrYgMzeL5XiKyVESWiMgCETkl7LmbRGS5iCwTkddFJEbfl4qm116z\n5VEvv7x4TalRtaolyXfftYFr11/fghtvtEGJiW7OHKsGmjTJeovNmGFjaJwLSmAJQ0SSgKeAs4BG\nQH8RaZRptw+BZqraHLgCeC50bA1gCNBKVZtg63pfFFSsRd2nn1pX044drVG4OHZZ7NHD+v+fe+6P\njBpljeIffBDvqPJn715rbO7c2RqZP/vMeioVp8Z9Fx9BXmG0Adaq6requgcYD/QK30FV01RVQw/L\nAxr2dEmgrIiUBMoBPwYYa5H1zTc2NXjt2vZNtKgsRp8fFSvCjTd+zUcf2e/hzDOtKue33+IdWeTW\nrrW5ke6/3wZcLl5s83s5Fwty8PM6yicW6Qt0V9WrQo8vA9qq6vWZ9usDPAhUA3qo6tzQ9huA+4Gd\nwAxVvSSb1xkIDASoXr16yvjx4/MVb1paGhWK2HJa27aV5LrrWvLHH8k89dQiatbcecjzRbHMucko\n8549JXjxxdqMH38slSrt5bbbVtG27ZZ4h5ctVZg+/SieeKIeJUsqN9+8ho4dN+d6XHF+j4uTgpS5\nc+fOC1W1VUQ7q2ogN6Av8FzY48uAJ3PY/zRgZuh+ZWAWcCSQDEwGLs3tNVNSUjS/Zs+ene9jC6Pd\nu1U7d1ZNTladMyfrfYpamSORucyLFqk2bWq/p2nT4hNTbrZsUb3gAlVQ7dRJdd26yI/197h4KEiZ\ngQUa4ed6kFVSG4FaYY9rhrZlSVXnAMeJSFWgK/Cdqm5W1b3AW0CCzc4fP6o2DmH2bBuZe+qp8Y6o\n8GrRwqbFbtLExqfMnh3viA710Uc28O3tt23525kzoVat3I9zLghBJoz5QH0RqSsipbBG6ynhO4hI\nPRFrghWRlkBp4FdgHdBORMqFnu8CrAww1iJlxAjrlz9sGFx6abyjKfwOP9x6GB1/vE2++Nln8Y7I\nGrbvuMMatsuWhblzbQS3N2y7eAosYahqOnA9MB37sJ+gqstFZJCIZKzKfD6wTESWYD2q+oWukuYB\nE4FFwFehOMcEFWtR8uab1mOmf38brOYiU7Wq9Zo65hibl2rhwvjF8vXXNhjuwQdtbqxFi6BVZDXM\nzgUq0KlBVHUaMC3TtqfD7o8ARmRz7HBgeJDxFTWffw5/+YutljZ2bPHsPlsQRx9taz+feqr1oMqo\nqoqVjBHbQ4ZYL66JE22yPucKCx/pXUR8/70ttXnMMVbfHatpIYqaWrUsaZQpY+s/r1kTm9fdsgUu\nuMCuKNq2tRHbnixcYeMJowj44w8bmLZnD0ydCkceGe+IEtvxx1vS2L/f1gr5/vtgX+/DD61h+7//\ntfanDz6I/qp2zkWDJ4wEN3eu1bmvWWMD8xo2jHdERUPDhvbBvX07nH46bMy2f1/+bdxos8t27WrL\non7+uU3N7mtEuMLK/zQTkKp1r+zc2RpHV6+Gl1+2DzYXPc2awfTptn5Ily6HLghUEHv22FrZDRrA\n5MnWOWHJEpsu3LnCzBNGAtm/36ot2rWDM86wq4rHHoMffrBvqi76Wre21evWr7ff+ZYCDgb/8ENL\nRLfeagl+xQqbFLJs2ejE61yQPGEkgPR0ePVVq+fu3du+8T7zjK31cNNNtjayC84pp1iiXrMGunWz\nNqO82rDBplnv2tWuMN59F6ZMKd7LyLrE4wmjENu922aXbdDg4AC8V16xKqiBA6F06fjGV5x07Wrd\nXJcssQ4G27dHdtyePdaQ3bChLWx09902a26PHsHG61wQPGEUQtu3W1XTccfB1VdDlSpW1710qa1s\nVtIX1o2Lc86xdUXmzrUuzLt25bz/Bx/YVeHQoZZwVqyw0ffe5dklKk8Yhchvv9lqeLVrw80325XF\nBx/AvHn2AeW9Z+Lvggtg3DiYNQv69rUriMzWr7f9zjzTqhOnTrWEX7duzMN1Lqr8u2oh8NNP8Pjj\nMHo0bNtm32TvuANOPjnekbmsXHYZ7NhhEzxecgm8/rpd9e3ebVeG991nPdnuvRduucWvKFzR4Qkj\nznbutHmCNm60RtHbb7deNK5wu/pqe+9uuskSwiWXwA03WMN4nz72BaB27XhH6Vx0ecKIs3HjrAfN\n++9bDxyXOG680dqb7rzTOiPUqwfvvQfdu8c7MueC4QkjjtLT4ZFHbO6gM8+MdzQuP/75T1v6dfdu\nGDzYe665os0TRhxNnGhjKR55xGeWTWSDB8c7Audiw/vdxImq9c9v0MB6QDnnXGHnVxhxMmOGDQJ7\n/nnvLuucSwz+URUn//oX1KhhvWuccy4ReMKIg3nzIDUV/v53byR1ziWOQBOGiHQXkdUislZEhmbx\nfC8RWSoiS0RkgYicEvbc4SIyUURWichKESkyw9hGjIDDD4e//S3ekTjnXOQCa8MQkSTgKeAMYAMw\nX0SmqOqKsN0+BKaoqopIU2ACkLEE0CjgfVXtKyKlgHJBxRpLq1bZNBH//Cccdli8o3HOucgFeYXR\nBlirqt+q6h5gPHBIfyBVTVNVDT0sDyiAiFQCTgOeD+23R1V/DzDWmHn4YauG8q6YzrlEE2QvqRrA\n+rDHG4C2mXcSkT7Ag0A1IGPS57rAZuAFEWkGLARuUNU/TSotIgOBgQDVq1cnNTU1X8GmpaXl+9hI\nbd5cipdeasc55/yPFSu+ZsWK3I8JUizKXNgUtzIXt/KClzlQqhrIDegLPBf2+DLgyRz2Pw2YGbrf\nCkgH2oYejwLuze01U1JSNL9mz56d72MjdfPNqklJqt9+G/hLRSQWZS5siluZi1t5Vb3MeQUs0Ag/\n14OsktoI1Ap7XDO0LUuqOgc4TkSqYlcjG1R1XujpiUDLoAKNhd9+s1Xy+vXzaa6dc4kpyIQxH6gv\nInVDjdYXAVPCdxCReiI2KYaItARKA7+q6iZgvYg0CO3aBYhzBU7BjB4NaWlw223xjsQ55/InsDYM\nVU0XkeuB6UASMFZVl4vIoNDzTwPnA38Rkb3ATqBf6BIJYDDwaijZfAv8NahYg7ZzJ4waBWedZSuw\nOedcIgp0ahBVnQZMy7Tt6bD7I4AR2Ry7BGvLSHgvvACbN9tSnc45l6h8pHfA0tOtK+3JJ8Opp8Y7\nGuecyz+ffDBgb74J338PI0f6FObOucTmVxgBUrVJBk88EXr2jHc0zjlXMH6FEaD334elS60Nw6cw\nd84lOv8YC9CIEVCzJlx8cbwjcc65gvOEEZDPP4ePPrIpzEuVinc0zjlXcJ4wAjJiBFSu7FOYO+eK\nDk8YAVi50qYwHzwYKlSIdzTOORcdnjAC8PDDULasT2HunCtaPGFE2fr18MorcNVVULVqvKNxzrno\n8YQRZY8/Dvv3W2O3c84VJZ4womjLFhgzBvr3hzp14h2Nc85FlyeMKHrqKdi+HW69Nd6ROOdc9HnC\niJIdO+CJJ6BHDzjppHhH45xz0ecJI0rGjoVffvEpzJ1zRZcnDOCrr+Dnn0uza1f+jt+7Fx55BNq3\nh1NOiW5szjlXWPjkg9haFdu3nwzYQLtq1eDII+0Wfj+rx2XLwoQJ8MMP8O9/x7kgzjkXoEAThoh0\nB0ZhS7Q+p6r/yvR8L+BeYD+QDtyoqp+EPZ8ELAA2quo5QcSoCq++Cp98sprKlRuwebOtjvfzz7Bh\nAyxebPf37s36+AoVYN8+aNTI2i+cc66oCixhhD7snwLOADYA80VkiqquCNvtQ2CKqqqINAUmAA3D\nnr8BWAlUDC5O6NULKlX6H506NchyH1XYuvVgIglPKps3w6+/wpVX+hTmzrmiLcgrjDbAWlX9FkBE\nxgO9gAMJQ1XTwvYvD2jGAxGpCfQA7gfiOgxOBCpVslu9evGMxDnn4ifIhFEDWB/2eAPQNvNOItIH\neBCohiWIDCOBW4HDcnoRERkIDASoXr06qamp+Qo2LS0t38cmKi9z0Vfcygte5iDFvdFbVd8G3haR\n07D2jK4icg7ws6ouFJFOuRw/BhgD0KpVK+3UKcfds5Wamkp+j01UXuair7iVF7zMQQqy1n0jUCvs\ncc3Qtiyp6hzgOBGpCnQAzhWR74HxwOki8kqAsTrnnMtFkAljPlBfROqKSCngImBK+A4iUk9EJHS/\nJVAa+FVVb1fVmqpaJ3TcLFW9NMBYnXPO5SKwKilVTReR64HpWLfasaq6XEQGhZ5/Gjgf+IuI7AV2\nAv1UVbM9qXPOubgJtA1DVacB0zJtezrs/ghgRC7nSAVSAwjPOedcHvjIAeeccxHxhOGccy4iUpSa\nDERkM/BDPg+vCvwSxXASgZe56Ctu5QUvc17VVtUjI9mxSCWMghCRBaraKt5xxJKXuegrbuUFL3OQ\nvErKOedcRDxhOOeci4gnjIPGxDuAOPAyF33FrbzgZQ6Mt2E455yLiF9hOOeci4gnDOeccxEpVglD\nRLqLyGoRWSsiQ7N4/hIRWSoiX4nIZyLSLB5xRlMEZe4VKvMSEVkgIqfEI85oyq3MYfu1FpF0Eekb\ny/iCEMH73ElE/gi9z0tEZFg84oymSN7nULmXiMhyEfko1jFGWwTv8z/C3uNlIrJPRI6IWgCqWixu\n2ASI3wDHAaWAL4FGmfZpD1QO3T8LmBfvuGNQ5gocbMtqCqyKd9xBlzlsv1nYXGd94x13DN7nTsC7\n8Y41xmU+HFvh89jQ42rxjjvoMmfavyc203fUYihOVxgHloxV1T3YOhu9wndQ1c9U9bfQw8+xNTwS\nWSRlTtPQXxeZlslNULmWOWQwMAn4OZbBBSTSMhclkZT5YuAtVV0HoKqJ/l7n9X3uD7wezQCKU8LI\nasnYGjnsfyXwXqARBS+iMotIHxFZBUwFrohRbEHJtcwiUgPoA/wnhnEFKdK/7fah6sf3RKRxbEIL\nTCRlPgGoLCKpIrJQRP4Ss+iCEfFnmIiUA7pjX4qiJu5LtBZGItIZSxgJX58fCc1imdw4hxS0kcBt\nqro/tH5XcbAIq5pJE5GzgclA/TjHFLSSQArQBSgLzBWRz1V1TXzDiomewKequiWaJy1OCSOiJWNF\npCnwHHCWqv4ao9iCkudlckXkOBGpqqqJOnlbJGVuBYwPJYuqwNkikq6qk2MTYtTlWmZV3Rp2f5qI\njC4G7/MGbAXP7cB2EZkDNAMSNWHk5f/5IqJcHQUUq0bvksC3QF0ONhg1zrTPscBaoH28441hmetx\nsNG7ZegPUOIde5BlzrT/OBK/0TuS9/mosPe5DbCuqL/PwInAh6F9ywHLgCbxjj3IMof2qwRsAcpH\nO4Zic4WhkS0ZOwyoAowOfftM1wSe9TLCMhepZXIjLHOREmGZ+wLXiEg69j5fVNTfZ1VdKSLvA0uB\n/cBzqrosflEXTB7+tvsAM9SurKLKpwZxzjkXkeLUS8o551wBeMJwzjkXEU8YzjnnIuIJwznnXEQ8\nYTjnnIuIJwxXaIhI89Ao5IzH5+Y022yUXnOIiKwUkVdz2S9VRHLsYi0iN4amZIgrERkgIk9msb20\niMwMzWTaL5/n7i0ijfJxXK7vpYgcIyIT8xOXi41iMw7DRYeIJKnqvgDOWxJojo3CngagqlOAKdF+\nrUyuBbqq6oYonOtG4BVgRxTOFYQWAKraPNIDsni/ewPvYrPAZt63pKqmZ3WeSN5LVf0RGy/iCim/\nwnAAiEgdEVklIq+GvnFPzPi2LCLfi8gIEVkEXBC6Evg8NJHd2yJSObRfqoiMCpuLv01o+xEiMjm0\n/+eh6VcQkbtE5GUR+RR4GbgH6JfxDTj8m3Iovlmhc3woIseGto8TkSfE1i/5VrJZ20JE/h6KaZmI\n3Bja9jQ2VfR7InJTpv3Lisj40O/ibWwuoozn/iO2dshyEbk7tG0IcAwwW0RmZ7dfFnEduHIRkaoi\n8n3ofmMR+SL0u1gqIvVD2y8N2/6MiCSFtv9VRNaIyBdAhyxepxqWzFqHjj1eRLqIyGKx9V/Gikjp\nrN7vsHO0B84FHg47R6qIjBSRBcANItJTROaFzjtTRKqHjg1/L7N8z0Lv8bKw/d8SkfdF5GsReSgs\njiszyioiz0oWV1MuIPEe7u63wnED6mBTm3cIPR4L3BK6/z1wa9i+S4GOofv3ACND91OBZ0P3TwOW\nhe7/Gxgeun86sCR0/y5gIVA29HgA8GTY6xx4DLwDXB66fwUwOXR/HPAm9uWnETb9c+aypQBfYdO3\nVwCWAy3CylY1i2P+jo2kBVsnJB1oFXp8ROhnUqjMTbM6V3b7ZXqd1LDzVgW+D/udXRK6XwpLWCeG\nfg/Joe2jgb8AR2NTfRwZ2vfT8N9j2Gt1IrQmBlAGm/n0hNDjl4Abs3q/M51jHGFTqYTiHx32uDIH\nBwRfBTyaxXuZ5XuG/Q0uC9v/W2yaizLAD9g8SseE4jsCSAY+zqqsfgvm5lcYLtx6Vf00dP8VDp2t\n9w0AEakEHK6qGauXvYglhwyvg01kCFQUkcND53k5tH0WUEVEKob2n6KqOyOI7WTgtdD9lzPFNllV\n96vqCqB6FseeArytqttVNQ14Czg1l9c7DfsdoKpLsSSZ4cLQt+/FQGPsQy8rke6XlbnAHSJyG1A7\n9DvqgiW/+SKyJPT4OKAtkKqqm9XWSXgjgvM3AL7TgzO3Zn4fIzlHVvvWBKaLyFfAP7ByZyW39wzg\nQ1X9Q1V3YVVgtbF5sD5S1S2quhdLPC5GPGG4cJnniQl/HOm8NDmdIyvRmO9md9j9QOcrF5G6wC1A\nF1Vtiq0hUia/+2FXLhn/hweeV9XXsOqfncA0ETkdK9uLqto8dGugqndFrXCHysv7Er7vv7Fv/CcB\nV5N1mSGy9yx8n314m2vcecJw4Y4VkZND9y8GPsm8g6r+AfwmIhnf0C8DwtdK7gcgtjb4H6H9PwYu\nCW3vBPyiYdNth9kGHJZNbJ9hUzYTOtfHEZaJ0L69RaSciJTHJmfL7fg52O8AEWmCVUsBVMQ+IP8I\n1c+flU38Oe0X7nvsqgHCGnxF5DjgW1V9Avhv6PU/BPqG2iMy2oZqA/OAjiJSRUSSCWt3yMFqoI6I\n1FGQ6Y8AAAFsSURBVAs9zvw+Zien9wisCiljyu3LIzhfXs3HylpZrKPE+QG8hsuGZ2wXbjVwnYiM\nxaoAsluR7nLgabFG8W+Bv4Y9t0tEFmP1yxmr990FjBWRpVgPouw+SGYDQ0PVLQ9mem4w8IKI/APY\nnOk1c6Sqi0RkHPBFaNNzqro4l8P+E3q9lcBKrK0FVf0yVL5VWBvAp2HHjAHeF5EfVbVzDvuFewSY\nICIDsauQDBcCl4nNIrwJeEBVt4jIncAMESkB7AWuU9XPReQurBrrd2BJLmVDVXeJyF+BN0MfvPOB\nSGbyHQ88G2rkz6qDwV2hc/6GrZleN4JzRkxVN4rIA9h7uQX7/f4Rzddw2fPZah1gPVSwBtEmBThH\nKtZQviBKYTn3JyJSQW3lwJLA21jnhLfjHVdx4FVSzrlEc1foKnQZ8B223KyLAb/CcM45FxG/wnDO\nORcRTxjOOeci4gnDOedcRDxhOOeci4gnDOeccxH5f8TGdJbZYgtJAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0xaf80f60>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "tp = np.linspace(0.2, 0.7, 20)\n",
    "trainerror = np.zeros(shape = (tp.size, 1))\n",
    "vaderror = np.zeros(shape = (tp.size, 1))\n",
    "for i in range(0, tp.size):\n",
    "    datain = readdata('train.csv', tp[i], 0.7, features)\n",
    "    X = np.array(datain['train'].ix[:,1:])\n",
    "    y = np.array(datain['train'].ix[:,0])\n",
    "    Xvad = np.array(datain['validation'].ix[:,1:])\n",
    "    yvad = np.array(datain['validation'].ix[:,0])\n",
    "    \n",
    "    Xq = titan.xquad(X)\n",
    "    Xvadq = titan.xquad(Xvad)\n",
    "    m = X.shape[0]\n",
    "    Xfull = titan.normdata(np.concatenate((Xq, Xvadq)))\n",
    "    Xq = np.concatenate((np.ones(shape=(Xq.shape[0], 1)), Xfull[:m, :]), axis = 1)\n",
    "    Xvadq = np.concatenate((np.ones(shape=(Xvadq.shape[0], 1)), Xfull[m:, :]), axis = 1)\n",
    "\n",
    "    nfeature = Xq.shape[1]\n",
    "    theta0 = np.zeros(shape = (nfeature, 1))\n",
    "    regpara = 1.0\n",
    "    theta = scop.fmin_l_bfgs_b(titan.lfCost, theta0, titan.lfGradient, (Xq, y, regpara))[0]\n",
    "    trainerror[i] = titan.lfCost(theta, Xq, y, 0.0)\n",
    "    vaderror[i] = titan.lfCost(theta, Xvadq, yvad, 0.0)\n",
    "plt.plot(tp, trainerror, '-b', tp, vaderror,'-r')\n",
    "plt.xlabel('proportion of data used for training')\n",
    "plt.ylabel('error')\n",
    "plt.legend(['training','validation'])\n",
    "plt.grid()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The learning curve indicates the model can benefit from more training examples as the gap between validation error and training error reduces with more training sets. Additionally, the training set error is smaller than the linear model, showing adding quadratic terms improves the model. The remaining gap between training and validation errors could suggest the model suffers from high variance (overfitting). I now use the 60/40 split on training/validation sets to test the learning curve for the regularization parameter. When the validation set is less than 40% the validation error starts to behave unpredictably, possibly due to small dataset size."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYwAAAEOCAYAAACaQSCZAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XmcFNW5//HPw7AJCrIpCuhgRFlkdVgUF1DxghoBAUEl\nCeYqggLBaCLml7jFNeF61UQxqLhcF0RU5HqNoAmIGEEGRMIqKCADLrgiiMLA8/vj9DA948zQw0xN\nzfJ9v179orv6VPfTZdvfqTpV55i7IyIisj/V4i5AREQqBgWGiIikRIEhIiIpUWCIiEhKFBgiIpIS\nBYaIiKREgSEiIilRYIiISEoUGCIikhIFhoiIpKR63AWUpsaNG3t6enrcZYiIVBiLFy/+3N2bpNK2\nUgVGeno6mZmZcZchIlJhmNnGVNvqkJSIiKREgSEiIilRYIiISEoqVR+GiFQeu3fvJisri++//z7u\nUiqF2rVr07x5c2rUqHHAr6HAEJFyKSsri0MOOYT09HTMLO5yKjR354svviArK4uWLVse8OvokJSI\nlEvff/89jRo1UliUAjOjUaNGJd5bU2AAzJgB69fHXYWI5KOwKD2lsS0VGN99B8OGwTHHQMeOcOON\nsGQJaK5zkSrt66+/5oEHHij2eueccw5ff/11kW1uuOEGXn/99QMtLTYKjDp1YPlymDgR6teHW2+F\nE0+E9HQYNw7+8Q/YvTvuKkWkjBUWGNnZ2UWu98orr3DooYcW2eaWW27hrLPOKlF9cVBgABx7LFxz\nDcybB598AlOmQKdO8NBDcNZZcNhhMHw4TJ8O334bd7UiUgYmTJjABx98QKdOnejatSunnnoq559/\nPm3btgVgwIABnHjiibRr147JkyfvWy89PZ3PP/+cDRs20KZNGy6//HLatWvH2Wefzc6dOwEYMWIE\n06dP39f+xhtvpEuXLrRv357Vq1cDsHXrVvr06UO7du247LLLOProo/n888/LeCvkpbOk8mvSBC69\nNNx27IDXXgt9HC+/DE89BbVqwZlnwoAB8NOfQtOmcVcsUumNHw9Ll5bua3bqBPfcU/jzd955J8uX\nL2fp0qXMnTuXc889l+XLl+87y2jKlCk0bNiQnTt30rVrVwYNGkSjRo3yvMbatWt55plneOihh7jw\nwgt5/vnnGT58+I/eq3HjxixZsoQHHniAiRMn8vDDD3PzzTdzxhlncP311/Pqq6/yyCOPlOrnPxDa\nwyhK3bohGB57LOx5zJ0LV14Jq1bByJFw5JFw8slw112wZk3c1YpIhLp165bnlNT77ruPjh070qNH\nDzZt2sTatWt/tE7Lli3p1KkTACeeeCIbNmwo8LUvuOCCH7WZP38+w4YNA6Bv3740aNCgFD/NgYl0\nD8PM+gL3AmnAw+5+Z77newEvATmnKL3g7reksm6Zq14dTj893P7rv0K/x4wZ4TZhQrgdf3wImAED\noFs3qKY8FikNRe0JlJW6devuuz937lxef/113n77berUqUOvXr0KPGW1Vq1a++6npaXtOyRVWLu0\ntLT99pHEKbJfNDNLA+4H+gFtgYvMrG0BTd90906J2y3FXDceZtC+PfzhD7B4MWzcCH/5C7RoEcLk\npJOgWTO44gp45RXQlaoiFc4hhxzCt4X0WX7zzTc0aNCAOnXqsHr1ahYsWFDq79+zZ0+mTZsGwOzZ\ns/nqq69K/T2KK8o/gbsB69z9Q3ffBUwF+pfBumXvqKNgzJjQ3/HZZ6Gv49RT4emn4dxzQ7/IkCFh\neTn4jy4i+9eoUSN69uzJCSecwG9+85s8z/Xt25fs7GzatGnDhAkT6NGjR6m//4033sjs2bM54YQT\neO6552jatCmHHHJIqb9PcZhHdL2BmQ0G+rr7ZYnHPwO6u/uYpDa9gBeALGAzcK27r0hl3YJkZGR4\nuZoP44cf4J//DIetZs4M/SA5h7YGDID+/cNeiYj8yKpVq2jTpk3cZcTmhx9+IC0tjerVq/P2228z\nevRolpaw57+gbWpmi909I5X14z5LaglwlLtvN7NzgBlAq+K8gJmNBEYCHHXUUaVfYUnUqgX9+oXb\npEnwzjshPF56CcaODbcuXUJwDBgQDnPpylYRAT766CMuvPBC9u7dS82aNXnooYfiLinSQ1KbgeQ/\nn5snlu3j7tvcfXvi/itADTNrnMq6Sa8x2d0z3D2jSZOUZhmMR7Vq0KMH3HlnOMtq9epwdlWtWnDT\nTeEq82OOgauvhjfegHLc8SUi0WvVqhXvvvsu7733HosWLaJr165xlxRpYCwCWplZSzOrCQwDZiY3\nMLOmlhjgxMy6Jer5IpV1K7zjj4ff/hb+9S/YsgUmT4Z27cKeSK9ecPjhMGIEvPhiuB5ERCRmkR2S\ncvdsMxsDzCKcGjsl0T8xKvH8g8BgYLSZZQM7gWEeOlUKXDeqWmPXtClcfnm4ffstzJoVDlu99BI8\n/jjUrg19+oQLBY8/Ho44IqwTcweYiFQtkXV6x6HcdXqX1O7d8Oabuf0eH32U9/m6dUN45ARIzv38\njxs10jUhUuFU9U7vKFT0Tm8pSo0acMYZ4XbvveFq8k2b4OOPwxlXH3+ce3vvPXj11YLHuqpePRzi\nKixQch43bQo1a5b95xSRCkGBUVGYQevW4VaUHTsKDpScxxs3wsKFsHVrwUO4N2q0/z2WI46Agw/W\nGV0iSQ4++GC2b9/Oli1bGDdu3L7BBZP16tWLiRMnkpFR+B/099xzDyNHjqROnTpAGC796aef3u8I\nuGVBgVHZ1K0bRt899tii2+3eHS4yLCpc3n8/3N+168fr16mz/z2WI46Axo11OEyqlCOPPLLAsEjV\nPffcw/Dhw/cFxiuvvFJapZWYAqOqqlEjDF/SrFnR7dzD1en5wyT58bJloaN+27Yfr1+9Ohx3HPz1\nr9C7dzSfRSQCEyZMoEWLFlx11VUA3HTTTVSvXp05c+bw1VdfsXv3bm699Vb69887CMWGDRs477zz\nWL58OTt37uTSSy/lvffeo3Xr1nnGkho9ejSLFi1i586dDB48mJtvvpn77ruPLVu20Lt3bxo3bsyc\nOXNIT08nMzOTxo0bc/fddzNlyhQALrvsMsaPH8+GDRvo168fp5xyCv/6179o1qwZL730EgcddFCp\nbxMFhhTNDBo2DLd27Ypu+913eQMl5/706WFI+F//Gm67LVx7IlIcMYxvPnToUMaPH78vMKZNm8as\nWbMYN24c9erV4/PPP6dHjx6cf/75hU5/OmnSJOrUqcOqVatYtmwZXbp02ffcbbfdRsOGDdmzZw9n\nnnkmy5YtY9y4cdx9993MmTOHxo0b53mtxYsX8+ijj7Jw4ULcne7du3P66afToEGDlIdRLykFhpSe\nOnXCxYfHHJN3+e9+B7/5TRiY8bXX4Mknw1XtIuVY586d+eyzz9iyZQtbt26lQYMGNG3alKuvvpp5\n8+ZRrVo1Nm/ezKeffkrTQubFmTdvHuPGjQOgQ4cOdOjQYd9z06ZNY/LkyWRnZ/Pxxx+zcuXKPM/n\nN3/+fAYOHLhv1NwLLriAN998k/PPPz/lYdRLSoEh0atbFx54IAzE+MtfQkYG3HFH+KtR/RuSipjG\nNx8yZAjTp0/nk08+YejQoTz11FNs3bqVxYsXU6NGDdLT0wsc1nx/1q9fz8SJE1m0aBENGjRgxIgR\nB/Q6OVIdRr2k9H+rlJ1zzw3ziPTrF6bE7dMHsrLirkqkUEOHDmXq1KlMnz6dIUOG8M0333DYYYdR\no0YN5syZw8aNG4tc/7TTTuPpp58GYPny5SxbtgyAbdu2UbduXerXr8+nn37K3//+933rFDas+qmn\nnsqMGTP47rvv2LFjBy+++CKnnnpqKX7a/VNgSNlq0iQMd/Lww+H03vbt4dln465KpEDt2rXj22+/\npVmzZhxxxBFccsklZGZm0r59e5544gla7+c099GjR7N9+3batGnDDTfcwIknnghAx44d6dy5M61b\nt+biiy+mZ8+e+9YZOXIkffv2pXe+k0S6dOnCiBEj6NatG927d+eyyy6jc+fOpf+hi6ArvSU+69bB\nz34GCxbA8OHhTKr69eOuSsoJXeld+kp6pbf2MCQ+xx4bhj65+WZ45hno0CGM1Csi5ZICQ+JVvTrc\ncAO89VYYlqR3b7juujD5lIiUKwoMKR+6d4d33w0j9v7pT+Hxiso7QLFIRaTAkPLj4IPhb38L09lu\n2QInnhgGXdy7N+7KJCaVqY81bqWxLRUYUv789Kfw73+H027Hj4e+fWFzgRMuSiVWu3ZtvvjiC4VG\nKXB3vvjiC2rXrl2i19GFe1I+HX542NN46KEwbW379mFWwsGD465Mykjz5s3Jyspi69atcZdSKdSu\nXZvmzZuX6DUiPa3WzPoC9xJmzXvY3e8spF1X4G3CjHvTE8uuBi4DHPg3cKm7F3kppE6rraTefz+c\nfvvOO/Dzn8Nf/gL16sVdlUilUC5OqzWzNOB+oB/QFrjIzNoW0u4uYHbSsmbAOCDD3U8gBM6wqGqV\ncu6442D+/HA21ZNPQseO4XRcESlTUfZhdAPWufuH7r4LmAr0L6DdWOB54LN8y6sDB5lZdaAOsCXC\nWqW8q1EjXK8xfz6kpcHpp8P11xc8V4eIRCLKwGgGbEp6nJVYtk9iT2IgMCl5ubtvBiYCHwEfA9+4\n+2xETjopDHP9n/8Jd94JPXrAqlVxVyVSJcR9ltQ9wHXunue8STNrQNgbaQkcCdQ1swIHdzezkWaW\naWaZ6hyrIg4+OHSGz5gR5jjv0iUMK6KzaUQiFWVgbAZaJD1unliWLAOYamYbgMHAA2Y2ADgLWO/u\nW919N/ACcHJBb+Luk909w90zmjRpUtqfQcqz/v3D6bdnnAFjx8I554QJm0QkElEGxiKglZm1NLOa\nhE7rmckN3L2lu6e7ezowHbjS3WcQDkX1MLM6FqayOhPQcQf5saZN4eWXw3wbb7wRTr994YW4qxKp\nlCILDHfPBsYAswg/9tPcfYWZjTKzUftZdyEhQJYQTqmtBkyOqlap4Mxg9OgwtEh6OgwaBJdeWvAc\n4yJywDS8uVQuu3eHs6nuuAOOPhr+538gaa4BEcmrXFyHIRKLGjXg1lth3rzw+LTT4Pe/D0EiIiWi\nwJDKqWdPeO89+MUv4Lbb4OSTYc2auKsSqdAUGFJ5HXIITJkCzz8P69dD584waZJOvxU5QAoMqfwu\nuCCcfnvaaXDllXDeefDJJ3FXJVLhKDCkajjiCPj738PAhf/8Zzj9dsaMuKsSqVAUGFJ1mMGYMbBk\nCbRoAQMHwmWXwfbtcVcmUiEoMKTqadMGFiwIgxdOmQKdOsHbb8ddlUi5p8CQqqlmTbj99nB1eHY2\nnHIK/O532tsQKYICQ6q2U0+FZcvCxEx33AGtWsEjj8CePXFXJlLuKDBE6tWDRx+Ff/0rDC1y2WVh\nBNzXXou7MpFyRYEhkuOkk0JoPPssfPstnH12GAF3xYq4KxMpFxQYIsnM4MILw6RMf/5zCJAOHcLg\nhp/lnxRSpGpRYIgUpFYtuPZaWLcOrroKHn4Yjj029HPs3Bl3dSKxUGCIFKVxY7jvPli+HHr3DmdS\ntW4NTz0Fe/fuf32RSkSBIZKK44+Hl14KV4k3agTDh4f5xN98M+7KRMqMAkOkOHr3hsxMePxx2LIl\njE81aFA4dCVSyUUaGGbW18zWmNk6M5tQRLuuZpZtZoOTlh1qZtPNbLWZrTKzk6KsVSRl1aqF6zbe\nfx9uuQVmzYK2beHqq+HLL+OuTiQykQWGmaUB9wP9gLbARWbWtpB2dwGz8z11L/Cqu7cGOqI5vaW8\nqVMH/vAHWLs2zLtx332hY/y//xt27Yq7OpFSF+UeRjdgnbt/6O67gKlA/wLajQWeB/ads2hm9YHT\ngEcA3H2Xu38dYa0iB+6II+Chh8Kc4hkZ8OtfQ7t28MILmntDKpUoA6MZsCnpcVZi2T5m1gwYCEzK\nt25LYCvwqJm9a2YPm1ndCGsVKbkOHcLhqVdeCWNVDRoEp58OixbFXZlIqYi70/se4Dp3z39+YnWg\nCzDJ3TsDO4AC+0DMbKSZZZpZ5tatW6OtVmR/zKBfvzA97IMPhmlhu3WDSy6Bjz6KuzqREokyMDYD\nLZIeN08sS5YBTDWzDcBg4AEzG0DYG8ly94WJdtMJAfIj7j7Z3TPcPaNJkyalWb/IgateHa64IvRv\nXH99ODx13HHhOo5t2+KuTuSARBkYi4BWZtbSzGoCw4CZyQ3cvaW7p7t7OiEUrnT3Ge7+CbDJzI5P\nND0TWBlhrSLRqFcvDKO+Zg0MHpw7Iu6DD4Zh1UUqkMgCw92zgTHALMIZTtPcfYWZjTKzUSm8xFjg\nKTNbBnQCbo+qVpHIHXUUPPkkvPNOuAhw9Gjo2DH0d6hjXCoI80r0Zc3IyPDMzMy4yxApmnuYT/y3\nvw0X/PXpAxMnhk5zkTJmZovdPSOVtnF3eotUPWZhPvEVK8I1G5mZ0LlzmIfj44/jrk6kUAoMkbjU\nrAnjx4e9jF/9Cp54IvRv3HIL7NgRd3UiP6LAEIlbw4Zw992wciX07Qs33hj6OR5/XCPiSrmiwBAp\nL449FqZPDyPgHnkkjBgRrhyfMyfuykQABYZI+XPKKbBgQZhz44sv4Iwz4Pzzw6m5IjFSYIiUR9Wq\nwcUXw+rV4dqNuXPhhBNg7Fj4/PO4q5MqSoEhUp4ddBBMmBA6xi+/HCZNCoeu/vxn+OGHuKuTKkaB\nIVIRHHYYPPAALFsGPXuGazhat4Znn9WFf1JmdOGeSEX0+utwzTUhQNLSoEGDcGvYMPX7DRuGPRip\n0opz4V71qIsRkQicdRYsWQLTpsHy5fDVV2G2v6++Ch3la9eG+19/XfSpubVqHVjQNGgANWqU3eeV\nckGBIVJRpaXBRRcV3Wbv3jA67ldf5Q2V5PvJy7Ky4N//Dvf3N6pu3bqpB02jRtCmTZilUCosBYZI\nZVatGhx6aLi1bFm8dbOzwx5KYeGS/37OXs2XX8LOnT9+vbS0MF5W9+7Qo0f497jjQo1SISgwRKRg\n1atD48bhVlw//JA3VD77LBxCW7AAnn46DO8OIci6dcsbIo0ale7nkFKjTm8RKVt794brSxYuDAGy\ncGE4DJbT13LssXkDpGPHMO6WRKI4nd4KDBGJ3/btsHhxboAsWJA7cm+tWtClS94QOfroMOqvlJgC\nQ0QqNvfQAZ8TIAsXhmHgv/8+PH/44XkDpGtXOOSQeGuuoMrNabVm1he4F0gDHnb3Owtp1xV4Gxjm\n7tOTlqcBmcBmdz8vylpFpBwxgxYtwm3IkLBs9+5w6Cp5L2TmzNz27dqF8GjZMpzyW7167i35cXGe\nq149dNanpYXO+eR/U1lWrVql2hOKLDASP/b3A32ALGCRmc1095UFtLsLmF3Ay/yKML1rvajqFJEK\nokaNcGiqSxe48sqw7Msvw7S3OQHywguhk708MSte6BQnkHKWNWoURjqOWJR7GN2Ade7+IYCZTQX6\nAyvztRsLPA90TV5oZs2Bc4HbgF9HWKeIVFQNG4Y5RPr2DY/dw55Idna4FXZ/f4+T7+/ZEzrkk/9N\ndVlx2x/oa+zeXSabO8rAaAZsSnqcBXRPbmBmzYCBQG/yBQZwD/BbQAcmRSQ1ZuGMKp1VFYm4r5i5\nB7jO3fOMXWBm5wGfufvi/b2AmY00s0wzy9y6dWtUdYqIlFvZ2WXzPlEGxmagRdLj5ollyTKAqWa2\nARgMPGBmA4CewPmJ5VOBM8zsyYLexN0nu3uGu2c0adKklD+CiEj59eWXMG4c9O5dNoMWRxkYi4BW\nZtbSzGoCw4CZyQ3cvaW7p7t7OjAduNLdZ7j79e7ePLF8GPBPdx8eYa0iIhVGdjbcfz+0ahX+bd8+\n94zjKEUWGO6eDYwBZhHOdJrm7ivMbJSZjYrqfUVEKrPXXoNOnWDMmPDv0qVhqpSyGKk+0usw3P0V\n4JV8yx4spO2IQpbPBeaWcmkiIhXKunVhCpSZM+GYY+DFF6F//7K9zGO/exhmlmZmV5dFMSIikte2\nbWGCxbZt4Z//hDvvhJUrYcCAsr8mcL+B4e57gP0Mui8iIqVpzx545JHQTzFxIgwfHkaQv+66MLxW\nHFI9JPWWmf0VeBbYkbPQ3ZdEUpWISBX25pvwq1/Bu++GKdz/7/8gI6XRnqKVamB0Svx7S9IyB84o\n3XJERKqujRvD4adp06B5c3jmGRg6tPwMR5VSYLh776gLERGpqnbsgLvugj//OYTDTTfBb35T/ma0\nTSkwzKw+cCNwWmLRG8At7v5NVIWJiFR27mECwuuug82bwxTtd90VBuktj1K9DmMK8C1wYeK2DXg0\nqqJERCq7d96Bk08OndlNm8L8+SE8ymtYQOp9GD9x90FJj282s6VRFCQiUplt2QLXXw9PPBGCYsoU\n+MUvwmjl5V2qJe40s1NyHphZT2BnNCWJiFQ+338Pt98Oxx0HU6fChAnw/vtw6aUVIywg9T2MUcAT\nib4MgK+AX0RTkohI5eEOzz8fOrE3bICBA0Pn9k9+EndlxbffwDCzasDx7t7RzOoBuPu2yCsTEang\nli6F8ePhjTfCAIH/+AecUYEvRkjlSu+9hImMcPdtCgsRkaJ99hlccUWYTXb58jA44JIlFTssIPU+\njNfN7Foza2FmDXNukVYmIlLB7NoFd98dhvOYMiVcrb12LYweDdUjHeq1bKT6EYYm/r0qaZkDx5Ru\nOSIiFY97GL7jmmtCR3a/fiE4WreOu7LSlWofxnB3f6sM6hERqVBWrYKrr4ZZs+D440NwnHNO3FVF\nI9U+jL+WQS0iIhXGl1+GQ07t28OCBWGPYtmyyhsWkHofxj/MbJBZ8YbAMrO+ZrbGzNaZ2YQi2nU1\ns2wzG5x43MLM5pjZSjNbYWa/Ks77iohEJTs7dGK3agV//Stcfnnop7j6aqhZM+7qopVqYFwBTAN+\nMLNtZvatmRV5tpSZpQH3A/2AtsBFZta2kHZ3AbOTFmcD17h7W6AHcFVB64qIlKXXX4fOneGqq6Bj\nxzD8+KRJ0KRJ3JWVjVQDoz4wArjV3esB7YA++1mnG7DO3T90913AVKB/Ae3GAs8Dn+UscPePc+ba\ncPdvCXOCN0uxVhGRUrVuXZjhrk+fMLLs88+Hayo6dIi7srKVamDcT/hLP2fmvW/Zf79GM2BT0uMs\n8v3om1kzYCAwqbAXMbN0oDOwMMVaRURKxbZtYSTZdu3C3sXtt4fpUS+4oPzMUVGWUj2ttru7dzGz\ndwHc/SszK42jdfcA17n73oK6R8zsYMLex/jCLhg0s5HASICjjjqqFEoSkaruhx/g4Yfhj3+ETz+F\nESNCWBxxRNyVxSvVwNid6GtwADNrAuzdzzqbgeSBepsnliXLAKYmwqIxcI6ZZbv7DDOrQQiLp9z9\nhcLexN0nA5MBMjIyPMXPIyLyI7t3w6OPwq23wqZNcMop8L//C127xl1Z+ZDqIan7gBeBw8zsNmA+\ncPt+1lkEtDKzlom9kWHAzOQG7t7S3dPdPR2YDlyZCAsDHgFWufvdqX8cEZHiy86Gxx4L11FccQUc\neSTMng3z5ikskqU6RetTZrYYOBMwYIC7r9rPOtlmNgaYBaQBU9x9hZmNSjz/YBGr9wR+Bvw7ad6N\n37n7K6nUKyKSij174Nln4eabwxXaXbrAX/4SrqWoin0U+5Py6CbuvhpYXZwXT/zAv5JvWYFB4e4j\nku7PJwSTiEip27sXXngBbrwxdGK3bx8eDxigoChKBZm2Q0Sk5NzhpZfCnsSQISE4nn02DEM+cKDC\nYn8UGCJS6bnD3/8O3bqFvYgdO+B//icMPX7hhRVnxru4aTOJSKXlHi6w69kz9Ets3QqPPBIGDBw+\nHNLS4q6wYlFgiEil9Oab0Ls3nHUWfPRRGMLj/ffhl7+sHHNTxEGBISKVyoIFcPbZcNppsGYN3Hdf\nGNpj1KjKPzhg1JSzIlLh7d0b5qO4555w/UTjxjBxYpjprk6duKurPBQYIlJhbdsGjz8erp1YuzYM\n3XHHHTBmDBx8cNzVVT4KDBGpcNauDXNRPPoofPst9OgRLr4bNEiHnaKkwBCRCmHvXnjttdAn8cor\nUKMGDB0KY8eG02UlegoMESnXtm+HJ54Ih51Wr4bDD4ebbgpjPjVtGnd1VYsCQ0TKpQ8+gPvvD9dN\nbNsGGRnhYrshQ6BWrbirq5oUGCJSbuRcaHffffDyy+HCuiFDYNw46N5dQ3fETYEhIrHLGarjL38J\ngwE2aQK//324duLII+OuTnIoMEQkNhs2hLOdHnkEvv46DAr4+ONhfKfateOuTvJTYIhImXKHuXPD\nYaeZM8NhpkGDwmGnk0/WYafyTIEhImXiu+/gqadCUCxfDo0awYQJ4Wrs5s3jrk5SEelYUmbW18zW\nmNk6M5tQRLuuZpZtZoOLu66IlF/u8N578NvfhlAYOTJ0ZE+ZEubMvu02hUVFEtkehpmlAfcDfYAs\nYJGZzXT3lQW0uwuYXdx1RaT8cQ97ENOmhdv774eQGDgwHHY65RQddqqoojwk1Q1Y5+4fApjZVKA/\nkP9HfyzwPND1ANYVkXJixYrckFi9OkxK1Ls3XHNNCIsmTeKuUEoqysBoBmxKepwFdE9uYGbNgIFA\nb/IGxn7XFZH4rVqVGxIrV4Y9h1694Fe/ggsugMMOi7tCKU1xd3rfA1zn7nvtAPdRzWwkMBLgqKOO\nKsXSRKQga9bkhsTy5SEkTjstXJV9wQUarqMyizIwNgMtkh43TyxLlgFMTYRFY+AcM8tOcV0A3H0y\nMBkgIyPDS6VyEclj7Vp47rkQEu+9F0LilFPChXaDBoVhxaXyizIwFgGtzKwl4cd+GHBxcgN3b5lz\n38weA1529xlmVn1/64pItD74IDck3n03LDv55DBJ0eDB0KxZvPVJ2YssMNw928zGALOANGCKu68w\ns1GJ5x8s7rpR1Soiwfr1uSGxeHFY1qMH3H13CIkWLYpeXyo3c688R3EyMjI8MzMz7jJEKpSNG3ND\nYtGisKxbtzA8x+DBcPTR8dYn0TKzxe6ekUrbuDu9RSQGmzblhsTChWFZRgb86U9hdNj09FjLk3JK\ngSFSRWRkc13OAAAShUlEQVRlwfTpISTefjss69IF7rwzhMQxx8Rbn5R/CgyRSmzLltyQeOutsKxT\nJ7j99hASxx4bb31SsSgwRCqZTz7JDYn588NQHR06wK23hpA47ri4K5SKSoEhUgl8+im88EIIiTfe\nCCFxwglw880hJFq3jrtCqQwUGCIV1PbtMHUqPPNMmF9i715o0wZuuCGc4dS2bdwVSmWjwBCpYFas\ngEmTwpSm27aFQ0z/7/+FkGjXTiPBSnQUGCIVwK5d4ZDTpEkwbx7UrBkCYvRoOOkkhYSUDQWGSDm2\ncSP87W9hzuvPPgunvv7pT3DppdC4cdzVSVWjwBApZ/bsgVmzwt7E//1f2Hs477ywN3H22WGeCZE4\nKDBEyonPPgtTl/7tb7BhAxx+eOibuPxy0Mj9Uh4oMERi5B4uqJs0KVw7sWtXmIDorrtgwIDQVyFS\nXigwRGKwbRs8+WQIiuXLoV49GDUq3Nq0ibs6kYIpMETK0LJlISSefDJcR9GlCzz0EFx0EdStG3d1\nIkVTYIhE7Pvvw+GmSZPgX/+C2rVh2LDQid21q06JlYpDgSESkQ8+CB3Yjz4Kn38OrVrBf/0XjBgB\nDRvGXZ1I8SkwRErRnj3hVNhJk+DVVyEtDfr3D3sTZ5yhU2KlYov062tmfc1sjZmtM7MJBTzf38yW\nmdlSM8s0s1OSnrvazFaY2XIze8bMakdZq0hJfPJJGA22ZcsQEMuWwY03hgvvnn8ezjpLYSEVX2R7\nGGaWBtwP9AGygEVmNtPdVyY1+wcw093dzDoA04DWZtYMGAe0dfedZjYNGAY8FlW9IsXlHkaGnTQp\nDNuRnR2C4Z574Kc/hRo14q5QpHRFeUiqG7DO3T8EMLOpQH9gX2C4+/ak9nWB5AnGqwMHmdluoA6w\nJcJaRVL29dfwxBPw4IOwahU0aADjxsEVV2iuCancogyMZsCmpMdZQPf8jcxsIHAHcBhwLoC7bzaz\nicBHwE5gtrvPLuhNzGwkMBLgKF0OKxFasiTsTTz9NHz3HXTrFjq0hw6Fgw6KuzqR6MV+VNXdX3T3\n1sAA4I8AZtaAsDfSEjgSqGtmwwtZf7K7Z7h7RpMmTcqqbKkidu6Exx6D7t3hxBNDWFx8MSxeDAsX\nhjOeFBZSVUS5h7EZaJH0uHliWYHcfZ6ZHWNmjYHewHp33wpgZi8AJwNPRlivyD5r14ZDTo8+Cl99\nFWasu/de+PnP4dBD465OJB5RBsYioJWZtSQExTDg4uQGZnYs8EGi07sLUAv4gnAoqoeZ1SEckjoT\nyIywVhGys2HmzHDY6fXXoXp1uOCCcErs6afrAjuRyALD3bPNbAwwC0gDprj7CjMblXj+QWAQ8PNE\nx/ZOYKi7O7DQzKYDS4Bs4F1gclS1StXlDkuXwnPPweOPw5Yt0KIF/PGPcNll0LRp3BWKlB8Wfp8r\nh4yMDM/M1I6IFM0d3n03hMRzz4UrstPSwlwTo0bBOeeEvQuRqsDMFrt7Ript9b+FVAnu4SynnJD4\n8MMQEmeeCRMmhKHENYOdSNEUGFJpuYezmXJCYv36sOdw5pnwu9+FkGjUKO4qRSoOBYZUKu6QmZkb\nEhs2hJA46yz4/e/DsB0KCZEDo8CQCs8dFi0KATF9em5I9OkDN9wQQkKjw4qUnAJDKiR3eOed3JDY\nuDGM3dSnTxj0r3//MGSHiJQeBYZUGO7h6uqckPjooxASZ58NN98M55+vkBCJkgJDyrW9e/OGxKZN\nULNmCIk//jGEhK68FikbCgwpd/buhQULckMiKyuExH/8B9x2WwiJ+vXjrlKk6lFgSLmwd2+Y7/q5\n58KEQ5s3h5Do2xfuuCPML6GQEImXAkNis3cvvPVWbkhs2QK1aoWQuOuuEBL16sVdpYjkUGBImdqz\nJ29IfPxxCIl+/WDIEDjvPIWESHmlwJDI7dkD8+fnhsQnn0Dt2nlD4pBD4q5SRPZHgSGRWL8e5s4N\nt9mzc0PinHNCSJx7rkJCpKJRYEip2LAhNyDmzg0X0kEY0O+MM8K8EueeCwcfHF+NIlIyCgw5IBs3\n5obDnDl5A+L00+E3v4FevaBtW008JFJZKDAkJR99lBsOc+eGPQoIA/n16gXXXpsbENVinyleRKIQ\naWCYWV/gXsKMew+7+535nu8P/BHYS5hZb7y7z088dyjwMHAC4MAv3f3tKOuVXJs25YbD3LmhTwJC\nQJx+Ovz61yEg2rVTQIhUFZEFhpmlAfcDfYAsYJGZzXT3lUnN/gHMTMzp3QGYBrROPHcv8Kq7Dzaz\nmkCdqGqVEBDJfRAffhiWN2wYAmL8eOjdWwEhUpVFuYfRDVjn7h8CmNlUoD+wLzDcfXtS+7qEPQnM\nrD5wGjAi0W4XsCvCWqucrKy8AfHBB2F5gwYhIMaNCwFxwgkKCBEJogyMZsCmpMdZQPf8jcxsIHAH\ncBhwbmJxS2Ar8KiZdQQWA79y9x0R1lupbd6cNyDWrQvLDz00BMTYseEQU/v2CggRKVjsnd7u/iLw\nopmdRujPOItQVxdgrLsvNLN7gQnAH/Kvb2YjgZEARx11VJnVXd5t2ZL3LKb8AXHVVbkBkZYWY6Ei\nUmFEGRibgRZJj5snlhXI3eeZ2TFm1piwN5Ll7gsTT08nBEZB600GJgNkZGR4aRReEW3ZAm+8kdtR\nvXZtWF6/fgiIK68MAdGhgwJCRA5MlIGxCGhlZi0JQTEMuDi5gZkdC3yQ6PTuAtQCvkg83mRmx7v7\nGuBMkvo+JIzBlHyI6f33w/L69eG002DUqBAQHTsqIESkdEQWGO6ebWZjgFmE02qnuPsKMxuVeP5B\nYBDwczPbDewEhrp7zl7CWOCpxBlSHwKXRlVreeceTmtduDDsRcydC2vWhOfq1QsBMXJk6KRWQIhI\nVCz397niy8jI8MzMzLjLKLFPP4VFi8Kc1YsWhdsXX4Tn6tWDU08N4dCrF3TqpIAQkQNnZovdPSOV\ntrF3eld127ZBZmZuMLzzTrgmAsLZSu3aQf/+0K0bdO0a+iCq67+aiMRAPz1l6Pvv4b338u49rFkT\nDjkB/OQn0LNnCIZu3aBzZ6hbN96aRURyKDAismcPrFyZu+ewaBEsWwa7d4fnmzYNwXDJJeHfjIww\n7IaISHmlwCgFOZ3SyXsOS5bAjsRlhvXqhVC45prcvYdmzTSKq4hULAqMA/DJJ3n3HJI7pWvVCoeS\nfvnL3H6HVq109bSIVHwKjP345htYvDjv3kP+TukBA0IwdO0arpyuUSPemkVEoqDASPL997B0ad49\nh9Wrc5/P6ZTO2XNQp7SIVCVVPjB27Qojs+Z0Smdnh+VNm4ZgUKe0iEhQ5QOjZk2YNw+OPDLMGpez\n96BOaRGRvKp8YACsWKFwEBHZH527g8JCRCQVCgwREUmJAkNERFKiwBARkZQoMEREJCUKDBERSYkC\nQ0REUqLAEBGRlFSqKVrNbCvwNfBNvqfqp7CsMfB5dNUVWUsU66fSrqg2hT2XyraMa9sW9N5RrVuS\n7VuSbVvQssr23U2lrb67B75u/rZHu3uTlNZ090p1AyYfyDIgM676olg/lXZFtSnsuRS3ZSzbtqTb\ntzjrlmT7lmTbxrl9y+q7m0pbfXej27ZF3SrjIan/LcGyslDS9011/VTaFdWmsOdS2ZZxbduSvndx\n1i3J9i3Jtk31vaNQVt/dVNrqu3vg6x7w+1SqQ1IlYWaZ7p4Rdx2VkbZttLR9o6Ntm1dl3MM4UJPj\nLqAS07aNlrZvdLRtk2gPQ0REUqI9DBERSYkCQ0REUqLAEBGRlCgwUmBmA8zsITN71szOjrueysTM\njjGzR8xsety1VAZmVtfMHk98Xy+Ju57Kpqp/Xyt9YJjZFDP7zMyW51ve18zWmNk6M5tQ1Gu4+wx3\nvxwYBQyNst6KpJS27Yfu/p/RVlqxFXM7XwBMT3xfzy/zYiug4mzfqv59rfSBATwG9E1eYGZpwP1A\nP6AtcJGZtTWz9mb2cr7bYUmr/j6xngSPUXrbVgr3GCluZ6A5sCnRbE8Z1liRPUbq27dKqx53AVFz\n93lmlp5vcTdgnbt/CGBmU4H+7n4HcF7+1zAzA+4E/u7uS6KtuOIojW0r+1ec7QxkEUJjKVXjD8IS\nK+b2XVm21ZUvVfUL1Yzcv8Ig/E/WrIj2Y4GzgMFmNirKwiqBYm1bM2tkZg8Cnc3s+qiLq0QK284v\nAIPMbBLxDnNR0RW4fav697XS72GUBne/D7gv7joqI3f/gtA3JKXA3XcAl8ZdR2VV1b+vVXUPYzPQ\nIulx88QyKTlt27Kh7Rwtbd8CVNXAWAS0MrOWZlYTGAbMjLmmykLbtmxoO0dL27cAlT4wzOwZ4G3g\neDPLMrP/dPdsYAwwC1gFTHP3FXHWWRFp25YNbedoafumToMPiohISir9HoaIiJQOBYaIiKREgSEi\nIilRYIiISEoUGCIikhIFhoiIpESBIeWKmW0/gHVeMbNDD2C98WZWp6SvU1GYWbqZXRzD+24ws8Zl\n/b5S+hQYcsAsiO07lPP+7n6Ou399AC8xHtgXGCV4nVJjZlGO75YOFCswIq5HKhgFhhRL4q/UNWb2\nBLAcaGFmZ5vZ22a2xMyeM7ODE23PMbPVZrbYzO4zs5cTy28ys2uTXnN5/uGlzexgM/tH4jX/bWb9\ni3j/DWbW2MxGmdnSxG29mc1JrDPJzDLNbIWZ3ZxYNg44EpiT1G7fX8Jm9utEXcvNbHzSe6+yMJvd\nCjObbWYHFbCNHjOzBxPv+b6ZnZe0/puJz7TEzE5OLO+VWD6TxPDZZjYjsd1WmNnIpNfebmZ/Tix/\n3cy6mdlcM/vQzM5PtElLtFlkZsvM7IrE6ncCpya2z9WFtSuonqT3H2Vmf056PMLM/lpUzfm+O8uT\nHl9rZjcl7v/EzF5NrP+mmbXOv76UA+6um24p3wh/pe4FeiQeNwbmAXUTj68DbgBqE4aHbplY/gzw\ncuL+TcC1Sa+5HEhP3N+e+Lc6UC/pPdYBlv/9E89vABonPa4BvAn8NPG4YeLfNGAu0KGQ9TYk3utE\n4N9AXeBgYAXQOfHe2UCnRPtpwPACttFjwKuEP8haEYbGrk3Ym6mdaNMKyEzc7wXsyNlW+Wo+KLF9\nGiUeO9Avcf9FYHbi83YEliaWjwR+n7hfC8gEWibe5+Wk9yiqXZ56ktZpQpgnIufx34FT9lNzznZN\nB5YnrXstcFPi/j+AVon73YF/xv1d1+3HN+1uyoHY6O4LEvd7EGYke8vMAGoSxuVpDXzo7usT7Z4h\n/EClyoDbzew0QkA0Aw4v4P0Lci/hBydnPogLE3/xVgeOSNS7rIj1TwFe9DBUOGb2AnAqYfC59e6+\nNNFuMeFHsCDT3H0vsNbMPiRsj/XAX82sE2E2vOOS2r+TtK0AxpnZwMT9FoSA+QLYRQgjCKH2g7vv\nNrN/J9VyNtDBzAYnHtdPrL8rX41FtctfDwDuvjWxN9MDWJv4XG/tp+YiJfZITwaeS3yHIASYlDMK\nDDkQO5LuG/Cau1+U3CDxo1iYbPIeDq1dQJtLCH/Nnpj4QdyQ1G5HAe1z3ncEcDRh4DjMrCXhL9mu\n7v6VmT1WyPul6oek+3sIf00XJP8gbQ5cDXxK2BuoBnyf9Py+z2RmvQgTdp3k7t+Z2dykmnd74s9w\nQpD+AODuey23v8GAse4+K7mAxOvmWVREu0K3MTAVuBBYTQhW30/NOQr7714N+Nrdi/rOSDmgPgwp\nqQVATzM7FsDM6prZccAa4JikvomhSetsALok2nchHAbJrz7wWSIsehNCoEhmdiIhHIYn/roHqEf4\n8fvGzA4nzNGc41vgkAJe6k1ggJnVMbO6wMDEsuIYYmbVzOwnwDGE7VEf+DhR288Ih8gKUh/4KvHD\n25qwF1ccs4DRZlYDwMyOS3yO/J+3sHb78yJhutKLCOGRas2fAodZmLWuFokpe919G7DezIYk6jAz\n61i8jyxlQXsYUiKJQxQjgGcSPwIQjou/b2ZXAq+a2Q7C/AI5ngd+bmYrgIXA+wW89FPA/yYOtWQS\n/prdnzFAQ0JHNoQ+gsvM7N3E+pvIPXwCMDlR3xZ37530mZYk9kTeSSx62N3ftR/P+1yUjxLr1wNG\nufv3ZvYA8LyZ/ZxwWKmwv+JfBUaZ2SpC0BR1+K0gDxMOTy2xsCG2AgMIh+H2mNl7hH6WewtpV6TE\nntoqoK2752yj/dacCP9bCNtlM3n/m14CTDKz3xP6ZKYC7xXvY0vUNLy5RMbMDnb37Ykfo/uBte7+\n33HXFbVE2Lzs7tPjrkWkNOmQlETpcjNbSjjLqD7wt5jrEZES0B6GiIikRHsYIiKSEgWGiIikRIEh\nIiIpUWCIiEhKFBgiIpISBYaIiKTk/wOfnIiKwweIDgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0xd4a5128>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "datain = readdata('train.csv', 0.6, 0.6, features)\n",
    "X = np.array(datain['train'].ix[:,1:])\n",
    "y = np.array(datain['train'].ix[:,0])\n",
    "Xvad = np.array(datain['validation'].ix[:,1:])\n",
    "yvad = np.array(datain['validation'].ix[:,0])\n",
    "    \n",
    "Xq = titan.xquad(X)\n",
    "Xvadq = titan.xquad(Xvad)\n",
    "m = X.shape[0]\n",
    "Xfull = titan.normdata(np.concatenate((Xq, Xvadq)))\n",
    "Xq = np.concatenate((np.ones(shape=(Xq.shape[0], 1)), Xfull[:m, :]), axis = 1)\n",
    "Xvadq = np.concatenate((np.ones(shape=(Xvadq.shape[0], 1)), Xfull[m:, :]), axis = 1)\n",
    "\n",
    "theta0 = np.zeros(shape = (Xq.shape[1], 1))  \n",
    "regpara = np.array([0, 0.001, 0.003, 0.01, 0.03, 0.1, 0.3, 1, 3, 6, 10, 30])\n",
    "trainerror = np.zeros(shape = (regpara.size, 1))\n",
    "vaderror = np.zeros(shape = (regpara.size, 1))\n",
    "\n",
    "for i in range(0, regpara.size):\n",
    "    theta = scop.fmin_l_bfgs_b(titan.lfCost, theta0, titan.lfGradient, (Xq, y, regpara[i]))[0]\n",
    "    trainerror[i] = titan.lfCost(theta, Xq, y, 0.0)\n",
    "    vaderror[i] = titan.lfCost(theta, Xvadq, yvad, 0.0)\n",
    "plt.semilogx(regpara, trainerror, '-b', regpara, vaderror,'-r')\n",
    "plt.xlabel('regularization parameter value')\n",
    "plt.ylabel('error')\n",
    "plt.legend(['training','validation'])\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The regularization learning curve suggests the model needs a relatively large regularization parameter (3.0), i.e., the model could be suffering from high variance with a small regularization parameter like 1.0. This agrees with results from the learning curve on training set size. \n",
    "\n",
    "Below I set up the model with the optimum parameters (70% training set and regularization parameter is 3.0) and the resulting accuracy is improved from the initial model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.83173076923076927"
      ]
     },
     "execution_count": 58,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "datain = readdata('train.csv', 0.7, 0.7, features)\n",
    "X = np.array(datain['train'].ix[:,1:])\n",
    "y = np.array(datain['train'].ix[:,0])\n",
    "Xvad = np.array(datain['validation'].ix[:,1:])\n",
    "yvad = np.array(datain['validation'].ix[:,0])\n",
    "    \n",
    "Xq = titan.xquad(X)\n",
    "Xvadq = titan.xquad(Xvad)\n",
    "m = X.shape[0]\n",
    "Xfull = titan.normdata(np.concatenate((Xq, Xvadq)))\n",
    "Xq = np.concatenate((np.ones(shape=(Xq.shape[0], 1)), Xfull[:m, :]), axis = 1)\n",
    "Xvadq = np.concatenate((np.ones(shape=(Xvadq.shape[0], 1)), Xfull[m:, :]), axis = 1)\n",
    "\n",
    "theta0 = np.zeros(shape = (Xq.shape[1], 1))  \n",
    "\n",
    "w = scop.fmin_l_bfgs_b(titan.lfCost, theta0, titan.lfGradient, (Xq, y, 3.0))\n",
    "titan.getacu(w[0], Xq, y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.84701492537313428"
      ]
     },
     "execution_count": 59,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "\"\"\"vadliation accuracy\"\"\"\n",
    "titan.getacu(w[0], Xvadq, yvad) "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Finally, to predict the results for the given test set I use all training set data with a regularization parameter of 3.0 to obtain the model parameters. Then I use those model parameters to estimate the outcome for the test set passengers."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 67,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "features = ['Survived', 'Pclass', 'Age', 'Sex', 'SibSp', 'Parch', 'Fare','Embarked','Name']\n",
    "datain = readdata('train.csv', 1.0, 1.0, features)\n",
    "X = np.array(datain['train'].ix[:,1:])\n",
    "y = np.array(datain['train'].ix[:,0])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>PassengerId</th>\n",
       "      <th>Survived</th>\n",
       "      <th>Pclass</th>\n",
       "      <th>Age</th>\n",
       "      <th>Sex</th>\n",
       "      <th>SibSp</th>\n",
       "      <th>Parch</th>\n",
       "      <th>Fare</th>\n",
       "      <th>Embarked</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>count</th>\n",
       "      <td>418.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>418.000000</td>\n",
       "      <td>332.000000</td>\n",
       "      <td>418.000000</td>\n",
       "      <td>418.000000</td>\n",
       "      <td>418.000000</td>\n",
       "      <td>417.000000</td>\n",
       "      <td>418.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>1100.500000</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2.265550</td>\n",
       "      <td>30.272590</td>\n",
       "      <td>-0.272727</td>\n",
       "      <td>0.447368</td>\n",
       "      <td>0.392344</td>\n",
       "      <td>35.627188</td>\n",
       "      <td>1.464115</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>120.810458</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.841838</td>\n",
       "      <td>14.181209</td>\n",
       "      <td>0.963244</td>\n",
       "      <td>0.896760</td>\n",
       "      <td>0.981429</td>\n",
       "      <td>55.907576</td>\n",
       "      <td>0.685516</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>892.000000</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.170000</td>\n",
       "      <td>-1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>996.250000</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>21.000000</td>\n",
       "      <td>-1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>7.895800</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>1100.500000</td>\n",
       "      <td>NaN</td>\n",
       "      <td>3.000000</td>\n",
       "      <td>27.000000</td>\n",
       "      <td>-1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>14.454200</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>1204.750000</td>\n",
       "      <td>NaN</td>\n",
       "      <td>3.000000</td>\n",
       "      <td>39.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>31.500000</td>\n",
       "      <td>2.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>1309.000000</td>\n",
       "      <td>NaN</td>\n",
       "      <td>3.000000</td>\n",
       "      <td>76.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>8.000000</td>\n",
       "      <td>9.000000</td>\n",
       "      <td>512.329200</td>\n",
       "      <td>3.000000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       PassengerId  Survived      Pclass         Age         Sex       SibSp  \\\n",
       "count   418.000000       0.0  418.000000  332.000000  418.000000  418.000000   \n",
       "mean   1100.500000       NaN    2.265550   30.272590   -0.272727    0.447368   \n",
       "std     120.810458       NaN    0.841838   14.181209    0.963244    0.896760   \n",
       "min     892.000000       NaN    1.000000    0.170000   -1.000000    0.000000   \n",
       "25%     996.250000       NaN    1.000000   21.000000   -1.000000    0.000000   \n",
       "50%    1100.500000       NaN    3.000000   27.000000   -1.000000    0.000000   \n",
       "75%    1204.750000       NaN    3.000000   39.000000    1.000000    1.000000   \n",
       "max    1309.000000       NaN    3.000000   76.000000    1.000000    8.000000   \n",
       "\n",
       "            Parch        Fare    Embarked  \n",
       "count  418.000000  417.000000  418.000000  \n",
       "mean     0.392344   35.627188    1.464115  \n",
       "std      0.981429   55.907576    0.685516  \n",
       "min      0.000000    0.000000    1.000000  \n",
       "25%      0.000000    7.895800    1.000000  \n",
       "50%      0.000000   14.454200    1.000000  \n",
       "75%      0.000000   31.500000    2.000000  \n",
       "max      9.000000  512.329200    3.000000  "
      ]
     },
     "execution_count": 68,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "features = ['PassengerId', 'Survived', 'Pclass', 'Age', 'Sex', 'SibSp', 'Parch', 'Fare','Embarked','Name']\n",
    "testin = titan.readdata('test.csv', 1.0, 1.0, features)\n",
    "testin['train'].describe()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 69,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "PassengerId    0.000000\n",
       "Survived            inf\n",
       "Pclass         0.000000\n",
       "Age            0.259036\n",
       "Sex            0.000000\n",
       "SibSp          0.000000\n",
       "Parch          0.000000\n",
       "Fare           0.002398\n",
       "Embarked       0.000000\n",
       "Name           0.000000\n",
       "dtype: float64"
      ]
     },
     "execution_count": 69,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "testin['train'].isnull().sum() * 1.0 / testin['train'].count()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Unfortunately there is a small fraction of passengers in the test database missing Fare information and just like in the training set a large portion of passengers missing Age information. First I use the mapping for age from training set to fill in the missing age in the test set."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 70,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "medage = datain['train'].groupby('title')['Age'].median()\n",
    "\n",
    "tmap = {' Mr':0, ' Mrs':1, ' Miss':2, ' Mme':1, ' Master':3}\n",
    "testtitle = testin['train'].Name.apply(titan.name2title).map(tmap).fillna(4).astype(int)\n",
    "ind = testin['train'].Age.isnull()\n",
    "testin['train'].loc[ind, 'Age'] = testtitle[ind].map(medage)\n",
    "testin['train'] = testin['train'].assign(title = testtitle)\n",
    "del testin['train']['Name']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 71,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>PassengerId</th>\n",
       "      <th>Survived</th>\n",
       "      <th>Pclass</th>\n",
       "      <th>Age</th>\n",
       "      <th>Sex</th>\n",
       "      <th>SibSp</th>\n",
       "      <th>Parch</th>\n",
       "      <th>Fare</th>\n",
       "      <th>Embarked</th>\n",
       "      <th>title</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>PassengerId</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>NaN</td>\n",
       "      <td>-0.026751</td>\n",
       "      <td>-0.059113</td>\n",
       "      <td>-0.023245</td>\n",
       "      <td>0.003818</td>\n",
       "      <td>0.043080</td>\n",
       "      <td>0.008211</td>\n",
       "      <td>-0.049863</td>\n",
       "      <td>0.015789</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Survived</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Pclass</th>\n",
       "      <td>-0.026751</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>-0.461438</td>\n",
       "      <td>-0.108615</td>\n",
       "      <td>0.001087</td>\n",
       "      <td>0.018721</td>\n",
       "      <td>-0.577147</td>\n",
       "      <td>0.031096</td>\n",
       "      <td>0.011875</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Age</th>\n",
       "      <td>-0.059113</td>\n",
       "      <td>NaN</td>\n",
       "      <td>-0.461438</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000537</td>\n",
       "      <td>-0.094570</td>\n",
       "      <td>-0.038589</td>\n",
       "      <td>0.332474</td>\n",
       "      <td>0.050405</td>\n",
       "      <td>-0.307685</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Sex</th>\n",
       "      <td>-0.023245</td>\n",
       "      <td>NaN</td>\n",
       "      <td>-0.108615</td>\n",
       "      <td>0.000537</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.099943</td>\n",
       "      <td>0.159120</td>\n",
       "      <td>0.191514</td>\n",
       "      <td>0.126779</td>\n",
       "      <td>0.580651</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>SibSp</th>\n",
       "      <td>0.003818</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.001087</td>\n",
       "      <td>-0.094570</td>\n",
       "      <td>0.099943</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.306895</td>\n",
       "      <td>0.171539</td>\n",
       "      <td>-0.100603</td>\n",
       "      <td>0.252831</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Parch</th>\n",
       "      <td>0.043080</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.018721</td>\n",
       "      <td>-0.038589</td>\n",
       "      <td>0.159120</td>\n",
       "      <td>0.306895</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.230046</td>\n",
       "      <td>-0.125164</td>\n",
       "      <td>0.218084</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Fare</th>\n",
       "      <td>0.008211</td>\n",
       "      <td>NaN</td>\n",
       "      <td>-0.577147</td>\n",
       "      <td>0.332474</td>\n",
       "      <td>0.191514</td>\n",
       "      <td>0.171539</td>\n",
       "      <td>0.230046</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.053588</td>\n",
       "      <td>0.150595</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Embarked</th>\n",
       "      <td>-0.049863</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.031096</td>\n",
       "      <td>0.050405</td>\n",
       "      <td>0.126779</td>\n",
       "      <td>-0.100603</td>\n",
       "      <td>-0.125164</td>\n",
       "      <td>0.053588</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.122205</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>title</th>\n",
       "      <td>0.015789</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.011875</td>\n",
       "      <td>-0.307685</td>\n",
       "      <td>0.580651</td>\n",
       "      <td>0.252831</td>\n",
       "      <td>0.218084</td>\n",
       "      <td>0.150595</td>\n",
       "      <td>0.122205</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "             PassengerId  Survived    Pclass       Age       Sex     SibSp  \\\n",
       "PassengerId     1.000000       NaN -0.026751 -0.059113 -0.023245  0.003818   \n",
       "Survived             NaN       NaN       NaN       NaN       NaN       NaN   \n",
       "Pclass         -0.026751       NaN  1.000000 -0.461438 -0.108615  0.001087   \n",
       "Age            -0.059113       NaN -0.461438  1.000000  0.000537 -0.094570   \n",
       "Sex            -0.023245       NaN -0.108615  0.000537  1.000000  0.099943   \n",
       "SibSp           0.003818       NaN  0.001087 -0.094570  0.099943  1.000000   \n",
       "Parch           0.043080       NaN  0.018721 -0.038589  0.159120  0.306895   \n",
       "Fare            0.008211       NaN -0.577147  0.332474  0.191514  0.171539   \n",
       "Embarked       -0.049863       NaN  0.031096  0.050405  0.126779 -0.100603   \n",
       "title           0.015789       NaN  0.011875 -0.307685  0.580651  0.252831   \n",
       "\n",
       "                Parch      Fare  Embarked     title  \n",
       "PassengerId  0.043080  0.008211 -0.049863  0.015789  \n",
       "Survived          NaN       NaN       NaN       NaN  \n",
       "Pclass       0.018721 -0.577147  0.031096  0.011875  \n",
       "Age         -0.038589  0.332474  0.050405 -0.307685  \n",
       "Sex          0.159120  0.191514  0.126779  0.580651  \n",
       "SibSp        0.306895  0.171539 -0.100603  0.252831  \n",
       "Parch        1.000000  0.230046 -0.125164  0.218084  \n",
       "Fare         0.230046  1.000000  0.053588  0.150595  \n",
       "Embarked    -0.125164  0.053588  1.000000  0.122205  \n",
       "title        0.218084  0.150595  0.122205  1.000000  "
      ]
     },
     "execution_count": 71,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "testin['train'].corr()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It seems the Fare correlates with Pclass very well, like in the training set example. As a result, we use a strategy similar to filling missing Age information. Here we use Pclass as the classifier and fill missing Fare with the median Fare from the corresponding Pclass category."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 72,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "medfare = datain['train'].groupby('Pclass')['Fare'].median()\n",
    "ind = testin['train'].Fare.isnull()\n",
    "testin['train'].loc[ind, 'Fare'] = testin['train'].Pclass[ind].map(medage)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 78,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "Xtest = np.array(testin['train'].ix[:,2:])\n",
    "\n",
    "Xq = titan.xquad(X)\n",
    "Xtestq = titan.xquad(Xtest)\n",
    "m = Xq.shape[0]\n",
    "Xfull = titan.normdata(np.concatenate((Xq, Xtestq)))\n",
    "Xq = np.concatenate((np.ones(shape=(Xq.shape[0], 1)), Xfull[:m, :]), axis = 1)\n",
    "Xtestq = np.concatenate((np.ones(shape=(Xtestq.shape[0], 1)), Xfull[m:, :]), axis = 1)\n",
    "\n",
    "theta0 = np.zeros(shape = (Xq.shape[1], 1))  \n",
    "w = scop.fmin_l_bfgs_b(titan.lfCost, theta0, titan.lfGradient, (Xq, y, 3.0))\n",
    "\n",
    "ytest = titan.sigmoid(np.dot(Xtestq, w[0]))\n",
    "ytest[ytest > 0.5] = 1\n",
    "ytest[ytest < 0.5] = 0\n",
    "\n",
    "testin['train'].Survived = ytest.astype(int)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 79,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>PassengerId</th>\n",
       "      <th>Survived</th>\n",
       "      <th>Pclass</th>\n",
       "      <th>Age</th>\n",
       "      <th>Sex</th>\n",
       "      <th>SibSp</th>\n",
       "      <th>Parch</th>\n",
       "      <th>Fare</th>\n",
       "      <th>Embarked</th>\n",
       "      <th>title</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>PassengerId</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.001085</td>\n",
       "      <td>-0.026751</td>\n",
       "      <td>-0.059113</td>\n",
       "      <td>-0.023245</td>\n",
       "      <td>0.003818</td>\n",
       "      <td>0.043080</td>\n",
       "      <td>0.008851</td>\n",
       "      <td>-0.049863</td>\n",
       "      <td>0.015789</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Survived</th>\n",
       "      <td>0.001085</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>-0.174442</td>\n",
       "      <td>-0.114955</td>\n",
       "      <td>0.820616</td>\n",
       "      <td>0.031786</td>\n",
       "      <td>0.112895</td>\n",
       "      <td>0.266820</td>\n",
       "      <td>0.145265</td>\n",
       "      <td>0.712244</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Pclass</th>\n",
       "      <td>-0.026751</td>\n",
       "      <td>-0.174442</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>-0.461438</td>\n",
       "      <td>-0.108615</td>\n",
       "      <td>0.001087</td>\n",
       "      <td>0.018721</td>\n",
       "      <td>-0.577594</td>\n",
       "      <td>0.031096</td>\n",
       "      <td>0.011875</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Age</th>\n",
       "      <td>-0.059113</td>\n",
       "      <td>-0.114955</td>\n",
       "      <td>-0.461438</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000537</td>\n",
       "      <td>-0.094570</td>\n",
       "      <td>-0.038589</td>\n",
       "      <td>0.326883</td>\n",
       "      <td>0.050405</td>\n",
       "      <td>-0.307685</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Sex</th>\n",
       "      <td>-0.023245</td>\n",
       "      <td>0.820616</td>\n",
       "      <td>-0.108615</td>\n",
       "      <td>0.000537</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.099943</td>\n",
       "      <td>0.159120</td>\n",
       "      <td>0.192348</td>\n",
       "      <td>0.126779</td>\n",
       "      <td>0.580651</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>SibSp</th>\n",
       "      <td>0.003818</td>\n",
       "      <td>0.031786</td>\n",
       "      <td>0.001087</td>\n",
       "      <td>-0.094570</td>\n",
       "      <td>0.099943</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.306895</td>\n",
       "      <td>0.172108</td>\n",
       "      <td>-0.100603</td>\n",
       "      <td>0.252831</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Parch</th>\n",
       "      <td>0.043080</td>\n",
       "      <td>0.112895</td>\n",
       "      <td>0.018721</td>\n",
       "      <td>-0.038589</td>\n",
       "      <td>0.159120</td>\n",
       "      <td>0.306895</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.230462</td>\n",
       "      <td>-0.125164</td>\n",
       "      <td>0.218084</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Fare</th>\n",
       "      <td>0.008851</td>\n",
       "      <td>0.266820</td>\n",
       "      <td>-0.577594</td>\n",
       "      <td>0.326883</td>\n",
       "      <td>0.192348</td>\n",
       "      <td>0.172108</td>\n",
       "      <td>0.230462</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.054471</td>\n",
       "      <td>0.151458</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Embarked</th>\n",
       "      <td>-0.049863</td>\n",
       "      <td>0.145265</td>\n",
       "      <td>0.031096</td>\n",
       "      <td>0.050405</td>\n",
       "      <td>0.126779</td>\n",
       "      <td>-0.100603</td>\n",
       "      <td>-0.125164</td>\n",
       "      <td>0.054471</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.122205</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>title</th>\n",
       "      <td>0.015789</td>\n",
       "      <td>0.712244</td>\n",
       "      <td>0.011875</td>\n",
       "      <td>-0.307685</td>\n",
       "      <td>0.580651</td>\n",
       "      <td>0.252831</td>\n",
       "      <td>0.218084</td>\n",
       "      <td>0.151458</td>\n",
       "      <td>0.122205</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "             PassengerId  Survived    Pclass       Age       Sex     SibSp  \\\n",
       "PassengerId     1.000000  0.001085 -0.026751 -0.059113 -0.023245  0.003818   \n",
       "Survived        0.001085  1.000000 -0.174442 -0.114955  0.820616  0.031786   \n",
       "Pclass         -0.026751 -0.174442  1.000000 -0.461438 -0.108615  0.001087   \n",
       "Age            -0.059113 -0.114955 -0.461438  1.000000  0.000537 -0.094570   \n",
       "Sex            -0.023245  0.820616 -0.108615  0.000537  1.000000  0.099943   \n",
       "SibSp           0.003818  0.031786  0.001087 -0.094570  0.099943  1.000000   \n",
       "Parch           0.043080  0.112895  0.018721 -0.038589  0.159120  0.306895   \n",
       "Fare            0.008851  0.266820 -0.577594  0.326883  0.192348  0.172108   \n",
       "Embarked       -0.049863  0.145265  0.031096  0.050405  0.126779 -0.100603   \n",
       "title           0.015789  0.712244  0.011875 -0.307685  0.580651  0.252831   \n",
       "\n",
       "                Parch      Fare  Embarked     title  \n",
       "PassengerId  0.043080  0.008851 -0.049863  0.015789  \n",
       "Survived     0.112895  0.266820  0.145265  0.712244  \n",
       "Pclass       0.018721 -0.577594  0.031096  0.011875  \n",
       "Age         -0.038589  0.326883  0.050405 -0.307685  \n",
       "Sex          0.159120  0.192348  0.126779  0.580651  \n",
       "SibSp        0.306895  0.172108 -0.100603  0.252831  \n",
       "Parch        1.000000  0.230462 -0.125164  0.218084  \n",
       "Fare         0.230462  1.000000  0.054471  0.151458  \n",
       "Embarked    -0.125164  0.054471  1.000000  0.122205  \n",
       "title        0.218084  0.151458  0.122205  1.000000  "
      ]
     },
     "execution_count": 79,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "testin['train'].corr()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 80,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "testin['train'].to_csv('titanic_v1.csv', columns = ['PassengerId', 'Survived'], index = False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}