Lesson_3.html

<!DOCTYPE html>
<html>
<head><meta charset="utf-8" />
<title>Lesson_3</title><script src="https://cdnjs.cloudflare.com/ajax/libs/require.js/2.1.10/require.min.js"></script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/jquery/2.0.3/jquery.min.js"></script>

<style type="text/css">
    /*!
*
* Twitter Bootstrap
*
*/
/*!
 * Bootstrap v3.3.7 (http://getbootstrap.com)
 * Copyright 2011-2016 Twitter, Inc.
 * Licensed under MIT (https://github.com/twbs/bootstrap/blob/master/LICENSE)
 */
/*! normalize.css v3.0.3 | MIT License | github.com/necolas/normalize.css */
html {
  font-family: sans-serif;
  -ms-text-size-adjust: 100%;
  -webkit-text-size-adjust: 100%;
}
body {
  margin: 0;
}
article,
aside,
details,
figcaption,
figure,
footer,
header,
hgroup,
main,
menu,
nav,
section,
summary {
  display: block;
}
audio,
canvas,
progress,
video {
  display: inline-block;
  vertical-align: baseline;
}
audio:not([controls]) {
  display: none;
  height: 0;
}
[hidden],
template {
  display: none;
}
a {
  background-color: transparent;
}
a:active,
a:hover {
  outline: 0;
}
abbr[title] {
  border-bottom: 1px dotted;
}
b,
strong {
  font-weight: bold;
}
dfn {
  font-style: italic;
}
h1 {
  font-size: 2em;
  margin: 0.67em 0;
}
mark {
  background: #ff0;
  color: #000;
}
small {
  font-size: 80%;
}
sub,
sup {
  font-size: 75%;
  line-height: 0;
  position: relative;
  vertical-align: baseline;
}
sup {
  top: -0.5em;
}
sub {
  bottom: -0.25em;
}
img {
  border: 0;
}
svg:not(:root) {
  overflow: hidden;
}
figure {
  margin: 1em 40px;
}
hr {
  box-sizing: content-box;
  height: 0;
}
pre {
  overflow: auto;
}
code,
kbd,
pre,
samp {
  font-family: monospace, monospace;
  font-size: 1em;
}
button,
input,
optgroup,
select,
textarea {
  color: inherit;
  font: inherit;
  margin: 0;
}
button {
  overflow: visible;
}
button,
select {
  text-transform: none;
}
button,
html input[type="button"],
input[type="reset"],
input[type="submit"] {
  -webkit-appearance: button;
  cursor: pointer;
}
button[disabled],
html input[disabled] {
  cursor: default;
}
button::-moz-focus-inner,
input::-moz-focus-inner {
  border: 0;
  padding: 0;
}
input {
  line-height: normal;
}
input[type="checkbox"],
input[type="radio"] {
  box-sizing: border-box;
  padding: 0;
}
input[type="number"]::-webkit-inner-spin-button,
input[type="number"]::-webkit-outer-spin-button {
  height: auto;
}
input[type="search"] {
  -webkit-appearance: textfield;
  box-sizing: content-box;
}
input[type="search"]::-webkit-search-cancel-button,
input[type="search"]::-webkit-search-decoration {
  -webkit-appearance: none;
}
fieldset {
  border: 1px solid #c0c0c0;
  margin: 0 2px;
  padding: 0.35em 0.625em 0.75em;
}
legend {
  border: 0;
  padding: 0;
}
textarea {
  overflow: auto;
}
optgroup {
  font-weight: bold;
}
table {
  border-collapse: collapse;
  border-spacing: 0;
}
td,
th {
  padding: 0;
}
/*! Source: https://github.com/h5bp/html5-boilerplate/blob/master/src/css/main.css */
@media print {
  *,
  *:before,
  *:after {
    background: transparent !important;
    color: #000 !important;
    box-shadow: none !important;
    text-shadow: none !important;
  }
  a,
  a:visited {
    text-decoration: underline;
  }
  a[href]:after {
    content: " (" attr(href) ")";
  }
  abbr[title]:after {
    content: " (" attr(title) ")";
  }
  a[href^="#"]:after,
  a[href^="javascript:"]:after {
    content: "";
  }
  pre,
  blockquote {
    border: 1px solid #999;
    page-break-inside: avoid;
  }
  thead {
    display: table-header-group;
  }
  tr,
  img {
    page-break-inside: avoid;
  }
  img {
    max-width: 100% !important;
  }
  p,
  h2,
  h3 {
    orphans: 3;
    widows: 3;
  }
  h2,
  h3 {
    page-break-after: avoid;
  }
  .navbar {
    display: none;
  }
  .btn > .caret,
  .dropup > .btn > .caret {
    border-top-color: #000 !important;
  }
  .label {
    border: 1px solid #000;
  }
  .table {
    border-collapse: collapse !important;
  }
  .table td,
  .table th {
    background-color: #fff !important;
  }
  .table-bordered th,
  .table-bordered td {
    border: 1px solid #ddd !important;
  }
}
@font-face {
  font-family: 'Glyphicons Halflings';
  src: url('../components/bootstrap/fonts/glyphicons-halflings-regular.eot');
  src: url('../components/bootstrap/fonts/glyphicons-halflings-regular.eot?#iefix') format('embedded-opentype'), url('../components/bootstrap/fonts/glyphicons-halflings-regular.woff2') format('woff2'), url('../components/bootstrap/fonts/glyphicons-halflings-regular.woff') format('woff'), url('../components/bootstrap/fonts/glyphicons-halflings-regular.ttf') format('truetype'), url('../components/bootstrap/fonts/glyphicons-halflings-regular.svg#glyphicons_halflingsregular') format('svg');
}
.glyphicon {
  position: relative;
  top: 1px;
  display: inline-block;
  font-family: 'Glyphicons Halflings';
  font-style: normal;
  font-weight: normal;
  line-height: 1;
  -webkit-font-smoothing: antialiased;
  -moz-osx-font-smoothing: grayscale;
}
.glyphicon-asterisk:before {
  content: "\002a";
}
.glyphicon-plus:before {
  content: "\002b";
}
.glyphicon-euro:before,
.glyphicon-eur:before {
  content: "\20ac";
}
.glyphicon-minus:before {
  content: "\2212";
}
.glyphicon-cloud:before {
  content: "\2601";
}
.glyphicon-envelope:before {
  content: "\2709";
}
.glyphicon-pencil:before {
  content: "\270f";
}
.glyphicon-glass:before {
  content: "\e001";
}
.glyphicon-music:before {
  content: "\e002";
}
.glyphicon-search:before {
  content: "\e003";
}
.glyphicon-heart:before {
  content: "\e005";
}
.glyphicon-star:before {
  content: "\e006";
}
.glyphicon-star-empty:before {
  content: "\e007";
}
.glyphicon-user:before {
  content: "\e008";
}
.glyphicon-film:before {
  content: "\e009";
}
.glyphicon-th-large:before {
  content: "\e010";
}
.glyphicon-th:before {
  content: "\e011";
}
.glyphicon-th-list:before {
  content: "\e012";
}
.glyphicon-ok:before {
  content: "\e013";
}
.glyphicon-remove:before {
  content: "\e014";
}
.glyphicon-zoom-in:before {
  content: "\e015";
}
.glyphicon-zoom-out:before {
  content: "\e016";
}
.glyphicon-off:before {
  content: "\e017";
}
.glyphicon-signal:before {
  content: "\e018";
}
.glyphicon-cog:before {
  content: "\e019";
}
.glyphicon-trash:before {
  content: "\e020";
}
.glyphicon-home:before {
  content: "\e021";
}
.glyphicon-file:before {
  content: "\e022";
}
.glyphicon-time:before {
  content: "\e023";
}
.glyphicon-road:before {
  content: "\e024";
}
.glyphicon-download-alt:before {
  content: "\e025";
}
.glyphicon-download:before {
  content: "\e026";
}
.glyphicon-upload:before {
  content: "\e027";
}
.glyphicon-inbox:before {
  content: "\e028";
}
.glyphicon-play-circle:before {
  content: "\e029";
}
.glyphicon-repeat:before {
  content: "\e030";
}
.glyphicon-refresh:before {
  content: "\e031";
}
.glyphicon-list-alt:before {
  content: "\e032";
}
.glyphicon-lock:before {
  content: "\e033";
}
.glyphicon-flag:before {
  content: "\e034";
}
.glyphicon-headphones:before {
  content: "\e035";
}
.glyphicon-volume-off:before {
  content: "\e036";
}
.glyphicon-volume-down:before {
  content: "\e037";
}
.glyphicon-volume-up:before {
  content: "\e038";
}
.glyphicon-qrcode:before {
  content: "\e039";
}
.glyphicon-barcode:before {
  content: "\e040";
}
.glyphicon-tag:before {
  content: "\e041";
}
.glyphicon-tags:before {
  content: "\e042";
}
.glyphicon-book:before {
  content: "\e043";
}
.glyphicon-bookmark:before {
  content: "\e044";
}
.glyphicon-print:before {
  content: "\e045";
}
.glyphicon-camera:before {
  content: "\e046";
}
.glyphicon-font:before {
  content: "\e047";
}
.glyphicon-bold:before {
  content: "\e048";
}
.glyphicon-italic:before {
  content: "\e049";
}
.glyphicon-text-height:before {
  content: "\e050";
}
.glyphicon-text-width:before {
  content: "\e051";
}
.glyphicon-align-left:before {
  content: "\e052";
}
.glyphicon-align-center:before {
  content: "\e053";
}
.glyphicon-align-right:before {
  content: "\e054";
}
.glyphicon-align-justify:before {
  content: "\e055";
}
.glyphicon-list:before {
  content: "\e056";
}
.glyphicon-indent-left:before {
  content: "\e057";
}
.glyphicon-indent-right:before {
  content: "\e058";
}
.glyphicon-facetime-video:before {
  content: "\e059";
}
.glyphicon-picture:before {
  content: "\e060";
}
.glyphicon-map-marker:before {
  content: "\e062";
}
.glyphicon-adjust:before {
  content: "\e063";
}
.glyphicon-tint:before {
  content: "\e064";
}
.glyphicon-edit:before {
  content: "\e065";
}
.glyphicon-share:before {
  content: "\e066";
}
.glyphicon-check:before {
  content: "\e067";
}
.glyphicon-move:before {
  content: "\e068";
}
.glyphicon-step-backward:before {
  content: "\e069";
}
.glyphicon-fast-backward:before {
  content: "\e070";
}
.glyphicon-backward:before {
  content: "\e071";
}
.glyphicon-play:before {
  content: "\e072";
}
.glyphicon-pause:before {
  content: "\e073";
}
.glyphicon-stop:before {
  content: "\e074";
}
.glyphicon-forward:before {
  content: "\e075";
}
.glyphicon-fast-forward:before {
  content: "\e076";
}
.glyphicon-step-forward:before {
  content: "\e077";
}
.glyphicon-eject:before {
  content: "\e078";
}
.glyphicon-chevron-left:before {
  content: "\e079";
}
.glyphicon-chevron-right:before {
  content: "\e080";
}
.glyphicon-plus-sign:before {
  content: "\e081";
}
.glyphicon-minus-sign:before {
  content: "\e082";
}
.glyphicon-remove-sign:before {
  content: "\e083";
}
.glyphicon-ok-sign:before {
  content: "\e084";
}
.glyphicon-question-sign:before {
  content: "\e085";
}
.glyphicon-info-sign:before {
  content: "\e086";
}
.glyphicon-screenshot:before {
  content: "\e087";
}
.glyphicon-remove-circle:before {
  content: "\e088";
}
.glyphicon-ok-circle:before {
  content: "\e089";
}
.glyphicon-ban-circle:before {
  content: "\e090";
}
.glyphicon-arrow-left:before {
  content: "\e091";
}
.glyphicon-arrow-right:before {
  content: "\e092";
}
.glyphicon-arrow-up:before {
  content: "\e093";
}
.glyphicon-arrow-down:before {
  content: "\e094";
}
.glyphicon-share-alt:before {
  content: "\e095";
}
.glyphicon-resize-full:before {
  content: "\e096";
}
.glyphicon-resize-small:before {
  content: "\e097";
}
.glyphicon-exclamation-sign:before {
  content: "\e101";
}
.glyphicon-gift:before {
  content: "\e102";
}
.glyphicon-leaf:before {
  content: "\e103";
}
.glyphicon-fire:before {
  content: "\e104";
}
.glyphicon-eye-open:before {
  content: "\e105";
}
.glyphicon-eye-close:before {
  content: "\e106";
}
.glyphicon-warning-sign:before {
  content: "\e107";
}
.glyphicon-plane:before {
  content: "\e108";
}
.glyphicon-calendar:before {
  content: "\e109";
}
.glyphicon-random:before {
  content: "\e110";
}
.glyphicon-comment:before {
  content: "\e111";
}
.glyphicon-magnet:before {
  content: "\e112";
}
.glyphicon-chevron-up:before {
  content: "\e113";
}
.glyphicon-chevron-down:before {
  content: "\e114";
}
.glyphicon-retweet:before {
  content: "\e115";
}
.glyphicon-shopping-cart:before {
  content: "\e116";
}
.glyphicon-folder-close:before {
  content: "\e117";
}
.glyphicon-folder-open:before {
  content: "\e118";
}
.glyphicon-resize-vertical:before {
  content: "\e119";
}
.glyphicon-resize-horizontal:before {
  content: "\e120";
}
.glyphicon-hdd:before {
  content: "\e121";
}
.glyphicon-bullhorn:before {
  content: "\e122";
}
.glyphicon-bell:before {
  content: "\e123";
}
.glyphicon-certificate:before {
  content: "\e124";
}
.glyphicon-thumbs-up:before {
  content: "\e125";
}
.glyphicon-thumbs-down:before {
  content: "\e126";
}
.glyphicon-hand-right:before {
  content: "\e127";
}
.glyphicon-hand-left:before {
  content: "\e128";
}
.glyphicon-hand-up:before {
  content: "\e129";
}
.glyphicon-hand-down:before {
  content: "\e130";
}
.glyphicon-circle-arrow-right:before {
  content: "\e131";
}
.glyphicon-circle-arrow-left:before {
  content: "\e132";
}
.glyphicon-circle-arrow-up:before {
  content: "\e133";
}
.glyphicon-circle-arrow-down:before {
  content: "\e134";
}
.glyphicon-globe:before {
  content: "\e135";
}
.glyphicon-wrench:before {
  content: "\e136";
}
.glyphicon-tasks:before {
  content: "\e137";
}
.glyphicon-filter:before {
  content: "\e138";
}
.glyphicon-briefcase:before {
  content: "\e139";
}
.glyphicon-fullscreen:before {
  content: "\e140";
}
.glyphicon-dashboard:before {
  content: "\e141";
}
.glyphicon-paperclip:before {
  content: "\e142";
}
.glyphicon-heart-empty:before {
  content: "\e143";
}
.glyphicon-link:before {
  content: "\e144";
}
.glyphicon-phone:before {
  content: "\e145";
}
.glyphicon-pushpin:before {
  content: "\e146";
}
.glyphicon-usd:before {
  content: "\e148";
}
.glyphicon-gbp:before {
  content: "\e149";
}
.glyphicon-sort:before {
  content: "\e150";
}
.glyphicon-sort-by-alphabet:before {
  content: "\e151";
}
.glyphicon-sort-by-alphabet-alt:before {
  content: "\e152";
}
.glyphicon-sort-by-order:before {
  content: "\e153";
}
.glyphicon-sort-by-order-alt:before {
  content: "\e154";
}
.glyphicon-sort-by-attributes:before {
  content: "\e155";
}
.glyphicon-sort-by-attributes-alt:before {
  content: "\e156";
}
.glyphicon-unchecked:before {
  content: "\e157";
}
.glyphicon-expand:before {
  content: "\e158";
}
.glyphicon-collapse-down:before {
  content: "\e159";
}
.glyphicon-collapse-up:before {
  content: "\e160";
}
.glyphicon-log-in:before {
  content: "\e161";
}
.glyphicon-flash:before {
  content: "\e162";
}
.glyphicon-log-out:before {
  content: "\e163";
}
.glyphicon-new-window:before {
  content: "\e164";
}
.glyphicon-record:before {
  content: "\e165";
}
.glyphicon-save:before {
  content: "\e166";
}
.glyphicon-open:before {
  content: "\e167";
}
.glyphicon-saved:before {
  content: "\e168";
}
.glyphicon-import:before {
  content: "\e169";
}
.glyphicon-export:before {
  content: "\e170";
}
.glyphicon-send:before {
  content: "\e171";
}
.glyphicon-floppy-disk:before {
  content: "\e172";
}
.glyphicon-floppy-saved:before {
  content: "\e173";
}
.glyphicon-floppy-remove:before {
  content: "\e174";
}
.glyphicon-floppy-save:before {
  content: "\e175";
}
.glyphicon-floppy-open:before {
  content: "\e176";
}
.glyphicon-credit-card:before {
  content: "\e177";
}
.glyphicon-transfer:before {
  content: "\e178";
}
.glyphicon-cutlery:before {
  content: "\e179";
}
.glyphicon-header:before {
  content: "\e180";
}
.glyphicon-compressed:before {
  content: "\e181";
}
.glyphicon-earphone:before {
  content: "\e182";
}
.glyphicon-phone-alt:before {
  content: "\e183";
}
.glyphicon-tower:before {
  content: "\e184";
}
.glyphicon-stats:before {
  content: "\e185";
}
.glyphicon-sd-video:before {
  content: "\e186";
}
.glyphicon-hd-video:before {
  content: "\e187";
}
.glyphicon-subtitles:before {
  content: "\e188";
}
.glyphicon-sound-stereo:before {
  content: "\e189";
}
.glyphicon-sound-dolby:before {
  content: "\e190";
}
.glyphicon-sound-5-1:before {
  content: "\e191";
}
.glyphicon-sound-6-1:before {
  content: "\e192";
}
.glyphicon-sound-7-1:before {
  content: "\e193";
}
.glyphicon-copyright-mark:before {
  content: "\e194";
}
.glyphicon-registration-mark:before {
  content: "\e195";
}
.glyphicon-cloud-download:before {
  content: "\e197";
}
.glyphicon-cloud-upload:before {
  content: "\e198";
}
.glyphicon-tree-conifer:before {
  content: "\e199";
}
.glyphicon-tree-deciduous:before {
  content: "\e200";
}
.glyphicon-cd:before {
  content: "\e201";
}
.glyphicon-save-file:before {
  content: "\e202";
}
.glyphicon-open-file:before {
  content: "\e203";
}
.glyphicon-level-up:before {
  content: "\e204";
}
.glyphicon-copy:before {
  content: "\e205";
}
.glyphicon-paste:before {
  content: "\e206";
}
.glyphicon-alert:before {
  content: "\e209";
}
.glyphicon-equalizer:before {
  content: "\e210";
}
.glyphicon-king:before {
  content: "\e211";
}
.glyphicon-queen:before {
  content: "\e212";
}
.glyphicon-pawn:before {
  content: "\e213";
}
.glyphicon-bishop:before {
  content: "\e214";
}
.glyphicon-knight:before {
  content: "\e215";
}
.glyphicon-baby-formula:before {
  content: "\e216";
}
.glyphicon-tent:before {
  content: "\26fa";
}
.glyphicon-blackboard:before {
  content: "\e218";
}
.glyphicon-bed:before {
  content: "\e219";
}
.glyphicon-apple:before {
  content: "\f8ff";
}
.glyphicon-erase:before {
  content: "\e221";
}
.glyphicon-hourglass:before {
  content: "\231b";
}
.glyphicon-lamp:before {
  content: "\e223";
}
.glyphicon-duplicate:before {
  content: "\e224";
}
.glyphicon-piggy-bank:before {
  content: "\e225";
}
.glyphicon-scissors:before {
  content: "\e226";
}
.glyphicon-bitcoin:before {
  content: "\e227";
}
.glyphicon-btc:before {
  content: "\e227";
}
.glyphicon-xbt:before {
  content: "\e227";
}
.glyphicon-yen:before {
  content: "\00a5";
}
.glyphicon-jpy:before {
  content: "\00a5";
}
.glyphicon-ruble:before {
  content: "\20bd";
}
.glyphicon-rub:before {
  content: "\20bd";
}
.glyphicon-scale:before {
  content: "\e230";
}
.glyphicon-ice-lolly:before {
  content: "\e231";
}
.glyphicon-ice-lolly-tasted:before {
  content: "\e232";
}
.glyphicon-education:before {
  content: "\e233";
}
.glyphicon-option-horizontal:before {
  content: "\e234";
}
.glyphicon-option-vertical:before {
  content: "\e235";
}
.glyphicon-menu-hamburger:before {
  content: "\e236";
}
.glyphicon-modal-window:before {
  content: "\e237";
}
.glyphicon-oil:before {
  content: "\e238";
}
.glyphicon-grain:before {
  content: "\e239";
}
.glyphicon-sunglasses:before {
  content: "\e240";
}
.glyphicon-text-size:before {
  content: "\e241";
}
.glyphicon-text-color:before {
  content: "\e242";
}
.glyphicon-text-background:before {
  content: "\e243";
}
.glyphicon-object-align-top:before {
  content: "\e244";
}
.glyphicon-object-align-bottom:before {
  content: "\e245";
}
.glyphicon-object-align-horizontal:before {
  content: "\e246";
}
.glyphicon-object-align-left:before {
  content: "\e247";
}
.glyphicon-object-align-vertical:before {
  content: "\e248";
}
.glyphicon-object-align-right:before {
  content: "\e249";
}
.glyphicon-triangle-right:before {
  content: "\e250";
}
.glyphicon-triangle-left:before {
  content: "\e251";
}
.glyphicon-triangle-bottom:before {
  content: "\e252";
}
.glyphicon-triangle-top:before {
  content: "\e253";
}
.glyphicon-console:before {
  content: "\e254";
}
.glyphicon-superscript:before {
  content: "\e255";
}
.glyphicon-subscript:before {
  content: "\e256";
}
.glyphicon-menu-left:before {
  content: "\e257";
}
.glyphicon-menu-right:before {
  content: "\e258";
}
.glyphicon-menu-down:before {
  content: "\e259";
}
.glyphicon-menu-up:before {
  content: "\e260";
}
* {
  -webkit-box-sizing: border-box;
  -moz-box-sizing: border-box;
  box-sizing: border-box;
}
*:before,
*:after {
  -webkit-box-sizing: border-box;
  -moz-box-sizing: border-box;
  box-sizing: border-box;
}
html {
  font-size: 10px;
  -webkit-tap-highlight-color: rgba(0, 0, 0, 0);
}
body {
  font-family: "Helvetica Neue", Helvetica, Arial, sans-serif;
  font-size: 13px;
  line-height: 1.42857143;
  color: #000;
  background-color: #fff;
}
input,
button,
select,
textarea {
  font-family: inherit;
  font-size: inherit;
  line-height: inherit;
}
a {
  color: #337ab7;
  text-decoration: none;
}
a:hover,
a:focus {
  color: #23527c;
  text-decoration: underline;
}
a:focus {
  outline: 5px auto -webkit-focus-ring-color;
  outline-offset: -2px;
}
figure {
  margin: 0;
}
img {
  vertical-align: middle;
}
.img-responsive,
.thumbnail > img,
.thumbnail a > img,
.carousel-inner > .item > img,
.carousel-inner > .item > a > img {
  display: block;
  max-width: 100%;
  height: auto;
}
.img-rounded {
  border-radius: 3px;
}
.img-thumbnail {
  padding: 4px;
  line-height: 1.42857143;
  background-color: #fff;
  border: 1px solid #ddd;
  border-radius: 2px;
  -webkit-transition: all 0.2s ease-in-out;
  -o-transition: all 0.2s ease-in-out;
  transition: all 0.2s ease-in-out;
  display: inline-block;
  max-width: 100%;
  height: auto;
}
.img-circle {
  border-radius: 50%;
}
hr {
  margin-top: 18px;
  margin-bottom: 18px;
  border: 0;
  border-top: 1px solid #eeeeee;
}
.sr-only {
  position: absolute;
  width: 1px;
  height: 1px;
  margin: -1px;
  padding: 0;
  overflow: hidden;
  clip: rect(0, 0, 0, 0);
  border: 0;
}
.sr-only-focusable:active,
.sr-only-focusable:focus {
  position: static;
  width: auto;
  height: auto;
  margin: 0;
  overflow: visible;
  clip: auto;
}
[role="button"] {
  cursor: pointer;
}
h1,
h2,
h3,
h4,
h5,
h6,
.h1,
.h2,
.h3,
.h4,
.h5,
.h6 {
  font-family: inherit;
  font-weight: 500;
  line-height: 1.1;
  color: inherit;
}
h1 small,
h2 small,
h3 small,
h4 small,
h5 small,
h6 small,
.h1 small,
.h2 small,
.h3 small,
.h4 small,
.h5 small,
.h6 small,
h1 .small,
h2 .small,
h3 .small,
h4 .small,
h5 .small,
h6 .small,
.h1 .small,
.h2 .small,
.h3 .small,
.h4 .small,
.h5 .small,
.h6 .small {
  font-weight: normal;
  line-height: 1;
  color: #777777;
}
h1,
.h1,
h2,
.h2,
h3,
.h3 {
  margin-top: 18px;
  margin-bottom: 9px;
}
h1 small,
.h1 small,
h2 small,
.h2 small,
h3 small,
.h3 small,
h1 .small,
.h1 .small,
h2 .small,
.h2 .small,
h3 .small,
.h3 .small {
  font-size: 65%;
}
h4,
.h4,
h5,
.h5,
h6,
.h6 {
  margin-top: 9px;
  margin-bottom: 9px;
}
h4 small,
.h4 small,
h5 small,
.h5 small,
h6 small,
.h6 small,
h4 .small,
.h4 .small,
h5 .small,
.h5 .small,
h6 .small,
.h6 .small {
  font-size: 75%;
}
h1,
.h1 {
  font-size: 33px;
}
h2,
.h2 {
  font-size: 27px;
}
h3,
.h3 {
  font-size: 23px;
}
h4,
.h4 {
  font-size: 17px;
}
h5,
.h5 {
  font-size: 13px;
}
h6,
.h6 {
  font-size: 12px;
}
p {
  margin: 0 0 9px;
}
.lead {
  margin-bottom: 18px;
  font-size: 14px;
  font-weight: 300;
  line-height: 1.4;
}
@media (min-width: 768px) {
  .lead {
    font-size: 19.5px;
  }
}
small,
.small {
  font-size: 92%;
}
mark,
.mark {
  background-color: #fcf8e3;
  padding: .2em;
}
.text-left {
  text-align: left;
}
.text-right {
  text-align: right;
}
.text-center {
  text-align: center;
}
.text-justify {
  text-align: justify;
}
.text-nowrap {
  white-space: nowrap;
}
.text-lowercase {
  text-transform: lowercase;
}
.text-uppercase {
  text-transform: uppercase;
}
.text-capitalize {
  text-transform: capitalize;
}
.text-muted {
  color: #777777;
}
.text-primary {
  color: #337ab7;
}
a.text-primary:hover,
a.text-primary:focus {
  color: #286090;
}
.text-success {
  color: #3c763d;
}
a.text-success:hover,
a.text-success:focus {
  color: #2b542c;
}
.text-info {
  color: #31708f;
}
a.text-info:hover,
a.text-info:focus {
  color: #245269;
}
.text-warning {
  color: #8a6d3b;
}
a.text-warning:hover,
a.text-warning:focus {
  color: #66512c;
}
.text-danger {
  color: #a94442;
}
a.text-danger:hover,
a.text-danger:focus {
  color: #843534;
}
.bg-primary {
  color: #fff;
  background-color: #337ab7;
}
a.bg-primary:hover,
a.bg-primary:focus {
  background-color: #286090;
}
.bg-success {
  background-color: #dff0d8;
}
a.bg-success:hover,
a.bg-success:focus {
  background-color: #c1e2b3;
}
.bg-info {
  background-color: #d9edf7;
}
a.bg-info:hover,
a.bg-info:focus {
  background-color: #afd9ee;
}
.bg-warning {
  background-color: #fcf8e3;
}
a.bg-warning:hover,
a.bg-warning:focus {
  background-color: #f7ecb5;
}
.bg-danger {
  background-color: #f2dede;
}
a.bg-danger:hover,
a.bg-danger:focus {
  background-color: #e4b9b9;
}
.page-header {
  padding-bottom: 8px;
  margin: 36px 0 18px;
  border-bottom: 1px solid #eeeeee;
}
ul,
ol {
  margin-top: 0;
  margin-bottom: 9px;
}
ul ul,
ol ul,
ul ol,
ol ol {
  margin-bottom: 0;
}
.list-unstyled {
  padding-left: 0;
  list-style: none;
}
.list-inline {
  padding-left: 0;
  list-style: none;
  margin-left: -5px;
}
.list-inline > li {
  display: inline-block;
  padding-left: 5px;
  padding-right: 5px;
}
dl {
  margin-top: 0;
  margin-bottom: 18px;
}
dt,
dd {
  line-height: 1.42857143;
}
dt {
  font-weight: bold;
}
dd {
  margin-left: 0;
}
@media (min-width: 541px) {
  .dl-horizontal dt {
    float: left;
    width: 160px;
    clear: left;
    text-align: right;
    overflow: hidden;
    text-overflow: ellipsis;
    white-space: nowrap;
  }
  .dl-horizontal dd {
    margin-left: 180px;
  }
}
abbr[title],
abbr[data-original-title] {
  cursor: help;
  border-bottom: 1px dotted #777777;
}
.initialism {
  font-size: 90%;
  text-transform: uppercase;
}
blockquote {
  padding: 9px 18px;
  margin: 0 0 18px;
  font-size: inherit;
  border-left: 5px solid #eeeeee;
}
blockquote p:last-child,
blockquote ul:last-child,
blockquote ol:last-child {
  margin-bottom: 0;
}
blockquote footer,
blockquote small,
blockquote .small {
  display: block;
  font-size: 80%;
  line-height: 1.42857143;
  color: #777777;
}
blockquote footer:before,
blockquote small:before,
blockquote .small:before {
  content: '\2014 \00A0';
}
.blockquote-reverse,
blockquote.pull-right {
  padding-right: 15px;
  padding-left: 0;
  border-right: 5px solid #eeeeee;
  border-left: 0;
  text-align: right;
}
.blockquote-reverse footer:before,
blockquote.pull-right footer:before,
.blockquote-reverse small:before,
blockquote.pull-right small:before,
.blockquote-reverse .small:before,
blockquote.pull-right .small:before {
  content: '';
}
.blockquote-reverse footer:after,
blockquote.pull-right footer:after,
.blockquote-reverse small:after,
blockquote.pull-right small:after,
.blockquote-reverse .small:after,
blockquote.pull-right .small:after {
  content: '\00A0 \2014';
}
address {
  margin-bottom: 18px;
  font-style: normal;
  line-height: 1.42857143;
}
code,
kbd,
pre,
samp {
  font-family: monospace;
}
code {
  padding: 2px 4px;
  font-size: 90%;
  color: #c7254e;
  background-color: #f9f2f4;
  border-radius: 2px;
}
kbd {
  padding: 2px 4px;
  font-size: 90%;
  color: #888;
  background-color: transparent;
  border-radius: 1px;
  box-shadow: inset 0 -1px 0 rgba(0, 0, 0, 0.25);
}
kbd kbd {
  padding: 0;
  font-size: 100%;
  font-weight: bold;
  box-shadow: none;
}
pre {
  display: block;
  padding: 8.5px;
  margin: 0 0 9px;
  font-size: 12px;
  line-height: 1.42857143;
  word-break: break-all;
  word-wrap: break-word;
  color: #333333;
  background-color: #f5f5f5;
  border: 1px solid #ccc;
  border-radius: 2px;
}
pre code {
  padding: 0;
  font-size: inherit;
  color: inherit;
  white-space: pre-wrap;
  background-color: transparent;
  border-radius: 0;
}
.pre-scrollable {
  max-height: 340px;
  overflow-y: scroll;
}
.container {
  margin-right: auto;
  margin-left: auto;
  padding-left: 0px;
  padding-right: 0px;
}
@media (min-width: 768px) {
  .container {
    width: 768px;
  }
}
@media (min-width: 992px) {
  .container {
    width: 940px;
  }
}
@media (min-width: 1200px) {
  .container {
    width: 1140px;
  }
}
.container-fluid {
  margin-right: auto;
  margin-left: auto;
  padding-left: 0px;
  padding-right: 0px;
}
.row {
  margin-left: 0px;
  margin-right: 0px;
}
.col-xs-1, .col-sm-1, .col-md-1, .col-lg-1, .col-xs-2, .col-sm-2, .col-md-2, .col-lg-2, .col-xs-3, .col-sm-3, .col-md-3, .col-lg-3, .col-xs-4, .col-sm-4, .col-md-4, .col-lg-4, .col-xs-5, .col-sm-5, .col-md-5, .col-lg-5, .col-xs-6, .col-sm-6, .col-md-6, .col-lg-6, .col-xs-7, .col-sm-7, .col-md-7, .col-lg-7, .col-xs-8, .col-sm-8, .col-md-8, .col-lg-8, .col-xs-9, .col-sm-9, .col-md-9, .col-lg-9, .col-xs-10, .col-sm-10, .col-md-10, .col-lg-10, .col-xs-11, .col-sm-11, .col-md-11, .col-lg-11, .col-xs-12, .col-sm-12, .col-md-12, .col-lg-12 {
  position: relative;
  min-height: 1px;
  padding-left: 0px;
  padding-right: 0px;
}
.col-xs-1, .col-xs-2, .col-xs-3, .col-xs-4, .col-xs-5, .col-xs-6, .col-xs-7, .col-xs-8, .col-xs-9, .col-xs-10, .col-xs-11, .col-xs-12 {
  float: left;
}
.col-xs-12 {
  width: 100%;
}
.col-xs-11 {
  width: 91.66666667%;
}
.col-xs-10 {
  width: 83.33333333%;
}
.col-xs-9 {
  width: 75%;
}
.col-xs-8 {
  width: 66.66666667%;
}
.col-xs-7 {
  width: 58.33333333%;
}
.col-xs-6 {
  width: 50%;
}
.col-xs-5 {
  width: 41.66666667%;
}
.col-xs-4 {
  width: 33.33333333%;
}
.col-xs-3 {
  width: 25%;
}
.col-xs-2 {
  width: 16.66666667%;
}
.col-xs-1 {
  width: 8.33333333%;
}
.col-xs-pull-12 {
  right: 100%;
}
.col-xs-pull-11 {
  right: 91.66666667%;
}
.col-xs-pull-10 {
  right: 83.33333333%;
}
.col-xs-pull-9 {
  right: 75%;
}
.col-xs-pull-8 {
  right: 66.66666667%;
}
.col-xs-pull-7 {
  right: 58.33333333%;
}
.col-xs-pull-6 {
  right: 50%;
}
.col-xs-pull-5 {
  right: 41.66666667%;
}
.col-xs-pull-4 {
  right: 33.33333333%;
}
.col-xs-pull-3 {
  right: 25%;
}
.col-xs-pull-2 {
  right: 16.66666667%;
}
.col-xs-pull-1 {
  right: 8.33333333%;
}
.col-xs-pull-0 {
  right: auto;
}
.col-xs-push-12 {
  left: 100%;
}
.col-xs-push-11 {
  left: 91.66666667%;
}
.col-xs-push-10 {
  left: 83.33333333%;
}
.col-xs-push-9 {
  left: 75%;
}
.col-xs-push-8 {
  left: 66.66666667%;
}
.col-xs-push-7 {
  left: 58.33333333%;
}
.col-xs-push-6 {
  left: 50%;
}
.col-xs-push-5 {
  left: 41.66666667%;
}
.col-xs-push-4 {
  left: 33.33333333%;
}
.col-xs-push-3 {
  left: 25%;
}
.col-xs-push-2 {
  left: 16.66666667%;
}
.col-xs-push-1 {
  left: 8.33333333%;
}
.col-xs-push-0 {
  left: auto;
}
.col-xs-offset-12 {
  margin-left: 100%;
}
.col-xs-offset-11 {
  margin-left: 91.66666667%;
}
.col-xs-offset-10 {
  margin-left: 83.33333333%;
}
.col-xs-offset-9 {
  margin-left: 75%;
}
.col-xs-offset-8 {
  margin-left: 66.66666667%;
}
.col-xs-offset-7 {
  margin-left: 58.33333333%;
}
.col-xs-offset-6 {
  margin-left: 50%;
}
.col-xs-offset-5 {
  margin-left: 41.66666667%;
}
.col-xs-offset-4 {
  margin-left: 33.33333333%;
}
.col-xs-offset-3 {
  margin-left: 25%;
}
.col-xs-offset-2 {
  margin-left: 16.66666667%;
}
.col-xs-offset-1 {
  margin-left: 8.33333333%;
}
.col-xs-offset-0 {
  margin-left: 0%;
}
@media (min-width: 768px) {
  .col-sm-1, .col-sm-2, .col-sm-3, .col-sm-4, .col-sm-5, .col-sm-6, .col-sm-7, .col-sm-8, .col-sm-9, .col-sm-10, .col-sm-11, .col-sm-12 {
    float: left;
  }
  .col-sm-12 {
    width: 100%;
  }
  .col-sm-11 {
    width: 91.66666667%;
  }
  .col-sm-10 {
    width: 83.33333333%;
  }
  .col-sm-9 {
    width: 75%;
  }
  .col-sm-8 {
    width: 66.66666667%;
  }
  .col-sm-7 {
    width: 58.33333333%;
  }
  .col-sm-6 {
    width: 50%;
  }
  .col-sm-5 {
    width: 41.66666667%;
  }
  .col-sm-4 {
    width: 33.33333333%;
  }
  .col-sm-3 {
    width: 25%;
  }
  .col-sm-2 {
    width: 16.66666667%;
  }
  .col-sm-1 {
    width: 8.33333333%;
  }
  .col-sm-pull-12 {
    right: 100%;
  }
  .col-sm-pull-11 {
    right: 91.66666667%;
  }
  .col-sm-pull-10 {
    right: 83.33333333%;
  }
  .col-sm-pull-9 {
    right: 75%;
  }
  .col-sm-pull-8 {
    right: 66.66666667%;
  }
  .col-sm-pull-7 {
    right: 58.33333333%;
  }
  .col-sm-pull-6 {
    right: 50%;
  }
  .col-sm-pull-5 {
    right: 41.66666667%;
  }
  .col-sm-pull-4 {
    right: 33.33333333%;
  }
  .col-sm-pull-3 {
    right: 25%;
  }
  .col-sm-pull-2 {
    right: 16.66666667%;
  }
  .col-sm-pull-1 {
    right: 8.33333333%;
  }
  .col-sm-pull-0 {
    right: auto;
  }
  .col-sm-push-12 {
    left: 100%;
  }
  .col-sm-push-11 {
    left: 91.66666667%;
  }
  .col-sm-push-10 {
    left: 83.33333333%;
  }
  .col-sm-push-9 {
    left: 75%;
  }
  .col-sm-push-8 {
    left: 66.66666667%;
  }
  .col-sm-push-7 {
    left: 58.33333333%;
  }
  .col-sm-push-6 {
    left: 50%;
  }
  .col-sm-push-5 {
    left: 41.66666667%;
  }
  .col-sm-push-4 {
    left: 33.33333333%;
  }
  .col-sm-push-3 {
    left: 25%;
  }
  .col-sm-push-2 {
    left: 16.66666667%;
  }
  .col-sm-push-1 {
    left: 8.33333333%;
  }
  .col-sm-push-0 {
    left: auto;
  }
  .col-sm-offset-12 {
    margin-left: 100%;
  }
  .col-sm-offset-11 {
    margin-left: 91.66666667%;
  }
  .col-sm-offset-10 {
    margin-left: 83.33333333%;
  }
  .col-sm-offset-9 {
    margin-left: 75%;
  }
  .col-sm-offset-8 {
    margin-left: 66.66666667%;
  }
  .col-sm-offset-7 {
    margin-left: 58.33333333%;
  }
  .col-sm-offset-6 {
    margin-left: 50%;
  }
  .col-sm-offset-5 {
    margin-left: 41.66666667%;
  }
  .col-sm-offset-4 {
    margin-left: 33.33333333%;
  }
  .col-sm-offset-3 {
    margin-left: 25%;
  }
  .col-sm-offset-2 {
    margin-left: 16.66666667%;
  }
  .col-sm-offset-1 {
    margin-left: 8.33333333%;
  }
  .col-sm-offset-0 {
    margin-left: 0%;
  }
}
@media (min-width: 992px) {
  .col-md-1, .col-md-2, .col-md-3, .col-md-4, .col-md-5, .col-md-6, .col-md-7, .col-md-8, .col-md-9, .col-md-10, .col-md-11, .col-md-12 {
    float: left;
  }
  .col-md-12 {
    width: 100%;
  }
  .col-md-11 {
    width: 91.66666667%;
  }
  .col-md-10 {
    width: 83.33333333%;
  }
  .col-md-9 {
    width: 75%;
  }
  .col-md-8 {
    width: 66.66666667%;
  }
  .col-md-7 {
    width: 58.33333333%;
  }
  .col-md-6 {
    width: 50%;
  }
  .col-md-5 {
    width: 41.66666667%;
  }
  .col-md-4 {
    width: 33.33333333%;
  }
  .col-md-3 {
    width: 25%;
  }
  .col-md-2 {
    width: 16.66666667%;
  }
  .col-md-1 {
    width: 8.33333333%;
  }
  .col-md-pull-12 {
    right: 100%;
  }
  .col-md-pull-11 {
    right: 91.66666667%;
  }
  .col-md-pull-10 {
    right: 83.33333333%;
  }
  .col-md-pull-9 {
    right: 75%;
  }
  .col-md-pull-8 {
    right: 66.66666667%;
  }
  .col-md-pull-7 {
    right: 58.33333333%;
  }
  .col-md-pull-6 {
    right: 50%;
  }
  .col-md-pull-5 {
    right: 41.66666667%;
  }
  .col-md-pull-4 {
    right: 33.33333333%;
  }
  .col-md-pull-3 {
    right: 25%;
  }
  .col-md-pull-2 {
    right: 16.66666667%;
  }
  .col-md-pull-1 {
    right: 8.33333333%;
  }
  .col-md-pull-0 {
    right: auto;
  }
  .col-md-push-12 {
    left: 100%;
  }
  .col-md-push-11 {
    left: 91.66666667%;
  }
  .col-md-push-10 {
    left: 83.33333333%;
  }
  .col-md-push-9 {
    left: 75%;
  }
  .col-md-push-8 {
    left: 66.66666667%;
  }
  .col-md-push-7 {
    left: 58.33333333%;
  }
  .col-md-push-6 {
    left: 50%;
  }
  .col-md-push-5 {
    left: 41.66666667%;
  }
  .col-md-push-4 {
    left: 33.33333333%;
  }
  .col-md-push-3 {
    left: 25%;
  }
  .col-md-push-2 {
    left: 16.66666667%;
  }
  .col-md-push-1 {
    left: 8.33333333%;
  }
  .col-md-push-0 {
    left: auto;
  }
  .col-md-offset-12 {
    margin-left: 100%;
  }
  .col-md-offset-11 {
    margin-left: 91.66666667%;
  }
  .col-md-offset-10 {
    margin-left: 83.33333333%;
  }
  .col-md-offset-9 {
    margin-left: 75%;
  }
  .col-md-offset-8 {
    margin-left: 66.66666667%;
  }
  .col-md-offset-7 {
    margin-left: 58.33333333%;
  }
  .col-md-offset-6 {
    margin-left: 50%;
  }
  .col-md-offset-5 {
    margin-left: 41.66666667%;
  }
  .col-md-offset-4 {
    margin-left: 33.33333333%;
  }
  .col-md-offset-3 {
    margin-left: 25%;
  }
  .col-md-offset-2 {
    margin-left: 16.66666667%;
  }
  .col-md-offset-1 {
    margin-left: 8.33333333%;
  }
  .col-md-offset-0 {
    margin-left: 0%;
  }
}
@media (min-width: 1200px) {
  .col-lg-1, .col-lg-2, .col-lg-3, .col-lg-4, .col-lg-5, .col-lg-6, .col-lg-7, .col-lg-8, .col-lg-9, .col-lg-10, .col-lg-11, .col-lg-12 {
    float: left;
  }
  .col-lg-12 {
    width: 100%;
  }
  .col-lg-11 {
    width: 91.66666667%;
  }
  .col-lg-10 {
    width: 83.33333333%;
  }
  .col-lg-9 {
    width: 75%;
  }
  .col-lg-8 {
    width: 66.66666667%;
  }
  .col-lg-7 {
    width: 58.33333333%;
  }
  .col-lg-6 {
    width: 50%;
  }
  .col-lg-5 {
    width: 41.66666667%;
  }
  .col-lg-4 {
    width: 33.33333333%;
  }
  .col-lg-3 {
    width: 25%;
  }
  .col-lg-2 {
    width: 16.66666667%;
  }
  .col-lg-1 {
    width: 8.33333333%;
  }
  .col-lg-pull-12 {
    right: 100%;
  }
  .col-lg-pull-11 {
    right: 91.66666667%;
  }
  .col-lg-pull-10 {
    right: 83.33333333%;
  }
  .col-lg-pull-9 {
    right: 75%;
  }
  .col-lg-pull-8 {
    right: 66.66666667%;
  }
  .col-lg-pull-7 {
    right: 58.33333333%;
  }
  .col-lg-pull-6 {
    right: 50%;
  }
  .col-lg-pull-5 {
    right: 41.66666667%;
  }
  .col-lg-pull-4 {
    right: 33.33333333%;
  }
  .col-lg-pull-3 {
    right: 25%;
  }
  .col-lg-pull-2 {
    right: 16.66666667%;
  }
  .col-lg-pull-1 {
    right: 8.33333333%;
  }
  .col-lg-pull-0 {
    right: auto;
  }
  .col-lg-push-12 {
    left: 100%;
  }
  .col-lg-push-11 {
    left: 91.66666667%;
  }
  .col-lg-push-10 {
    left: 83.33333333%;
  }
  .col-lg-push-9 {
    left: 75%;
  }
  .col-lg-push-8 {
    left: 66.66666667%;
  }
  .col-lg-push-7 {
    left: 58.33333333%;
  }
  .col-lg-push-6 {
    left: 50%;
  }
  .col-lg-push-5 {
    left: 41.66666667%;
  }
  .col-lg-push-4 {
    left: 33.33333333%;
  }
  .col-lg-push-3 {
    left: 25%;
  }
  .col-lg-push-2 {
    left: 16.66666667%;
  }
  .col-lg-push-1 {
    left: 8.33333333%;
  }
  .col-lg-push-0 {
    left: auto;
  }
  .col-lg-offset-12 {
    margin-left: 100%;
  }
  .col-lg-offset-11 {
    margin-left: 91.66666667%;
  }
  .col-lg-offset-10 {
    margin-left: 83.33333333%;
  }
  .col-lg-offset-9 {
    margin-left: 75%;
  }
  .col-lg-offset-8 {
    margin-left: 66.66666667%;
  }
  .col-lg-offset-7 {
    margin-left: 58.33333333%;
  }
  .col-lg-offset-6 {
    margin-left: 50%;
  }
  .col-lg-offset-5 {
    margin-left: 41.66666667%;
  }
  .col-lg-offset-4 {
    margin-left: 33.33333333%;
  }
  .col-lg-offset-3 {
    margin-left: 25%;
  }
  .col-lg-offset-2 {
    margin-left: 16.66666667%;
  }
  .col-lg-offset-1 {
    margin-left: 8.33333333%;
  }
  .col-lg-offset-0 {
    margin-left: 0%;
  }
}
table {
  background-color: transparent;
}
caption {
  padding-top: 8px;
  padding-bottom: 8px;
  color: #777777;
  text-align: left;
}
th {
  text-align: left;
}
.table {
  width: 100%;
  max-width: 100%;
  margin-bottom: 18px;
}
.table > thead > tr > th,
.table > tbody > tr > th,
.table > tfoot > tr > th,
.table > thead > tr > td,
.table > tbody > tr > td,
.table > tfoot > tr > td {
  padding: 8px;
  line-height: 1.42857143;
  vertical-align: top;
  border-top: 1px solid #ddd;
}
.table > thead > tr > th {
  vertical-align: bottom;
  border-bottom: 2px solid #ddd;
}
.table > caption + thead > tr:first-child > th,
.table > colgroup + thead > tr:first-child > th,
.table > thead:first-child > tr:first-child > th,
.table > caption + thead > tr:first-child > td,
.table > colgroup + thead > tr:first-child > td,
.table > thead:first-child > tr:first-child > td {
  border-top: 0;
}
.table > tbody + tbody {
  border-top: 2px solid #ddd;
}
.table .table {
  background-color: #fff;
}
.table-condensed > thead > tr > th,
.table-condensed > tbody > tr > th,
.table-condensed > tfoot > tr > th,
.table-condensed > thead > tr > td,
.table-condensed > tbody > tr > td,
.table-condensed > tfoot > tr > td {
  padding: 5px;
}
.table-bordered {
  border: 1px solid #ddd;
}
.table-bordered > thead > tr > th,
.table-bordered > tbody > tr > th,
.table-bordered > tfoot > tr > th,
.table-bordered > thead > tr > td,
.table-bordered > tbody > tr > td,
.table-bordered > tfoot > tr > td {
  border: 1px solid #ddd;
}
.table-bordered > thead > tr > th,
.table-bordered > thead > tr > td {
  border-bottom-width: 2px;
}
.table-striped > tbody > tr:nth-of-type(odd) {
  background-color: #f9f9f9;
}
.table-hover > tbody > tr:hover {
  background-color: #f5f5f5;
}
table col[class*="col-"] {
  position: static;
  float: none;
  display: table-column;
}
table td[class*="col-"],
table th[class*="col-"] {
  position: static;
  float: none;
  display: table-cell;
}
.table > thead > tr > td.active,
.table > tbody > tr > td.active,
.table > tfoot > tr > td.active,
.table > thead > tr > th.active,
.table > tbody > tr > th.active,
.table > tfoot > tr > th.active,
.table > thead > tr.active > td,
.table > tbody > tr.active > td,
.table > tfoot > tr.active > td,
.table > thead > tr.active > th,
.table > tbody > tr.active > th,
.table > tfoot > tr.active > th {
  background-color: #f5f5f5;
}
.table-hover > tbody > tr > td.active:hover,
.table-hover > tbody > tr > th.active:hover,
.table-hover > tbody > tr.active:hover > td,
.table-hover > tbody > tr:hover > .active,
.table-hover > tbody > tr.active:hover > th {
  background-color: #e8e8e8;
}
.table > thead > tr > td.success,
.table > tbody > tr > td.success,
.table > tfoot > tr > td.success,
.table > thead > tr > th.success,
.table > tbody > tr > th.success,
.table > tfoot > tr > th.success,
.table > thead > tr.success > td,
.table > tbody > tr.success > td,
.table > tfoot > tr.success > td,
.table > thead > tr.success > th,
.table > tbody > tr.success > th,
.table > tfoot > tr.success > th {
  background-color: #dff0d8;
}
.table-hover > tbody > tr > td.success:hover,
.table-hover > tbody > tr > th.success:hover,
.table-hover > tbody > tr.success:hover > td,
.table-hover > tbody > tr:hover > .success,
.table-hover > tbody > tr.success:hover > th {
  background-color: #d0e9c6;
}
.table > thead > tr > td.info,
.table > tbody > tr > td.info,
.table > tfoot > tr > td.info,
.table > thead > tr > th.info,
.table > tbody > tr > th.info,
.table > tfoot > tr > th.info,
.table > thead > tr.info > td,
.table > tbody > tr.info > td,
.table > tfoot > tr.info > td,
.table > thead > tr.info > th,
.table > tbody > tr.info > th,
.table > tfoot > tr.info > th {
  background-color: #d9edf7;
}
.table-hover > tbody > tr > td.info:hover,
.table-hover > tbody > tr > th.info:hover,
.table-hover > tbody > tr.info:hover > td,
.table-hover > tbody > tr:hover > .info,
.table-hover > tbody > tr.info:hover > th {
  background-color: #c4e3f3;
}
.table > thead > tr > td.warning,
.table > tbody > tr > td.warning,
.table > tfoot > tr > td.warning,
.table > thead > tr > th.warning,
.table > tbody > tr > th.warning,
.table > tfoot > tr > th.warning,
.table > thead > tr.warning > td,
.table > tbody > tr.warning > td,
.table > tfoot > tr.warning > td,
.table > thead > tr.warning > th,
.table > tbody > tr.warning > th,
.table > tfoot > tr.warning > th {
  background-color: #fcf8e3;
}
.table-hover > tbody > tr > td.warning:hover,
.table-hover > tbody > tr > th.warning:hover,
.table-hover > tbody > tr.warning:hover > td,
.table-hover > tbody > tr:hover > .warning,
.table-hover > tbody > tr.warning:hover > th {
  background-color: #faf2cc;
}
.table > thead > tr > td.danger,
.table > tbody > tr > td.danger,
.table > tfoot > tr > td.danger,
.table > thead > tr > th.danger,
.table > tbody > tr > th.danger,
.table > tfoot > tr > th.danger,
.table > thead > tr.danger > td,
.table > tbody > tr.danger > td,
.table > tfoot > tr.danger > td,
.table > thead > tr.danger > th,
.table > tbody > tr.danger > th,
.table > tfoot > tr.danger > th {
  background-color: #f2dede;
}
.table-hover > tbody > tr > td.danger:hover,
.table-hover > tbody > tr > th.danger:hover,
.table-hover > tbody > tr.danger:hover > td,
.table-hover > tbody > tr:hover > .danger,
.table-hover > tbody > tr.danger:hover > th {
  background-color: #ebcccc;
}
.table-responsive {
  overflow-x: auto;
  min-height: 0.01%;
}
@media screen and (max-width: 767px) {
  .table-responsive {
    width: 100%;
    margin-bottom: 13.5px;
    overflow-y: hidden;
    -ms-overflow-style: -ms-autohiding-scrollbar;
    border: 1px solid #ddd;
  }
  .table-responsive > .table {
    margin-bottom: 0;
  }
  .table-responsive > .table > thead > tr > th,
  .table-responsive > .table > tbody > tr > th,
  .table-responsive > .table > tfoot > tr > th,
  .table-responsive > .table > thead > tr > td,
  .table-responsive > .table > tbody > tr > td,
  .table-responsive > .table > tfoot > tr > td {
    white-space: nowrap;
  }
  .table-responsive > .table-bordered {
    border: 0;
  }
  .table-responsive > .table-bordered > thead > tr > th:first-child,
  .table-responsive > .table-bordered > tbody > tr > th:first-child,
  .table-responsive > .table-bordered > tfoot > tr > th:first-child,
  .table-responsive > .table-bordered > thead > tr > td:first-child,
  .table-responsive > .table-bordered > tbody > tr > td:first-child,
  .table-responsive > .table-bordered > tfoot > tr > td:first-child {
    border-left: 0;
  }
  .table-responsive > .table-bordered > thead > tr > th:last-child,
  .table-responsive > .table-bordered > tbody > tr > th:last-child,
  .table-responsive > .table-bordered > tfoot > tr > th:last-child,
  .table-responsive > .table-bordered > thead > tr > td:last-child,
  .table-responsive > .table-bordered > tbody > tr > td:last-child,
  .table-responsive > .table-bordered > tfoot > tr > td:last-child {
    border-right: 0;
  }
  .table-responsive > .table-bordered > tbody > tr:last-child > th,
  .table-responsive > .table-bordered > tfoot > tr:last-child > th,
  .table-responsive > .table-bordered > tbody > tr:last-child > td,
  .table-responsive > .table-bordered > tfoot > tr:last-child > td {
    border-bottom: 0;
  }
}
fieldset {
  padding: 0;
  margin: 0;
  border: 0;
  min-width: 0;
}
legend {
  display: block;
  width: 100%;
  padding: 0;
  margin-bottom: 18px;
  font-size: 19.5px;
  line-height: inherit;
  color: #333333;
  border: 0;
  border-bottom: 1px solid #e5e5e5;
}
label {
  display: inline-block;
  max-width: 100%;
  margin-bottom: 5px;
  font-weight: bold;
}
input[type="search"] {
  -webkit-box-sizing: border-box;
  -moz-box-sizing: border-box;
  box-sizing: border-box;
}
input[type="radio"],
input[type="checkbox"] {
  margin: 4px 0 0;
  margin-top: 1px \9;
  line-height: normal;
}
input[type="file"] {
  display: block;
}
input[type="range"] {
  display: block;
  width: 100%;
}
select[multiple],
select[size] {
  height: auto;
}
input[type="file"]:focus,
input[type="radio"]:focus,
input[type="checkbox"]:focus {
  outline: 5px auto -webkit-focus-ring-color;
  outline-offset: -2px;
}
output {
  display: block;
  padding-top: 7px;
  font-size: 13px;
  line-height: 1.42857143;
  color: #555555;
}
.form-control {
  display: block;
  width: 100%;
  height: 32px;
  padding: 6px 12px;
  font-size: 13px;
  line-height: 1.42857143;
  color: #555555;
  background-color: #fff;
  background-image: none;
  border: 1px solid #ccc;
  border-radius: 2px;
  -webkit-box-shadow: inset 0 1px 1px rgba(0, 0, 0, 0.075);
  box-shadow: inset 0 1px 1px rgba(0, 0, 0, 0.075);
  -webkit-transition: border-color ease-in-out .15s, box-shadow ease-in-out .15s;
  -o-transition: border-color ease-in-out .15s, box-shadow ease-in-out .15s;
  transition: border-color ease-in-out .15s, box-shadow ease-in-out .15s;
}
.form-control:focus {
  border-color: #66afe9;
  outline: 0;
  -webkit-box-shadow: inset 0 1px 1px rgba(0,0,0,.075), 0 0 8px rgba(102, 175, 233, 0.6);
  box-shadow: inset 0 1px 1px rgba(0,0,0,.075), 0 0 8px rgba(102, 175, 233, 0.6);
}
.form-control::-moz-placeholder {
  color: #999;
  opacity: 1;
}
.form-control:-ms-input-placeholder {
  color: #999;
}
.form-control::-webkit-input-placeholder {
  color: #999;
}
.form-control::-ms-expand {
  border: 0;
  background-color: transparent;
}
.form-control[disabled],
.form-control[readonly],
fieldset[disabled] .form-control {
  background-color: #eeeeee;
  opacity: 1;
}
.form-control[disabled],
fieldset[disabled] .form-control {
  cursor: not-allowed;
}
textarea.form-control {
  height: auto;
}
input[type="search"] {
  -webkit-appearance: none;
}
@media screen and (-webkit-min-device-pixel-ratio: 0) {
  input[type="date"].form-control,
  input[type="time"].form-control,
  input[type="datetime-local"].form-control,
  input[type="month"].form-control {
    line-height: 32px;
  }
  input[type="date"].input-sm,
  input[type="time"].input-sm,
  input[type="datetime-local"].input-sm,
  input[type="month"].input-sm,
  .input-group-sm input[type="date"],
  .input-group-sm input[type="time"],
  .input-group-sm input[type="datetime-local"],
  .input-group-sm input[type="month"] {
    line-height: 30px;
  }
  input[type="date"].input-lg,
  input[type="time"].input-lg,
  input[type="datetime-local"].input-lg,
  input[type="month"].input-lg,
  .input-group-lg input[type="date"],
  .input-group-lg input[type="time"],
  .input-group-lg input[type="datetime-local"],
  .input-group-lg input[type="month"] {
    line-height: 45px;
  }
}
.form-group {
  margin-bottom: 15px;
}
.radio,
.checkbox {
  position: relative;
  display: block;
  margin-top: 10px;
  margin-bottom: 10px;
}
.radio label,
.checkbox label {
  min-height: 18px;
  padding-left: 20px;
  margin-bottom: 0;
  font-weight: normal;
  cursor: pointer;
}
.radio input[type="radio"],
.radio-inline input[type="radio"],
.checkbox input[type="checkbox"],
.checkbox-inline input[type="checkbox"] {
  position: absolute;
  margin-left: -20px;
  margin-top: 4px \9;
}
.radio + .radio,
.checkbox + .checkbox {
  margin-top: -5px;
}
.radio-inline,
.checkbox-inline {
  position: relative;
  display: inline-block;
  padding-left: 20px;
  margin-bottom: 0;
  vertical-align: middle;
  font-weight: normal;
  cursor: pointer;
}
.radio-inline + .radio-inline,
.checkbox-inline + .checkbox-inline {
  margin-top: 0;
  margin-left: 10px;
}
input[type="radio"][disabled],
input[type="checkbox"][disabled],
input[type="radio"].disabled,
input[type="checkbox"].disabled,
fieldset[disabled] input[type="radio"],
fieldset[disabled] input[type="checkbox"] {
  cursor: not-allowed;
}
.radio-inline.disabled,
.checkbox-inline.disabled,
fieldset[disabled] .radio-inline,
fieldset[disabled] .checkbox-inline {
  cursor: not-allowed;
}
.radio.disabled label,
.checkbox.disabled label,
fieldset[disabled] .radio label,
fieldset[disabled] .checkbox label {
  cursor: not-allowed;
}
.form-control-static {
  padding-top: 7px;
  padding-bottom: 7px;
  margin-bottom: 0;
  min-height: 31px;
}
.form-control-static.input-lg,
.form-control-static.input-sm {
  padding-left: 0;
  padding-right: 0;
}
.input-sm {
  height: 30px;
  padding: 5px 10px;
  font-size: 12px;
  line-height: 1.5;
  border-radius: 1px;
}
select.input-sm {
  height: 30px;
  line-height: 30px;
}
textarea.input-sm,
select[multiple].input-sm {
  height: auto;
}
.form-group-sm .form-control {
  height: 30px;
  padding: 5px 10px;
  font-size: 12px;
  line-height: 1.5;
  border-radius: 1px;
}
.form-group-sm select.form-control {
  height: 30px;
  line-height: 30px;
}
.form-group-sm textarea.form-control,
.form-group-sm select[multiple].form-control {
  height: auto;
}
.form-group-sm .form-control-static {
  height: 30px;
  min-height: 30px;
  padding: 6px 10px;
  font-size: 12px;
  line-height: 1.5;
}
.input-lg {
  height: 45px;
  padding: 10px 16px;
  font-size: 17px;
  line-height: 1.3333333;
  border-radius: 3px;
}
select.input-lg {
  height: 45px;
  line-height: 45px;
}
textarea.input-lg,
select[multiple].input-lg {
  height: auto;
}
.form-group-lg .form-control {
  height: 45px;
  padding: 10px 16px;
  font-size: 17px;
  line-height: 1.3333333;
  border-radius: 3px;
}
.form-group-lg select.form-control {
  height: 45px;
  line-height: 45px;
}
.form-group-lg textarea.form-control,
.form-group-lg select[multiple].form-control {
  height: auto;
}
.form-group-lg .form-control-static {
  height: 45px;
  min-height: 35px;
  padding: 11px 16px;
  font-size: 17px;
  line-height: 1.3333333;
}
.has-feedback {
  position: relative;
}
.has-feedback .form-control {
  padding-right: 40px;
}
.form-control-feedback {
  position: absolute;
  top: 0;
  right: 0;
  z-index: 2;
  display: block;
  width: 32px;
  height: 32px;
  line-height: 32px;
  text-align: center;
  pointer-events: none;
}
.input-lg + .form-control-feedback,
.input-group-lg + .form-control-feedback,
.form-group-lg .form-control + .form-control-feedback {
  width: 45px;
  height: 45px;
  line-height: 45px;
}
.input-sm + .form-control-feedback,
.input-group-sm + .form-control-feedback,
.form-group-sm .form-control + .form-control-feedback {
  width: 30px;
  height: 30px;
  line-height: 30px;
}
.has-success .help-block,
.has-success .control-label,
.has-success .radio,
.has-success .checkbox,
.has-success .radio-inline,
.has-success .checkbox-inline,
.has-success.radio label,
.has-success.checkbox label,
.has-success.radio-inline label,
.has-success.checkbox-inline label {
  color: #3c763d;
}
.has-success .form-control {
  border-color: #3c763d;
  -webkit-box-shadow: inset 0 1px 1px rgba(0, 0, 0, 0.075);
  box-shadow: inset 0 1px 1px rgba(0, 0, 0, 0.075);
}
.has-success .form-control:focus {
  border-color: #2b542c;
  -webkit-box-shadow: inset 0 1px 1px rgba(0, 0, 0, 0.075), 0 0 6px #67b168;
  box-shadow: inset 0 1px 1px rgba(0, 0, 0, 0.075), 0 0 6px #67b168;
}
.has-success .input-group-addon {
  color: #3c763d;
  border-color: #3c763d;
  background-color: #dff0d8;
}
.has-success .form-control-feedback {
  color: #3c763d;
}
.has-warning .help-block,
.has-warning .control-label,
.has-warning .radio,
.has-warning .checkbox,
.has-warning .radio-inline,
.has-warning .checkbox-inline,
.has-warning.radio label,
.has-warning.checkbox label,
.has-warning.radio-inline label,
.has-warning.checkbox-inline label {
  color: #8a6d3b;
}
.has-warning .form-control {
  border-color: #8a6d3b;
  -webkit-box-shadow: inset 0 1px 1px rgba(0, 0, 0, 0.075);
  box-shadow: inset 0 1px 1px rgba(0, 0, 0, 0.075);
}
.has-warning .form-control:focus {
  border-color: #66512c;
  -webkit-box-shadow: inset 0 1px 1px rgba(0, 0, 0, 0.075), 0 0 6px #c0a16b;
  box-shadow: inset 0 1px 1px rgba(0, 0, 0, 0.075), 0 0 6px #c0a16b;
}
.has-warning .input-group-addon {
  color: #8a6d3b;
  border-color: #8a6d3b;
  background-color: #fcf8e3;
}
.has-warning .form-control-feedback {
  color: #8a6d3b;
}
.has-error .help-block,
.has-error .control-label,
.has-error .radio,
.has-error .checkbox,
.has-error .radio-inline,
.has-error .checkbox-inline,
.has-error.radio label,
.has-error.checkbox label,
.has-error.radio-inline label,
.has-error.checkbox-inline label {
  color: #a94442;
}
.has-error .form-control {
  border-color: #a94442;
  -webkit-box-shadow: inset 0 1px 1px rgba(0, 0, 0, 0.075);
  box-shadow: inset 0 1px 1px rgba(0, 0, 0, 0.075);
}
.has-error .form-control:focus {
  border-color: #843534;
  -webkit-box-shadow: inset 0 1px 1px rgba(0, 0, 0, 0.075), 0 0 6px #ce8483;
  box-shadow: inset 0 1px 1px rgba(0, 0, 0, 0.075), 0 0 6px #ce8483;
}
.has-error .input-group-addon {
  color: #a94442;
  border-color: #a94442;
  background-color: #f2dede;
}
.has-error .form-control-feedback {
  color: #a94442;
}
.has-feedback label ~ .form-control-feedback {
  top: 23px;
}
.has-feedback label.sr-only ~ .form-control-feedback {
  top: 0;
}
.help-block {
  display: block;
  margin-top: 5px;
  margin-bottom: 10px;
  color: #404040;
}
@media (min-width: 768px) {
  .form-inline .form-group {
    display: inline-block;
    margin-bottom: 0;
    vertical-align: middle;
  }
  .form-inline .form-control {
    display: inline-block;
    width: auto;
    vertical-align: middle;
  }
  .form-inline .form-control-static {
    display: inline-block;
  }
  .form-inline .input-group {
    display: inline-table;
    vertical-align: middle;
  }
  .form-inline .input-group .input-group-addon,
  .form-inline .input-group .input-group-btn,
  .form-inline .input-group .form-control {
    width: auto;
  }
  .form-inline .input-group > .form-control {
    width: 100%;
  }
  .form-inline .control-label {
    margin-bottom: 0;
    vertical-align: middle;
  }
  .form-inline .radio,
  .form-inline .checkbox {
    display: inline-block;
    margin-top: 0;
    margin-bottom: 0;
    vertical-align: middle;
  }
  .form-inline .radio label,
  .form-inline .checkbox label {
    padding-left: 0;
  }
  .form-inline .radio input[type="radio"],
  .form-inline .checkbox input[type="checkbox"] {
    position: relative;
    margin-left: 0;
  }
  .form-inline .has-feedback .form-control-feedback {
    top: 0;
  }
}
.form-horizontal .radio,
.form-horizontal .checkbox,
.form-horizontal .radio-inline,
.form-horizontal .checkbox-inline {
  margin-top: 0;
  margin-bottom: 0;
  padding-top: 7px;
}
.form-horizontal .radio,
.form-horizontal .checkbox {
  min-height: 25px;
}
.form-horizontal .form-group {
  margin-left: 0px;
  margin-right: 0px;
}
@media (min-width: 768px) {
  .form-horizontal .control-label {
    text-align: right;
    margin-bottom: 0;
    padding-top: 7px;
  }
}
.form-horizontal .has-feedback .form-control-feedback {
  right: 0px;
}
@media (min-width: 768px) {
  .form-horizontal .form-group-lg .control-label {
    padding-top: 11px;
    font-size: 17px;
  }
}
@media (min-width: 768px) {
  .form-horizontal .form-group-sm .control-label {
    padding-top: 6px;
    font-size: 12px;
  }
}
.btn {
  display: inline-block;
  margin-bottom: 0;
  font-weight: normal;
  text-align: center;
  vertical-align: middle;
  touch-action: manipulation;
  cursor: pointer;
  background-image: none;
  border: 1px solid transparent;
  white-space: nowrap;
  padding: 6px 12px;
  font-size: 13px;
  line-height: 1.42857143;
  border-radius: 2px;
  -webkit-user-select: none;
  -moz-user-select: none;
  -ms-user-select: none;
  user-select: none;
}
.btn:focus,
.btn:active:focus,
.btn.active:focus,
.btn.focus,
.btn:active.focus,
.btn.active.focus {
  outline: 5px auto -webkit-focus-ring-color;
  outline-offset: -2px;
}
.btn:hover,
.btn:focus,
.btn.focus {
  color: #333;
  text-decoration: none;
}
.btn:active,
.btn.active {
  outline: 0;
  background-image: none;
  -webkit-box-shadow: inset 0 3px 5px rgba(0, 0, 0, 0.125);
  box-shadow: inset 0 3px 5px rgba(0, 0, 0, 0.125);
}
.btn.disabled,
.btn[disabled],
fieldset[disabled] .btn {
  cursor: not-allowed;
  opacity: 0.65;
  filter: alpha(opacity=65);
  -webkit-box-shadow: none;
  box-shadow: none;
}
a.btn.disabled,
fieldset[disabled] a.btn {
  pointer-events: none;
}
.btn-default {
  color: #333;
  background-color: #fff;
  border-color: #ccc;
}
.btn-default:focus,
.btn-default.focus {
  color: #333;
  background-color: #e6e6e6;
  border-color: #8c8c8c;
}
.btn-default:hover {
  color: #333;
  background-color: #e6e6e6;
  border-color: #adadad;
}
.btn-default:active,
.btn-default.active,
.open > .dropdown-toggle.btn-default {
  color: #333;
  background-color: #e6e6e6;
  border-color: #adadad;
}
.btn-default:active:hover,
.btn-default.active:hover,
.open > .dropdown-toggle.btn-default:hover,
.btn-default:active:focus,
.btn-default.active:focus,
.open > .dropdown-toggle.btn-default:focus,
.btn-default:active.focus,
.btn-default.active.focus,
.open > .dropdown-toggle.btn-default.focus {
  color: #333;
  background-color: #d4d4d4;
  border-color: #8c8c8c;
}
.btn-default:active,
.btn-default.active,
.open > .dropdown-toggle.btn-default {
  background-image: none;
}
.btn-default.disabled:hover,
.btn-default[disabled]:hover,
fieldset[disabled] .btn-default:hover,
.btn-default.disabled:focus,
.btn-default[disabled]:focus,
fieldset[disabled] .btn-default:focus,
.btn-default.disabled.focus,
.btn-default[disabled].focus,
fieldset[disabled] .btn-default.focus {
  background-color: #fff;
  border-color: #ccc;
}
.btn-default .badge {
  color: #fff;
  background-color: #333;
}
.btn-primary {
  color: #fff;
  background-color: #337ab7;
  border-color: #2e6da4;
}
.btn-primary:focus,
.btn-primary.focus {
  color: #fff;
  background-color: #286090;
  border-color: #122b40;
}
.btn-primary:hover {
  color: #fff;
  background-color: #286090;
  border-color: #204d74;
}
.btn-primary:active,
.btn-primary.active,
.open > .dropdown-toggle.btn-primary {
  color: #fff;
  background-color: #286090;
  border-color: #204d74;
}
.btn-primary:active:hover,
.btn-primary.active:hover,
.open > .dropdown-toggle.btn-primary:hover,
.btn-primary:active:focus,
.btn-primary.active:focus,
.open > .dropdown-toggle.btn-primary:focus,
.btn-primary:active.focus,
.btn-primary.active.focus,
.open > .dropdown-toggle.btn-primary.focus {
  color: #fff;
  background-color: #204d74;
  border-color: #122b40;
}
.btn-primary:active,
.btn-primary.active,
.open > .dropdown-toggle.btn-primary {
  background-image: none;
}
.btn-primary.disabled:hover,
.btn-primary[disabled]:hover,
fieldset[disabled] .btn-primary:hover,
.btn-primary.disabled:focus,
.btn-primary[disabled]:focus,
fieldset[disabled] .btn-primary:focus,
.btn-primary.disabled.focus,
.btn-primary[disabled].focus,
fieldset[disabled] .btn-primary.focus {
  background-color: #337ab7;
  border-color: #2e6da4;
}
.btn-primary .badge {
  color: #337ab7;
  background-color: #fff;
}
.btn-success {
  color: #fff;
  background-color: #5cb85c;
  border-color: #4cae4c;
}
.btn-success:focus,
.btn-success.focus {
  color: #fff;
  background-color: #449d44;
  border-color: #255625;
}
.btn-success:hover {
  color: #fff;
  background-color: #449d44;
  border-color: #398439;
}
.btn-success:active,
.btn-success.active,
.open > .dropdown-toggle.btn-success {
  color: #fff;
  background-color: #449d44;
  border-color: #398439;
}
.btn-success:active:hover,
.btn-success.active:hover,
.open > .dropdown-toggle.btn-success:hover,
.btn-success:active:focus,
.btn-success.active:focus,
.open > .dropdown-toggle.btn-success:focus,
.btn-success:active.focus,
.btn-success.active.focus,
.open > .dropdown-toggle.btn-success.focus {
  color: #fff;
  background-color: #398439;
  border-color: #255625;
}
.btn-success:active,
.btn-success.active,
.open > .dropdown-toggle.btn-success {
  background-image: none;
}
.btn-success.disabled:hover,
.btn-success[disabled]:hover,
fieldset[disabled] .btn-success:hover,
.btn-success.disabled:focus,
.btn-success[disabled]:focus,
fieldset[disabled] .btn-success:focus,
.btn-success.disabled.focus,
.btn-success[disabled].focus,
fieldset[disabled] .btn-success.focus {
  background-color: #5cb85c;
  border-color: #4cae4c;
}
.btn-success .badge {
  color: #5cb85c;
  background-color: #fff;
}
.btn-info {
  color: #fff;
  background-color: #5bc0de;
  border-color: #46b8da;
}
.btn-info:focus,
.btn-info.focus {
  color: #fff;
  background-color: #31b0d5;
  border-color: #1b6d85;
}
.btn-info:hover {
  color: #fff;
  background-color: #31b0d5;
  border-color: #269abc;
}
.btn-info:active,
.btn-info.active,
.open > .dropdown-toggle.btn-info {
  color: #fff;
  background-color: #31b0d5;
  border-color: #269abc;
}
.btn-info:active:hover,
.btn-info.active:hover,
.open > .dropdown-toggle.btn-info:hover,
.btn-info:active:focus,
.btn-info.active:focus,
.open > .dropdown-toggle.btn-info:focus,
.btn-info:active.focus,
.btn-info.active.focus,
.open > .dropdown-toggle.btn-info.focus {
  color: #fff;
  background-color: #269abc;
  border-color: #1b6d85;
}
.btn-info:active,
.btn-info.active,
.open > .dropdown-toggle.btn-info {
  background-image: none;
}
.btn-info.disabled:hover,
.btn-info[disabled]:hover,
fieldset[disabled] .btn-info:hover,
.btn-info.disabled:focus,
.btn-info[disabled]:focus,
fieldset[disabled] .btn-info:focus,
.btn-info.disabled.focus,
.btn-info[disabled].focus,
fieldset[disabled] .btn-info.focus {
  background-color: #5bc0de;
  border-color: #46b8da;
}
.btn-info .badge {
  color: #5bc0de;
  background-color: #fff;
}
.btn-warning {
  color: #fff;
  background-color: #f0ad4e;
  border-color: #eea236;
}
.btn-warning:focus,
.btn-warning.focus {
  color: #fff;
  background-color: #ec971f;
  border-color: #985f0d;
}
.btn-warning:hover {
  color: #fff;
  background-color: #ec971f;
  border-color: #d58512;
}
.btn-warning:active,
.btn-warning.active,
.open > .dropdown-toggle.btn-warning {
  color: #fff;
  background-color: #ec971f;
  border-color: #d58512;
}
.btn-warning:active:hover,
.btn-warning.active:hover,
.open > .dropdown-toggle.btn-warning:hover,
.btn-warning:active:focus,
.btn-warning.active:focus,
.open > .dropdown-toggle.btn-warning:focus,
.btn-warning:active.focus,
.btn-warning.active.focus,
.open > .dropdown-toggle.btn-warning.focus {
  color: #fff;
  background-color: #d58512;
  border-color: #985f0d;
}
.btn-warning:active,
.btn-warning.active,
.open > .dropdown-toggle.btn-warning {
  background-image: none;
}
.btn-warning.disabled:hover,
.btn-warning[disabled]:hover,
fieldset[disabled] .btn-warning:hover,
.btn-warning.disabled:focus,
.btn-warning[disabled]:focus,
fieldset[disabled] .btn-warning:focus,
.btn-warning.disabled.focus,
.btn-warning[disabled].focus,
fieldset[disabled] .btn-warning.focus {
  background-color: #f0ad4e;
  border-color: #eea236;
}
.btn-warning .badge {
  color: #f0ad4e;
  background-color: #fff;
}
.btn-danger {
  color: #fff;
  background-color: #d9534f;
  border-color: #d43f3a;
}
.btn-danger:focus,
.btn-danger.focus {
  color: #fff;
  background-color: #c9302c;
  border-color: #761c19;
}
.btn-danger:hover {
  color: #fff;
  background-color: #c9302c;
  border-color: #ac2925;
}
.btn-danger:active,
.btn-danger.active,
.open > .dropdown-toggle.btn-danger {
  color: #fff;
  background-color: #c9302c;
  border-color: #ac2925;
}
.btn-danger:active:hover,
.btn-danger.active:hover,
.open > .dropdown-toggle.btn-danger:hover,
.btn-danger:active:focus,
.btn-danger.active:focus,
.open > .dropdown-toggle.btn-danger:focus,
.btn-danger:active.focus,
.btn-danger.active.focus,
.open > .dropdown-toggle.btn-danger.focus {
  color: #fff;
  background-color: #ac2925;
  border-color: #761c19;
}
.btn-danger:active,
.btn-danger.active,
.open > .dropdown-toggle.btn-danger {
  background-image: none;
}
.btn-danger.disabled:hover,
.btn-danger[disabled]:hover,
fieldset[disabled] .btn-danger:hover,
.btn-danger.disabled:focus,
.btn-danger[disabled]:focus,
fieldset[disabled] .btn-danger:focus,
.btn-danger.disabled.focus,
.btn-danger[disabled].focus,
fieldset[disabled] .btn-danger.focus {
  background-color: #d9534f;
  border-color: #d43f3a;
}
.btn-danger .badge {
  color: #d9534f;
  background-color: #fff;
}
.btn-link {
  color: #337ab7;
  font-weight: normal;
  border-radius: 0;
}
.btn-link,
.btn-link:active,
.btn-link.active,
.btn-link[disabled],
fieldset[disabled] .btn-link {
  background-color: transparent;
  -webkit-box-shadow: none;
  box-shadow: none;
}
.btn-link,
.btn-link:hover,
.btn-link:focus,
.btn-link:active {
  border-color: transparent;
}
.btn-link:hover,
.btn-link:focus {
  color: #23527c;
  text-decoration: underline;
  background-color: transparent;
}
.btn-link[disabled]:hover,
fieldset[disabled] .btn-link:hover,
.btn-link[disabled]:focus,
fieldset[disabled] .btn-link:focus {
  color: #777777;
  text-decoration: none;
}
.btn-lg,
.btn-group-lg > .btn {
  padding: 10px 16px;
  font-size: 17px;
  line-height: 1.3333333;
  border-radius: 3px;
}
.btn-sm,
.btn-group-sm > .btn {
  padding: 5px 10px;
  font-size: 12px;
  line-height: 1.5;
  border-radius: 1px;
}
.btn-xs,
.btn-group-xs > .btn {
  padding: 1px 5px;
  font-size: 12px;
  line-height: 1.5;
  border-radius: 1px;
}
.btn-block {
  display: block;
  width: 100%;
}
.btn-block + .btn-block {
  margin-top: 5px;
}
input[type="submit"].btn-block,
input[type="reset"].btn-block,
input[type="button"].btn-block {
  width: 100%;
}
.fade {
  opacity: 0;
  -webkit-transition: opacity 0.15s linear;
  -o-transition: opacity 0.15s linear;
  transition: opacity 0.15s linear;
}
.fade.in {
  opacity: 1;
}
.collapse {
  display: none;
}
.collapse.in {
  display: block;
}
tr.collapse.in {
  display: table-row;
}
tbody.collapse.in {
  display: table-row-group;
}
.collapsing {
  position: relative;
  height: 0;
  overflow: hidden;
  -webkit-transition-property: height, visibility;
  transition-property: height, visibility;
  -webkit-transition-duration: 0.35s;
  transition-duration: 0.35s;
  -webkit-transition-timing-function: ease;
  transition-timing-function: ease;
}
.caret {
  display: inline-block;
  width: 0;
  height: 0;
  margin-left: 2px;
  vertical-align: middle;
  border-top: 4px dashed;
  border-top: 4px solid \9;
  border-right: 4px solid transparent;
  border-left: 4px solid transparent;
}
.dropup,
.dropdown {
  position: relative;
}
.dropdown-toggle:focus {
  outline: 0;
}
.dropdown-menu {
  position: absolute;
  top: 100%;
  left: 0;
  z-index: 1000;
  display: none;
  float: left;
  min-width: 160px;
  padding: 5px 0;
  margin: 2px 0 0;
  list-style: none;
  font-size: 13px;
  text-align: left;
  background-color: #fff;
  border: 1px solid #ccc;
  border: 1px solid rgba(0, 0, 0, 0.15);
  border-radius: 2px;
  -webkit-box-shadow: 0 6px 12px rgba(0, 0, 0, 0.175);
  box-shadow: 0 6px 12px rgba(0, 0, 0, 0.175);
  background-clip: padding-box;
}
.dropdown-menu.pull-right {
  right: 0;
  left: auto;
}
.dropdown-menu .divider {
  height: 1px;
  margin: 8px 0;
  overflow: hidden;
  background-color: #e5e5e5;
}
.dropdown-menu > li > a {
  display: block;
  padding: 3px 20px;
  clear: both;
  font-weight: normal;
  line-height: 1.42857143;
  color: #333333;
  white-space: nowrap;
}
.dropdown-menu > li > a:hover,
.dropdown-menu > li > a:focus {
  text-decoration: none;
  color: #262626;
  background-color: #f5f5f5;
}
.dropdown-menu > .active > a,
.dropdown-menu > .active > a:hover,
.dropdown-menu > .active > a:focus {
  color: #fff;
  text-decoration: none;
  outline: 0;
  background-color: #337ab7;
}
.dropdown-menu > .disabled > a,
.dropdown-menu > .disabled > a:hover,
.dropdown-menu > .disabled > a:focus {
  color: #777777;
}
.dropdown-menu > .disabled > a:hover,
.dropdown-menu > .disabled > a:focus {
  text-decoration: none;
  background-color: transparent;
  background-image: none;
  filter: progid:DXImageTransform.Microsoft.gradient(enabled = false);
  cursor: not-allowed;
}
.open > .dropdown-menu {
  display: block;
}
.open > a {
  outline: 0;
}
.dropdown-menu-right {
  left: auto;
  right: 0;
}
.dropdown-menu-left {
  left: 0;
  right: auto;
}
.dropdown-header {
  display: block;
  padding: 3px 20px;
  font-size: 12px;
  line-height: 1.42857143;
  color: #777777;
  white-space: nowrap;
}
.dropdown-backdrop {
  position: fixed;
  left: 0;
  right: 0;
  bottom: 0;
  top: 0;
  z-index: 990;
}
.pull-right > .dropdown-menu {
  right: 0;
  left: auto;
}
.dropup .caret,
.navbar-fixed-bottom .dropdown .caret {
  border-top: 0;
  border-bottom: 4px dashed;
  border-bottom: 4px solid \9;
  content: "";
}
.dropup .dropdown-menu,
.navbar-fixed-bottom .dropdown .dropdown-menu {
  top: auto;
  bottom: 100%;
  margin-bottom: 2px;
}
@media (min-width: 541px) {
  .navbar-right .dropdown-menu {
    left: auto;
    right: 0;
  }
  .navbar-right .dropdown-menu-left {
    left: 0;
    right: auto;
  }
}
.btn-group,
.btn-group-vertical {
  position: relative;
  display: inline-block;
  vertical-align: middle;
}
.btn-group > .btn,
.btn-group-vertical > .btn {
  position: relative;
  float: left;
}
.btn-group > .btn:hover,
.btn-group-vertical > .btn:hover,
.btn-group > .btn:focus,
.btn-group-vertical > .btn:focus,
.btn-group > .btn:active,
.btn-group-vertical > .btn:active,
.btn-group > .btn.active,
.btn-group-vertical > .btn.active {
  z-index: 2;
}
.btn-group .btn + .btn,
.btn-group .btn + .btn-group,
.btn-group .btn-group + .btn,
.btn-group .btn-group + .btn-group {
  margin-left: -1px;
}
.btn-toolbar {
  margin-left: -5px;
}
.btn-toolbar .btn,
.btn-toolbar .btn-group,
.btn-toolbar .input-group {
  float: left;
}
.btn-toolbar > .btn,
.btn-toolbar > .btn-group,
.btn-toolbar > .input-group {
  margin-left: 5px;
}
.btn-group > .btn:not(:first-child):not(:last-child):not(.dropdown-toggle) {
  border-radius: 0;
}
.btn-group > .btn:first-child {
  margin-left: 0;
}
.btn-group > .btn:first-child:not(:last-child):not(.dropdown-toggle) {
  border-bottom-right-radius: 0;
  border-top-right-radius: 0;
}
.btn-group > .btn:last-child:not(:first-child),
.btn-group > .dropdown-toggle:not(:first-child) {
  border-bottom-left-radius: 0;
  border-top-left-radius: 0;
}
.btn-group > .btn-group {
  float: left;
}
.btn-group > .btn-group:not(:first-child):not(:last-child) > .btn {
  border-radius: 0;
}
.btn-group > .btn-group:first-child:not(:last-child) > .btn:last-child,
.btn-group > .btn-group:first-child:not(:last-child) > .dropdown-toggle {
  border-bottom-right-radius: 0;
  border-top-right-radius: 0;
}
.btn-group > .btn-group:last-child:not(:first-child) > .btn:first-child {
  border-bottom-left-radius: 0;
  border-top-left-radius: 0;
}
.btn-group .dropdown-toggle:active,
.btn-group.open .dropdown-toggle {
  outline: 0;
}
.btn-group > .btn + .dropdown-toggle {
  padding-left: 8px;
  padding-right: 8px;
}
.btn-group > .btn-lg + .dropdown-toggle {
  padding-left: 12px;
  padding-right: 12px;
}
.btn-group.open .dropdown-toggle {
  -webkit-box-shadow: inset 0 3px 5px rgba(0, 0, 0, 0.125);
  box-shadow: inset 0 3px 5px rgba(0, 0, 0, 0.125);
}
.btn-group.open .dropdown-toggle.btn-link {
  -webkit-box-shadow: none;
  box-shadow: none;
}
.btn .caret {
  margin-left: 0;
}
.btn-lg .caret {
  border-width: 5px 5px 0;
  border-bottom-width: 0;
}
.dropup .btn-lg .caret {
  border-width: 0 5px 5px;
}
.btn-group-vertical > .btn,
.btn-group-vertical > .btn-group,
.btn-group-vertical > .btn-group > .btn {
  display: block;
  float: none;
  width: 100%;
  max-width: 100%;
}
.btn-group-vertical > .btn-group > .btn {
  float: none;
}
.btn-group-vertical > .btn + .btn,
.btn-group-vertical > .btn + .btn-group,
.btn-group-vertical > .btn-group + .btn,
.btn-group-vertical > .btn-group + .btn-group {
  margin-top: -1px;
  margin-left: 0;
}
.btn-group-vertical > .btn:not(:first-child):not(:last-child) {
  border-radius: 0;
}
.btn-group-vertical > .btn:first-child:not(:last-child) {
  border-top-right-radius: 2px;
  border-top-left-radius: 2px;
  border-bottom-right-radius: 0;
  border-bottom-left-radius: 0;
}
.btn-group-vertical > .btn:last-child:not(:first-child) {
  border-top-right-radius: 0;
  border-top-left-radius: 0;
  border-bottom-right-radius: 2px;
  border-bottom-left-radius: 2px;
}
.btn-group-vertical > .btn-group:not(:first-child):not(:last-child) > .btn {
  border-radius: 0;
}
.btn-group-vertical > .btn-group:first-child:not(:last-child) > .btn:last-child,
.btn-group-vertical > .btn-group:first-child:not(:last-child) > .dropdown-toggle {
  border-bottom-right-radius: 0;
  border-bottom-left-radius: 0;
}
.btn-group-vertical > .btn-group:last-child:not(:first-child) > .btn:first-child {
  border-top-right-radius: 0;
  border-top-left-radius: 0;
}
.btn-group-justified {
  display: table;
  width: 100%;
  table-layout: fixed;
  border-collapse: separate;
}
.btn-group-justified > .btn,
.btn-group-justified > .btn-group {
  float: none;
  display: table-cell;
  width: 1%;
}
.btn-group-justified > .btn-group .btn {
  width: 100%;
}
.btn-group-justified > .btn-group .dropdown-menu {
  left: auto;
}
[data-toggle="buttons"] > .btn input[type="radio"],
[data-toggle="buttons"] > .btn-group > .btn input[type="radio"],
[data-toggle="buttons"] > .btn input[type="checkbox"],
[data-toggle="buttons"] > .btn-group > .btn input[type="checkbox"] {
  position: absolute;
  clip: rect(0, 0, 0, 0);
  pointer-events: none;
}
.input-group {
  position: relative;
  display: table;
  border-collapse: separate;
}
.input-group[class*="col-"] {
  float: none;
  padding-left: 0;
  padding-right: 0;
}
.input-group .form-control {
  position: relative;
  z-index: 2;
  float: left;
  width: 100%;
  margin-bottom: 0;
}
.input-group .form-control:focus {
  z-index: 3;
}
.input-group-lg > .form-control,
.input-group-lg > .input-group-addon,
.input-group-lg > .input-group-btn > .btn {
  height: 45px;
  padding: 10px 16px;
  font-size: 17px;
  line-height: 1.3333333;
  border-radius: 3px;
}
select.input-group-lg > .form-control,
select.input-group-lg > .input-group-addon,
select.input-group-lg > .input-group-btn > .btn {
  height: 45px;
  line-height: 45px;
}
textarea.input-group-lg > .form-control,
textarea.input-group-lg > .input-group-addon,
textarea.input-group-lg > .input-group-btn > .btn,
select[multiple].input-group-lg > .form-control,
select[multiple].input-group-lg > .input-group-addon,
select[multiple].input-group-lg > .input-group-btn > .btn {
  height: auto;
}
.input-group-sm > .form-control,
.input-group-sm > .input-group-addon,
.input-group-sm > .input-group-btn > .btn {
  height: 30px;
  padding: 5px 10px;
  font-size: 12px;
  line-height: 1.5;
  border-radius: 1px;
}
select.input-group-sm > .form-control,
select.input-group-sm > .input-group-addon,
select.input-group-sm > .input-group-btn > .btn {
  height: 30px;
  line-height: 30px;
}
textarea.input-group-sm > .form-control,
textarea.input-group-sm > .input-group-addon,
textarea.input-group-sm > .input-group-btn > .btn,
select[multiple].input-group-sm > .form-control,
select[multiple].input-group-sm > .input-group-addon,
select[multiple].input-group-sm > .input-group-btn > .btn {
  height: auto;
}
.input-group-addon,
.input-group-btn,
.input-group .form-control {
  display: table-cell;
}
.input-group-addon:not(:first-child):not(:last-child),
.input-group-btn:not(:first-child):not(:last-child),
.input-group .form-control:not(:first-child):not(:last-child) {
  border-radius: 0;
}
.input-group-addon,
.input-group-btn {
  width: 1%;
  white-space: nowrap;
  vertical-align: middle;
}
.input-group-addon {
  padding: 6px 12px;
  font-size: 13px;
  font-weight: normal;
  line-height: 1;
  color: #555555;
  text-align: center;
  background-color: #eeeeee;
  border: 1px solid #ccc;
  border-radius: 2px;
}
.input-group-addon.input-sm {
  padding: 5px 10px;
  font-size: 12px;
  border-radius: 1px;
}
.input-group-addon.input-lg {
  padding: 10px 16px;
  font-size: 17px;
  border-radius: 3px;
}
.input-group-addon input[type="radio"],
.input-group-addon input[type="checkbox"] {
  margin-top: 0;
}
.input-group .form-control:first-child,
.input-group-addon:first-child,
.input-group-btn:first-child > .btn,
.input-group-btn:first-child > .btn-group > .btn,
.input-group-btn:first-child > .dropdown-toggle,
.input-group-btn:last-child > .btn:not(:last-child):not(.dropdown-toggle),
.input-group-btn:last-child > .btn-group:not(:last-child) > .btn {
  border-bottom-right-radius: 0;
  border-top-right-radius: 0;
}
.input-group-addon:first-child {
  border-right: 0;
}
.input-group .form-control:last-child,
.input-group-addon:last-child,
.input-group-btn:last-child > .btn,
.input-group-btn:last-child > .btn-group > .btn,
.input-group-btn:last-child > .dropdown-toggle,
.input-group-btn:first-child > .btn:not(:first-child),
.input-group-btn:first-child > .btn-group:not(:first-child) > .btn {
  border-bottom-left-radius: 0;
  border-top-left-radius: 0;
}
.input-group-addon:last-child {
  border-left: 0;
}
.input-group-btn {
  position: relative;
  font-size: 0;
  white-space: nowrap;
}
.input-group-btn > .btn {
  position: relative;
}
.input-group-btn > .btn + .btn {
  margin-left: -1px;
}
.input-group-btn > .btn:hover,
.input-group-btn > .btn:focus,
.input-group-btn > .btn:active {
  z-index: 2;
}
.input-group-btn:first-child > .btn,
.input-group-btn:first-child > .btn-group {
  margin-right: -1px;
}
.input-group-btn:last-child > .btn,
.input-group-btn:last-child > .btn-group {
  z-index: 2;
  margin-left: -1px;
}
.nav {
  margin-bottom: 0;
  padding-left: 0;
  list-style: none;
}
.nav > li {
  position: relative;
  display: block;
}
.nav > li > a {
  position: relative;
  display: block;
  padding: 10px 15px;
}
.nav > li > a:hover,
.nav > li > a:focus {
  text-decoration: none;
  background-color: #eeeeee;
}
.nav > li.disabled > a {
  color: #777777;
}
.nav > li.disabled > a:hover,
.nav > li.disabled > a:focus {
  color: #777777;
  text-decoration: none;
  background-color: transparent;
  cursor: not-allowed;
}
.nav .open > a,
.nav .open > a:hover,
.nav .open > a:focus {
  background-color: #eeeeee;
  border-color: #337ab7;
}
.nav .nav-divider {
  height: 1px;
  margin: 8px 0;
  overflow: hidden;
  background-color: #e5e5e5;
}
.nav > li > a > img {
  max-width: none;
}
.nav-tabs {
  border-bottom: 1px solid #ddd;
}
.nav-tabs > li {
  float: left;
  margin-bottom: -1px;
}
.nav-tabs > li > a {
  margin-right: 2px;
  line-height: 1.42857143;
  border: 1px solid transparent;
  border-radius: 2px 2px 0 0;
}
.nav-tabs > li > a:hover {
  border-color: #eeeeee #eeeeee #ddd;
}
.nav-tabs > li.active > a,
.nav-tabs > li.active > a:hover,
.nav-tabs > li.active > a:focus {
  color: #555555;
  background-color: #fff;
  border: 1px solid #ddd;
  border-bottom-color: transparent;
  cursor: default;
}
.nav-tabs.nav-justified {
  width: 100%;
  border-bottom: 0;
}
.nav-tabs.nav-justified > li {
  float: none;
}
.nav-tabs.nav-justified > li > a {
  text-align: center;
  margin-bottom: 5px;
}
.nav-tabs.nav-justified > .dropdown .dropdown-menu {
  top: auto;
  left: auto;
}
@media (min-width: 768px) {
  .nav-tabs.nav-justified > li {
    display: table-cell;
    width: 1%;
  }
  .nav-tabs.nav-justified > li > a {
    margin-bottom: 0;
  }
}
.nav-tabs.nav-justified > li > a {
  margin-right: 0;
  border-radius: 2px;
}
.nav-tabs.nav-justified > .active > a,
.nav-tabs.nav-justified > .active > a:hover,
.nav-tabs.nav-justified > .active > a:focus {
  border: 1px solid #ddd;
}
@media (min-width: 768px) {
  .nav-tabs.nav-justified > li > a {
    border-bottom: 1px solid #ddd;
    border-radius: 2px 2px 0 0;
  }
  .nav-tabs.nav-justified > .active > a,
  .nav-tabs.nav-justified > .active > a:hover,
  .nav-tabs.nav-justified > .active > a:focus {
    border-bottom-color: #fff;
  }
}
.nav-pills > li {
  float: left;
}
.nav-pills > li > a {
  border-radius: 2px;
}
.nav-pills > li + li {
  margin-left: 2px;
}
.nav-pills > li.active > a,
.nav-pills > li.active > a:hover,
.nav-pills > li.active > a:focus {
  color: #fff;
  background-color: #337ab7;
}
.nav-stacked > li {
  float: none;
}
.nav-stacked > li + li {
  margin-top: 2px;
  margin-left: 0;
}
.nav-justified {
  width: 100%;
}
.nav-justified > li {
  float: none;
}
.nav-justified > li > a {
  text-align: center;
  margin-bottom: 5px;
}
.nav-justified > .dropdown .dropdown-menu {
  top: auto;
  left: auto;
}
@media (min-width: 768px) {
  .nav-justified > li {
    display: table-cell;
    width: 1%;
  }
  .nav-justified > li > a {
    margin-bottom: 0;
  }
}
.nav-tabs-justified {
  border-bottom: 0;
}
.nav-tabs-justified > li > a {
  margin-right: 0;
  border-radius: 2px;
}
.nav-tabs-justified > .active > a,
.nav-tabs-justified > .active > a:hover,
.nav-tabs-justified > .active > a:focus {
  border: 1px solid #ddd;
}
@media (min-width: 768px) {
  .nav-tabs-justified > li > a {
    border-bottom: 1px solid #ddd;
    border-radius: 2px 2px 0 0;
  }
  .nav-tabs-justified > .active > a,
  .nav-tabs-justified > .active > a:hover,
  .nav-tabs-justified > .active > a:focus {
    border-bottom-color: #fff;
  }
}
.tab-content > .tab-pane {
  display: none;
}
.tab-content > .active {
  display: block;
}
.nav-tabs .dropdown-menu {
  margin-top: -1px;
  border-top-right-radius: 0;
  border-top-left-radius: 0;
}
.navbar {
  position: relative;
  min-height: 30px;
  margin-bottom: 18px;
  border: 1px solid transparent;
}
@media (min-width: 541px) {
  .navbar {
    border-radius: 2px;
  }
}
@media (min-width: 541px) {
  .navbar-header {
    float: left;
  }
}
.navbar-collapse {
  overflow-x: visible;
  padding-right: 0px;
  padding-left: 0px;
  border-top: 1px solid transparent;
  box-shadow: inset 0 1px 0 rgba(255, 255, 255, 0.1);
  -webkit-overflow-scrolling: touch;
}
.navbar-collapse.in {
  overflow-y: auto;
}
@media (min-width: 541px) {
  .navbar-collapse {
    width: auto;
    border-top: 0;
    box-shadow: none;
  }
  .navbar-collapse.collapse {
    display: block !important;
    height: auto !important;
    padding-bottom: 0;
    overflow: visible !important;
  }
  .navbar-collapse.in {
    overflow-y: visible;
  }
  .navbar-fixed-top .navbar-collapse,
  .navbar-static-top .navbar-collapse,
  .navbar-fixed-bottom .navbar-collapse {
    padding-left: 0;
    padding-right: 0;
  }
}
.navbar-fixed-top .navbar-collapse,
.navbar-fixed-bottom .navbar-collapse {
  max-height: 340px;
}
@media (max-device-width: 540px) and (orientation: landscape) {
  .navbar-fixed-top .navbar-collapse,
  .navbar-fixed-bottom .navbar-collapse {
    max-height: 200px;
  }
}
.container > .navbar-header,
.container-fluid > .navbar-header,
.container > .navbar-collapse,
.container-fluid > .navbar-collapse {
  margin-right: 0px;
  margin-left: 0px;
}
@media (min-width: 541px) {
  .container > .navbar-header,
  .container-fluid > .navbar-header,
  .container > .navbar-collapse,
  .container-fluid > .navbar-collapse {
    margin-right: 0;
    margin-left: 0;
  }
}
.navbar-static-top {
  z-index: 1000;
  border-width: 0 0 1px;
}
@media (min-width: 541px) {
  .navbar-static-top {
    border-radius: 0;
  }
}
.navbar-fixed-top,
.navbar-fixed-bottom {
  position: fixed;
  right: 0;
  left: 0;
  z-index: 1030;
}
@media (min-width: 541px) {
  .navbar-fixed-top,
  .navbar-fixed-bottom {
    border-radius: 0;
  }
}
.navbar-fixed-top {
  top: 0;
  border-width: 0 0 1px;
}
.navbar-fixed-bottom {
  bottom: 0;
  margin-bottom: 0;
  border-width: 1px 0 0;
}
.navbar-brand {
  float: left;
  padding: 6px 0px;
  font-size: 17px;
  line-height: 18px;
  height: 30px;
}
.navbar-brand:hover,
.navbar-brand:focus {
  text-decoration: none;
}
.navbar-brand > img {
  display: block;
}
@media (min-width: 541px) {
  .navbar > .container .navbar-brand,
  .navbar > .container-fluid .navbar-brand {
    margin-left: 0px;
  }
}
.navbar-toggle {
  position: relative;
  float: right;
  margin-right: 0px;
  padding: 9px 10px;
  margin-top: -2px;
  margin-bottom: -2px;
  background-color: transparent;
  background-image: none;
  border: 1px solid transparent;
  border-radius: 2px;
}
.navbar-toggle:focus {
  outline: 0;
}
.navbar-toggle .icon-bar {
  display: block;
  width: 22px;
  height: 2px;
  border-radius: 1px;
}
.navbar-toggle .icon-bar + .icon-bar {
  margin-top: 4px;
}
@media (min-width: 541px) {
  .navbar-toggle {
    display: none;
  }
}
.navbar-nav {
  margin: 3px 0px;
}
.navbar-nav > li > a {
  padding-top: 10px;
  padding-bottom: 10px;
  line-height: 18px;
}
@media (max-width: 540px) {
  .navbar-nav .open .dropdown-menu {
    position: static;
    float: none;
    width: auto;
    margin-top: 0;
    background-color: transparent;
    border: 0;
    box-shadow: none;
  }
  .navbar-nav .open .dropdown-menu > li > a,
  .navbar-nav .open .dropdown-menu .dropdown-header {
    padding: 5px 15px 5px 25px;
  }
  .navbar-nav .open .dropdown-menu > li > a {
    line-height: 18px;
  }
  .navbar-nav .open .dropdown-menu > li > a:hover,
  .navbar-nav .open .dropdown-menu > li > a:focus {
    background-image: none;
  }
}
@media (min-width: 541px) {
  .navbar-nav {
    float: left;
    margin: 0;
  }
  .navbar-nav > li {
    float: left;
  }
  .navbar-nav > li > a {
    padding-top: 6px;
    padding-bottom: 6px;
  }
}
.navbar-form {
  margin-left: 0px;
  margin-right: 0px;
  padding: 10px 0px;
  border-top: 1px solid transparent;
  border-bottom: 1px solid transparent;
  -webkit-box-shadow: inset 0 1px 0 rgba(255, 255, 255, 0.1), 0 1px 0 rgba(255, 255, 255, 0.1);
  box-shadow: inset 0 1px 0 rgba(255, 255, 255, 0.1), 0 1px 0 rgba(255, 255, 255, 0.1);
  margin-top: -1px;
  margin-bottom: -1px;
}
@media (min-width: 768px) {
  .navbar-form .form-group {
    display: inline-block;
    margin-bottom: 0;
    vertical-align: middle;
  }
  .navbar-form .form-control {
    display: inline-block;
    width: auto;
    vertical-align: middle;
  }
  .navbar-form .form-control-static {
    display: inline-block;
  }
  .navbar-form .input-group {
    display: inline-table;
    vertical-align: middle;
  }
  .navbar-form .input-group .input-group-addon,
  .navbar-form .input-group .input-group-btn,
  .navbar-form .input-group .form-control {
    width: auto;
  }
  .navbar-form .input-group > .form-control {
    width: 100%;
  }
  .navbar-form .control-label {
    margin-bottom: 0;
    vertical-align: middle;
  }
  .navbar-form .radio,
  .navbar-form .checkbox {
    display: inline-block;
    margin-top: 0;
    margin-bottom: 0;
    vertical-align: middle;
  }
  .navbar-form .radio label,
  .navbar-form .checkbox label {
    padding-left: 0;
  }
  .navbar-form .radio input[type="radio"],
  .navbar-form .checkbox input[type="checkbox"] {
    position: relative;
    margin-left: 0;
  }
  .navbar-form .has-feedback .form-control-feedback {
    top: 0;
  }
}
@media (max-width: 540px) {
  .navbar-form .form-group {
    margin-bottom: 5px;
  }
  .navbar-form .form-group:last-child {
    margin-bottom: 0;
  }
}
@media (min-width: 541px) {
  .navbar-form {
    width: auto;
    border: 0;
    margin-left: 0;
    margin-right: 0;
    padding-top: 0;
    padding-bottom: 0;
    -webkit-box-shadow: none;
    box-shadow: none;
  }
}
.navbar-nav > li > .dropdown-menu {
  margin-top: 0;
  border-top-right-radius: 0;
  border-top-left-radius: 0;
}
.navbar-fixed-bottom .navbar-nav > li > .dropdown-menu {
  margin-bottom: 0;
  border-top-right-radius: 2px;
  border-top-left-radius: 2px;
  border-bottom-right-radius: 0;
  border-bottom-left-radius: 0;
}
.navbar-btn {
  margin-top: -1px;
  margin-bottom: -1px;
}
.navbar-btn.btn-sm {
  margin-top: 0px;
  margin-bottom: 0px;
}
.navbar-btn.btn-xs {
  margin-top: 4px;
  margin-bottom: 4px;
}
.navbar-text {
  margin-top: 6px;
  margin-bottom: 6px;
}
@media (min-width: 541px) {
  .navbar-text {
    float: left;
    margin-left: 0px;
    margin-right: 0px;
  }
}
@media (min-width: 541px) {
  .navbar-left {
    float: left !important;
    float: left;
  }
  .navbar-right {
    float: right !important;
    float: right;
    margin-right: 0px;
  }
  .navbar-right ~ .navbar-right {
    margin-right: 0;
  }
}
.navbar-default {
  background-color: #f8f8f8;
  border-color: #e7e7e7;
}
.navbar-default .navbar-brand {
  color: #777;
}
.navbar-default .navbar-brand:hover,
.navbar-default .navbar-brand:focus {
  color: #5e5e5e;
  background-color: transparent;
}
.navbar-default .navbar-text {
  color: #777;
}
.navbar-default .navbar-nav > li > a {
  color: #777;
}
.navbar-default .navbar-nav > li > a:hover,
.navbar-default .navbar-nav > li > a:focus {
  color: #333;
  background-color: transparent;
}
.navbar-default .navbar-nav > .active > a,
.navbar-default .navbar-nav > .active > a:hover,
.navbar-default .navbar-nav > .active > a:focus {
  color: #555;
  background-color: #e7e7e7;
}
.navbar-default .navbar-nav > .disabled > a,
.navbar-default .navbar-nav > .disabled > a:hover,
.navbar-default .navbar-nav > .disabled > a:focus {
  color: #ccc;
  background-color: transparent;
}
.navbar-default .navbar-toggle {
  border-color: #ddd;
}
.navbar-default .navbar-toggle:hover,
.navbar-default .navbar-toggle:focus {
  background-color: #ddd;
}
.navbar-default .navbar-toggle .icon-bar {
  background-color: #888;
}
.navbar-default .navbar-collapse,
.navbar-default .navbar-form {
  border-color: #e7e7e7;
}
.navbar-default .navbar-nav > .open > a,
.navbar-default .navbar-nav > .open > a:hover,
.navbar-default .navbar-nav > .open > a:focus {
  background-color: #e7e7e7;
  color: #555;
}
@media (max-width: 540px) {
  .navbar-default .navbar-nav .open .dropdown-menu > li > a {
    color: #777;
  }
  .navbar-default .navbar-nav .open .dropdown-menu > li > a:hover,
  .navbar-default .navbar-nav .open .dropdown-menu > li > a:focus {
    color: #333;
    background-color: transparent;
  }
  .navbar-default .navbar-nav .open .dropdown-menu > .active > a,
  .navbar-default .navbar-nav .open .dropdown-menu > .active > a:hover,
  .navbar-default .navbar-nav .open .dropdown-menu > .active > a:focus {
    color: #555;
    background-color: #e7e7e7;
  }
  .navbar-default .navbar-nav .open .dropdown-menu > .disabled > a,
  .navbar-default .navbar-nav .open .dropdown-menu > .disabled > a:hover,
  .navbar-default .navbar-nav .open .dropdown-menu > .disabled > a:focus {
    color: #ccc;
    background-color: transparent;
  }
}
.navbar-default .navbar-link {
  color: #777;
}
.navbar-default .navbar-link:hover {
  color: #333;
}
.navbar-default .btn-link {
  color: #777;
}
.navbar-default .btn-link:hover,
.navbar-default .btn-link:focus {
  color: #333;
}
.navbar-default .btn-link[disabled]:hover,
fieldset[disabled] .navbar-default .btn-link:hover,
.navbar-default .btn-link[disabled]:focus,
fieldset[disabled] .navbar-default .btn-link:focus {
  color: #ccc;
}
.navbar-inverse {
  background-color: #222;
  border-color: #080808;
}
.navbar-inverse .navbar-brand {
  color: #9d9d9d;
}
.navbar-inverse .navbar-brand:hover,
.navbar-inverse .navbar-brand:focus {
  color: #fff;
  background-color: transparent;
}
.navbar-inverse .navbar-text {
  color: #9d9d9d;
}
.navbar-inverse .navbar-nav > li > a {
  color: #9d9d9d;
}
.navbar-inverse .navbar-nav > li > a:hover,
.navbar-inverse .navbar-nav > li > a:focus {
  color: #fff;
  background-color: transparent;
}
.navbar-inverse .navbar-nav > .active > a,
.navbar-inverse .navbar-nav > .active > a:hover,
.navbar-inverse .navbar-nav > .active > a:focus {
  color: #fff;
  background-color: #080808;
}
.navbar-inverse .navbar-nav > .disabled > a,
.navbar-inverse .navbar-nav > .disabled > a:hover,
.navbar-inverse .navbar-nav > .disabled > a:focus {
  color: #444;
  background-color: transparent;
}
.navbar-inverse .navbar-toggle {
  border-color: #333;
}
.navbar-inverse .navbar-toggle:hover,
.navbar-inverse .navbar-toggle:focus {
  background-color: #333;
}
.navbar-inverse .navbar-toggle .icon-bar {
  background-color: #fff;
}
.navbar-inverse .navbar-collapse,
.navbar-inverse .navbar-form {
  border-color: #101010;
}
.navbar-inverse .navbar-nav > .open > a,
.navbar-inverse .navbar-nav > .open > a:hover,
.navbar-inverse .navbar-nav > .open > a:focus {
  background-color: #080808;
  color: #fff;
}
@media (max-width: 540px) {
  .navbar-inverse .navbar-nav .open .dropdown-menu > .dropdown-header {
    border-color: #080808;
  }
  .navbar-inverse .navbar-nav .open .dropdown-menu .divider {
    background-color: #080808;
  }
  .navbar-inverse .navbar-nav .open .dropdown-menu > li > a {
    color: #9d9d9d;
  }
  .navbar-inverse .navbar-nav .open .dropdown-menu > li > a:hover,
  .navbar-inverse .navbar-nav .open .dropdown-menu > li > a:focus {
    color: #fff;
    background-color: transparent;
  }
  .navbar-inverse .navbar-nav .open .dropdown-menu > .active > a,
  .navbar-inverse .navbar-nav .open .dropdown-menu > .active > a:hover,
  .navbar-inverse .navbar-nav .open .dropdown-menu > .active > a:focus {
    color: #fff;
    background-color: #080808;
  }
  .navbar-inverse .navbar-nav .open .dropdown-menu > .disabled > a,
  .navbar-inverse .navbar-nav .open .dropdown-menu > .disabled > a:hover,
  .navbar-inverse .navbar-nav .open .dropdown-menu > .disabled > a:focus {
    color: #444;
    background-color: transparent;
  }
}
.navbar-inverse .navbar-link {
  color: #9d9d9d;
}
.navbar-inverse .navbar-link:hover {
  color: #fff;
}
.navbar-inverse .btn-link {
  color: #9d9d9d;
}
.navbar-inverse .btn-link:hover,
.navbar-inverse .btn-link:focus {
  color: #fff;
}
.navbar-inverse .btn-link[disabled]:hover,
fieldset[disabled] .navbar-inverse .btn-link:hover,
.navbar-inverse .btn-link[disabled]:focus,
fieldset[disabled] .navbar-inverse .btn-link:focus {
  color: #444;
}
.breadcrumb {
  padding: 8px 15px;
  margin-bottom: 18px;
  list-style: none;
  background-color: #f5f5f5;
  border-radius: 2px;
}
.breadcrumb > li {
  display: inline-block;
}
.breadcrumb > li + li:before {
  content: "/\00a0";
  padding: 0 5px;
  color: #5e5e5e;
}
.breadcrumb > .active {
  color: #777777;
}
.pagination {
  display: inline-block;
  padding-left: 0;
  margin: 18px 0;
  border-radius: 2px;
}
.pagination > li {
  display: inline;
}
.pagination > li > a,
.pagination > li > span {
  position: relative;
  float: left;
  padding: 6px 12px;
  line-height: 1.42857143;
  text-decoration: none;
  color: #337ab7;
  background-color: #fff;
  border: 1px solid #ddd;
  margin-left: -1px;
}
.pagination > li:first-child > a,
.pagination > li:first-child > span {
  margin-left: 0;
  border-bottom-left-radius: 2px;
  border-top-left-radius: 2px;
}
.pagination > li:last-child > a,
.pagination > li:last-child > span {
  border-bottom-right-radius: 2px;
  border-top-right-radius: 2px;
}
.pagination > li > a:hover,
.pagination > li > span:hover,
.pagination > li > a:focus,
.pagination > li > span:focus {
  z-index: 2;
  color: #23527c;
  background-color: #eeeeee;
  border-color: #ddd;
}
.pagination > .active > a,
.pagination > .active > span,
.pagination > .active > a:hover,
.pagination > .active > span:hover,
.pagination > .active > a:focus,
.pagination > .active > span:focus {
  z-index: 3;
  color: #fff;
  background-color: #337ab7;
  border-color: #337ab7;
  cursor: default;
}
.pagination > .disabled > span,
.pagination > .disabled > span:hover,
.pagination > .disabled > span:focus,
.pagination > .disabled > a,
.pagination > .disabled > a:hover,
.pagination > .disabled > a:focus {
  color: #777777;
  background-color: #fff;
  border-color: #ddd;
  cursor: not-allowed;
}
.pagination-lg > li > a,
.pagination-lg > li > span {
  padding: 10px 16px;
  font-size: 17px;
  line-height: 1.3333333;
}
.pagination-lg > li:first-child > a,
.pagination-lg > li:first-child > span {
  border-bottom-left-radius: 3px;
  border-top-left-radius: 3px;
}
.pagination-lg > li:last-child > a,
.pagination-lg > li:last-child > span {
  border-bottom-right-radius: 3px;
  border-top-right-radius: 3px;
}
.pagination-sm > li > a,
.pagination-sm > li > span {
  padding: 5px 10px;
  font-size: 12px;
  line-height: 1.5;
}
.pagination-sm > li:first-child > a,
.pagination-sm > li:first-child > span {
  border-bottom-left-radius: 1px;
  border-top-left-radius: 1px;
}
.pagination-sm > li:last-child > a,
.pagination-sm > li:last-child > span {
  border-bottom-right-radius: 1px;
  border-top-right-radius: 1px;
}
.pager {
  padding-left: 0;
  margin: 18px 0;
  list-style: none;
  text-align: center;
}
.pager li {
  display: inline;
}
.pager li > a,
.pager li > span {
  display: inline-block;
  padding: 5px 14px;
  background-color: #fff;
  border: 1px solid #ddd;
  border-radius: 15px;
}
.pager li > a:hover,
.pager li > a:focus {
  text-decoration: none;
  background-color: #eeeeee;
}
.pager .next > a,
.pager .next > span {
  float: right;
}
.pager .previous > a,
.pager .previous > span {
  float: left;
}
.pager .disabled > a,
.pager .disabled > a:hover,
.pager .disabled > a:focus,
.pager .disabled > span {
  color: #777777;
  background-color: #fff;
  cursor: not-allowed;
}
.label {
  display: inline;
  padding: .2em .6em .3em;
  font-size: 75%;
  font-weight: bold;
  line-height: 1;
  color: #fff;
  text-align: center;
  white-space: nowrap;
  vertical-align: baseline;
  border-radius: .25em;
}
a.label:hover,
a.label:focus {
  color: #fff;
  text-decoration: none;
  cursor: pointer;
}
.label:empty {
  display: none;
}
.btn .label {
  position: relative;
  top: -1px;
}
.label-default {
  background-color: #777777;
}
.label-default[href]:hover,
.label-default[href]:focus {
  background-color: #5e5e5e;
}
.label-primary {
  background-color: #337ab7;
}
.label-primary[href]:hover,
.label-primary[href]:focus {
  background-color: #286090;
}
.label-success {
  background-color: #5cb85c;
}
.label-success[href]:hover,
.label-success[href]:focus {
  background-color: #449d44;
}
.label-info {
  background-color: #5bc0de;
}
.label-info[href]:hover,
.label-info[href]:focus {
  background-color: #31b0d5;
}
.label-warning {
  background-color: #f0ad4e;
}
.label-warning[href]:hover,
.label-warning[href]:focus {
  background-color: #ec971f;
}
.label-danger {
  background-color: #d9534f;
}
.label-danger[href]:hover,
.label-danger[href]:focus {
  background-color: #c9302c;
}
.badge {
  display: inline-block;
  min-width: 10px;
  padding: 3px 7px;
  font-size: 12px;
  font-weight: bold;
  color: #fff;
  line-height: 1;
  vertical-align: middle;
  white-space: nowrap;
  text-align: center;
  background-color: #777777;
  border-radius: 10px;
}
.badge:empty {
  display: none;
}
.btn .badge {
  position: relative;
  top: -1px;
}
.btn-xs .badge,
.btn-group-xs > .btn .badge {
  top: 0;
  padding: 1px 5px;
}
a.badge:hover,
a.badge:focus {
  color: #fff;
  text-decoration: none;
  cursor: pointer;
}
.list-group-item.active > .badge,
.nav-pills > .active > a > .badge {
  color: #337ab7;
  background-color: #fff;
}
.list-group-item > .badge {
  float: right;
}
.list-group-item > .badge + .badge {
  margin-right: 5px;
}
.nav-pills > li > a > .badge {
  margin-left: 3px;
}
.jumbotron {
  padding-top: 30px;
  padding-bottom: 30px;
  margin-bottom: 30px;
  color: inherit;
  background-color: #eeeeee;
}
.jumbotron h1,
.jumbotron .h1 {
  color: inherit;
}
.jumbotron p {
  margin-bottom: 15px;
  font-size: 20px;
  font-weight: 200;
}
.jumbotron > hr {
  border-top-color: #d5d5d5;
}
.container .jumbotron,
.container-fluid .jumbotron {
  border-radius: 3px;
  padding-left: 0px;
  padding-right: 0px;
}
.jumbotron .container {
  max-width: 100%;
}
@media screen and (min-width: 768px) {
  .jumbotron {
    padding-top: 48px;
    padding-bottom: 48px;
  }
  .container .jumbotron,
  .container-fluid .jumbotron {
    padding-left: 60px;
    padding-right: 60px;
  }
  .jumbotron h1,
  .jumbotron .h1 {
    font-size: 59px;
  }
}
.thumbnail {
  display: block;
  padding: 4px;
  margin-bottom: 18px;
  line-height: 1.42857143;
  background-color: #fff;
  border: 1px solid #ddd;
  border-radius: 2px;
  -webkit-transition: border 0.2s ease-in-out;
  -o-transition: border 0.2s ease-in-out;
  transition: border 0.2s ease-in-out;
}
.thumbnail > img,
.thumbnail a > img {
  margin-left: auto;
  margin-right: auto;
}
a.thumbnail:hover,
a.thumbnail:focus,
a.thumbnail.active {
  border-color: #337ab7;
}
.thumbnail .caption {
  padding: 9px;
  color: #000;
}
.alert {
  padding: 15px;
  margin-bottom: 18px;
  border: 1px solid transparent;
  border-radius: 2px;
}
.alert h4 {
  margin-top: 0;
  color: inherit;
}
.alert .alert-link {
  font-weight: bold;
}
.alert > p,
.alert > ul {
  margin-bottom: 0;
}
.alert > p + p {
  margin-top: 5px;
}
.alert-dismissable,
.alert-dismissible {
  padding-right: 35px;
}
.alert-dismissable .close,
.alert-dismissible .close {
  position: relative;
  top: -2px;
  right: -21px;
  color: inherit;
}
.alert-success {
  background-color: #dff0d8;
  border-color: #d6e9c6;
  color: #3c763d;
}
.alert-success hr {
  border-top-color: #c9e2b3;
}
.alert-success .alert-link {
  color: #2b542c;
}
.alert-info {
  background-color: #d9edf7;
  border-color: #bce8f1;
  color: #31708f;
}
.alert-info hr {
  border-top-color: #a6e1ec;
}
.alert-info .alert-link {
  color: #245269;
}
.alert-warning {
  background-color: #fcf8e3;
  border-color: #faebcc;
  color: #8a6d3b;
}
.alert-warning hr {
  border-top-color: #f7e1b5;
}
.alert-warning .alert-link {
  color: #66512c;
}
.alert-danger {
  background-color: #f2dede;
  border-color: #ebccd1;
  color: #a94442;
}
.alert-danger hr {
  border-top-color: #e4b9c0;
}
.alert-danger .alert-link {
  color: #843534;
}
@-webkit-keyframes progress-bar-stripes {
  from {
    background-position: 40px 0;
  }
  to {
    background-position: 0 0;
  }
}
@keyframes progress-bar-stripes {
  from {
    background-position: 40px 0;
  }
  to {
    background-position: 0 0;
  }
}
.progress {
  overflow: hidden;
  height: 18px;
  margin-bottom: 18px;
  background-color: #f5f5f5;
  border-radius: 2px;
  -webkit-box-shadow: inset 0 1px 2px rgba(0, 0, 0, 0.1);
  box-shadow: inset 0 1px 2px rgba(0, 0, 0, 0.1);
}
.progress-bar {
  float: left;
  width: 0%;
  height: 100%;
  font-size: 12px;
  line-height: 18px;
  color: #fff;
  text-align: center;
  background-color: #337ab7;
  -webkit-box-shadow: inset 0 -1px 0 rgba(0, 0, 0, 0.15);
  box-shadow: inset 0 -1px 0 rgba(0, 0, 0, 0.15);
  -webkit-transition: width 0.6s ease;
  -o-transition: width 0.6s ease;
  transition: width 0.6s ease;
}
.progress-striped .progress-bar,
.progress-bar-striped {
  background-image: -webkit-linear-gradient(45deg, rgba(255, 255, 255, 0.15) 25%, transparent 25%, transparent 50%, rgba(255, 255, 255, 0.15) 50%, rgba(255, 255, 255, 0.15) 75%, transparent 75%, transparent);
  background-image: -o-linear-gradient(45deg, rgba(255, 255, 255, 0.15) 25%, transparent 25%, transparent 50%, rgba(255, 255, 255, 0.15) 50%, rgba(255, 255, 255, 0.15) 75%, transparent 75%, transparent);
  background-image: linear-gradient(45deg, rgba(255, 255, 255, 0.15) 25%, transparent 25%, transparent 50%, rgba(255, 255, 255, 0.15) 50%, rgba(255, 255, 255, 0.15) 75%, transparent 75%, transparent);
  background-size: 40px 40px;
}
.progress.active .progress-bar,
.progress-bar.active {
  -webkit-animation: progress-bar-stripes 2s linear infinite;
  -o-animation: progress-bar-stripes 2s linear infinite;
  animation: progress-bar-stripes 2s linear infinite;
}
.progress-bar-success {
  background-color: #5cb85c;
}
.progress-striped .progress-bar-success {
  background-image: -webkit-linear-gradient(45deg, rgba(255, 255, 255, 0.15) 25%, transparent 25%, transparent 50%, rgba(255, 255, 255, 0.15) 50%, rgba(255, 255, 255, 0.15) 75%, transparent 75%, transparent);
  background-image: -o-linear-gradient(45deg, rgba(255, 255, 255, 0.15) 25%, transparent 25%, transparent 50%, rgba(255, 255, 255, 0.15) 50%, rgba(255, 255, 255, 0.15) 75%, transparent 75%, transparent);
  background-image: linear-gradient(45deg, rgba(255, 255, 255, 0.15) 25%, transparent 25%, transparent 50%, rgba(255, 255, 255, 0.15) 50%, rgba(255, 255, 255, 0.15) 75%, transparent 75%, transparent);
}
.progress-bar-info {
  background-color: #5bc0de;
}
.progress-striped .progress-bar-info {
  background-image: -webkit-linear-gradient(45deg, rgba(255, 255, 255, 0.15) 25%, transparent 25%, transparent 50%, rgba(255, 255, 255, 0.15) 50%, rgba(255, 255, 255, 0.15) 75%, transparent 75%, transparent);
  background-image: -o-linear-gradient(45deg, rgba(255, 255, 255, 0.15) 25%, transparent 25%, transparent 50%, rgba(255, 255, 255, 0.15) 50%, rgba(255, 255, 255, 0.15) 75%, transparent 75%, transparent);
  background-image: linear-gradient(45deg, rgba(255, 255, 255, 0.15) 25%, transparent 25%, transparent 50%, rgba(255, 255, 255, 0.15) 50%, rgba(255, 255, 255, 0.15) 75%, transparent 75%, transparent);
}
.progress-bar-warning {
  background-color: #f0ad4e;
}
.progress-striped .progress-bar-warning {
  background-image: -webkit-linear-gradient(45deg, rgba(255, 255, 255, 0.15) 25%, transparent 25%, transparent 50%, rgba(255, 255, 255, 0.15) 50%, rgba(255, 255, 255, 0.15) 75%, transparent 75%, transparent);
  background-image: -o-linear-gradient(45deg, rgba(255, 255, 255, 0.15) 25%, transparent 25%, transparent 50%, rgba(255, 255, 255, 0.15) 50%, rgba(255, 255, 255, 0.15) 75%, transparent 75%, transparent);
  background-image: linear-gradient(45deg, rgba(255, 255, 255, 0.15) 25%, transparent 25%, transparent 50%, rgba(255, 255, 255, 0.15) 50%, rgba(255, 255, 255, 0.15) 75%, transparent 75%, transparent);
}
.progress-bar-danger {
  background-color: #d9534f;
}
.progress-striped .progress-bar-danger {
  background-image: -webkit-linear-gradient(45deg, rgba(255, 255, 255, 0.15) 25%, transparent 25%, transparent 50%, rgba(255, 255, 255, 0.15) 50%, rgba(255, 255, 255, 0.15) 75%, transparent 75%, transparent);
  background-image: -o-linear-gradient(45deg, rgba(255, 255, 255, 0.15) 25%, transparent 25%, transparent 50%, rgba(255, 255, 255, 0.15) 50%, rgba(255, 255, 255, 0.15) 75%, transparent 75%, transparent);
  background-image: linear-gradient(45deg, rgba(255, 255, 255, 0.15) 25%, transparent 25%, transparent 50%, rgba(255, 255, 255, 0.15) 50%, rgba(255, 255, 255, 0.15) 75%, transparent 75%, transparent);
}
.media {
  margin-top: 15px;
}
.media:first-child {
  margin-top: 0;
}
.media,
.media-body {
  zoom: 1;
  overflow: hidden;
}
.media-body {
  width: 10000px;
}
.media-object {
  display: block;
}
.media-object.img-thumbnail {
  max-width: none;
}
.media-right,
.media > .pull-right {
  padding-left: 10px;
}
.media-left,
.media > .pull-left {
  padding-right: 10px;
}
.media-left,
.media-right,
.media-body {
  display: table-cell;
  vertical-align: top;
}
.media-middle {
  vertical-align: middle;
}
.media-bottom {
  vertical-align: bottom;
}
.media-heading {
  margin-top: 0;
  margin-bottom: 5px;
}
.media-list {
  padding-left: 0;
  list-style: none;
}
.list-group {
  margin-bottom: 20px;
  padding-left: 0;
}
.list-group-item {
  position: relative;
  display: block;
  padding: 10px 15px;
  margin-bottom: -1px;
  background-color: #fff;
  border: 1px solid #ddd;
}
.list-group-item:first-child {
  border-top-right-radius: 2px;
  border-top-left-radius: 2px;
}
.list-group-item:last-child {
  margin-bottom: 0;
  border-bottom-right-radius: 2px;
  border-bottom-left-radius: 2px;
}
a.list-group-item,
button.list-group-item {
  color: #555;
}
a.list-group-item .list-group-item-heading,
button.list-group-item .list-group-item-heading {
  color: #333;
}
a.list-group-item:hover,
button.list-group-item:hover,
a.list-group-item:focus,
button.list-group-item:focus {
  text-decoration: none;
  color: #555;
  background-color: #f5f5f5;
}
button.list-group-item {
  width: 100%;
  text-align: left;
}
.list-group-item.disabled,
.list-group-item.disabled:hover,
.list-group-item.disabled:focus {
  background-color: #eeeeee;
  color: #777777;
  cursor: not-allowed;
}
.list-group-item.disabled .list-group-item-heading,
.list-group-item.disabled:hover .list-group-item-heading,
.list-group-item.disabled:focus .list-group-item-heading {
  color: inherit;
}
.list-group-item.disabled .list-group-item-text,
.list-group-item.disabled:hover .list-group-item-text,
.list-group-item.disabled:focus .list-group-item-text {
  color: #777777;
}
.list-group-item.active,
.list-group-item.active:hover,
.list-group-item.active:focus {
  z-index: 2;
  color: #fff;
  background-color: #337ab7;
  border-color: #337ab7;
}
.list-group-item.active .list-group-item-heading,
.list-group-item.active:hover .list-group-item-heading,
.list-group-item.active:focus .list-group-item-heading,
.list-group-item.active .list-group-item-heading > small,
.list-group-item.active:hover .list-group-item-heading > small,
.list-group-item.active:focus .list-group-item-heading > small,
.list-group-item.active .list-group-item-heading > .small,
.list-group-item.active:hover .list-group-item-heading > .small,
.list-group-item.active:focus .list-group-item-heading > .small {
  color: inherit;
}
.list-group-item.active .list-group-item-text,
.list-group-item.active:hover .list-group-item-text,
.list-group-item.active:focus .list-group-item-text {
  color: #c7ddef;
}
.list-group-item-success {
  color: #3c763d;
  background-color: #dff0d8;
}
a.list-group-item-success,
button.list-group-item-success {
  color: #3c763d;
}
a.list-group-item-success .list-group-item-heading,
button.list-group-item-success .list-group-item-heading {
  color: inherit;
}
a.list-group-item-success:hover,
button.list-group-item-success:hover,
a.list-group-item-success:focus,
button.list-group-item-success:focus {
  color: #3c763d;
  background-color: #d0e9c6;
}
a.list-group-item-success.active,
button.list-group-item-success.active,
a.list-group-item-success.active:hover,
button.list-group-item-success.active:hover,
a.list-group-item-success.active:focus,
button.list-group-item-success.active:focus {
  color: #fff;
  background-color: #3c763d;
  border-color: #3c763d;
}
.list-group-item-info {
  color: #31708f;
  background-color: #d9edf7;
}
a.list-group-item-info,
button.list-group-item-info {
  color: #31708f;
}
a.list-group-item-info .list-group-item-heading,
button.list-group-item-info .list-group-item-heading {
  color: inherit;
}
a.list-group-item-info:hover,
button.list-group-item-info:hover,
a.list-group-item-info:focus,
button.list-group-item-info:focus {
  color: #31708f;
  background-color: #c4e3f3;
}
a.list-group-item-info.active,
button.list-group-item-info.active,
a.list-group-item-info.active:hover,
button.list-group-item-info.active:hover,
a.list-group-item-info.active:focus,
button.list-group-item-info.active:focus {
  color: #fff;
  background-color: #31708f;
  border-color: #31708f;
}
.list-group-item-warning {
  color: #8a6d3b;
  background-color: #fcf8e3;
}
a.list-group-item-warning,
button.list-group-item-warning {
  color: #8a6d3b;
}
a.list-group-item-warning .list-group-item-heading,
button.list-group-item-warning .list-group-item-heading {
  color: inherit;
}
a.list-group-item-warning:hover,
button.list-group-item-warning:hover,
a.list-group-item-warning:focus,
button.list-group-item-warning:focus {
  color: #8a6d3b;
  background-color: #faf2cc;
}
a.list-group-item-warning.active,
button.list-group-item-warning.active,
a.list-group-item-warning.active:hover,
button.list-group-item-warning.active:hover,
a.list-group-item-warning.active:focus,
button.list-group-item-warning.active:focus {
  color: #fff;
  background-color: #8a6d3b;
  border-color: #8a6d3b;
}
.list-group-item-danger {
  color: #a94442;
  background-color: #f2dede;
}
a.list-group-item-danger,
button.list-group-item-danger {
  color: #a94442;
}
a.list-group-item-danger .list-group-item-heading,
button.list-group-item-danger .list-group-item-heading {
  color: inherit;
}
a.list-group-item-danger:hover,
button.list-group-item-danger:hover,
a.list-group-item-danger:focus,
button.list-group-item-danger:focus {
  color: #a94442;
  background-color: #ebcccc;
}
a.list-group-item-danger.active,
button.list-group-item-danger.active,
a.list-group-item-danger.active:hover,
button.list-group-item-danger.active:hover,
a.list-group-item-danger.active:focus,
button.list-group-item-danger.active:focus {
  color: #fff;
  background-color: #a94442;
  border-color: #a94442;
}
.list-group-item-heading {
  margin-top: 0;
  margin-bottom: 5px;
}
.list-group-item-text {
  margin-bottom: 0;
  line-height: 1.3;
}
.panel {
  margin-bottom: 18px;
  background-color: #fff;
  border: 1px solid transparent;
  border-radius: 2px;
  -webkit-box-shadow: 0 1px 1px rgba(0, 0, 0, 0.05);
  box-shadow: 0 1px 1px rgba(0, 0, 0, 0.05);
}
.panel-body {
  padding: 15px;
}
.panel-heading {
  padding: 10px 15px;
  border-bottom: 1px solid transparent;
  border-top-right-radius: 1px;
  border-top-left-radius: 1px;
}
.panel-heading > .dropdown .dropdown-toggle {
  color: inherit;
}
.panel-title {
  margin-top: 0;
  margin-bottom: 0;
  font-size: 15px;
  color: inherit;
}
.panel-title > a,
.panel-title > small,
.panel-title > .small,
.panel-title > small > a,
.panel-title > .small > a {
  color: inherit;
}
.panel-footer {
  padding: 10px 15px;
  background-color: #f5f5f5;
  border-top: 1px solid #ddd;
  border-bottom-right-radius: 1px;
  border-bottom-left-radius: 1px;
}
.panel > .list-group,
.panel > .panel-collapse > .list-group {
  margin-bottom: 0;
}
.panel > .list-group .list-group-item,
.panel > .panel-collapse > .list-group .list-group-item {
  border-width: 1px 0;
  border-radius: 0;
}
.panel > .list-group:first-child .list-group-item:first-child,
.panel > .panel-collapse > .list-group:first-child .list-group-item:first-child {
  border-top: 0;
  border-top-right-radius: 1px;
  border-top-left-radius: 1px;
}
.panel > .list-group:last-child .list-group-item:last-child,
.panel > .panel-collapse > .list-group:last-child .list-group-item:last-child {
  border-bottom: 0;
  border-bottom-right-radius: 1px;
  border-bottom-left-radius: 1px;
}
.panel > .panel-heading + .panel-collapse > .list-group .list-group-item:first-child {
  border-top-right-radius: 0;
  border-top-left-radius: 0;
}
.panel-heading + .list-group .list-group-item:first-child {
  border-top-width: 0;
}
.list-group + .panel-footer {
  border-top-width: 0;
}
.panel > .table,
.panel > .table-responsive > .table,
.panel > .panel-collapse > .table {
  margin-bottom: 0;
}
.panel > .table caption,
.panel > .table-responsive > .table caption,
.panel > .panel-collapse > .table caption {
  padding-left: 15px;
  padding-right: 15px;
}
.panel > .table:first-child,
.panel > .table-responsive:first-child > .table:first-child {
  border-top-right-radius: 1px;
  border-top-left-radius: 1px;
}
.panel > .table:first-child > thead:first-child > tr:first-child,
.panel > .table-responsive:first-child > .table:first-child > thead:first-child > tr:first-child,
.panel > .table:first-child > tbody:first-child > tr:first-child,
.panel > .table-responsive:first-child > .table:first-child > tbody:first-child > tr:first-child {
  border-top-left-radius: 1px;
  border-top-right-radius: 1px;
}
.panel > .table:first-child > thead:first-child > tr:first-child td:first-child,
.panel > .table-responsive:first-child > .table:first-child > thead:first-child > tr:first-child td:first-child,
.panel > .table:first-child > tbody:first-child > tr:first-child td:first-child,
.panel > .table-responsive:first-child > .table:first-child > tbody:first-child > tr:first-child td:first-child,
.panel > .table:first-child > thead:first-child > tr:first-child th:first-child,
.panel > .table-responsive:first-child > .table:first-child > thead:first-child > tr:first-child th:first-child,
.panel > .table:first-child > tbody:first-child > tr:first-child th:first-child,
.panel > .table-responsive:first-child > .table:first-child > tbody:first-child > tr:first-child th:first-child {
  border-top-left-radius: 1px;
}
.panel > .table:first-child > thead:first-child > tr:first-child td:last-child,
.panel > .table-responsive:first-child > .table:first-child > thead:first-child > tr:first-child td:last-child,
.panel > .table:first-child > tbody:first-child > tr:first-child td:last-child,
.panel > .table-responsive:first-child > .table:first-child > tbody:first-child > tr:first-child td:last-child,
.panel > .table:first-child > thead:first-child > tr:first-child th:last-child,
.panel > .table-responsive:first-child > .table:first-child > thead:first-child > tr:first-child th:last-child,
.panel > .table:first-child > tbody:first-child > tr:first-child th:last-child,
.panel > .table-responsive:first-child > .table:first-child > tbody:first-child > tr:first-child th:last-child {
  border-top-right-radius: 1px;
}
.panel > .table:last-child,
.panel > .table-responsive:last-child > .table:last-child {
  border-bottom-right-radius: 1px;
  border-bottom-left-radius: 1px;
}
.panel > .table:last-child > tbody:last-child > tr:last-child,
.panel > .table-responsive:last-child > .table:last-child > tbody:last-child > tr:last-child,
.panel > .table:last-child > tfoot:last-child > tr:last-child,
.panel > .table-responsive:last-child > .table:last-child > tfoot:last-child > tr:last-child {
  border-bottom-left-radius: 1px;
  border-bottom-right-radius: 1px;
}
.panel > .table:last-child > tbody:last-child > tr:last-child td:first-child,
.panel > .table-responsive:last-child > .table:last-child > tbody:last-child > tr:last-child td:first-child,
.panel > .table:last-child > tfoot:last-child > tr:last-child td:first-child,
.panel > .table-responsive:last-child > .table:last-child > tfoot:last-child > tr:last-child td:first-child,
.panel > .table:last-child > tbody:last-child > tr:last-child th:first-child,
.panel > .table-responsive:last-child > .table:last-child > tbody:last-child > tr:last-child th:first-child,
.panel > .table:last-child > tfoot:last-child > tr:last-child th:first-child,
.panel > .table-responsive:last-child > .table:last-child > tfoot:last-child > tr:last-child th:first-child {
  border-bottom-left-radius: 1px;
}
.panel > .table:last-child > tbody:last-child > tr:last-child td:last-child,
.panel > .table-responsive:last-child > .table:last-child > tbody:last-child > tr:last-child td:last-child,
.panel > .table:last-child > tfoot:last-child > tr:last-child td:last-child,
.panel > .table-responsive:last-child > .table:last-child > tfoot:last-child > tr:last-child td:last-child,
.panel > .table:last-child > tbody:last-child > tr:last-child th:last-child,
.panel > .table-responsive:last-child > .table:last-child > tbody:last-child > tr:last-child th:last-child,
.panel > .table:last-child > tfoot:last-child > tr:last-child th:last-child,
.panel > .table-responsive:last-child > .table:last-child > tfoot:last-child > tr:last-child th:last-child {
  border-bottom-right-radius: 1px;
}
.panel > .panel-body + .table,
.panel > .panel-body + .table-responsive,
.panel > .table + .panel-body,
.panel > .table-responsive + .panel-body {
  border-top: 1px solid #ddd;
}
.panel > .table > tbody:first-child > tr:first-child th,
.panel > .table > tbody:first-child > tr:first-child td {
  border-top: 0;
}
.panel > .table-bordered,
.panel > .table-responsive > .table-bordered {
  border: 0;
}
.panel > .table-bordered > thead > tr > th:first-child,
.panel > .table-responsive > .table-bordered > thead > tr > th:first-child,
.panel > .table-bordered > tbody > tr > th:first-child,
.panel > .table-responsive > .table-bordered > tbody > tr > th:first-child,
.panel > .table-bordered > tfoot > tr > th:first-child,
.panel > .table-responsive > .table-bordered > tfoot > tr > th:first-child,
.panel > .table-bordered > thead > tr > td:first-child,
.panel > .table-responsive > .table-bordered > thead > tr > td:first-child,
.panel > .table-bordered > tbody > tr > td:first-child,
.panel > .table-responsive > .table-bordered > tbody > tr > td:first-child,
.panel > .table-bordered > tfoot > tr > td:first-child,
.panel > .table-responsive > .table-bordered > tfoot > tr > td:first-child {
  border-left: 0;
}
.panel > .table-bordered > thead > tr > th:last-child,
.panel > .table-responsive > .table-bordered > thead > tr > th:last-child,
.panel > .table-bordered > tbody > tr > th:last-child,
.panel > .table-responsive > .table-bordered > tbody > tr > th:last-child,
.panel > .table-bordered > tfoot > tr > th:last-child,
.panel > .table-responsive > .table-bordered > tfoot > tr > th:last-child,
.panel > .table-bordered > thead > tr > td:last-child,
.panel > .table-responsive > .table-bordered > thead > tr > td:last-child,
.panel > .table-bordered > tbody > tr > td:last-child,
.panel > .table-responsive > .table-bordered > tbody > tr > td:last-child,
.panel > .table-bordered > tfoot > tr > td:last-child,
.panel > .table-responsive > .table-bordered > tfoot > tr > td:last-child {
  border-right: 0;
}
.panel > .table-bordered > thead > tr:first-child > td,
.panel > .table-responsive > .table-bordered > thead > tr:first-child > td,
.panel > .table-bordered > tbody > tr:first-child > td,
.panel > .table-responsive > .table-bordered > tbody > tr:first-child > td,
.panel > .table-bordered > thead > tr:first-child > th,
.panel > .table-responsive > .table-bordered > thead > tr:first-child > th,
.panel > .table-bordered > tbody > tr:first-child > th,
.panel > .table-responsive > .table-bordered > tbody > tr:first-child > th {
  border-bottom: 0;
}
.panel > .table-bordered > tbody > tr:last-child > td,
.panel > .table-responsive > .table-bordered > tbody > tr:last-child > td,
.panel > .table-bordered > tfoot > tr:last-child > td,
.panel > .table-responsive > .table-bordered > tfoot > tr:last-child > td,
.panel > .table-bordered > tbody > tr:last-child > th,
.panel > .table-responsive > .table-bordered > tbody > tr:last-child > th,
.panel > .table-bordered > tfoot > tr:last-child > th,
.panel > .table-responsive > .table-bordered > tfoot > tr:last-child > th {
  border-bottom: 0;
}
.panel > .table-responsive {
  border: 0;
  margin-bottom: 0;
}
.panel-group {
  margin-bottom: 18px;
}
.panel-group .panel {
  margin-bottom: 0;
  border-radius: 2px;
}
.panel-group .panel + .panel {
  margin-top: 5px;
}
.panel-group .panel-heading {
  border-bottom: 0;
}
.panel-group .panel-heading + .panel-collapse > .panel-body,
.panel-group .panel-heading + .panel-collapse > .list-group {
  border-top: 1px solid #ddd;
}
.panel-group .panel-footer {
  border-top: 0;
}
.panel-group .panel-footer + .panel-collapse .panel-body {
  border-bottom: 1px solid #ddd;
}
.panel-default {
  border-color: #ddd;
}
.panel-default > .panel-heading {
  color: #333333;
  background-color: #f5f5f5;
  border-color: #ddd;
}
.panel-default > .panel-heading + .panel-collapse > .panel-body {
  border-top-color: #ddd;
}
.panel-default > .panel-heading .badge {
  color: #f5f5f5;
  background-color: #333333;
}
.panel-default > .panel-footer + .panel-collapse > .panel-body {
  border-bottom-color: #ddd;
}
.panel-primary {
  border-color: #337ab7;
}
.panel-primary > .panel-heading {
  color: #fff;
  background-color: #337ab7;
  border-color: #337ab7;
}
.panel-primary > .panel-heading + .panel-collapse > .panel-body {
  border-top-color: #337ab7;
}
.panel-primary > .panel-heading .badge {
  color: #337ab7;
  background-color: #fff;
}
.panel-primary > .panel-footer + .panel-collapse > .panel-body {
  border-bottom-color: #337ab7;
}
.panel-success {
  border-color: #d6e9c6;
}
.panel-success > .panel-heading {
  color: #3c763d;
  background-color: #dff0d8;
  border-color: #d6e9c6;
}
.panel-success > .panel-heading + .panel-collapse > .panel-body {
  border-top-color: #d6e9c6;
}
.panel-success > .panel-heading .badge {
  color: #dff0d8;
  background-color: #3c763d;
}
.panel-success > .panel-footer + .panel-collapse > .panel-body {
  border-bottom-color: #d6e9c6;
}
.panel-info {
  border-color: #bce8f1;
}
.panel-info > .panel-heading {
  color: #31708f;
  background-color: #d9edf7;
  border-color: #bce8f1;
}
.panel-info > .panel-heading + .panel-collapse > .panel-body {
  border-top-color: #bce8f1;
}
.panel-info > .panel-heading .badge {
  color: #d9edf7;
  background-color: #31708f;
}
.panel-info > .panel-footer + .panel-collapse > .panel-body {
  border-bottom-color: #bce8f1;
}
.panel-warning {
  border-color: #faebcc;
}
.panel-warning > .panel-heading {
  color: #8a6d3b;
  background-color: #fcf8e3;
  border-color: #faebcc;
}
.panel-warning > .panel-heading + .panel-collapse > .panel-body {
  border-top-color: #faebcc;
}
.panel-warning > .panel-heading .badge {
  color: #fcf8e3;
  background-color: #8a6d3b;
}
.panel-warning > .panel-footer + .panel-collapse > .panel-body {
  border-bottom-color: #faebcc;
}
.panel-danger {
  border-color: #ebccd1;
}
.panel-danger > .panel-heading {
  color: #a94442;
  background-color: #f2dede;
  border-color: #ebccd1;
}
.panel-danger > .panel-heading + .panel-collapse > .panel-body {
  border-top-color: #ebccd1;
}
.panel-danger > .panel-heading .badge {
  color: #f2dede;
  background-color: #a94442;
}
.panel-danger > .panel-footer + .panel-collapse > .panel-body {
  border-bottom-color: #ebccd1;
}
.embed-responsive {
  position: relative;
  display: block;
  height: 0;
  padding: 0;
  overflow: hidden;
}
.embed-responsive .embed-responsive-item,
.embed-responsive iframe,
.embed-responsive embed,
.embed-responsive object,
.embed-responsive video {
  position: absolute;
  top: 0;
  left: 0;
  bottom: 0;
  height: 100%;
  width: 100%;
  border: 0;
}
.embed-responsive-16by9 {
  padding-bottom: 56.25%;
}
.embed-responsive-4by3 {
  padding-bottom: 75%;
}
.well {
  min-height: 20px;
  padding: 19px;
  margin-bottom: 20px;
  background-color: #f5f5f5;
  border: 1px solid #e3e3e3;
  border-radius: 2px;
  -webkit-box-shadow: inset 0 1px 1px rgba(0, 0, 0, 0.05);
  box-shadow: inset 0 1px 1px rgba(0, 0, 0, 0.05);
}
.well blockquote {
  border-color: #ddd;
  border-color: rgba(0, 0, 0, 0.15);
}
.well-lg {
  padding: 24px;
  border-radius: 3px;
}
.well-sm {
  padding: 9px;
  border-radius: 1px;
}
.close {
  float: right;
  font-size: 19.5px;
  font-weight: bold;
  line-height: 1;
  color: #000;
  text-shadow: 0 1px 0 #fff;
  opacity: 0.2;
  filter: alpha(opacity=20);
}
.close:hover,
.close:focus {
  color: #000;
  text-decoration: none;
  cursor: pointer;
  opacity: 0.5;
  filter: alpha(opacity=50);
}
button.close {
  padding: 0;
  cursor: pointer;
  background: transparent;
  border: 0;
  -webkit-appearance: none;
}
.modal-open {
  overflow: hidden;
}
.modal {
  display: none;
  overflow: hidden;
  position: fixed;
  top: 0;
  right: 0;
  bottom: 0;
  left: 0;
  z-index: 1050;
  -webkit-overflow-scrolling: touch;
  outline: 0;
}
.modal.fade .modal-dialog {
  -webkit-transform: translate(0, -25%);
  -ms-transform: translate(0, -25%);
  -o-transform: translate(0, -25%);
  transform: translate(0, -25%);
  -webkit-transition: -webkit-transform 0.3s ease-out;
  -moz-transition: -moz-transform 0.3s ease-out;
  -o-transition: -o-transform 0.3s ease-out;
  transition: transform 0.3s ease-out;
}
.modal.in .modal-dialog {
  -webkit-transform: translate(0, 0);
  -ms-transform: translate(0, 0);
  -o-transform: translate(0, 0);
  transform: translate(0, 0);
}
.modal-open .modal {
  overflow-x: hidden;
  overflow-y: auto;
}
.modal-dialog {
  position: relative;
  width: auto;
  margin: 10px;
}
.modal-content {
  position: relative;
  background-color: #fff;
  border: 1px solid #999;
  border: 1px solid rgba(0, 0, 0, 0.2);
  border-radius: 3px;
  -webkit-box-shadow: 0 3px 9px rgba(0, 0, 0, 0.5);
  box-shadow: 0 3px 9px rgba(0, 0, 0, 0.5);
  background-clip: padding-box;
  outline: 0;
}
.modal-backdrop {
  position: fixed;
  top: 0;
  right: 0;
  bottom: 0;
  left: 0;
  z-index: 1040;
  background-color: #000;
}
.modal-backdrop.fade {
  opacity: 0;
  filter: alpha(opacity=0);
}
.modal-backdrop.in {
  opacity: 0.5;
  filter: alpha(opacity=50);
}
.modal-header {
  padding: 15px;
  border-bottom: 1px solid #e5e5e5;
}
.modal-header .close {
  margin-top: -2px;
}
.modal-title {
  margin: 0;
  line-height: 1.42857143;
}
.modal-body {
  position: relative;
  padding: 15px;
}
.modal-footer {
  padding: 15px;
  text-align: right;
  border-top: 1px solid #e5e5e5;
}
.modal-footer .btn + .btn {
  margin-left: 5px;
  margin-bottom: 0;
}
.modal-footer .btn-group .btn + .btn {
  margin-left: -1px;
}
.modal-footer .btn-block + .btn-block {
  margin-left: 0;
}
.modal-scrollbar-measure {
  position: absolute;
  top: -9999px;
  width: 50px;
  height: 50px;
  overflow: scroll;
}
@media (min-width: 768px) {
  .modal-dialog {
    width: 600px;
    margin: 30px auto;
  }
  .modal-content {
    -webkit-box-shadow: 0 5px 15px rgba(0, 0, 0, 0.5);
    box-shadow: 0 5px 15px rgba(0, 0, 0, 0.5);
  }
  .modal-sm {
    width: 300px;
  }
}
@media (min-width: 992px) {
  .modal-lg {
    width: 900px;
  }
}
.tooltip {
  position: absolute;
  z-index: 1070;
  display: block;
  font-family: "Helvetica Neue", Helvetica, Arial, sans-serif;
  font-style: normal;
  font-weight: normal;
  letter-spacing: normal;
  line-break: auto;
  line-height: 1.42857143;
  text-align: left;
  text-align: start;
  text-decoration: none;
  text-shadow: none;
  text-transform: none;
  white-space: normal;
  word-break: normal;
  word-spacing: normal;
  word-wrap: normal;
  font-size: 12px;
  opacity: 0;
  filter: alpha(opacity=0);
}
.tooltip.in {
  opacity: 0.9;
  filter: alpha(opacity=90);
}
.tooltip.top {
  margin-top: -3px;
  padding: 5px 0;
}
.tooltip.right {
  margin-left: 3px;
  padding: 0 5px;
}
.tooltip.bottom {
  margin-top: 3px;
  padding: 5px 0;
}
.tooltip.left {
  margin-left: -3px;
  padding: 0 5px;
}
.tooltip-inner {
  max-width: 200px;
  padding: 3px 8px;
  color: #fff;
  text-align: center;
  background-color: #000;
  border-radius: 2px;
}
.tooltip-arrow {
  position: absolute;
  width: 0;
  height: 0;
  border-color: transparent;
  border-style: solid;
}
.tooltip.top .tooltip-arrow {
  bottom: 0;
  left: 50%;
  margin-left: -5px;
  border-width: 5px 5px 0;
  border-top-color: #000;
}
.tooltip.top-left .tooltip-arrow {
  bottom: 0;
  right: 5px;
  margin-bottom: -5px;
  border-width: 5px 5px 0;
  border-top-color: #000;
}
.tooltip.top-right .tooltip-arrow {
  bottom: 0;
  left: 5px;
  margin-bottom: -5px;
  border-width: 5px 5px 0;
  border-top-color: #000;
}
.tooltip.right .tooltip-arrow {
  top: 50%;
  left: 0;
  margin-top: -5px;
  border-width: 5px 5px 5px 0;
  border-right-color: #000;
}
.tooltip.left .tooltip-arrow {
  top: 50%;
  right: 0;
  margin-top: -5px;
  border-width: 5px 0 5px 5px;
  border-left-color: #000;
}
.tooltip.bottom .tooltip-arrow {
  top: 0;
  left: 50%;
  margin-left: -5px;
  border-width: 0 5px 5px;
  border-bottom-color: #000;
}
.tooltip.bottom-left .tooltip-arrow {
  top: 0;
  right: 5px;
  margin-top: -5px;
  border-width: 0 5px 5px;
  border-bottom-color: #000;
}
.tooltip.bottom-right .tooltip-arrow {
  top: 0;
  left: 5px;
  margin-top: -5px;
  border-width: 0 5px 5px;
  border-bottom-color: #000;
}
.popover {
  position: absolute;
  top: 0;
  left: 0;
  z-index: 1060;
  display: none;
  max-width: 276px;
  padding: 1px;
  font-family: "Helvetica Neue", Helvetica, Arial, sans-serif;
  font-style: normal;
  font-weight: normal;
  letter-spacing: normal;
  line-break: auto;
  line-height: 1.42857143;
  text-align: left;
  text-align: start;
  text-decoration: none;
  text-shadow: none;
  text-transform: none;
  white-space: normal;
  word-break: normal;
  word-spacing: normal;
  word-wrap: normal;
  font-size: 13px;
  background-color: #fff;
  background-clip: padding-box;
  border: 1px solid #ccc;
  border: 1px solid rgba(0, 0, 0, 0.2);
  border-radius: 3px;
  -webkit-box-shadow: 0 5px 10px rgba(0, 0, 0, 0.2);
  box-shadow: 0 5px 10px rgba(0, 0, 0, 0.2);
}
.popover.top {
  margin-top: -10px;
}
.popover.right {
  margin-left: 10px;
}
.popover.bottom {
  margin-top: 10px;
}
.popover.left {
  margin-left: -10px;
}
.popover-title {
  margin: 0;
  padding: 8px 14px;
  font-size: 13px;
  background-color: #f7f7f7;
  border-bottom: 1px solid #ebebeb;
  border-radius: 2px 2px 0 0;
}
.popover-content {
  padding: 9px 14px;
}
.popover > .arrow,
.popover > .arrow:after {
  position: absolute;
  display: block;
  width: 0;
  height: 0;
  border-color: transparent;
  border-style: solid;
}
.popover > .arrow {
  border-width: 11px;
}
.popover > .arrow:after {
  border-width: 10px;
  content: "";
}
.popover.top > .arrow {
  left: 50%;
  margin-left: -11px;
  border-bottom-width: 0;
  border-top-color: #999999;
  border-top-color: rgba(0, 0, 0, 0.25);
  bottom: -11px;
}
.popover.top > .arrow:after {
  content: " ";
  bottom: 1px;
  margin-left: -10px;
  border-bottom-width: 0;
  border-top-color: #fff;
}
.popover.right > .arrow {
  top: 50%;
  left: -11px;
  margin-top: -11px;
  border-left-width: 0;
  border-right-color: #999999;
  border-right-color: rgba(0, 0, 0, 0.25);
}
.popover.right > .arrow:after {
  content: " ";
  left: 1px;
  bottom: -10px;
  border-left-width: 0;
  border-right-color: #fff;
}
.popover.bottom > .arrow {
  left: 50%;
  margin-left: -11px;
  border-top-width: 0;
  border-bottom-color: #999999;
  border-bottom-color: rgba(0, 0, 0, 0.25);
  top: -11px;
}
.popover.bottom > .arrow:after {
  content: " ";
  top: 1px;
  margin-left: -10px;
  border-top-width: 0;
  border-bottom-color: #fff;
}
.popover.left > .arrow {
  top: 50%;
  right: -11px;
  margin-top: -11px;
  border-right-width: 0;
  border-left-color: #999999;
  border-left-color: rgba(0, 0, 0, 0.25);
}
.popover.left > .arrow:after {
  content: " ";
  right: 1px;
  border-right-width: 0;
  border-left-color: #fff;
  bottom: -10px;
}
.carousel {
  position: relative;
}
.carousel-inner {
  position: relative;
  overflow: hidden;
  width: 100%;
}
.carousel-inner > .item {
  display: none;
  position: relative;
  -webkit-transition: 0.6s ease-in-out left;
  -o-transition: 0.6s ease-in-out left;
  transition: 0.6s ease-in-out left;
}
.carousel-inner > .item > img,
.carousel-inner > .item > a > img {
  line-height: 1;
}
@media all and (transform-3d), (-webkit-transform-3d) {
  .carousel-inner > .item {
    -webkit-transition: -webkit-transform 0.6s ease-in-out;
    -moz-transition: -moz-transform 0.6s ease-in-out;
    -o-transition: -o-transform 0.6s ease-in-out;
    transition: transform 0.6s ease-in-out;
    -webkit-backface-visibility: hidden;
    -moz-backface-visibility: hidden;
    backface-visibility: hidden;
    -webkit-perspective: 1000px;
    -moz-perspective: 1000px;
    perspective: 1000px;
  }
  .carousel-inner > .item.next,
  .carousel-inner > .item.active.right {
    -webkit-transform: translate3d(100%, 0, 0);
    transform: translate3d(100%, 0, 0);
    left: 0;
  }
  .carousel-inner > .item.prev,
  .carousel-inner > .item.active.left {
    -webkit-transform: translate3d(-100%, 0, 0);
    transform: translate3d(-100%, 0, 0);
    left: 0;
  }
  .carousel-inner > .item.next.left,
  .carousel-inner > .item.prev.right,
  .carousel-inner > .item.active {
    -webkit-transform: translate3d(0, 0, 0);
    transform: translate3d(0, 0, 0);
    left: 0;
  }
}
.carousel-inner > .active,
.carousel-inner > .next,
.carousel-inner > .prev {
  display: block;
}
.carousel-inner > .active {
  left: 0;
}
.carousel-inner > .next,
.carousel-inner > .prev {
  position: absolute;
  top: 0;
  width: 100%;
}
.carousel-inner > .next {
  left: 100%;
}
.carousel-inner > .prev {
  left: -100%;
}
.carousel-inner > .next.left,
.carousel-inner > .prev.right {
  left: 0;
}
.carousel-inner > .active.left {
  left: -100%;
}
.carousel-inner > .active.right {
  left: 100%;
}
.carousel-control {
  position: absolute;
  top: 0;
  left: 0;
  bottom: 0;
  width: 15%;
  opacity: 0.5;
  filter: alpha(opacity=50);
  font-size: 20px;
  color: #fff;
  text-align: center;
  text-shadow: 0 1px 2px rgba(0, 0, 0, 0.6);
  background-color: rgba(0, 0, 0, 0);
}
.carousel-control.left {
  background-image: -webkit-linear-gradient(left, rgba(0, 0, 0, 0.5) 0%, rgba(0, 0, 0, 0.0001) 100%);
  background-image: -o-linear-gradient(left, rgba(0, 0, 0, 0.5) 0%, rgba(0, 0, 0, 0.0001) 100%);
  background-image: linear-gradient(to right, rgba(0, 0, 0, 0.5) 0%, rgba(0, 0, 0, 0.0001) 100%);
  background-repeat: repeat-x;
  filter: progid:DXImageTransform.Microsoft.gradient(startColorstr='#80000000', endColorstr='#00000000', GradientType=1);
}
.carousel-control.right {
  left: auto;
  right: 0;
  background-image: -webkit-linear-gradient(left, rgba(0, 0, 0, 0.0001) 0%, rgba(0, 0, 0, 0.5) 100%);
  background-image: -o-linear-gradient(left, rgba(0, 0, 0, 0.0001) 0%, rgba(0, 0, 0, 0.5) 100%);
  background-image: linear-gradient(to right, rgba(0, 0, 0, 0.0001) 0%, rgba(0, 0, 0, 0.5) 100%);
  background-repeat: repeat-x;
  filter: progid:DXImageTransform.Microsoft.gradient(startColorstr='#00000000', endColorstr='#80000000', GradientType=1);
}
.carousel-control:hover,
.carousel-control:focus {
  outline: 0;
  color: #fff;
  text-decoration: none;
  opacity: 0.9;
  filter: alpha(opacity=90);
}
.carousel-control .icon-prev,
.carousel-control .icon-next,
.carousel-control .glyphicon-chevron-left,
.carousel-control .glyphicon-chevron-right {
  position: absolute;
  top: 50%;
  margin-top: -10px;
  z-index: 5;
  display: inline-block;
}
.carousel-control .icon-prev,
.carousel-control .glyphicon-chevron-left {
  left: 50%;
  margin-left: -10px;
}
.carousel-control .icon-next,
.carousel-control .glyphicon-chevron-right {
  right: 50%;
  margin-right: -10px;
}
.carousel-control .icon-prev,
.carousel-control .icon-next {
  width: 20px;
  height: 20px;
  line-height: 1;
  font-family: serif;
}
.carousel-control .icon-prev:before {
  content: '\2039';
}
.carousel-control .icon-next:before {
  content: '\203a';
}
.carousel-indicators {
  position: absolute;
  bottom: 10px;
  left: 50%;
  z-index: 15;
  width: 60%;
  margin-left: -30%;
  padding-left: 0;
  list-style: none;
  text-align: center;
}
.carousel-indicators li {
  display: inline-block;
  width: 10px;
  height: 10px;
  margin: 1px;
  text-indent: -999px;
  border: 1px solid #fff;
  border-radius: 10px;
  cursor: pointer;
  background-color: #000 \9;
  background-color: rgba(0, 0, 0, 0);
}
.carousel-indicators .active {
  margin: 0;
  width: 12px;
  height: 12px;
  background-color: #fff;
}
.carousel-caption {
  position: absolute;
  left: 15%;
  right: 15%;
  bottom: 20px;
  z-index: 10;
  padding-top: 20px;
  padding-bottom: 20px;
  color: #fff;
  text-align: center;
  text-shadow: 0 1px 2px rgba(0, 0, 0, 0.6);
}
.carousel-caption .btn {
  text-shadow: none;
}
@media screen and (min-width: 768px) {
  .carousel-control .glyphicon-chevron-left,
  .carousel-control .glyphicon-chevron-right,
  .carousel-control .icon-prev,
  .carousel-control .icon-next {
    width: 30px;
    height: 30px;
    margin-top: -10px;
    font-size: 30px;
  }
  .carousel-control .glyphicon-chevron-left,
  .carousel-control .icon-prev {
    margin-left: -10px;
  }
  .carousel-control .glyphicon-chevron-right,
  .carousel-control .icon-next {
    margin-right: -10px;
  }
  .carousel-caption {
    left: 20%;
    right: 20%;
    padding-bottom: 30px;
  }
  .carousel-indicators {
    bottom: 20px;
  }
}
.clearfix:before,
.clearfix:after,
.dl-horizontal dd:before,
.dl-horizontal dd:after,
.container:before,
.container:after,
.container-fluid:before,
.container-fluid:after,
.row:before,
.row:after,
.form-horizontal .form-group:before,
.form-horizontal .form-group:after,
.btn-toolbar:before,
.btn-toolbar:after,
.btn-group-vertical > .btn-group:before,
.btn-group-vertical > .btn-group:after,
.nav:before,
.nav:after,
.navbar:before,
.navbar:after,
.navbar-header:before,
.navbar-header:after,
.navbar-collapse:before,
.navbar-collapse:after,
.pager:before,
.pager:after,
.panel-body:before,
.panel-body:after,
.modal-header:before,
.modal-header:after,
.modal-footer:before,
.modal-footer:after,
.item_buttons:before,
.item_buttons:after {
  content: " ";
  display: table;
}
.clearfix:after,
.dl-horizontal dd:after,
.container:after,
.container-fluid:after,
.row:after,
.form-horizontal .form-group:after,
.btn-toolbar:after,
.btn-group-vertical > .btn-group:after,
.nav:after,
.navbar:after,
.navbar-header:after,
.navbar-collapse:after,
.pager:after,
.panel-body:after,
.modal-header:after,
.modal-footer:after,
.item_buttons:after {
  clear: both;
}
.center-block {
  display: block;
  margin-left: auto;
  margin-right: auto;
}
.pull-right {
  float: right !important;
}
.pull-left {
  float: left !important;
}
.hide {
  display: none !important;
}
.show {
  display: block !important;
}
.invisible {
  visibility: hidden;
}
.text-hide {
  font: 0/0 a;
  color: transparent;
  text-shadow: none;
  background-color: transparent;
  border: 0;
}
.hidden {
  display: none !important;
}
.affix {
  position: fixed;
}
@-ms-viewport {
  width: device-width;
}
.visible-xs,
.visible-sm,
.visible-md,
.visible-lg {
  display: none !important;
}
.visible-xs-block,
.visible-xs-inline,
.visible-xs-inline-block,
.visible-sm-block,
.visible-sm-inline,
.visible-sm-inline-block,
.visible-md-block,
.visible-md-inline,
.visible-md-inline-block,
.visible-lg-block,
.visible-lg-inline,
.visible-lg-inline-block {
  display: none !important;
}
@media (max-width: 767px) {
  .visible-xs {
    display: block !important;
  }
  table.visible-xs {
    display: table !important;
  }
  tr.visible-xs {
    display: table-row !important;
  }
  th.visible-xs,
  td.visible-xs {
    display: table-cell !important;
  }
}
@media (max-width: 767px) {
  .visible-xs-block {
    display: block !important;
  }
}
@media (max-width: 767px) {
  .visible-xs-inline {
    display: inline !important;
  }
}
@media (max-width: 767px) {
  .visible-xs-inline-block {
    display: inline-block !important;
  }
}
@media (min-width: 768px) and (max-width: 991px) {
  .visible-sm {
    display: block !important;
  }
  table.visible-sm {
    display: table !important;
  }
  tr.visible-sm {
    display: table-row !important;
  }
  th.visible-sm,
  td.visible-sm {
    display: table-cell !important;
  }
}
@media (min-width: 768px) and (max-width: 991px) {
  .visible-sm-block {
    display: block !important;
  }
}
@media (min-width: 768px) and (max-width: 991px) {
  .visible-sm-inline {
    display: inline !important;
  }
}
@media (min-width: 768px) and (max-width: 991px) {
  .visible-sm-inline-block {
    display: inline-block !important;
  }
}
@media (min-width: 992px) and (max-width: 1199px) {
  .visible-md {
    display: block !important;
  }
  table.visible-md {
    display: table !important;
  }
  tr.visible-md {
    display: table-row !important;
  }
  th.visible-md,
  td.visible-md {
    display: table-cell !important;
  }
}
@media (min-width: 992px) and (max-width: 1199px) {
  .visible-md-block {
    display: block !important;
  }
}
@media (min-width: 992px) and (max-width: 1199px) {
  .visible-md-inline {
    display: inline !important;
  }
}
@media (min-width: 992px) and (max-width: 1199px) {
  .visible-md-inline-block {
    display: inline-block !important;
  }
}
@media (min-width: 1200px) {
  .visible-lg {
    display: block !important;
  }
  table.visible-lg {
    display: table !important;
  }
  tr.visible-lg {
    display: table-row !important;
  }
  th.visible-lg,
  td.visible-lg {
    display: table-cell !important;
  }
}
@media (min-width: 1200px) {
  .visible-lg-block {
    display: block !important;
  }
}
@media (min-width: 1200px) {
  .visible-lg-inline {
    display: inline !important;
  }
}
@media (min-width: 1200px) {
  .visible-lg-inline-block {
    display: inline-block !important;
  }
}
@media (max-width: 767px) {
  .hidden-xs {
    display: none !important;
  }
}
@media (min-width: 768px) and (max-width: 991px) {
  .hidden-sm {
    display: none !important;
  }
}
@media (min-width: 992px) and (max-width: 1199px) {
  .hidden-md {
    display: none !important;
  }
}
@media (min-width: 1200px) {
  .hidden-lg {
    display: none !important;
  }
}
.visible-print {
  display: none !important;
}
@media print {
  .visible-print {
    display: block !important;
  }
  table.visible-print {
    display: table !important;
  }
  tr.visible-print {
    display: table-row !important;
  }
  th.visible-print,
  td.visible-print {
    display: table-cell !important;
  }
}
.visible-print-block {
  display: none !important;
}
@media print {
  .visible-print-block {
    display: block !important;
  }
}
.visible-print-inline {
  display: none !important;
}
@media print {
  .visible-print-inline {
    display: inline !important;
  }
}
.visible-print-inline-block {
  display: none !important;
}
@media print {
  .visible-print-inline-block {
    display: inline-block !important;
  }
}
@media print {
  .hidden-print {
    display: none !important;
  }
}
/*!
*
* Font Awesome
*
*/
/*!
 *  Font Awesome 4.2.0 by @davegandy - http://fontawesome.io - @fontawesome
 *  License - http://fontawesome.io/license (Font: SIL OFL 1.1, CSS: MIT License)
 */
/* FONT PATH
 * -------------------------- */
@font-face {
  font-family: 'FontAwesome';
  src: url('../components/font-awesome/fonts/fontawesome-webfont.eot?v=4.2.0');
  src: url('../components/font-awesome/fonts/fontawesome-webfont.eot?#iefix&v=4.2.0') format('embedded-opentype'), url('../components/font-awesome/fonts/fontawesome-webfont.woff?v=4.2.0') format('woff'), url('../components/font-awesome/fonts/fontawesome-webfont.ttf?v=4.2.0') format('truetype'), url('../components/font-awesome/fonts/fontawesome-webfont.svg?v=4.2.0#fontawesomeregular') format('svg');
  font-weight: normal;
  font-style: normal;
}
.fa {
  display: inline-block;
  font: normal normal normal 14px/1 FontAwesome;
  font-size: inherit;
  text-rendering: auto;
  -webkit-font-smoothing: antialiased;
  -moz-osx-font-smoothing: grayscale;
}
/* makes the font 33% larger relative to the icon container */
.fa-lg {
  font-size: 1.33333333em;
  line-height: 0.75em;
  vertical-align: -15%;
}
.fa-2x {
  font-size: 2em;
}
.fa-3x {
  font-size: 3em;
}
.fa-4x {
  font-size: 4em;
}
.fa-5x {
  font-size: 5em;
}
.fa-fw {
  width: 1.28571429em;
  text-align: center;
}
.fa-ul {
  padding-left: 0;
  margin-left: 2.14285714em;
  list-style-type: none;
}
.fa-ul > li {
  position: relative;
}
.fa-li {
  position: absolute;
  left: -2.14285714em;
  width: 2.14285714em;
  top: 0.14285714em;
  text-align: center;
}
.fa-li.fa-lg {
  left: -1.85714286em;
}
.fa-border {
  padding: .2em .25em .15em;
  border: solid 0.08em #eee;
  border-radius: .1em;
}
.pull-right {
  float: right;
}
.pull-left {
  float: left;
}
.fa.pull-left {
  margin-right: .3em;
}
.fa.pull-right {
  margin-left: .3em;
}
.fa-spin {
  -webkit-animation: fa-spin 2s infinite linear;
  animation: fa-spin 2s infinite linear;
}
@-webkit-keyframes fa-spin {
  0% {
    -webkit-transform: rotate(0deg);
    transform: rotate(0deg);
  }
  100% {
    -webkit-transform: rotate(359deg);
    transform: rotate(359deg);
  }
}
@keyframes fa-spin {
  0% {
    -webkit-transform: rotate(0deg);
    transform: rotate(0deg);
  }
  100% {
    -webkit-transform: rotate(359deg);
    transform: rotate(359deg);
  }
}
.fa-rotate-90 {
  filter: progid:DXImageTransform.Microsoft.BasicImage(rotation=1);
  -webkit-transform: rotate(90deg);
  -ms-transform: rotate(90deg);
  transform: rotate(90deg);
}
.fa-rotate-180 {
  filter: progid:DXImageTransform.Microsoft.BasicImage(rotation=2);
  -webkit-transform: rotate(180deg);
  -ms-transform: rotate(180deg);
  transform: rotate(180deg);
}
.fa-rotate-270 {
  filter: progid:DXImageTransform.Microsoft.BasicImage(rotation=3);
  -webkit-transform: rotate(270deg);
  -ms-transform: rotate(270deg);
  transform: rotate(270deg);
}
.fa-flip-horizontal {
  filter: progid:DXImageTransform.Microsoft.BasicImage(rotation=0, mirror=1);
  -webkit-transform: scale(-1, 1);
  -ms-transform: scale(-1, 1);
  transform: scale(-1, 1);
}
.fa-flip-vertical {
  filter: progid:DXImageTransform.Microsoft.BasicImage(rotation=2, mirror=1);
  -webkit-transform: scale(1, -1);
  -ms-transform: scale(1, -1);
  transform: scale(1, -1);
}
:root .fa-rotate-90,
:root .fa-rotate-180,
:root .fa-rotate-270,
:root .fa-flip-horizontal,
:root .fa-flip-vertical {
  filter: none;
}
.fa-stack {
  position: relative;
  display: inline-block;
  width: 2em;
  height: 2em;
  line-height: 2em;
  vertical-align: middle;
}
.fa-stack-1x,
.fa-stack-2x {
  position: absolute;
  left: 0;
  width: 100%;
  text-align: center;
}
.fa-stack-1x {
  line-height: inherit;
}
.fa-stack-2x {
  font-size: 2em;
}
.fa-inverse {
  color: #fff;
}
/* Font Awesome uses the Unicode Private Use Area (PUA) to ensure screen
   readers do not read off random characters that represent icons */
.fa-glass:before {
  content: "\f000";
}
.fa-music:before {
  content: "\f001";
}
.fa-search:before {
  content: "\f002";
}
.fa-envelope-o:before {
  content: "\f003";
}
.fa-heart:before {
  content: "\f004";
}
.fa-star:before {
  content: "\f005";
}
.fa-star-o:before {
  content: "\f006";
}
.fa-user:before {
  content: "\f007";
}
.fa-film:before {
  content: "\f008";
}
.fa-th-large:before {
  content: "\f009";
}
.fa-th:before {
  content: "\f00a";
}
.fa-th-list:before {
  content: "\f00b";
}
.fa-check:before {
  content: "\f00c";
}
.fa-remove:before,
.fa-close:before,
.fa-times:before {
  content: "\f00d";
}
.fa-search-plus:before {
  content: "\f00e";
}
.fa-search-minus:before {
  content: "\f010";
}
.fa-power-off:before {
  content: "\f011";
}
.fa-signal:before {
  content: "\f012";
}
.fa-gear:before,
.fa-cog:before {
  content: "\f013";
}
.fa-trash-o:before {
  content: "\f014";
}
.fa-home:before {
  content: "\f015";
}
.fa-file-o:before {
  content: "\f016";
}
.fa-clock-o:before {
  content: "\f017";
}
.fa-road:before {
  content: "\f018";
}
.fa-download:before {
  content: "\f019";
}
.fa-arrow-circle-o-down:before {
  content: "\f01a";
}
.fa-arrow-circle-o-up:before {
  content: "\f01b";
}
.fa-inbox:before {
  content: "\f01c";
}
.fa-play-circle-o:before {
  content: "\f01d";
}
.fa-rotate-right:before,
.fa-repeat:before {
  content: "\f01e";
}
.fa-refresh:before {
  content: "\f021";
}
.fa-list-alt:before {
  content: "\f022";
}
.fa-lock:before {
  content: "\f023";
}
.fa-flag:before {
  content: "\f024";
}
.fa-headphones:before {
  content: "\f025";
}
.fa-volume-off:before {
  content: "\f026";
}
.fa-volume-down:before {
  content: "\f027";
}
.fa-volume-up:before {
  content: "\f028";
}
.fa-qrcode:before {
  content: "\f029";
}
.fa-barcode:before {
  content: "\f02a";
}
.fa-tag:before {
  content: "\f02b";
}
.fa-tags:before {
  content: "\f02c";
}
.fa-book:before {
  content: "\f02d";
}
.fa-bookmark:before {
  content: "\f02e";
}
.fa-print:before {
  content: "\f02f";
}
.fa-camera:before {
  content: "\f030";
}
.fa-font:before {
  content: "\f031";
}
.fa-bold:before {
  content: "\f032";
}
.fa-italic:before {
  content: "\f033";
}
.fa-text-height:before {
  content: "\f034";
}
.fa-text-width:before {
  content: "\f035";
}
.fa-align-left:before {
  content: "\f036";
}
.fa-align-center:before {
  content: "\f037";
}
.fa-align-right:before {
  content: "\f038";
}
.fa-align-justify:before {
  content: "\f039";
}
.fa-list:before {
  content: "\f03a";
}
.fa-dedent:before,
.fa-outdent:before {
  content: "\f03b";
}
.fa-indent:before {
  content: "\f03c";
}
.fa-video-camera:before {
  content: "\f03d";
}
.fa-photo:before,
.fa-image:before,
.fa-picture-o:before {
  content: "\f03e";
}
.fa-pencil:before {
  content: "\f040";
}
.fa-map-marker:before {
  content: "\f041";
}
.fa-adjust:before {
  content: "\f042";
}
.fa-tint:before {
  content: "\f043";
}
.fa-edit:before,
.fa-pencil-square-o:before {
  content: "\f044";
}
.fa-share-square-o:before {
  content: "\f045";
}
.fa-check-square-o:before {
  content: "\f046";
}
.fa-arrows:before {
  content: "\f047";
}
.fa-step-backward:before {
  content: "\f048";
}
.fa-fast-backward:before {
  content: "\f049";
}
.fa-backward:before {
  content: "\f04a";
}
.fa-play:before {
  content: "\f04b";
}
.fa-pause:before {
  content: "\f04c";
}
.fa-stop:before {
  content: "\f04d";
}
.fa-forward:before {
  content: "\f04e";
}
.fa-fast-forward:before {
  content: "\f050";
}
.fa-step-forward:before {
  content: "\f051";
}
.fa-eject:before {
  content: "\f052";
}
.fa-chevron-left:before {
  content: "\f053";
}
.fa-chevron-right:before {
  content: "\f054";
}
.fa-plus-circle:before {
  content: "\f055";
}
.fa-minus-circle:before {
  content: "\f056";
}
.fa-times-circle:before {
  content: "\f057";
}
.fa-check-circle:before {
  content: "\f058";
}
.fa-question-circle:before {
  content: "\f059";
}
.fa-info-circle:before {
  content: "\f05a";
}
.fa-crosshairs:before {
  content: "\f05b";
}
.fa-times-circle-o:before {
  content: "\f05c";
}
.fa-check-circle-o:before {
  content: "\f05d";
}
.fa-ban:before {
  content: "\f05e";
}
.fa-arrow-left:before {
  content: "\f060";
}
.fa-arrow-right:before {
  content: "\f061";
}
.fa-arrow-up:before {
  content: "\f062";
}
.fa-arrow-down:before {
  content: "\f063";
}
.fa-mail-forward:before,
.fa-share:before {
  content: "\f064";
}
.fa-expand:before {
  content: "\f065";
}
.fa-compress:before {
  content: "\f066";
}
.fa-plus:before {
  content: "\f067";
}
.fa-minus:before {
  content: "\f068";
}
.fa-asterisk:before {
  content: "\f069";
}
.fa-exclamation-circle:before {
  content: "\f06a";
}
.fa-gift:before {
  content: "\f06b";
}
.fa-leaf:before {
  content: "\f06c";
}
.fa-fire:before {
  content: "\f06d";
}
.fa-eye:before {
  content: "\f06e";
}
.fa-eye-slash:before {
  content: "\f070";
}
.fa-warning:before,
.fa-exclamation-triangle:before {
  content: "\f071";
}
.fa-plane:before {
  content: "\f072";
}
.fa-calendar:before {
  content: "\f073";
}
.fa-random:before {
  content: "\f074";
}
.fa-comment:before {
  content: "\f075";
}
.fa-magnet:before {
  content: "\f076";
}
.fa-chevron-up:before {
  content: "\f077";
}
.fa-chevron-down:before {
  content: "\f078";
}
.fa-retweet:before {
  content: "\f079";
}
.fa-shopping-cart:before {
  content: "\f07a";
}
.fa-folder:before {
  content: "\f07b";
}
.fa-folder-open:before {
  content: "\f07c";
}
.fa-arrows-v:before {
  content: "\f07d";
}
.fa-arrows-h:before {
  content: "\f07e";
}
.fa-bar-chart-o:before,
.fa-bar-chart:before {
  content: "\f080";
}
.fa-twitter-square:before {
  content: "\f081";
}
.fa-facebook-square:before {
  content: "\f082";
}
.fa-camera-retro:before {
  content: "\f083";
}
.fa-key:before {
  content: "\f084";
}
.fa-gears:before,
.fa-cogs:before {
  content: "\f085";
}
.fa-comments:before {
  content: "\f086";
}
.fa-thumbs-o-up:before {
  content: "\f087";
}
.fa-thumbs-o-down:before {
  content: "\f088";
}
.fa-star-half:before {
  content: "\f089";
}
.fa-heart-o:before {
  content: "\f08a";
}
.fa-sign-out:before {
  content: "\f08b";
}
.fa-linkedin-square:before {
  content: "\f08c";
}
.fa-thumb-tack:before {
  content: "\f08d";
}
.fa-external-link:before {
  content: "\f08e";
}
.fa-sign-in:before {
  content: "\f090";
}
.fa-trophy:before {
  content: "\f091";
}
.fa-github-square:before {
  content: "\f092";
}
.fa-upload:before {
  content: "\f093";
}
.fa-lemon-o:before {
  content: "\f094";
}
.fa-phone:before {
  content: "\f095";
}
.fa-square-o:before {
  content: "\f096";
}
.fa-bookmark-o:before {
  content: "\f097";
}
.fa-phone-square:before {
  content: "\f098";
}
.fa-twitter:before {
  content: "\f099";
}
.fa-facebook:before {
  content: "\f09a";
}
.fa-github:before {
  content: "\f09b";
}
.fa-unlock:before {
  content: "\f09c";
}
.fa-credit-card:before {
  content: "\f09d";
}
.fa-rss:before {
  content: "\f09e";
}
.fa-hdd-o:before {
  content: "\f0a0";
}
.fa-bullhorn:before {
  content: "\f0a1";
}
.fa-bell:before {
  content: "\f0f3";
}
.fa-certificate:before {
  content: "\f0a3";
}
.fa-hand-o-right:before {
  content: "\f0a4";
}
.fa-hand-o-left:before {
  content: "\f0a5";
}
.fa-hand-o-up:before {
  content: "\f0a6";
}
.fa-hand-o-down:before {
  content: "\f0a7";
}
.fa-arrow-circle-left:before {
  content: "\f0a8";
}
.fa-arrow-circle-right:before {
  content: "\f0a9";
}
.fa-arrow-circle-up:before {
  content: "\f0aa";
}
.fa-arrow-circle-down:before {
  content: "\f0ab";
}
.fa-globe:before {
  content: "\f0ac";
}
.fa-wrench:before {
  content: "\f0ad";
}
.fa-tasks:before {
  content: "\f0ae";
}
.fa-filter:before {
  content: "\f0b0";
}
.fa-briefcase:before {
  content: "\f0b1";
}
.fa-arrows-alt:before {
  content: "\f0b2";
}
.fa-group:before,
.fa-users:before {
  content: "\f0c0";
}
.fa-chain:before,
.fa-link:before {
  content: "\f0c1";
}
.fa-cloud:before {
  content: "\f0c2";
}
.fa-flask:before {
  content: "\f0c3";
}
.fa-cut:before,
.fa-scissors:before {
  content: "\f0c4";
}
.fa-copy:before,
.fa-files-o:before {
  content: "\f0c5";
}
.fa-paperclip:before {
  content: "\f0c6";
}
.fa-save:before,
.fa-floppy-o:before {
  content: "\f0c7";
}
.fa-square:before {
  content: "\f0c8";
}
.fa-navicon:before,
.fa-reorder:before,
.fa-bars:before {
  content: "\f0c9";
}
.fa-list-ul:before {
  content: "\f0ca";
}
.fa-list-ol:before {
  content: "\f0cb";
}
.fa-strikethrough:before {
  content: "\f0cc";
}
.fa-underline:before {
  content: "\f0cd";
}
.fa-table:before {
  content: "\f0ce";
}
.fa-magic:before {
  content: "\f0d0";
}
.fa-truck:before {
  content: "\f0d1";
}
.fa-pinterest:before {
  content: "\f0d2";
}
.fa-pinterest-square:before {
  content: "\f0d3";
}
.fa-google-plus-square:before {
  content: "\f0d4";
}
.fa-google-plus:before {
  content: "\f0d5";
}
.fa-money:before {
  content: "\f0d6";
}
.fa-caret-down:before {
  content: "\f0d7";
}
.fa-caret-up:before {
  content: "\f0d8";
}
.fa-caret-left:before {
  content: "\f0d9";
}
.fa-caret-right:before {
  content: "\f0da";
}
.fa-columns:before {
  content: "\f0db";
}
.fa-unsorted:before,
.fa-sort:before {
  content: "\f0dc";
}
.fa-sort-down:before,
.fa-sort-desc:before {
  content: "\f0dd";
}
.fa-sort-up:before,
.fa-sort-asc:before {
  content: "\f0de";
}
.fa-envelope:before {
  content: "\f0e0";
}
.fa-linkedin:before {
  content: "\f0e1";
}
.fa-rotate-left:before,
.fa-undo:before {
  content: "\f0e2";
}
.fa-legal:before,
.fa-gavel:before {
  content: "\f0e3";
}
.fa-dashboard:before,
.fa-tachometer:before {
  content: "\f0e4";
}
.fa-comment-o:before {
  content: "\f0e5";
}
.fa-comments-o:before {
  content: "\f0e6";
}
.fa-flash:before,
.fa-bolt:before {
  content: "\f0e7";
}
.fa-sitemap:before {
  content: "\f0e8";
}
.fa-umbrella:before {
  content: "\f0e9";
}
.fa-paste:before,
.fa-clipboard:before {
  content: "\f0ea";
}
.fa-lightbulb-o:before {
  content: "\f0eb";
}
.fa-exchange:before {
  content: "\f0ec";
}
.fa-cloud-download:before {
  content: "\f0ed";
}
.fa-cloud-upload:before {
  content: "\f0ee";
}
.fa-user-md:before {
  content: "\f0f0";
}
.fa-stethoscope:before {
  content: "\f0f1";
}
.fa-suitcase:before {
  content: "\f0f2";
}
.fa-bell-o:before {
  content: "\f0a2";
}
.fa-coffee:before {
  content: "\f0f4";
}
.fa-cutlery:before {
  content: "\f0f5";
}
.fa-file-text-o:before {
  content: "\f0f6";
}
.fa-building-o:before {
  content: "\f0f7";
}
.fa-hospital-o:before {
  content: "\f0f8";
}
.fa-ambulance:before {
  content: "\f0f9";
}
.fa-medkit:before {
  content: "\f0fa";
}
.fa-fighter-jet:before {
  content: "\f0fb";
}
.fa-beer:before {
  content: "\f0fc";
}
.fa-h-square:before {
  content: "\f0fd";
}
.fa-plus-square:before {
  content: "\f0fe";
}
.fa-angle-double-left:before {
  content: "\f100";
}
.fa-angle-double-right:before {
  content: "\f101";
}
.fa-angle-double-up:before {
  content: "\f102";
}
.fa-angle-double-down:before {
  content: "\f103";
}
.fa-angle-left:before {
  content: "\f104";
}
.fa-angle-right:before {
  content: "\f105";
}
.fa-angle-up:before {
  content: "\f106";
}
.fa-angle-down:before {
  content: "\f107";
}
.fa-desktop:before {
  content: "\f108";
}
.fa-laptop:before {
  content: "\f109";
}
.fa-tablet:before {
  content: "\f10a";
}
.fa-mobile-phone:before,
.fa-mobile:before {
  content: "\f10b";
}
.fa-circle-o:before {
  content: "\f10c";
}
.fa-quote-left:before {
  content: "\f10d";
}
.fa-quote-right:before {
  content: "\f10e";
}
.fa-spinner:before {
  content: "\f110";
}
.fa-circle:before {
  content: "\f111";
}
.fa-mail-reply:before,
.fa-reply:before {
  content: "\f112";
}
.fa-github-alt:before {
  content: "\f113";
}
.fa-folder-o:before {
  content: "\f114";
}
.fa-folder-open-o:before {
  content: "\f115";
}
.fa-smile-o:before {
  content: "\f118";
}
.fa-frown-o:before {
  content: "\f119";
}
.fa-meh-o:before {
  content: "\f11a";
}
.fa-gamepad:before {
  content: "\f11b";
}
.fa-keyboard-o:before {
  content: "\f11c";
}
.fa-flag-o:before {
  content: "\f11d";
}
.fa-flag-checkered:before {
  content: "\f11e";
}
.fa-terminal:before {
  content: "\f120";
}
.fa-code:before {
  content: "\f121";
}
.fa-mail-reply-all:before,
.fa-reply-all:before {
  content: "\f122";
}
.fa-star-half-empty:before,
.fa-star-half-full:before,
.fa-star-half-o:before {
  content: "\f123";
}
.fa-location-arrow:before {
  content: "\f124";
}
.fa-crop:before {
  content: "\f125";
}
.fa-code-fork:before {
  content: "\f126";
}
.fa-unlink:before,
.fa-chain-broken:before {
  content: "\f127";
}
.fa-question:before {
  content: "\f128";
}
.fa-info:before {
  content: "\f129";
}
.fa-exclamation:before {
  content: "\f12a";
}
.fa-superscript:before {
  content: "\f12b";
}
.fa-subscript:before {
  content: "\f12c";
}
.fa-eraser:before {
  content: "\f12d";
}
.fa-puzzle-piece:before {
  content: "\f12e";
}
.fa-microphone:before {
  content: "\f130";
}
.fa-microphone-slash:before {
  content: "\f131";
}
.fa-shield:before {
  content: "\f132";
}
.fa-calendar-o:before {
  content: "\f133";
}
.fa-fire-extinguisher:before {
  content: "\f134";
}
.fa-rocket:before {
  content: "\f135";
}
.fa-maxcdn:before {
  content: "\f136";
}
.fa-chevron-circle-left:before {
  content: "\f137";
}
.fa-chevron-circle-right:before {
  content: "\f138";
}
.fa-chevron-circle-up:before {
  content: "\f139";
}
.fa-chevron-circle-down:before {
  content: "\f13a";
}
.fa-html5:before {
  content: "\f13b";
}
.fa-css3:before {
  content: "\f13c";
}
.fa-anchor:before {
  content: "\f13d";
}
.fa-unlock-alt:before {
  content: "\f13e";
}
.fa-bullseye:before {
  content: "\f140";
}
.fa-ellipsis-h:before {
  content: "\f141";
}
.fa-ellipsis-v:before {
  content: "\f142";
}
.fa-rss-square:before {
  content: "\f143";
}
.fa-play-circle:before {
  content: "\f144";
}
.fa-ticket:before {
  content: "\f145";
}
.fa-minus-square:before {
  content: "\f146";
}
.fa-minus-square-o:before {
  content: "\f147";
}
.fa-level-up:before {
  content: "\f148";
}
.fa-level-down:before {
  content: "\f149";
}
.fa-check-square:before {
  content: "\f14a";
}
.fa-pencil-square:before {
  content: "\f14b";
}
.fa-external-link-square:before {
  content: "\f14c";
}
.fa-share-square:before {
  content: "\f14d";
}
.fa-compass:before {
  content: "\f14e";
}
.fa-toggle-down:before,
.fa-caret-square-o-down:before {
  content: "\f150";
}
.fa-toggle-up:before,
.fa-caret-square-o-up:before {
  content: "\f151";
}
.fa-toggle-right:before,
.fa-caret-square-o-right:before {
  content: "\f152";
}
.fa-euro:before,
.fa-eur:before {
  content: "\f153";
}
.fa-gbp:before {
  content: "\f154";
}
.fa-dollar:before,
.fa-usd:before {
  content: "\f155";
}
.fa-rupee:before,
.fa-inr:before {
  content: "\f156";
}
.fa-cny:before,
.fa-rmb:before,
.fa-yen:before,
.fa-jpy:before {
  content: "\f157";
}
.fa-ruble:before,
.fa-rouble:before,
.fa-rub:before {
  content: "\f158";
}
.fa-won:before,
.fa-krw:before {
  content: "\f159";
}
.fa-bitcoin:before,
.fa-btc:before {
  content: "\f15a";
}
.fa-file:before {
  content: "\f15b";
}
.fa-file-text:before {
  content: "\f15c";
}
.fa-sort-alpha-asc:before {
  content: "\f15d";
}
.fa-sort-alpha-desc:before {
  content: "\f15e";
}
.fa-sort-amount-asc:before {
  content: "\f160";
}
.fa-sort-amount-desc:before {
  content: "\f161";
}
.fa-sort-numeric-asc:before {
  content: "\f162";
}
.fa-sort-numeric-desc:before {
  content: "\f163";
}
.fa-thumbs-up:before {
  content: "\f164";
}
.fa-thumbs-down:before {
  content: "\f165";
}
.fa-youtube-square:before {
  content: "\f166";
}
.fa-youtube:before {
  content: "\f167";
}
.fa-xing:before {
  content: "\f168";
}
.fa-xing-square:before {
  content: "\f169";
}
.fa-youtube-play:before {
  content: "\f16a";
}
.fa-dropbox:before {
  content: "\f16b";
}
.fa-stack-overflow:before {
  content: "\f16c";
}
.fa-instagram:before {
  content: "\f16d";
}
.fa-flickr:before {
  content: "\f16e";
}
.fa-adn:before {
  content: "\f170";
}
.fa-bitbucket:before {
  content: "\f171";
}
.fa-bitbucket-square:before {
  content: "\f172";
}
.fa-tumblr:before {
  content: "\f173";
}
.fa-tumblr-square:before {
  content: "\f174";
}
.fa-long-arrow-down:before {
  content: "\f175";
}
.fa-long-arrow-up:before {
  content: "\f176";
}
.fa-long-arrow-left:before {
  content: "\f177";
}
.fa-long-arrow-right:before {
  content: "\f178";
}
.fa-apple:before {
  content: "\f179";
}
.fa-windows:before {
  content: "\f17a";
}
.fa-android:before {
  content: "\f17b";
}
.fa-linux:before {
  content: "\f17c";
}
.fa-dribbble:before {
  content: "\f17d";
}
.fa-skype:before {
  content: "\f17e";
}
.fa-foursquare:before {
  content: "\f180";
}
.fa-trello:before {
  content: "\f181";
}
.fa-female:before {
  content: "\f182";
}
.fa-male:before {
  content: "\f183";
}
.fa-gittip:before {
  content: "\f184";
}
.fa-sun-o:before {
  content: "\f185";
}
.fa-moon-o:before {
  content: "\f186";
}
.fa-archive:before {
  content: "\f187";
}
.fa-bug:before {
  content: "\f188";
}
.fa-vk:before {
  content: "\f189";
}
.fa-weibo:before {
  content: "\f18a";
}
.fa-renren:before {
  content: "\f18b";
}
.fa-pagelines:before {
  content: "\f18c";
}
.fa-stack-exchange:before {
  content: "\f18d";
}
.fa-arrow-circle-o-right:before {
  content: "\f18e";
}
.fa-arrow-circle-o-left:before {
  content: "\f190";
}
.fa-toggle-left:before,
.fa-caret-square-o-left:before {
  content: "\f191";
}
.fa-dot-circle-o:before {
  content: "\f192";
}
.fa-wheelchair:before {
  content: "\f193";
}
.fa-vimeo-square:before {
  content: "\f194";
}
.fa-turkish-lira:before,
.fa-try:before {
  content: "\f195";
}
.fa-plus-square-o:before {
  content: "\f196";
}
.fa-space-shuttle:before {
  content: "\f197";
}
.fa-slack:before {
  content: "\f198";
}
.fa-envelope-square:before {
  content: "\f199";
}
.fa-wordpress:before {
  content: "\f19a";
}
.fa-openid:before {
  content: "\f19b";
}
.fa-institution:before,
.fa-bank:before,
.fa-university:before {
  content: "\f19c";
}
.fa-mortar-board:before,
.fa-graduation-cap:before {
  content: "\f19d";
}
.fa-yahoo:before {
  content: "\f19e";
}
.fa-google:before {
  content: "\f1a0";
}
.fa-reddit:before {
  content: "\f1a1";
}
.fa-reddit-square:before {
  content: "\f1a2";
}
.fa-stumbleupon-circle:before {
  content: "\f1a3";
}
.fa-stumbleupon:before {
  content: "\f1a4";
}
.fa-delicious:before {
  content: "\f1a5";
}
.fa-digg:before {
  content: "\f1a6";
}
.fa-pied-piper:before {
  content: "\f1a7";
}
.fa-pied-piper-alt:before {
  content: "\f1a8";
}
.fa-drupal:before {
  content: "\f1a9";
}
.fa-joomla:before {
  content: "\f1aa";
}
.fa-language:before {
  content: "\f1ab";
}
.fa-fax:before {
  content: "\f1ac";
}
.fa-building:before {
  content: "\f1ad";
}
.fa-child:before {
  content: "\f1ae";
}
.fa-paw:before {
  content: "\f1b0";
}
.fa-spoon:before {
  content: "\f1b1";
}
.fa-cube:before {
  content: "\f1b2";
}
.fa-cubes:before {
  content: "\f1b3";
}
.fa-behance:before {
  content: "\f1b4";
}
.fa-behance-square:before {
  content: "\f1b5";
}
.fa-steam:before {
  content: "\f1b6";
}
.fa-steam-square:before {
  content: "\f1b7";
}
.fa-recycle:before {
  content: "\f1b8";
}
.fa-automobile:before,
.fa-car:before {
  content: "\f1b9";
}
.fa-cab:before,
.fa-taxi:before {
  content: "\f1ba";
}
.fa-tree:before {
  content: "\f1bb";
}
.fa-spotify:before {
  content: "\f1bc";
}
.fa-deviantart:before {
  content: "\f1bd";
}
.fa-soundcloud:before {
  content: "\f1be";
}
.fa-database:before {
  content: "\f1c0";
}
.fa-file-pdf-o:before {
  content: "\f1c1";
}
.fa-file-word-o:before {
  content: "\f1c2";
}
.fa-file-excel-o:before {
  content: "\f1c3";
}
.fa-file-powerpoint-o:before {
  content: "\f1c4";
}
.fa-file-photo-o:before,
.fa-file-picture-o:before,
.fa-file-image-o:before {
  content: "\f1c5";
}
.fa-file-zip-o:before,
.fa-file-archive-o:before {
  content: "\f1c6";
}
.fa-file-sound-o:before,
.fa-file-audio-o:before {
  content: "\f1c7";
}
.fa-file-movie-o:before,
.fa-file-video-o:before {
  content: "\f1c8";
}
.fa-file-code-o:before {
  content: "\f1c9";
}
.fa-vine:before {
  content: "\f1ca";
}
.fa-codepen:before {
  content: "\f1cb";
}
.fa-jsfiddle:before {
  content: "\f1cc";
}
.fa-life-bouy:before,
.fa-life-buoy:before,
.fa-life-saver:before,
.fa-support:before,
.fa-life-ring:before {
  content: "\f1cd";
}
.fa-circle-o-notch:before {
  content: "\f1ce";
}
.fa-ra:before,
.fa-rebel:before {
  content: "\f1d0";
}
.fa-ge:before,
.fa-empire:before {
  content: "\f1d1";
}
.fa-git-square:before {
  content: "\f1d2";
}
.fa-git:before {
  content: "\f1d3";
}
.fa-hacker-news:before {
  content: "\f1d4";
}
.fa-tencent-weibo:before {
  content: "\f1d5";
}
.fa-qq:before {
  content: "\f1d6";
}
.fa-wechat:before,
.fa-weixin:before {
  content: "\f1d7";
}
.fa-send:before,
.fa-paper-plane:before {
  content: "\f1d8";
}
.fa-send-o:before,
.fa-paper-plane-o:before {
  content: "\f1d9";
}
.fa-history:before {
  content: "\f1da";
}
.fa-circle-thin:before {
  content: "\f1db";
}
.fa-header:before {
  content: "\f1dc";
}
.fa-paragraph:before {
  content: "\f1dd";
}
.fa-sliders:before {
  content: "\f1de";
}
.fa-share-alt:before {
  content: "\f1e0";
}
.fa-share-alt-square:before {
  content: "\f1e1";
}
.fa-bomb:before {
  content: "\f1e2";
}
.fa-soccer-ball-o:before,
.fa-futbol-o:before {
  content: "\f1e3";
}
.fa-tty:before {
  content: "\f1e4";
}
.fa-binoculars:before {
  content: "\f1e5";
}
.fa-plug:before {
  content: "\f1e6";
}
.fa-slideshare:before {
  content: "\f1e7";
}
.fa-twitch:before {
  content: "\f1e8";
}
.fa-yelp:before {
  content: "\f1e9";
}
.fa-newspaper-o:before {
  content: "\f1ea";
}
.fa-wifi:before {
  content: "\f1eb";
}
.fa-calculator:before {
  content: "\f1ec";
}
.fa-paypal:before {
  content: "\f1ed";
}
.fa-google-wallet:before {
  content: "\f1ee";
}
.fa-cc-visa:before {
  content: "\f1f0";
}
.fa-cc-mastercard:before {
  content: "\f1f1";
}
.fa-cc-discover:before {
  content: "\f1f2";
}
.fa-cc-amex:before {
  content: "\f1f3";
}
.fa-cc-paypal:before {
  content: "\f1f4";
}
.fa-cc-stripe:before {
  content: "\f1f5";
}
.fa-bell-slash:before {
  content: "\f1f6";
}
.fa-bell-slash-o:before {
  content: "\f1f7";
}
.fa-trash:before {
  content: "\f1f8";
}
.fa-copyright:before {
  content: "\f1f9";
}
.fa-at:before {
  content: "\f1fa";
}
.fa-eyedropper:before {
  content: "\f1fb";
}
.fa-paint-brush:before {
  content: "\f1fc";
}
.fa-birthday-cake:before {
  content: "\f1fd";
}
.fa-area-chart:before {
  content: "\f1fe";
}
.fa-pie-chart:before {
  content: "\f200";
}
.fa-line-chart:before {
  content: "\f201";
}
.fa-lastfm:before {
  content: "\f202";
}
.fa-lastfm-square:before {
  content: "\f203";
}
.fa-toggle-off:before {
  content: "\f204";
}
.fa-toggle-on:before {
  content: "\f205";
}
.fa-bicycle:before {
  content: "\f206";
}
.fa-bus:before {
  content: "\f207";
}
.fa-ioxhost:before {
  content: "\f208";
}
.fa-angellist:before {
  content: "\f209";
}
.fa-cc:before {
  content: "\f20a";
}
.fa-shekel:before,
.fa-sheqel:before,
.fa-ils:before {
  content: "\f20b";
}
.fa-meanpath:before {
  content: "\f20c";
}
/*!
*
* IPython base
*
*/
.modal.fade .modal-dialog {
  -webkit-transform: translate(0, 0);
  -ms-transform: translate(0, 0);
  -o-transform: translate(0, 0);
  transform: translate(0, 0);
}
code {
  color: #000;
}
pre {
  font-size: inherit;
  line-height: inherit;
}
label {
  font-weight: normal;
}
/* Make the page background atleast 100% the height of the view port */
/* Make the page itself atleast 70% the height of the view port */
.border-box-sizing {
  box-sizing: border-box;
  -moz-box-sizing: border-box;
  -webkit-box-sizing: border-box;
}
.corner-all {
  border-radius: 2px;
}
.no-padding {
  padding: 0px;
}
/* Flexible box model classes */
/* Taken from Alex Russell http://infrequently.org/2009/08/css-3-progress/ */
/* This file is a compatability layer.  It allows the usage of flexible box 
model layouts accross multiple browsers, including older browsers.  The newest,
universal implementation of the flexible box model is used when available (see
`Modern browsers` comments below).  Browsers that are known to implement this 
new spec completely include:

    Firefox 28.0+
    Chrome 29.0+
    Internet Explorer 11+ 
    Opera 17.0+

Browsers not listed, including Safari, are supported via the styling under the
`Old browsers` comments below.
*/
.hbox {
  /* Old browsers */
  display: -webkit-box;
  -webkit-box-orient: horizontal;
  -webkit-box-align: stretch;
  display: -moz-box;
  -moz-box-orient: horizontal;
  -moz-box-align: stretch;
  display: box;
  box-orient: horizontal;
  box-align: stretch;
  /* Modern browsers */
  display: flex;
  flex-direction: row;
  align-items: stretch;
}
.hbox > * {
  /* Old browsers */
  -webkit-box-flex: 0;
  -moz-box-flex: 0;
  box-flex: 0;
  /* Modern browsers */
  flex: none;
}
.vbox {
  /* Old browsers */
  display: -webkit-box;
  -webkit-box-orient: vertical;
  -webkit-box-align: stretch;
  display: -moz-box;
  -moz-box-orient: vertical;
  -moz-box-align: stretch;
  display: box;
  box-orient: vertical;
  box-align: stretch;
  /* Modern browsers */
  display: flex;
  flex-direction: column;
  align-items: stretch;
}
.vbox > * {
  /* Old browsers */
  -webkit-box-flex: 0;
  -moz-box-flex: 0;
  box-flex: 0;
  /* Modern browsers */
  flex: none;
}
.hbox.reverse,
.vbox.reverse,
.reverse {
  /* Old browsers */
  -webkit-box-direction: reverse;
  -moz-box-direction: reverse;
  box-direction: reverse;
  /* Modern browsers */
  flex-direction: row-reverse;
}
.hbox.box-flex0,
.vbox.box-flex0,
.box-flex0 {
  /* Old browsers */
  -webkit-box-flex: 0;
  -moz-box-flex: 0;
  box-flex: 0;
  /* Modern browsers */
  flex: none;
  width: auto;
}
.hbox.box-flex1,
.vbox.box-flex1,
.box-flex1 {
  /* Old browsers */
  -webkit-box-flex: 1;
  -moz-box-flex: 1;
  box-flex: 1;
  /* Modern browsers */
  flex: 1;
}
.hbox.box-flex,
.vbox.box-flex,
.box-flex {
  /* Old browsers */
  /* Old browsers */
  -webkit-box-flex: 1;
  -moz-box-flex: 1;
  box-flex: 1;
  /* Modern browsers */
  flex: 1;
}
.hbox.box-flex2,
.vbox.box-flex2,
.box-flex2 {
  /* Old browsers */
  -webkit-box-flex: 2;
  -moz-box-flex: 2;
  box-flex: 2;
  /* Modern browsers */
  flex: 2;
}
.box-group1 {
  /*  Deprecated */
  -webkit-box-flex-group: 1;
  -moz-box-flex-group: 1;
  box-flex-group: 1;
}
.box-group2 {
  /* Deprecated */
  -webkit-box-flex-group: 2;
  -moz-box-flex-group: 2;
  box-flex-group: 2;
}
.hbox.start,
.vbox.start,
.start {
  /* Old browsers */
  -webkit-box-pack: start;
  -moz-box-pack: start;
  box-pack: start;
  /* Modern browsers */
  justify-content: flex-start;
}
.hbox.end,
.vbox.end,
.end {
  /* Old browsers */
  -webkit-box-pack: end;
  -moz-box-pack: end;
  box-pack: end;
  /* Modern browsers */
  justify-content: flex-end;
}
.hbox.center,
.vbox.center,
.center {
  /* Old browsers */
  -webkit-box-pack: center;
  -moz-box-pack: center;
  box-pack: center;
  /* Modern browsers */
  justify-content: center;
}
.hbox.baseline,
.vbox.baseline,
.baseline {
  /* Old browsers */
  -webkit-box-pack: baseline;
  -moz-box-pack: baseline;
  box-pack: baseline;
  /* Modern browsers */
  justify-content: baseline;
}
.hbox.stretch,
.vbox.stretch,
.stretch {
  /* Old browsers */
  -webkit-box-pack: stretch;
  -moz-box-pack: stretch;
  box-pack: stretch;
  /* Modern browsers */
  justify-content: stretch;
}
.hbox.align-start,
.vbox.align-start,
.align-start {
  /* Old browsers */
  -webkit-box-align: start;
  -moz-box-align: start;
  box-align: start;
  /* Modern browsers */
  align-items: flex-start;
}
.hbox.align-end,
.vbox.align-end,
.align-end {
  /* Old browsers */
  -webkit-box-align: end;
  -moz-box-align: end;
  box-align: end;
  /* Modern browsers */
  align-items: flex-end;
}
.hbox.align-center,
.vbox.align-center,
.align-center {
  /* Old browsers */
  -webkit-box-align: center;
  -moz-box-align: center;
  box-align: center;
  /* Modern browsers */
  align-items: center;
}
.hbox.align-baseline,
.vbox.align-baseline,
.align-baseline {
  /* Old browsers */
  -webkit-box-align: baseline;
  -moz-box-align: baseline;
  box-align: baseline;
  /* Modern browsers */
  align-items: baseline;
}
.hbox.align-stretch,
.vbox.align-stretch,
.align-stretch {
  /* Old browsers */
  -webkit-box-align: stretch;
  -moz-box-align: stretch;
  box-align: stretch;
  /* Modern browsers */
  align-items: stretch;
}
div.error {
  margin: 2em;
  text-align: center;
}
div.error > h1 {
  font-size: 500%;
  line-height: normal;
}
div.error > p {
  font-size: 200%;
  line-height: normal;
}
div.traceback-wrapper {
  text-align: left;
  max-width: 800px;
  margin: auto;
}
/**
 * Primary styles
 *
 * Author: Jupyter Development Team
 */
body {
  background-color: #fff;
  /* This makes sure that the body covers the entire window and needs to
       be in a different element than the display: box in wrapper below */
  position: absolute;
  left: 0px;
  right: 0px;
  top: 0px;
  bottom: 0px;
  overflow: visible;
}
body > #header {
  /* Initially hidden to prevent FLOUC */
  display: none;
  background-color: #fff;
  /* Display over codemirror */
  position: relative;
  z-index: 100;
}
body > #header #header-container {
  padding-bottom: 5px;
  padding-top: 5px;
  box-sizing: border-box;
  -moz-box-sizing: border-box;
  -webkit-box-sizing: border-box;
}
body > #header .header-bar {
  width: 100%;
  height: 1px;
  background: #e7e7e7;
  margin-bottom: -1px;
}
@media print {
  body > #header {
    display: none !important;
  }
}
#header-spacer {
  width: 100%;
  visibility: hidden;
}
@media print {
  #header-spacer {
    display: none;
  }
}
#ipython_notebook {
  padding-left: 0px;
  padding-top: 1px;
  padding-bottom: 1px;
}
@media (max-width: 991px) {
  #ipython_notebook {
    margin-left: 10px;
  }
}
[dir="rtl"] #ipython_notebook {
  float: right !important;
}
#noscript {
  width: auto;
  padding-top: 16px;
  padding-bottom: 16px;
  text-align: center;
  font-size: 22px;
  color: red;
  font-weight: bold;
}
#ipython_notebook img {
  height: 28px;
}
#site {
  width: 100%;
  display: none;
  box-sizing: border-box;
  -moz-box-sizing: border-box;
  -webkit-box-sizing: border-box;
  overflow: auto;
}
@media print {
  #site {
    height: auto !important;
  }
}
/* Smaller buttons */
.ui-button .ui-button-text {
  padding: 0.2em 0.8em;
  font-size: 77%;
}
input.ui-button {
  padding: 0.3em 0.9em;
}
span#login_widget {
  float: right;
}
span#login_widget > .button,
#logout {
  color: #333;
  background-color: #fff;
  border-color: #ccc;
}
span#login_widget > .button:focus,
#logout:focus,
span#login_widget > .button.focus,
#logout.focus {
  color: #333;
  background-color: #e6e6e6;
  border-color: #8c8c8c;
}
span#login_widget > .button:hover,
#logout:hover {
  color: #333;
  background-color: #e6e6e6;
  border-color: #adadad;
}
span#login_widget > .button:active,
#logout:active,
span#login_widget > .button.active,
#logout.active,
.open > .dropdown-togglespan#login_widget > .button,
.open > .dropdown-toggle#logout {
  color: #333;
  background-color: #e6e6e6;
  border-color: #adadad;
}
span#login_widget > .button:active:hover,
#logout:active:hover,
span#login_widget > .button.active:hover,
#logout.active:hover,
.open > .dropdown-togglespan#login_widget > .button:hover,
.open > .dropdown-toggle#logout:hover,
span#login_widget > .button:active:focus,
#logout:active:focus,
span#login_widget > .button.active:focus,
#logout.active:focus,
.open > .dropdown-togglespan#login_widget > .button:focus,
.open > .dropdown-toggle#logout:focus,
span#login_widget > .button:active.focus,
#logout:active.focus,
span#login_widget > .button.active.focus,
#logout.active.focus,
.open > .dropdown-togglespan#login_widget > .button.focus,
.open > .dropdown-toggle#logout.focus {
  color: #333;
  background-color: #d4d4d4;
  border-color: #8c8c8c;
}
span#login_widget > .button:active,
#logout:active,
span#login_widget > .button.active,
#logout.active,
.open > .dropdown-togglespan#login_widget > .button,
.open > .dropdown-toggle#logout {
  background-image: none;
}
span#login_widget > .button.disabled:hover,
#logout.disabled:hover,
span#login_widget > .button[disabled]:hover,
#logout[disabled]:hover,
fieldset[disabled] span#login_widget > .button:hover,
fieldset[disabled] #logout:hover,
span#login_widget > .button.disabled:focus,
#logout.disabled:focus,
span#login_widget > .button[disabled]:focus,
#logout[disabled]:focus,
fieldset[disabled] span#login_widget > .button:focus,
fieldset[disabled] #logout:focus,
span#login_widget > .button.disabled.focus,
#logout.disabled.focus,
span#login_widget > .button[disabled].focus,
#logout[disabled].focus,
fieldset[disabled] span#login_widget > .button.focus,
fieldset[disabled] #logout.focus {
  background-color: #fff;
  border-color: #ccc;
}
span#login_widget > .button .badge,
#logout .badge {
  color: #fff;
  background-color: #333;
}
.nav-header {
  text-transform: none;
}
#header > span {
  margin-top: 10px;
}
.modal_stretch .modal-dialog {
  /* Old browsers */
  display: -webkit-box;
  -webkit-box-orient: vertical;
  -webkit-box-align: stretch;
  display: -moz-box;
  -moz-box-orient: vertical;
  -moz-box-align: stretch;
  display: box;
  box-orient: vertical;
  box-align: stretch;
  /* Modern browsers */
  display: flex;
  flex-direction: column;
  align-items: stretch;
  min-height: 80vh;
}
.modal_stretch .modal-dialog .modal-body {
  max-height: calc(100vh - 200px);
  overflow: auto;
  flex: 1;
}
@media (min-width: 768px) {
  .modal .modal-dialog {
    width: 700px;
  }
}
@media (min-width: 768px) {
  select.form-control {
    margin-left: 12px;
    margin-right: 12px;
  }
}
/*!
*
* IPython auth
*
*/
.center-nav {
  display: inline-block;
  margin-bottom: -4px;
}
/*!
*
* IPython tree view
*
*/
/* We need an invisible input field on top of the sentense*/
/* "Drag file onto the list ..." */
.alternate_upload {
  background-color: none;
  display: inline;
}
.alternate_upload.form {
  padding: 0;
  margin: 0;
}
.alternate_upload input.fileinput {
  text-align: center;
  vertical-align: middle;
  display: inline;
  opacity: 0;
  z-index: 2;
  width: 12ex;
  margin-right: -12ex;
}
.alternate_upload .btn-upload {
  height: 22px;
}
/**
 * Primary styles
 *
 * Author: Jupyter Development Team
 */
[dir="rtl"] #tabs li {
  float: right;
}
ul#tabs {
  margin-bottom: 4px;
}
[dir="rtl"] ul#tabs {
  margin-right: 0px;
}
ul#tabs a {
  padding-top: 6px;
  padding-bottom: 4px;
}
ul.breadcrumb a:focus,
ul.breadcrumb a:hover {
  text-decoration: none;
}
ul.breadcrumb i.icon-home {
  font-size: 16px;
  margin-right: 4px;
}
ul.breadcrumb span {
  color: #5e5e5e;
}
.list_toolbar {
  padding: 4px 0 4px 0;
  vertical-align: middle;
}
.list_toolbar .tree-buttons {
  padding-top: 1px;
}
[dir="rtl"] .list_toolbar .tree-buttons {
  float: left !important;
}
[dir="rtl"] .list_toolbar .pull-right {
  padding-top: 1px;
  float: left !important;
}
[dir="rtl"] .list_toolbar .pull-left {
  float: right !important;
}
.dynamic-buttons {
  padding-top: 3px;
  display: inline-block;
}
.list_toolbar [class*="span"] {
  min-height: 24px;
}
.list_header {
  font-weight: bold;
  background-color: #EEE;
}
.list_placeholder {
  font-weight: bold;
  padding-top: 4px;
  padding-bottom: 4px;
  padding-left: 7px;
  padding-right: 7px;
}
.list_container {
  margin-top: 4px;
  margin-bottom: 20px;
  border: 1px solid #ddd;
  border-radius: 2px;
}
.list_container > div {
  border-bottom: 1px solid #ddd;
}
.list_container > div:hover .list-item {
  background-color: red;
}
.list_container > div:last-child {
  border: none;
}
.list_item:hover .list_item {
  background-color: #ddd;
}
.list_item a {
  text-decoration: none;
}
.list_item:hover {
  background-color: #fafafa;
}
.list_header > div,
.list_item > div {
  padding-top: 4px;
  padding-bottom: 4px;
  padding-left: 7px;
  padding-right: 7px;
  line-height: 22px;
}
.list_header > div input,
.list_item > div input {
  margin-right: 7px;
  margin-left: 14px;
  vertical-align: baseline;
  line-height: 22px;
  position: relative;
  top: -1px;
}
.list_header > div .item_link,
.list_item > div .item_link {
  margin-left: -1px;
  vertical-align: baseline;
  line-height: 22px;
}
.new-file input[type=checkbox] {
  visibility: hidden;
}
.item_name {
  line-height: 22px;
  height: 24px;
}
.item_icon {
  font-size: 14px;
  color: #5e5e5e;
  margin-right: 7px;
  margin-left: 7px;
  line-height: 22px;
  vertical-align: baseline;
}
.item_buttons {
  line-height: 1em;
  margin-left: -5px;
}
.item_buttons .btn,
.item_buttons .btn-group,
.item_buttons .input-group {
  float: left;
}
.item_buttons > .btn,
.item_buttons > .btn-group,
.item_buttons > .input-group {
  margin-left: 5px;
}
.item_buttons .btn {
  min-width: 13ex;
}
.item_buttons .running-indicator {
  padding-top: 4px;
  color: #5cb85c;
}
.item_buttons .kernel-name {
  padding-top: 4px;
  color: #5bc0de;
  margin-right: 7px;
  float: left;
}
.toolbar_info {
  height: 24px;
  line-height: 24px;
}
.list_item input:not([type=checkbox]) {
  padding-top: 3px;
  padding-bottom: 3px;
  height: 22px;
  line-height: 14px;
  margin: 0px;
}
.highlight_text {
  color: blue;
}
#project_name {
  display: inline-block;
  padding-left: 7px;
  margin-left: -2px;
}
#project_name > .breadcrumb {
  padding: 0px;
  margin-bottom: 0px;
  background-color: transparent;
  font-weight: bold;
}
#tree-selector {
  padding-right: 0px;
}
[dir="rtl"] #tree-selector a {
  float: right;
}
#button-select-all {
  min-width: 50px;
}
#select-all {
  margin-left: 7px;
  margin-right: 2px;
}
.menu_icon {
  margin-right: 2px;
}
.tab-content .row {
  margin-left: 0px;
  margin-right: 0px;
}
.folder_icon:before {
  display: inline-block;
  font: normal normal normal 14px/1 FontAwesome;
  font-size: inherit;
  text-rendering: auto;
  -webkit-font-smoothing: antialiased;
  -moz-osx-font-smoothing: grayscale;
  content: "\f114";
}
.folder_icon:before.pull-left {
  margin-right: .3em;
}
.folder_icon:before.pull-right {
  margin-left: .3em;
}
.notebook_icon:before {
  display: inline-block;
  font: normal normal normal 14px/1 FontAwesome;
  font-size: inherit;
  text-rendering: auto;
  -webkit-font-smoothing: antialiased;
  -moz-osx-font-smoothing: grayscale;
  content: "\f02d";
  position: relative;
  top: -1px;
}
.notebook_icon:before.pull-left {
  margin-right: .3em;
}
.notebook_icon:before.pull-right {
  margin-left: .3em;
}
.running_notebook_icon:before {
  display: inline-block;
  font: normal normal normal 14px/1 FontAwesome;
  font-size: inherit;
  text-rendering: auto;
  -webkit-font-smoothing: antialiased;
  -moz-osx-font-smoothing: grayscale;
  content: "\f02d";
  position: relative;
  top: -1px;
  color: #5cb85c;
}
.running_notebook_icon:before.pull-left {
  margin-right: .3em;
}
.running_notebook_icon:before.pull-right {
  margin-left: .3em;
}
.file_icon:before {
  display: inline-block;
  font: normal normal normal 14px/1 FontAwesome;
  font-size: inherit;
  text-rendering: auto;
  -webkit-font-smoothing: antialiased;
  -moz-osx-font-smoothing: grayscale;
  content: "\f016";
  position: relative;
  top: -2px;
}
.file_icon:before.pull-left {
  margin-right: .3em;
}
.file_icon:before.pull-right {
  margin-left: .3em;
}
#notebook_toolbar .pull-right {
  padding-top: 0px;
  margin-right: -1px;
}
ul#new-menu {
  left: auto;
  right: 0;
}
[dir="rtl"] #new-menu {
  text-align: right;
}
.kernel-menu-icon {
  padding-right: 12px;
  width: 24px;
  content: "\f096";
}
.kernel-menu-icon:before {
  content: "\f096";
}
.kernel-menu-icon-current:before {
  content: "\f00c";
}
#tab_content {
  padding-top: 20px;
}
#running .panel-group .panel {
  margin-top: 3px;
  margin-bottom: 1em;
}
#running .panel-group .panel .panel-heading {
  background-color: #EEE;
  padding-top: 4px;
  padding-bottom: 4px;
  padding-left: 7px;
  padding-right: 7px;
  line-height: 22px;
}
#running .panel-group .panel .panel-heading a:focus,
#running .panel-group .panel .panel-heading a:hover {
  text-decoration: none;
}
#running .panel-group .panel .panel-body {
  padding: 0px;
}
#running .panel-group .panel .panel-body .list_container {
  margin-top: 0px;
  margin-bottom: 0px;
  border: 0px;
  border-radius: 0px;
}
#running .panel-group .panel .panel-body .list_container .list_item {
  border-bottom: 1px solid #ddd;
}
#running .panel-group .panel .panel-body .list_container .list_item:last-child {
  border-bottom: 0px;
}
[dir="rtl"] #running .col-sm-8 {
  float: right !important;
}
.delete-button {
  display: none;
}
.duplicate-button {
  display: none;
}
.rename-button {
  display: none;
}
.shutdown-button {
  display: none;
}
.dynamic-instructions {
  display: inline-block;
  padding-top: 4px;
}
/*!
*
* IPython text editor webapp
*
*/
.selected-keymap i.fa {
  padding: 0px 5px;
}
.selected-keymap i.fa:before {
  content: "\f00c";
}
#mode-menu {
  overflow: auto;
  max-height: 20em;
}
.edit_app #header {
  -webkit-box-shadow: 0px 0px 12px 1px rgba(87, 87, 87, 0.2);
  box-shadow: 0px 0px 12px 1px rgba(87, 87, 87, 0.2);
}
.edit_app #menubar .navbar {
  /* Use a negative 1 bottom margin, so the border overlaps the border of the
    header */
  margin-bottom: -1px;
}
.dirty-indicator {
  display: inline-block;
  font: normal normal normal 14px/1 FontAwesome;
  font-size: inherit;
  text-rendering: auto;
  -webkit-font-smoothing: antialiased;
  -moz-osx-font-smoothing: grayscale;
  width: 20px;
}
.dirty-indicator.pull-left {
  margin-right: .3em;
}
.dirty-indicator.pull-right {
  margin-left: .3em;
}
.dirty-indicator-dirty {
  display: inline-block;
  font: normal normal normal 14px/1 FontAwesome;
  font-size: inherit;
  text-rendering: auto;
  -webkit-font-smoothing: antialiased;
  -moz-osx-font-smoothing: grayscale;
  width: 20px;
}
.dirty-indicator-dirty.pull-left {
  margin-right: .3em;
}
.dirty-indicator-dirty.pull-right {
  margin-left: .3em;
}
.dirty-indicator-clean {
  display: inline-block;
  font: normal normal normal 14px/1 FontAwesome;
  font-size: inherit;
  text-rendering: auto;
  -webkit-font-smoothing: antialiased;
  -moz-osx-font-smoothing: grayscale;
  width: 20px;
}
.dirty-indicator-clean.pull-left {
  margin-right: .3em;
}
.dirty-indicator-clean.pull-right {
  margin-left: .3em;
}
.dirty-indicator-clean:before {
  display: inline-block;
  font: normal normal normal 14px/1 FontAwesome;
  font-size: inherit;
  text-rendering: auto;
  -webkit-font-smoothing: antialiased;
  -moz-osx-font-smoothing: grayscale;
  content: "\f00c";
}
.dirty-indicator-clean:before.pull-left {
  margin-right: .3em;
}
.dirty-indicator-clean:before.pull-right {
  margin-left: .3em;
}
#filename {
  font-size: 16pt;
  display: table;
  padding: 0px 5px;
}
#current-mode {
  padding-left: 5px;
  padding-right: 5px;
}
#texteditor-backdrop {
  padding-top: 20px;
  padding-bottom: 20px;
}
@media not print {
  #texteditor-backdrop {
    background-color: #EEE;
  }
}
@media print {
  #texteditor-backdrop #texteditor-container .CodeMirror-gutter,
  #texteditor-backdrop #texteditor-container .CodeMirror-gutters {
    background-color: #fff;
  }
}
@media not print {
  #texteditor-backdrop #texteditor-container .CodeMirror-gutter,
  #texteditor-backdrop #texteditor-container .CodeMirror-gutters {
    background-color: #fff;
  }
}
@media not print {
  #texteditor-backdrop #texteditor-container {
    padding: 0px;
    background-color: #fff;
    -webkit-box-shadow: 0px 0px 12px 1px rgba(87, 87, 87, 0.2);
    box-shadow: 0px 0px 12px 1px rgba(87, 87, 87, 0.2);
  }
}
/*!
*
* IPython notebook
*
*/
/* CSS font colors for translated ANSI colors. */
.ansibold {
  font-weight: bold;
}
/* use dark versions for foreground, to improve visibility */
.ansiblack {
  color: black;
}
.ansired {
  color: darkred;
}
.ansigreen {
  color: darkgreen;
}
.ansiyellow {
  color: #c4a000;
}
.ansiblue {
  color: darkblue;
}
.ansipurple {
  color: darkviolet;
}
.ansicyan {
  color: steelblue;
}
.ansigray {
  color: gray;
}
/* and light for background, for the same reason */
.ansibgblack {
  background-color: black;
}
.ansibgred {
  background-color: red;
}
.ansibggreen {
  background-color: green;
}
.ansibgyellow {
  background-color: yellow;
}
.ansibgblue {
  background-color: blue;
}
.ansibgpurple {
  background-color: magenta;
}
.ansibgcyan {
  background-color: cyan;
}
.ansibggray {
  background-color: gray;
}
div.cell {
  /* Old browsers */
  display: -webkit-box;
  -webkit-box-orient: vertical;
  -webkit-box-align: stretch;
  display: -moz-box;
  -moz-box-orient: vertical;
  -moz-box-align: stretch;
  display: box;
  box-orient: vertical;
  box-align: stretch;
  /* Modern browsers */
  display: flex;
  flex-direction: column;
  align-items: stretch;
  border-radius: 2px;
  box-sizing: border-box;
  -moz-box-sizing: border-box;
  -webkit-box-sizing: border-box;
  border-width: 1px;
  border-style: solid;
  border-color: transparent;
  width: 100%;
  padding: 5px;
  /* This acts as a spacer between cells, that is outside the border */
  margin: 0px;
  outline: none;
  border-left-width: 1px;
  padding-left: 5px;
  background: linear-gradient(to right, transparent -40px, transparent 1px, transparent 1px, transparent 100%);
}
div.cell.jupyter-soft-selected {
  border-left-color: #90CAF9;
  border-left-color: #E3F2FD;
  border-left-width: 1px;
  padding-left: 5px;
  border-right-color: #E3F2FD;
  border-right-width: 1px;
  background: #E3F2FD;
}
@media print {
  div.cell.jupyter-soft-selected {
    border-color: transparent;
  }
}
div.cell.selected {
  border-color: #ababab;
  border-left-width: 0px;
  padding-left: 6px;
  background: linear-gradient(to right, #42A5F5 -40px, #42A5F5 5px, transparent 5px, transparent 100%);
}
@media print {
  div.cell.selected {
    border-color: transparent;
  }
}
div.cell.selected.jupyter-soft-selected {
  border-left-width: 0;
  padding-left: 6px;
  background: linear-gradient(to right, #42A5F5 -40px, #42A5F5 7px, #E3F2FD 7px, #E3F2FD 100%);
}
.edit_mode div.cell.selected {
  border-color: #66BB6A;
  border-left-width: 0px;
  padding-left: 6px;
  background: linear-gradient(to right, #66BB6A -40px, #66BB6A 5px, transparent 5px, transparent 100%);
}
@media print {
  .edit_mode div.cell.selected {
    border-color: transparent;
  }
}
.prompt {
  /* This needs to be wide enough for 3 digit prompt numbers: In[100]: */
  min-width: 14ex;
  /* This padding is tuned to match the padding on the CodeMirror editor. */
  padding: 0.4em;
  margin: 0px;
  font-family: monospace;
  text-align: right;
  /* This has to match that of the the CodeMirror class line-height below */
  line-height: 1.21429em;
  /* Don't highlight prompt number selection */
  -webkit-touch-callout: none;
  -webkit-user-select: none;
  -khtml-user-select: none;
  -moz-user-select: none;
  -ms-user-select: none;
  user-select: none;
  /* Use default cursor */
  cursor: default;
}
@media (max-width: 540px) {
  .prompt {
    text-align: left;
  }
}
div.inner_cell {
  min-width: 0;
  /* Old browsers */
  display: -webkit-box;
  -webkit-box-orient: vertical;
  -webkit-box-align: stretch;
  display: -moz-box;
  -moz-box-orient: vertical;
  -moz-box-align: stretch;
  display: box;
  box-orient: vertical;
  box-align: stretch;
  /* Modern browsers */
  display: flex;
  flex-direction: column;
  align-items: stretch;
  /* Old browsers */
  -webkit-box-flex: 1;
  -moz-box-flex: 1;
  box-flex: 1;
  /* Modern browsers */
  flex: 1;
}
/* input_area and input_prompt must match in top border and margin for alignment */
div.input_area {
  border: 1px solid #cfcfcf;
  border-radius: 2px;
  background: #f7f7f7;
  line-height: 1.21429em;
}
/* This is needed so that empty prompt areas can collapse to zero height when there
   is no content in the output_subarea and the prompt. The main purpose of this is
   to make sure that empty JavaScript output_subareas have no height. */
div.prompt:empty {
  padding-top: 0;
  padding-bottom: 0;
}
div.unrecognized_cell {
  padding: 5px 5px 5px 0px;
  /* Old browsers */
  display: -webkit-box;
  -webkit-box-orient: horizontal;
  -webkit-box-align: stretch;
  display: -moz-box;
  -moz-box-orient: horizontal;
  -moz-box-align: stretch;
  display: box;
  box-orient: horizontal;
  box-align: stretch;
  /* Modern browsers */
  display: flex;
  flex-direction: row;
  align-items: stretch;
}
div.unrecognized_cell .inner_cell {
  border-radius: 2px;
  padding: 5px;
  font-weight: bold;
  color: red;
  border: 1px solid #cfcfcf;
  background: #eaeaea;
}
div.unrecognized_cell .inner_cell a {
  color: inherit;
  text-decoration: none;
}
div.unrecognized_cell .inner_cell a:hover {
  color: inherit;
  text-decoration: none;
}
@media (max-width: 540px) {
  div.unrecognized_cell > div.prompt {
    display: none;
  }
}
div.code_cell {
  /* avoid page breaking on code cells when printing */
}
@media print {
  div.code_cell {
    page-break-inside: avoid;
  }
}
/* any special styling for code cells that are currently running goes here */
div.input {
  page-break-inside: avoid;
  /* Old browsers */
  display: -webkit-box;
  -webkit-box-orient: horizontal;
  -webkit-box-align: stretch;
  display: -moz-box;
  -moz-box-orient: horizontal;
  -moz-box-align: stretch;
  display: box;
  box-orient: horizontal;
  box-align: stretch;
  /* Modern browsers */
  display: flex;
  flex-direction: row;
  align-items: stretch;
}
@media (max-width: 540px) {
  div.input {
    /* Old browsers */
    display: -webkit-box;
    -webkit-box-orient: vertical;
    -webkit-box-align: stretch;
    display: -moz-box;
    -moz-box-orient: vertical;
    -moz-box-align: stretch;
    display: box;
    box-orient: vertical;
    box-align: stretch;
    /* Modern browsers */
    display: flex;
    flex-direction: column;
    align-items: stretch;
  }
}
/* input_area and input_prompt must match in top border and margin for alignment */
div.input_prompt {
  color: #303F9F;
  border-top: 1px solid transparent;
}
div.input_area > div.highlight {
  margin: 0.4em;
  border: none;
  padding: 0px;
  background-color: transparent;
}
div.input_area > div.highlight > pre {
  margin: 0px;
  border: none;
  padding: 0px;
  background-color: transparent;
}
/* The following gets added to the <head> if it is detected that the user has a
 * monospace font with inconsistent normal/bold/italic height.  See
 * notebookmain.js.  Such fonts will have keywords vertically offset with
 * respect to the rest of the text.  The user should select a better font.
 * See: https://github.com/ipython/ipython/issues/1503
 *
 * .CodeMirror span {
 *      vertical-align: bottom;
 * }
 */
.CodeMirror {
  line-height: 1.21429em;
  /* Changed from 1em to our global default */
  font-size: 14px;
  height: auto;
  /* Changed to auto to autogrow */
  background: none;
  /* Changed from white to allow our bg to show through */
}
.CodeMirror-scroll {
  /*  The CodeMirror docs are a bit fuzzy on if overflow-y should be hidden or visible.*/
  /*  We have found that if it is visible, vertical scrollbars appear with font size changes.*/
  overflow-y: hidden;
  overflow-x: auto;
}
.CodeMirror-lines {
  /* In CM2, this used to be 0.4em, but in CM3 it went to 4px. We need the em value because */
  /* we have set a different line-height and want this to scale with that. */
  padding: 0.4em;
}
.CodeMirror-linenumber {
  padding: 0 8px 0 4px;
}
.CodeMirror-gutters {
  border-bottom-left-radius: 2px;
  border-top-left-radius: 2px;
}
.CodeMirror pre {
  /* In CM3 this went to 4px from 0 in CM2. We need the 0 value because of how we size */
  /* .CodeMirror-lines */
  padding: 0;
  border: 0;
  border-radius: 0;
}
/*

Original style from softwaremaniacs.org (c) Ivan Sagalaev <Maniac@SoftwareManiacs.Org>
Adapted from GitHub theme

*/
.highlight-base {
  color: #000;
}
.highlight-variable {
  color: #000;
}
.highlight-variable-2 {
  color: #1a1a1a;
}
.highlight-variable-3 {
  color: #333333;
}
.highlight-string {
  color: #BA2121;
}
.highlight-comment {
  color: #408080;
  font-style: italic;
}
.highlight-number {
  color: #080;
}
.highlight-atom {
  color: #88F;
}
.highlight-keyword {
  color: #008000;
  font-weight: bold;
}
.highlight-builtin {
  color: #008000;
}
.highlight-error {
  color: #f00;
}
.highlight-operator {
  color: #AA22FF;
  font-weight: bold;
}
.highlight-meta {
  color: #AA22FF;
}
/* previously not defined, copying from default codemirror */
.highlight-def {
  color: #00f;
}
.highlight-string-2 {
  color: #f50;
}
.highlight-qualifier {
  color: #555;
}
.highlight-bracket {
  color: #997;
}
.highlight-tag {
  color: #170;
}
.highlight-attribute {
  color: #00c;
}
.highlight-header {
  color: blue;
}
.highlight-quote {
  color: #090;
}
.highlight-link {
  color: #00c;
}
/* apply the same style to codemirror */
.cm-s-ipython span.cm-keyword {
  color: #008000;
  font-weight: bold;
}
.cm-s-ipython span.cm-atom {
  color: #88F;
}
.cm-s-ipython span.cm-number {
  color: #080;
}
.cm-s-ipython span.cm-def {
  color: #00f;
}
.cm-s-ipython span.cm-variable {
  color: #000;
}
.cm-s-ipython span.cm-operator {
  color: #AA22FF;
  font-weight: bold;
}
.cm-s-ipython span.cm-variable-2 {
  color: #1a1a1a;
}
.cm-s-ipython span.cm-variable-3 {
  color: #333333;
}
.cm-s-ipython span.cm-comment {
  color: #408080;
  font-style: italic;
}
.cm-s-ipython span.cm-string {
  color: #BA2121;
}
.cm-s-ipython span.cm-string-2 {
  color: #f50;
}
.cm-s-ipython span.cm-meta {
  color: #AA22FF;
}
.cm-s-ipython span.cm-qualifier {
  color: #555;
}
.cm-s-ipython span.cm-builtin {
  color: #008000;
}
.cm-s-ipython span.cm-bracket {
  color: #997;
}
.cm-s-ipython span.cm-tag {
  color: #170;
}
.cm-s-ipython span.cm-attribute {
  color: #00c;
}
.cm-s-ipython span.cm-header {
  color: blue;
}
.cm-s-ipython span.cm-quote {
  color: #090;
}
.cm-s-ipython span.cm-link {
  color: #00c;
}
.cm-s-ipython span.cm-error {
  color: #f00;
}
.cm-s-ipython span.cm-tab {
  background: url(data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAADAAAAAMCAYAAAAkuj5RAAAAAXNSR0IArs4c6QAAAGFJREFUSMft1LsRQFAQheHPowAKoACx3IgEKtaEHujDjORSgWTH/ZOdnZOcM/sgk/kFFWY0qV8foQwS4MKBCS3qR6ixBJvElOobYAtivseIE120FaowJPN75GMu8j/LfMwNjh4HUpwg4LUAAAAASUVORK5CYII=);
  background-position: right;
  background-repeat: no-repeat;
}
div.output_wrapper {
  /* this position must be relative to enable descendents to be absolute within it */
  position: relative;
  /* Old browsers */
  display: -webkit-box;
  -webkit-box-orient: vertical;
  -webkit-box-align: stretch;
  display: -moz-box;
  -moz-box-orient: vertical;
  -moz-box-align: stretch;
  display: box;
  box-orient: vertical;
  box-align: stretch;
  /* Modern browsers */
  display: flex;
  flex-direction: column;
  align-items: stretch;
  z-index: 1;
}
/* class for the output area when it should be height-limited */
div.output_scroll {
  /* ideally, this would be max-height, but FF barfs all over that */
  height: 24em;
  /* FF needs this *and the wrapper* to specify full width, or it will shrinkwrap */
  width: 100%;
  overflow: auto;
  border-radius: 2px;
  -webkit-box-shadow: inset 0 2px 8px rgba(0, 0, 0, 0.8);
  box-shadow: inset 0 2px 8px rgba(0, 0, 0, 0.8);
  display: block;
}
/* output div while it is collapsed */
div.output_collapsed {
  margin: 0px;
  padding: 0px;
  /* Old browsers */
  display: -webkit-box;
  -webkit-box-orient: vertical;
  -webkit-box-align: stretch;
  display: -moz-box;
  -moz-box-orient: vertical;
  -moz-box-align: stretch;
  display: box;
  box-orient: vertical;
  box-align: stretch;
  /* Modern browsers */
  display: flex;
  flex-direction: column;
  align-items: stretch;
}
div.out_prompt_overlay {
  height: 100%;
  padding: 0px 0.4em;
  position: absolute;
  border-radius: 2px;
}
div.out_prompt_overlay:hover {
  /* use inner shadow to get border that is computed the same on WebKit/FF */
  -webkit-box-shadow: inset 0 0 1px #000;
  box-shadow: inset 0 0 1px #000;
  background: rgba(240, 240, 240, 0.5);
}
div.output_prompt {
  color: #D84315;
}
/* This class is the outer container of all output sections. */
div.output_area {
  padding: 0px;
  page-break-inside: avoid;
  /* Old browsers */
  display: -webkit-box;
  -webkit-box-orient: horizontal;
  -webkit-box-align: stretch;
  display: -moz-box;
  -moz-box-orient: horizontal;
  -moz-box-align: stretch;
  display: box;
  box-orient: horizontal;
  box-align: stretch;
  /* Modern browsers */
  display: flex;
  flex-direction: row;
  align-items: stretch;
}
div.output_area .MathJax_Display {
  text-align: left !important;
}
div.output_area .rendered_html table {
  margin-left: 0;
  margin-right: 0;
}
div.output_area .rendered_html img {
  margin-left: 0;
  margin-right: 0;
}
div.output_area img,
div.output_area svg {
  max-width: 100%;
  height: auto;
}
div.output_area img.unconfined,
div.output_area svg.unconfined {
  max-width: none;
}
/* This is needed to protect the pre formating from global settings such
   as that of bootstrap */
.output {
  /* Old browsers */
  display: -webkit-box;
  -webkit-box-orient: vertical;
  -webkit-box-align: stretch;
  display: -moz-box;
  -moz-box-orient: vertical;
  -moz-box-align: stretch;
  display: box;
  box-orient: vertical;
  box-align: stretch;
  /* Modern browsers */
  display: flex;
  flex-direction: column;
  align-items: stretch;
}
@media (max-width: 540px) {
  div.output_area {
    /* Old browsers */
    display: -webkit-box;
    -webkit-box-orient: vertical;
    -webkit-box-align: stretch;
    display: -moz-box;
    -moz-box-orient: vertical;
    -moz-box-align: stretch;
    display: box;
    box-orient: vertical;
    box-align: stretch;
    /* Modern browsers */
    display: flex;
    flex-direction: column;
    align-items: stretch;
  }
}
div.output_area pre {
  margin: 0;
  padding: 0;
  border: 0;
  vertical-align: baseline;
  color: black;
  background-color: transparent;
  border-radius: 0;
}
/* This class is for the output subarea inside the output_area and after
   the prompt div. */
div.output_subarea {
  overflow-x: auto;
  padding: 0.4em;
  /* Old browsers */
  -webkit-box-flex: 1;
  -moz-box-flex: 1;
  box-flex: 1;
  /* Modern browsers */
  flex: 1;
  max-width: calc(100% - 14ex);
}
div.output_scroll div.output_subarea {
  overflow-x: visible;
}
/* The rest of the output_* classes are for special styling of the different
   output types */
/* all text output has this class: */
div.output_text {
  text-align: left;
  color: #000;
  /* This has to match that of the the CodeMirror class line-height below */
  line-height: 1.21429em;
}
/* stdout/stderr are 'text' as well as 'stream', but execute_result/error are *not* streams */
div.output_stderr {
  background: #fdd;
  /* very light red background for stderr */
}
div.output_latex {
  text-align: left;
}
/* Empty output_javascript divs should have no height */
div.output_javascript:empty {
  padding: 0;
}
.js-error {
  color: darkred;
}
/* raw_input styles */
div.raw_input_container {
  line-height: 1.21429em;
  padding-top: 5px;
}
pre.raw_input_prompt {
  /* nothing needed here. */
}
input.raw_input {
  font-family: monospace;
  font-size: inherit;
  color: inherit;
  width: auto;
  /* make sure input baseline aligns with prompt */
  vertical-align: baseline;
  /* padding + margin = 0.5em between prompt and cursor */
  padding: 0em 0.25em;
  margin: 0em 0.25em;
}
input.raw_input:focus {
  box-shadow: none;
}
p.p-space {
  margin-bottom: 10px;
}
div.output_unrecognized {
  padding: 5px;
  font-weight: bold;
  color: red;
}
div.output_unrecognized a {
  color: inherit;
  text-decoration: none;
}
div.output_unrecognized a:hover {
  color: inherit;
  text-decoration: none;
}
.rendered_html {
  color: #000;
  /* any extras will just be numbers: */
}
.rendered_html em {
  font-style: italic;
}
.rendered_html strong {
  font-weight: bold;
}
.rendered_html u {
  text-decoration: underline;
}
.rendered_html :link {
  text-decoration: underline;
}
.rendered_html :visited {
  text-decoration: underline;
}
.rendered_html h1 {
  font-size: 185.7%;
  margin: 1.08em 0 0 0;
  font-weight: bold;
  line-height: 1.0;
}
.rendered_html h2 {
  font-size: 157.1%;
  margin: 1.27em 0 0 0;
  font-weight: bold;
  line-height: 1.0;
}
.rendered_html h3 {
  font-size: 128.6%;
  margin: 1.55em 0 0 0;
  font-weight: bold;
  line-height: 1.0;
}
.rendered_html h4 {
  font-size: 100%;
  margin: 2em 0 0 0;
  font-weight: bold;
  line-height: 1.0;
}
.rendered_html h5 {
  font-size: 100%;
  margin: 2em 0 0 0;
  font-weight: bold;
  line-height: 1.0;
  font-style: italic;
}
.rendered_html h6 {
  font-size: 100%;
  margin: 2em 0 0 0;
  font-weight: bold;
  line-height: 1.0;
  font-style: italic;
}
.rendered_html h1:first-child {
  margin-top: 0.538em;
}
.rendered_html h2:first-child {
  margin-top: 0.636em;
}
.rendered_html h3:first-child {
  margin-top: 0.777em;
}
.rendered_html h4:first-child {
  margin-top: 1em;
}
.rendered_html h5:first-child {
  margin-top: 1em;
}
.rendered_html h6:first-child {
  margin-top: 1em;
}
.rendered_html ul {
  list-style: disc;
  margin: 0em 2em;
  padding-left: 0px;
}
.rendered_html ul ul {
  list-style: square;
  margin: 0em 2em;
}
.rendered_html ul ul ul {
  list-style: circle;
  margin: 0em 2em;
}
.rendered_html ol {
  list-style: decimal;
  margin: 0em 2em;
  padding-left: 0px;
}
.rendered_html ol ol {
  list-style: upper-alpha;
  margin: 0em 2em;
}
.rendered_html ol ol ol {
  list-style: lower-alpha;
  margin: 0em 2em;
}
.rendered_html ol ol ol ol {
  list-style: lower-roman;
  margin: 0em 2em;
}
.rendered_html ol ol ol ol ol {
  list-style: decimal;
  margin: 0em 2em;
}
.rendered_html * + ul {
  margin-top: 1em;
}
.rendered_html * + ol {
  margin-top: 1em;
}
.rendered_html hr {
  color: black;
  background-color: black;
}
.rendered_html pre {
  margin: 1em 2em;
}
.rendered_html pre,
.rendered_html code {
  border: 0;
  background-color: #fff;
  color: #000;
  font-size: 100%;
  padding: 0px;
}
.rendered_html blockquote {
  margin: 1em 2em;
}
.rendered_html table {
  margin-left: auto;
  margin-right: auto;
  border: 1px solid black;
  border-collapse: collapse;
}
.rendered_html tr,
.rendered_html th,
.rendered_html td {
  border: 1px solid black;
  border-collapse: collapse;
  margin: 1em 2em;
}
.rendered_html td,
.rendered_html th {
  text-align: left;
  vertical-align: middle;
  padding: 4px;
}
.rendered_html th {
  font-weight: bold;
}
.rendered_html * + table {
  margin-top: 1em;
}
.rendered_html p {
  text-align: left;
}
.rendered_html * + p {
  margin-top: 1em;
}
.rendered_html img {
  display: block;
  margin-left: auto;
  margin-right: auto;
}
.rendered_html * + img {
  margin-top: 1em;
}
.rendered_html img,
.rendered_html svg {
  max-width: 100%;
  height: auto;
}
.rendered_html img.unconfined,
.rendered_html svg.unconfined {
  max-width: none;
}
div.text_cell {
  /* Old browsers */
  display: -webkit-box;
  -webkit-box-orient: horizontal;
  -webkit-box-align: stretch;
  display: -moz-box;
  -moz-box-orient: horizontal;
  -moz-box-align: stretch;
  display: box;
  box-orient: horizontal;
  box-align: stretch;
  /* Modern browsers */
  display: flex;
  flex-direction: row;
  align-items: stretch;
}
@media (max-width: 540px) {
  div.text_cell > div.prompt {
    display: none;
  }
}
div.text_cell_render {
  /*font-family: "Helvetica Neue", Arial, Helvetica, Geneva, sans-serif;*/
  outline: none;
  resize: none;
  width: inherit;
  border-style: none;
  padding: 0.5em 0.5em 0.5em 0.4em;
  color: #000;
  box-sizing: border-box;
  -moz-box-sizing: border-box;
  -webkit-box-sizing: border-box;
}
a.anchor-link:link {
  text-decoration: none;
  padding: 0px 20px;
  visibility: hidden;
}
h1:hover .anchor-link,
h2:hover .anchor-link,
h3:hover .anchor-link,
h4:hover .anchor-link,
h5:hover .anchor-link,
h6:hover .anchor-link {
  visibility: visible;
}
.text_cell.rendered .input_area {
  display: none;
}
.text_cell.rendered .rendered_html {
  overflow-x: auto;
  overflow-y: hidden;
}
.text_cell.unrendered .text_cell_render {
  display: none;
}
.cm-header-1,
.cm-header-2,
.cm-header-3,
.cm-header-4,
.cm-header-5,
.cm-header-6 {
  font-weight: bold;
  font-family: "Helvetica Neue", Helvetica, Arial, sans-serif;
}
.cm-header-1 {
  font-size: 185.7%;
}
.cm-header-2 {
  font-size: 157.1%;
}
.cm-header-3 {
  font-size: 128.6%;
}
.cm-header-4 {
  font-size: 110%;
}
.cm-header-5 {
  font-size: 100%;
  font-style: italic;
}
.cm-header-6 {
  font-size: 100%;
  font-style: italic;
}
/*!
*
* IPython notebook webapp
*
*/
@media (max-width: 767px) {
  .notebook_app {
    padding-left: 0px;
    padding-right: 0px;
  }
}
#ipython-main-app {
  box-sizing: border-box;
  -moz-box-sizing: border-box;
  -webkit-box-sizing: border-box;
  height: 100%;
}
div#notebook_panel {
  margin: 0px;
  padding: 0px;
  box-sizing: border-box;
  -moz-box-sizing: border-box;
  -webkit-box-sizing: border-box;
  height: 100%;
}
div#notebook {
  font-size: 14px;
  line-height: 20px;
  overflow-y: hidden;
  overflow-x: auto;
  width: 100%;
  /* This spaces the page away from the edge of the notebook area */
  padding-top: 20px;
  margin: 0px;
  outline: none;
  box-sizing: border-box;
  -moz-box-sizing: border-box;
  -webkit-box-sizing: border-box;
  min-height: 100%;
}
@media not print {
  #notebook-container {
    padding: 15px;
    background-color: #fff;
    min-height: 0;
    -webkit-box-shadow: 0px 0px 12px 1px rgba(87, 87, 87, 0.2);
    box-shadow: 0px 0px 12px 1px rgba(87, 87, 87, 0.2);
  }
}
@media print {
  #notebook-container {
    width: 100%;
  }
}
div.ui-widget-content {
  border: 1px solid #ababab;
  outline: none;
}
pre.dialog {
  background-color: #f7f7f7;
  border: 1px solid #ddd;
  border-radius: 2px;
  padding: 0.4em;
  padding-left: 2em;
}
p.dialog {
  padding: 0.2em;
}
/* Word-wrap output correctly.  This is the CSS3 spelling, though Firefox seems
   to not honor it correctly.  Webkit browsers (Chrome, rekonq, Safari) do.
 */
pre,
code,
kbd,
samp {
  white-space: pre-wrap;
}
#fonttest {
  font-family: monospace;
}
p {
  margin-bottom: 0;
}
.end_space {
  min-height: 100px;
  transition: height .2s ease;
}
.notebook_app > #header {
  -webkit-box-shadow: 0px 0px 12px 1px rgba(87, 87, 87, 0.2);
  box-shadow: 0px 0px 12px 1px rgba(87, 87, 87, 0.2);
}
@media not print {
  .notebook_app {
    background-color: #EEE;
  }
}
kbd {
  border-style: solid;
  border-width: 1px;
  box-shadow: none;
  margin: 2px;
  padding-left: 2px;
  padding-right: 2px;
  padding-top: 1px;
  padding-bottom: 1px;
}
/* CSS for the cell toolbar */
.celltoolbar {
  border: thin solid #CFCFCF;
  border-bottom: none;
  background: #EEE;
  border-radius: 2px 2px 0px 0px;
  width: 100%;
  height: 29px;
  padding-right: 4px;
  /* Old browsers */
  display: -webkit-box;
  -webkit-box-orient: horizontal;
  -webkit-box-align: stretch;
  display: -moz-box;
  -moz-box-orient: horizontal;
  -moz-box-align: stretch;
  display: box;
  box-orient: horizontal;
  box-align: stretch;
  /* Modern browsers */
  display: flex;
  flex-direction: row;
  align-items: stretch;
  /* Old browsers */
  -webkit-box-pack: end;
  -moz-box-pack: end;
  box-pack: end;
  /* Modern browsers */
  justify-content: flex-end;
  display: -webkit-flex;
}
@media print {
  .celltoolbar {
    display: none;
  }
}
.ctb_hideshow {
  display: none;
  vertical-align: bottom;
}
/* ctb_show is added to the ctb_hideshow div to show the cell toolbar.
   Cell toolbars are only shown when the ctb_global_show class is also set.
*/
.ctb_global_show .ctb_show.ctb_hideshow {
  display: block;
}
.ctb_global_show .ctb_show + .input_area,
.ctb_global_show .ctb_show + div.text_cell_input,
.ctb_global_show .ctb_show ~ div.text_cell_render {
  border-top-right-radius: 0px;
  border-top-left-radius: 0px;
}
.ctb_global_show .ctb_show ~ div.text_cell_render {
  border: 1px solid #cfcfcf;
}
.celltoolbar {
  font-size: 87%;
  padding-top: 3px;
}
.celltoolbar select {
  display: block;
  width: 100%;
  height: 32px;
  padding: 6px 12px;
  font-size: 13px;
  line-height: 1.42857143;
  color: #555555;
  background-color: #fff;
  background-image: none;
  border: 1px solid #ccc;
  border-radius: 2px;
  -webkit-box-shadow: inset 0 1px 1px rgba(0, 0, 0, 0.075);
  box-shadow: inset 0 1px 1px rgba(0, 0, 0, 0.075);
  -webkit-transition: border-color ease-in-out .15s, box-shadow ease-in-out .15s;
  -o-transition: border-color ease-in-out .15s, box-shadow ease-in-out .15s;
  transition: border-color ease-in-out .15s, box-shadow ease-in-out .15s;
  height: 30px;
  padding: 5px 10px;
  font-size: 12px;
  line-height: 1.5;
  border-radius: 1px;
  width: inherit;
  font-size: inherit;
  height: 22px;
  padding: 0px;
  display: inline-block;
}
.celltoolbar select:focus {
  border-color: #66afe9;
  outline: 0;
  -webkit-box-shadow: inset 0 1px 1px rgba(0,0,0,.075), 0 0 8px rgba(102, 175, 233, 0.6);
  box-shadow: inset 0 1px 1px rgba(0,0,0,.075), 0 0 8px rgba(102, 175, 233, 0.6);
}
.celltoolbar select::-moz-placeholder {
  color: #999;
  opacity: 1;
}
.celltoolbar select:-ms-input-placeholder {
  color: #999;
}
.celltoolbar select::-webkit-input-placeholder {
  color: #999;
}
.celltoolbar select::-ms-expand {
  border: 0;
  background-color: transparent;
}
.celltoolbar select[disabled],
.celltoolbar select[readonly],
fieldset[disabled] .celltoolbar select {
  background-color: #eeeeee;
  opacity: 1;
}
.celltoolbar select[disabled],
fieldset[disabled] .celltoolbar select {
  cursor: not-allowed;
}
textarea.celltoolbar select {
  height: auto;
}
select.celltoolbar select {
  height: 30px;
  line-height: 30px;
}
textarea.celltoolbar select,
select[multiple].celltoolbar select {
  height: auto;
}
.celltoolbar label {
  margin-left: 5px;
  margin-right: 5px;
}
.completions {
  position: absolute;
  z-index: 110;
  overflow: hidden;
  border: 1px solid #ababab;
  border-radius: 2px;
  -webkit-box-shadow: 0px 6px 10px -1px #adadad;
  box-shadow: 0px 6px 10px -1px #adadad;
  line-height: 1;
}
.completions select {
  background: white;
  outline: none;
  border: none;
  padding: 0px;
  margin: 0px;
  overflow: auto;
  font-family: monospace;
  font-size: 110%;
  color: #000;
  width: auto;
}
.completions select option.context {
  color: #286090;
}
#kernel_logo_widget {
  float: right !important;
  float: right;
}
#kernel_logo_widget .current_kernel_logo {
  display: none;
  margin-top: -1px;
  margin-bottom: -1px;
  width: 32px;
  height: 32px;
}
#menubar {
  box-sizing: border-box;
  -moz-box-sizing: border-box;
  -webkit-box-sizing: border-box;
  margin-top: 1px;
}
#menubar .navbar {
  border-top: 1px;
  border-radius: 0px 0px 2px 2px;
  margin-bottom: 0px;
}
#menubar .navbar-toggle {
  float: left;
  padding-top: 7px;
  padding-bottom: 7px;
  border: none;
}
#menubar .navbar-collapse {
  clear: left;
}
.nav-wrapper {
  border-bottom: 1px solid #e7e7e7;
}
i.menu-icon {
  padding-top: 4px;
}
ul#help_menu li a {
  overflow: hidden;
  padding-right: 2.2em;
}
ul#help_menu li a i {
  margin-right: -1.2em;
}
.dropdown-submenu {
  position: relative;
}
.dropdown-submenu > .dropdown-menu {
  top: 0;
  left: 100%;
  margin-top: -6px;
  margin-left: -1px;
}
.dropdown-submenu:hover > .dropdown-menu {
  display: block;
}
.dropdown-submenu > a:after {
  display: inline-block;
  font: normal normal normal 14px/1 FontAwesome;
  font-size: inherit;
  text-rendering: auto;
  -webkit-font-smoothing: antialiased;
  -moz-osx-font-smoothing: grayscale;
  display: block;
  content: "\f0da";
  float: right;
  color: #333333;
  margin-top: 2px;
  margin-right: -10px;
}
.dropdown-submenu > a:after.pull-left {
  margin-right: .3em;
}
.dropdown-submenu > a:after.pull-right {
  margin-left: .3em;
}
.dropdown-submenu:hover > a:after {
  color: #262626;
}
.dropdown-submenu.pull-left {
  float: none;
}
.dropdown-submenu.pull-left > .dropdown-menu {
  left: -100%;
  margin-left: 10px;
}
#notification_area {
  float: right !important;
  float: right;
  z-index: 10;
}
.indicator_area {
  float: right !important;
  float: right;
  color: #777;
  margin-left: 5px;
  margin-right: 5px;
  width: 11px;
  z-index: 10;
  text-align: center;
  width: auto;
}
#kernel_indicator {
  float: right !important;
  float: right;
  color: #777;
  margin-left: 5px;
  margin-right: 5px;
  width: 11px;
  z-index: 10;
  text-align: center;
  width: auto;
  border-left: 1px solid;
}
#kernel_indicator .kernel_indicator_name {
  padding-left: 5px;
  padding-right: 5px;
}
#modal_indicator {
  float: right !important;
  float: right;
  color: #777;
  margin-left: 5px;
  margin-right: 5px;
  width: 11px;
  z-index: 10;
  text-align: center;
  width: auto;
}
#readonly-indicator {
  float: right !important;
  float: right;
  color: #777;
  margin-left: 5px;
  margin-right: 5px;
  width: 11px;
  z-index: 10;
  text-align: center;
  width: auto;
  margin-top: 2px;
  margin-bottom: 0px;
  margin-left: 0px;
  margin-right: 0px;
  display: none;
}
.modal_indicator:before {
  width: 1.28571429em;
  text-align: center;
}
.edit_mode .modal_indicator:before {
  display: inline-block;
  font: normal normal normal 14px/1 FontAwesome;
  font-size: inherit;
  text-rendering: auto;
  -webkit-font-smoothing: antialiased;
  -moz-osx-font-smoothing: grayscale;
  content: "\f040";
}
.edit_mode .modal_indicator:before.pull-left {
  margin-right: .3em;
}
.edit_mode .modal_indicator:before.pull-right {
  margin-left: .3em;
}
.command_mode .modal_indicator:before {
  display: inline-block;
  font: normal normal normal 14px/1 FontAwesome;
  font-size: inherit;
  text-rendering: auto;
  -webkit-font-smoothing: antialiased;
  -moz-osx-font-smoothing: grayscale;
  content: ' ';
}
.command_mode .modal_indicator:before.pull-left {
  margin-right: .3em;
}
.command_mode .modal_indicator:before.pull-right {
  margin-left: .3em;
}
.kernel_idle_icon:before {
  display: inline-block;
  font: normal normal normal 14px/1 FontAwesome;
  font-size: inherit;
  text-rendering: auto;
  -webkit-font-smoothing: antialiased;
  -moz-osx-font-smoothing: grayscale;
  content: "\f10c";
}
.kernel_idle_icon:before.pull-left {
  margin-right: .3em;
}
.kernel_idle_icon:before.pull-right {
  margin-left: .3em;
}
.kernel_busy_icon:before {
  display: inline-block;
  font: normal normal normal 14px/1 FontAwesome;
  font-size: inherit;
  text-rendering: auto;
  -webkit-font-smoothing: antialiased;
  -moz-osx-font-smoothing: grayscale;
  content: "\f111";
}
.kernel_busy_icon:before.pull-left {
  margin-right: .3em;
}
.kernel_busy_icon:before.pull-right {
  margin-left: .3em;
}
.kernel_dead_icon:before {
  display: inline-block;
  font: normal normal normal 14px/1 FontAwesome;
  font-size: inherit;
  text-rendering: auto;
  -webkit-font-smoothing: antialiased;
  -moz-osx-font-smoothing: grayscale;
  content: "\f1e2";
}
.kernel_dead_icon:before.pull-left {
  margin-right: .3em;
}
.kernel_dead_icon:before.pull-right {
  margin-left: .3em;
}
.kernel_disconnected_icon:before {
  display: inline-block;
  font: normal normal normal 14px/1 FontAwesome;
  font-size: inherit;
  text-rendering: auto;
  -webkit-font-smoothing: antialiased;
  -moz-osx-font-smoothing: grayscale;
  content: "\f127";
}
.kernel_disconnected_icon:before.pull-left {
  margin-right: .3em;
}
.kernel_disconnected_icon:before.pull-right {
  margin-left: .3em;
}
.notification_widget {
  color: #777;
  z-index: 10;
  background: rgba(240, 240, 240, 0.5);
  margin-right: 4px;
  color: #333;
  background-color: #fff;
  border-color: #ccc;
}
.notification_widget:focus,
.notification_widget.focus {
  color: #333;
  background-color: #e6e6e6;
  border-color: #8c8c8c;
}
.notification_widget:hover {
  color: #333;
  background-color: #e6e6e6;
  border-color: #adadad;
}
.notification_widget:active,
.notification_widget.active,
.open > .dropdown-toggle.notification_widget {
  color: #333;
  background-color: #e6e6e6;
  border-color: #adadad;
}
.notification_widget:active:hover,
.notification_widget.active:hover,
.open > .dropdown-toggle.notification_widget:hover,
.notification_widget:active:focus,
.notification_widget.active:focus,
.open > .dropdown-toggle.notification_widget:focus,
.notification_widget:active.focus,
.notification_widget.active.focus,
.open > .dropdown-toggle.notification_widget.focus {
  color: #333;
  background-color: #d4d4d4;
  border-color: #8c8c8c;
}
.notification_widget:active,
.notification_widget.active,
.open > .dropdown-toggle.notification_widget {
  background-image: none;
}
.notification_widget.disabled:hover,
.notification_widget[disabled]:hover,
fieldset[disabled] .notification_widget:hover,
.notification_widget.disabled:focus,
.notification_widget[disabled]:focus,
fieldset[disabled] .notification_widget:focus,
.notification_widget.disabled.focus,
.notification_widget[disabled].focus,
fieldset[disabled] .notification_widget.focus {
  background-color: #fff;
  border-color: #ccc;
}
.notification_widget .badge {
  color: #fff;
  background-color: #333;
}
.notification_widget.warning {
  color: #fff;
  background-color: #f0ad4e;
  border-color: #eea236;
}
.notification_widget.warning:focus,
.notification_widget.warning.focus {
  color: #fff;
  background-color: #ec971f;
  border-color: #985f0d;
}
.notification_widget.warning:hover {
  color: #fff;
  background-color: #ec971f;
  border-color: #d58512;
}
.notification_widget.warning:active,
.notification_widget.warning.active,
.open > .dropdown-toggle.notification_widget.warning {
  color: #fff;
  background-color: #ec971f;
  border-color: #d58512;
}
.notification_widget.warning:active:hover,
.notification_widget.warning.active:hover,
.open > .dropdown-toggle.notification_widget.warning:hover,
.notification_widget.warning:active:focus,
.notification_widget.warning.active:focus,
.open > .dropdown-toggle.notification_widget.warning:focus,
.notification_widget.warning:active.focus,
.notification_widget.warning.active.focus,
.open > .dropdown-toggle.notification_widget.warning.focus {
  color: #fff;
  background-color: #d58512;
  border-color: #985f0d;
}
.notification_widget.warning:active,
.notification_widget.warning.active,
.open > .dropdown-toggle.notification_widget.warning {
  background-image: none;
}
.notification_widget.warning.disabled:hover,
.notification_widget.warning[disabled]:hover,
fieldset[disabled] .notification_widget.warning:hover,
.notification_widget.warning.disabled:focus,
.notification_widget.warning[disabled]:focus,
fieldset[disabled] .notification_widget.warning:focus,
.notification_widget.warning.disabled.focus,
.notification_widget.warning[disabled].focus,
fieldset[disabled] .notification_widget.warning.focus {
  background-color: #f0ad4e;
  border-color: #eea236;
}
.notification_widget.warning .badge {
  color: #f0ad4e;
  background-color: #fff;
}
.notification_widget.success {
  color: #fff;
  background-color: #5cb85c;
  border-color: #4cae4c;
}
.notification_widget.success:focus,
.notification_widget.success.focus {
  color: #fff;
  background-color: #449d44;
  border-color: #255625;
}
.notification_widget.success:hover {
  color: #fff;
  background-color: #449d44;
  border-color: #398439;
}
.notification_widget.success:active,
.notification_widget.success.active,
.open > .dropdown-toggle.notification_widget.success {
  color: #fff;
  background-color: #449d44;
  border-color: #398439;
}
.notification_widget.success:active:hover,
.notification_widget.success.active:hover,
.open > .dropdown-toggle.notification_widget.success:hover,
.notification_widget.success:active:focus,
.notification_widget.success.active:focus,
.open > .dropdown-toggle.notification_widget.success:focus,
.notification_widget.success:active.focus,
.notification_widget.success.active.focus,
.open > .dropdown-toggle.notification_widget.success.focus {
  color: #fff;
  background-color: #398439;
  border-color: #255625;
}
.notification_widget.success:active,
.notification_widget.success.active,
.open > .dropdown-toggle.notification_widget.success {
  background-image: none;
}
.notification_widget.success.disabled:hover,
.notification_widget.success[disabled]:hover,
fieldset[disabled] .notification_widget.success:hover,
.notification_widget.success.disabled:focus,
.notification_widget.success[disabled]:focus,
fieldset[disabled] .notification_widget.success:focus,
.notification_widget.success.disabled.focus,
.notification_widget.success[disabled].focus,
fieldset[disabled] .notification_widget.success.focus {
  background-color: #5cb85c;
  border-color: #4cae4c;
}
.notification_widget.success .badge {
  color: #5cb85c;
  background-color: #fff;
}
.notification_widget.info {
  color: #fff;
  background-color: #5bc0de;
  border-color: #46b8da;
}
.notification_widget.info:focus,
.notification_widget.info.focus {
  color: #fff;
  background-color: #31b0d5;
  border-color: #1b6d85;
}
.notification_widget.info:hover {
  color: #fff;
  background-color: #31b0d5;
  border-color: #269abc;
}
.notification_widget.info:active,
.notification_widget.info.active,
.open > .dropdown-toggle.notification_widget.info {
  color: #fff;
  background-color: #31b0d5;
  border-color: #269abc;
}
.notification_widget.info:active:hover,
.notification_widget.info.active:hover,
.open > .dropdown-toggle.notification_widget.info:hover,
.notification_widget.info:active:focus,
.notification_widget.info.active:focus,
.open > .dropdown-toggle.notification_widget.info:focus,
.notification_widget.info:active.focus,
.notification_widget.info.active.focus,
.open > .dropdown-toggle.notification_widget.info.focus {
  color: #fff;
  background-color: #269abc;
  border-color: #1b6d85;
}
.notification_widget.info:active,
.notification_widget.info.active,
.open > .dropdown-toggle.notification_widget.info {
  background-image: none;
}
.notification_widget.info.disabled:hover,
.notification_widget.info[disabled]:hover,
fieldset[disabled] .notification_widget.info:hover,
.notification_widget.info.disabled:focus,
.notification_widget.info[disabled]:focus,
fieldset[disabled] .notification_widget.info:focus,
.notification_widget.info.disabled.focus,
.notification_widget.info[disabled].focus,
fieldset[disabled] .notification_widget.info.focus {
  background-color: #5bc0de;
  border-color: #46b8da;
}
.notification_widget.info .badge {
  color: #5bc0de;
  background-color: #fff;
}
.notification_widget.danger {
  color: #fff;
  background-color: #d9534f;
  border-color: #d43f3a;
}
.notification_widget.danger:focus,
.notification_widget.danger.focus {
  color: #fff;
  background-color: #c9302c;
  border-color: #761c19;
}
.notification_widget.danger:hover {
  color: #fff;
  background-color: #c9302c;
  border-color: #ac2925;
}
.notification_widget.danger:active,
.notification_widget.danger.active,
.open > .dropdown-toggle.notification_widget.danger {
  color: #fff;
  background-color: #c9302c;
  border-color: #ac2925;
}
.notification_widget.danger:active:hover,
.notification_widget.danger.active:hover,
.open > .dropdown-toggle.notification_widget.danger:hover,
.notification_widget.danger:active:focus,
.notification_widget.danger.active:focus,
.open > .dropdown-toggle.notification_widget.danger:focus,
.notification_widget.danger:active.focus,
.notification_widget.danger.active.focus,
.open > .dropdown-toggle.notification_widget.danger.focus {
  color: #fff;
  background-color: #ac2925;
  border-color: #761c19;
}
.notification_widget.danger:active,
.notification_widget.danger.active,
.open > .dropdown-toggle.notification_widget.danger {
  background-image: none;
}
.notification_widget.danger.disabled:hover,
.notification_widget.danger[disabled]:hover,
fieldset[disabled] .notification_widget.danger:hover,
.notification_widget.danger.disabled:focus,
.notification_widget.danger[disabled]:focus,
fieldset[disabled] .notification_widget.danger:focus,
.notification_widget.danger.disabled.focus,
.notification_widget.danger[disabled].focus,
fieldset[disabled] .notification_widget.danger.focus {
  background-color: #d9534f;
  border-color: #d43f3a;
}
.notification_widget.danger .badge {
  color: #d9534f;
  background-color: #fff;
}
div#pager {
  background-color: #fff;
  font-size: 14px;
  line-height: 20px;
  overflow: hidden;
  display: none;
  position: fixed;
  bottom: 0px;
  width: 100%;
  max-height: 50%;
  padding-top: 8px;
  -webkit-box-shadow: 0px 0px 12px 1px rgba(87, 87, 87, 0.2);
  box-shadow: 0px 0px 12px 1px rgba(87, 87, 87, 0.2);
  /* Display over codemirror */
  z-index: 100;
  /* Hack which prevents jquery ui resizable from changing top. */
  top: auto !important;
}
div#pager pre {
  line-height: 1.21429em;
  color: #000;
  background-color: #f7f7f7;
  padding: 0.4em;
}
div#pager #pager-button-area {
  position: absolute;
  top: 8px;
  right: 20px;
}
div#pager #pager-contents {
  position: relative;
  overflow: auto;
  width: 100%;
  height: 100%;
}
div#pager #pager-contents #pager-container {
  position: relative;
  padding: 15px 0px;
  box-sizing: border-box;
  -moz-box-sizing: border-box;
  -webkit-box-sizing: border-box;
}
div#pager .ui-resizable-handle {
  top: 0px;
  height: 8px;
  background: #f7f7f7;
  border-top: 1px solid #cfcfcf;
  border-bottom: 1px solid #cfcfcf;
  /* This injects handle bars (a short, wide = symbol) for 
        the resize handle. */
}
div#pager .ui-resizable-handle::after {
  content: '';
  top: 2px;
  left: 50%;
  height: 3px;
  width: 30px;
  margin-left: -15px;
  position: absolute;
  border-top: 1px solid #cfcfcf;
}
.quickhelp {
  /* Old browsers */
  display: -webkit-box;
  -webkit-box-orient: horizontal;
  -webkit-box-align: stretch;
  display: -moz-box;
  -moz-box-orient: horizontal;
  -moz-box-align: stretch;
  display: box;
  box-orient: horizontal;
  box-align: stretch;
  /* Modern browsers */
  display: flex;
  flex-direction: row;
  align-items: stretch;
  line-height: 1.8em;
}
.shortcut_key {
  display: inline-block;
  width: 21ex;
  text-align: right;
  font-family: monospace;
}
.shortcut_descr {
  display: inline-block;
  /* Old browsers */
  -webkit-box-flex: 1;
  -moz-box-flex: 1;
  box-flex: 1;
  /* Modern browsers */
  flex: 1;
}
span.save_widget {
  margin-top: 6px;
}
span.save_widget span.filename {
  height: 1em;
  line-height: 1em;
  padding: 3px;
  margin-left: 16px;
  border: none;
  font-size: 146.5%;
  border-radius: 2px;
}
span.save_widget span.filename:hover {
  background-color: #e6e6e6;
}
span.checkpoint_status,
span.autosave_status {
  font-size: small;
}
@media (max-width: 767px) {
  span.save_widget {
    font-size: small;
  }
  span.checkpoint_status,
  span.autosave_status {
    display: none;
  }
}
@media (min-width: 768px) and (max-width: 991px) {
  span.checkpoint_status {
    display: none;
  }
  span.autosave_status {
    font-size: x-small;
  }
}
.toolbar {
  padding: 0px;
  margin-left: -5px;
  margin-top: 2px;
  margin-bottom: 5px;
  box-sizing: border-box;
  -moz-box-sizing: border-box;
  -webkit-box-sizing: border-box;
}
.toolbar select,
.toolbar label {
  width: auto;
  vertical-align: middle;
  margin-right: 2px;
  margin-bottom: 0px;
  display: inline;
  font-size: 92%;
  margin-left: 0.3em;
  margin-right: 0.3em;
  padding: 0px;
  padding-top: 3px;
}
.toolbar .btn {
  padding: 2px 8px;
}
.toolbar .btn-group {
  margin-top: 0px;
  margin-left: 5px;
}
#maintoolbar {
  margin-bottom: -3px;
  margin-top: -8px;
  border: 0px;
  min-height: 27px;
  margin-left: 0px;
  padding-top: 11px;
  padding-bottom: 3px;
}
#maintoolbar .navbar-text {
  float: none;
  vertical-align: middle;
  text-align: right;
  margin-left: 5px;
  margin-right: 0px;
  margin-top: 0px;
}
.select-xs {
  height: 24px;
}
.pulse,
.dropdown-menu > li > a.pulse,
li.pulse > a.dropdown-toggle,
li.pulse.open > a.dropdown-toggle {
  background-color: #F37626;
  color: white;
}
/**
 * Primary styles
 *
 * Author: Jupyter Development Team
 */
/** WARNING IF YOU ARE EDITTING THIS FILE, if this is a .css file, It has a lot
 * of chance of beeing generated from the ../less/[samename].less file, you can
 * try to get back the less file by reverting somme commit in history
 **/
/*
 * We'll try to get something pretty, so we
 * have some strange css to have the scroll bar on
 * the left with fix button on the top right of the tooltip
 */
@-moz-keyframes fadeOut {
  from {
    opacity: 1;
  }
  to {
    opacity: 0;
  }
}
@-webkit-keyframes fadeOut {
  from {
    opacity: 1;
  }
  to {
    opacity: 0;
  }
}
@-moz-keyframes fadeIn {
  from {
    opacity: 0;
  }
  to {
    opacity: 1;
  }
}
@-webkit-keyframes fadeIn {
  from {
    opacity: 0;
  }
  to {
    opacity: 1;
  }
}
/*properties of tooltip after "expand"*/
.bigtooltip {
  overflow: auto;
  height: 200px;
  -webkit-transition-property: height;
  -webkit-transition-duration: 500ms;
  -moz-transition-property: height;
  -moz-transition-duration: 500ms;
  transition-property: height;
  transition-duration: 500ms;
}
/*properties of tooltip before "expand"*/
.smalltooltip {
  -webkit-transition-property: height;
  -webkit-transition-duration: 500ms;
  -moz-transition-property: height;
  -moz-transition-duration: 500ms;
  transition-property: height;
  transition-duration: 500ms;
  text-overflow: ellipsis;
  overflow: hidden;
  height: 80px;
}
.tooltipbuttons {
  position: absolute;
  padding-right: 15px;
  top: 0px;
  right: 0px;
}
.tooltiptext {
  /*avoid the button to overlap on some docstring*/
  padding-right: 30px;
}
.ipython_tooltip {
  max-width: 700px;
  /*fade-in animation when inserted*/
  -webkit-animation: fadeOut 400ms;
  -moz-animation: fadeOut 400ms;
  animation: fadeOut 400ms;
  -webkit-animation: fadeIn 400ms;
  -moz-animation: fadeIn 400ms;
  animation: fadeIn 400ms;
  vertical-align: middle;
  background-color: #f7f7f7;
  overflow: visible;
  border: #ababab 1px solid;
  outline: none;
  padding: 3px;
  margin: 0px;
  padding-left: 7px;
  font-family: monospace;
  min-height: 50px;
  -moz-box-shadow: 0px 6px 10px -1px #adadad;
  -webkit-box-shadow: 0px 6px 10px -1px #adadad;
  box-shadow: 0px 6px 10px -1px #adadad;
  border-radius: 2px;
  position: absolute;
  z-index: 1000;
}
.ipython_tooltip a {
  float: right;
}
.ipython_tooltip .tooltiptext pre {
  border: 0;
  border-radius: 0;
  font-size: 100%;
  background-color: #f7f7f7;
}
.pretooltiparrow {
  left: 0px;
  margin: 0px;
  top: -16px;
  width: 40px;
  height: 16px;
  overflow: hidden;
  position: absolute;
}
.pretooltiparrow:before {
  background-color: #f7f7f7;
  border: 1px #ababab solid;
  z-index: 11;
  content: "";
  position: absolute;
  left: 15px;
  top: 10px;
  width: 25px;
  height: 25px;
  -webkit-transform: rotate(45deg);
  -moz-transform: rotate(45deg);
  -ms-transform: rotate(45deg);
  -o-transform: rotate(45deg);
}
ul.typeahead-list i {
  margin-left: -10px;
  width: 18px;
}
ul.typeahead-list {
  max-height: 80vh;
  overflow: auto;
}
ul.typeahead-list > li > a {
  /** Firefox bug **/
  /* see https://github.com/jupyter/notebook/issues/559 */
  white-space: normal;
}
.cmd-palette .modal-body {
  padding: 7px;
}
.cmd-palette form {
  background: white;
}
.cmd-palette input {
  outline: none;
}
.no-shortcut {
  display: none;
}
.command-shortcut:before {
  content: "(command)";
  padding-right: 3px;
  color: #777777;
}
.edit-shortcut:before {
  content: "(edit)";
  padding-right: 3px;
  color: #777777;
}
#find-and-replace #replace-preview .match,
#find-and-replace #replace-preview .insert {
  background-color: #BBDEFB;
  border-color: #90CAF9;
  border-style: solid;
  border-width: 1px;
  border-radius: 0px;
}
#find-and-replace #replace-preview .replace .match {
  background-color: #FFCDD2;
  border-color: #EF9A9A;
  border-radius: 0px;
}
#find-and-replace #replace-preview .replace .insert {
  background-color: #C8E6C9;
  border-color: #A5D6A7;
  border-radius: 0px;
}
#find-and-replace #replace-preview {
  max-height: 60vh;
  overflow: auto;
}
#find-and-replace #replace-preview pre {
  padding: 5px 10px;
}
.terminal-app {
  background: #EEE;
}
.terminal-app #header {
  background: #fff;
  -webkit-box-shadow: 0px 0px 12px 1px rgba(87, 87, 87, 0.2);
  box-shadow: 0px 0px 12px 1px rgba(87, 87, 87, 0.2);
}
.terminal-app .terminal {
  width: 100%;
  float: left;
  font-family: monospace;
  color: white;
  background: black;
  padding: 0.4em;
  border-radius: 2px;
  -webkit-box-shadow: 0px 0px 12px 1px rgba(87, 87, 87, 0.4);
  box-shadow: 0px 0px 12px 1px rgba(87, 87, 87, 0.4);
}
.terminal-app .terminal,
.terminal-app .terminal dummy-screen {
  line-height: 1em;
  font-size: 14px;
}
.terminal-app .terminal .xterm-rows {
  padding: 10px;
}
.terminal-app .terminal-cursor {
  color: black;
  background: white;
}
.terminal-app #terminado-container {
  margin-top: 20px;
}
/*# sourceMappingURL=style.min.css.map */
    </style>
<style type="text/css">
    .highlight .hll { background-color: #ffffcc }
.highlight  { background: #f8f8f8; }
.highlight .c { color: #408080; font-style: italic } /* Comment */
.highlight .err { border: 1px solid #FF0000 } /* Error */
.highlight .k { color: #008000; font-weight: bold } /* Keyword */
.highlight .o { color: #666666 } /* Operator */
.highlight .ch { color: #408080; font-style: italic } /* Comment.Hashbang */
.highlight .cm { color: #408080; font-style: italic } /* Comment.Multiline */
.highlight .cp { color: #BC7A00 } /* Comment.Preproc */
.highlight .cpf { color: #408080; font-style: italic } /* Comment.PreprocFile */
.highlight .c1 { color: #408080; font-style: italic } /* Comment.Single */
.highlight .cs { color: #408080; font-style: italic } /* Comment.Special */
.highlight .gd { color: #A00000 } /* Generic.Deleted */
.highlight .ge { font-style: italic } /* Generic.Emph */
.highlight .gr { color: #FF0000 } /* Generic.Error */
.highlight .gh { color: #000080; font-weight: bold } /* Generic.Heading */
.highlight .gi { color: #00A000 } /* Generic.Inserted */
.highlight .go { color: #888888 } /* Generic.Output */
.highlight .gp { color: #000080; font-weight: bold } /* Generic.Prompt */
.highlight .gs { font-weight: bold } /* Generic.Strong */
.highlight .gu { color: #800080; font-weight: bold } /* Generic.Subheading */
.highlight .gt { color: #0044DD } /* Generic.Traceback */
.highlight .kc { color: #008000; font-weight: bold } /* Keyword.Constant */
.highlight .kd { color: #008000; font-weight: bold } /* Keyword.Declaration */
.highlight .kn { color: #008000; font-weight: bold } /* Keyword.Namespace */
.highlight .kp { color: #008000 } /* Keyword.Pseudo */
.highlight .kr { color: #008000; font-weight: bold } /* Keyword.Reserved */
.highlight .kt { color: #B00040 } /* Keyword.Type */
.highlight .m { color: #666666 } /* Literal.Number */
.highlight .s { color: #BA2121 } /* Literal.String */
.highlight .na { color: #7D9029 } /* Name.Attribute */
.highlight .nb { color: #008000 } /* Name.Builtin */
.highlight .nc { color: #0000FF; font-weight: bold } /* Name.Class */
.highlight .no { color: #880000 } /* Name.Constant */
.highlight .nd { color: #AA22FF } /* Name.Decorator */
.highlight .ni { color: #999999; font-weight: bold } /* Name.Entity */
.highlight .ne { color: #D2413A; font-weight: bold } /* Name.Exception */
.highlight .nf { color: #0000FF } /* Name.Function */
.highlight .nl { color: #A0A000 } /* Name.Label */
.highlight .nn { color: #0000FF; font-weight: bold } /* Name.Namespace */
.highlight .nt { color: #008000; font-weight: bold } /* Name.Tag */
.highlight .nv { color: #19177C } /* Name.Variable */
.highlight .ow { color: #AA22FF; font-weight: bold } /* Operator.Word */
.highlight .w { color: #bbbbbb } /* Text.Whitespace */
.highlight .mb { color: #666666 } /* Literal.Number.Bin */
.highlight .mf { color: #666666 } /* Literal.Number.Float */
.highlight .mh { color: #666666 } /* Literal.Number.Hex */
.highlight .mi { color: #666666 } /* Literal.Number.Integer */
.highlight .mo { color: #666666 } /* Literal.Number.Oct */
.highlight .sa { color: #BA2121 } /* Literal.String.Affix */
.highlight .sb { color: #BA2121 } /* Literal.String.Backtick */
.highlight .sc { color: #BA2121 } /* Literal.String.Char */
.highlight .dl { color: #BA2121 } /* Literal.String.Delimiter */
.highlight .sd { color: #BA2121; font-style: italic } /* Literal.String.Doc */
.highlight .s2 { color: #BA2121 } /* Literal.String.Double */
.highlight .se { color: #BB6622; font-weight: bold } /* Literal.String.Escape */
.highlight .sh { color: #BA2121 } /* Literal.String.Heredoc */
.highlight .si { color: #BB6688; font-weight: bold } /* Literal.String.Interpol */
.highlight .sx { color: #008000 } /* Literal.String.Other */
.highlight .sr { color: #BB6688 } /* Literal.String.Regex */
.highlight .s1 { color: #BA2121 } /* Literal.String.Single */
.highlight .ss { color: #19177C } /* Literal.String.Symbol */
.highlight .bp { color: #008000 } /* Name.Builtin.Pseudo */
.highlight .fm { color: #0000FF } /* Name.Function.Magic */
.highlight .vc { color: #19177C } /* Name.Variable.Class */
.highlight .vg { color: #19177C } /* Name.Variable.Global */
.highlight .vi { color: #19177C } /* Name.Variable.Instance */
.highlight .vm { color: #19177C } /* Name.Variable.Magic */
.highlight .il { color: #666666 } /* Literal.Number.Integer.Long */
    </style>
<style type="text/css">
    
/* Temporary definitions which will become obsolete with Notebook release 5.0 */
.ansi-black-fg { color: #3E424D; }
.ansi-black-bg { background-color: #3E424D; }
.ansi-black-intense-fg { color: #282C36; }
.ansi-black-intense-bg { background-color: #282C36; }
.ansi-red-fg { color: #E75C58; }
.ansi-red-bg { background-color: #E75C58; }
.ansi-red-intense-fg { color: #B22B31; }
.ansi-red-intense-bg { background-color: #B22B31; }
.ansi-green-fg { color: #00A250; }
.ansi-green-bg { background-color: #00A250; }
.ansi-green-intense-fg { color: #007427; }
.ansi-green-intense-bg { background-color: #007427; }
.ansi-yellow-fg { color: #DDB62B; }
.ansi-yellow-bg { background-color: #DDB62B; }
.ansi-yellow-intense-fg { color: #B27D12; }
.ansi-yellow-intense-bg { background-color: #B27D12; }
.ansi-blue-fg { color: #208FFB; }
.ansi-blue-bg { background-color: #208FFB; }
.ansi-blue-intense-fg { color: #0065CA; }
.ansi-blue-intense-bg { background-color: #0065CA; }
.ansi-magenta-fg { color: #D160C4; }
.ansi-magenta-bg { background-color: #D160C4; }
.ansi-magenta-intense-fg { color: #A03196; }
.ansi-magenta-intense-bg { background-color: #A03196; }
.ansi-cyan-fg { color: #60C6C8; }
.ansi-cyan-bg { background-color: #60C6C8; }
.ansi-cyan-intense-fg { color: #258F8F; }
.ansi-cyan-intense-bg { background-color: #258F8F; }
.ansi-white-fg { color: #C5C1B4; }
.ansi-white-bg { background-color: #C5C1B4; }
.ansi-white-intense-fg { color: #A1A6B2; }
.ansi-white-intense-bg { background-color: #A1A6B2; }

.ansi-bold { font-weight: bold; }

    </style>


<style type="text/css">
/* Overrides of notebook CSS for static HTML export */
body {
  overflow: visible;
  padding: 8px;
}

div#notebook {
  overflow: visible;
  border-top: none;
}@media print {
  div.cell {
    display: block;
    page-break-inside: avoid;
  } 
  div.output_wrapper { 
    display: block;
    page-break-inside: avoid; 
  }
  div.output { 
    display: block;
    page-break-inside: avoid; 
  }
}
</style>

<!-- Custom stylesheet, it must be in the same directory as the html file -->
<link rel="stylesheet" href="custom.css">

<!-- Loading mathjax macro -->
<!-- Load mathjax -->
    <script src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.1/MathJax.js?config=TeX-AMS_HTML"></script>
    <!-- MathJax configuration -->
    <script type="text/x-mathjax-config">
    MathJax.Hub.Config({
        tex2jax: {
            inlineMath: [ ['$','$'], ["\\(","\\)"] ],
            displayMath: [ ['$$','$$'], ["\\[","\\]"] ],
            processEscapes: true,
            processEnvironments: true
        },
        // Center justify equations in code and markdown cells. Elsewhere
        // we use CSS to left justify single line equations in code cells.
        displayAlign: 'center',
        "HTML-CSS": {
            styles: {'.MathJax_Display': {"margin": 0}},
            linebreaks: { automatic: true }
        }
    });
    </script>
    <!-- End of mathjax configuration --></head>
<body>
  <div tabindex="-1" id="notebook" class="border-box-sizing">
    <div class="container" id="notebook-container">

<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div>
<div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<h1 id="Lesson-3---Recurrent-Neural-Networks">Lesson 3 - Recurrent Neural Networks<a class="anchor-link" href="#Lesson-3---Recurrent-Neural-Networks">&#182;</a></h1><p>This lesson introduces the important class of Recurrent Neural Networks, or RNNs.
The student will understand how compact mutual connections in a layer can store information about the past and integrate information about the present, and will learn how to extend the backpropagation algorithm to handle time-dependent tasks.
This knowledge will be applied for regressing Arterial Blood Pressure from two other physiological signals.
At the end of the lesson, the fundamental statistical problem of regularization will be introduced along with the most important solutions to it in the field of ANN.</p>
<h3 id="Summary">Summary<a class="anchor-link" href="#Summary">&#182;</a></h3><ul>
<li><a href="#timepatterns">Finding patterns in time</a></li>
<li><a href="#bptt">BackPropagation Through Time</a></li>
<li><a href="#statefulstateless"><em>Stateful</em> vs <em>stateless</em></a></li>
<li><a href="#bloodpressure">Predicting Arterial Blood Pressure</a></li>
<li><a href="#explodevanish">The exploding/vanishing gradient problem</a></li>
<li><a href="#overfitregularize">Overfitting and regularization</a></li>
</ul>

</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div>
<div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p><a id=timepatterns></a></p>
<h3 id="Finding-patterns-in-time">Finding patterns in time<a class="anchor-link" href="#Finding-patterns-in-time">&#182;</a></h3><p>Feedforward models, as the one built during the previous lesson, struggle to capture time-dependent relationships hidden in sequential data.
This happens because, after each forward and backward pass, all the internal information and extracted features are flushed and replaced by the next sample's, and as a consequence all the memory about what was in the input is lost.</p>
<p>Suppose $X$ is a set of sequences $x = (x_1, x_2, \dots x_T), T \in \mathbb{N}$.
There are two ways to extract information from such sequences: looking at a complete sequence as a whole, or trying to get an understanding of it piece after piece, as they are revealed.</p>
<p>The first approach corresponds to a simultaneous analysis of the sequence points.
This solution requires connections for each of the timesteps $x_t$, a concern that brings along two issues:</p>
<ul>
<li>since $T \in \mathbb{N}$ could be arbitrarily large, a feedforward ANN to process long sequences could require too much memory to hold all the $T$ weights for just the first layer;</li>
<li>from a more theoretical standpoint, posing design constraints on $T$ could be unreasonable: for example, natural language sentences don't have rigid constraints on the number of words they contain!</li>
</ul>
<p>The first results on the capability of fixed-size ANNs to hold memories, in the basic sense of being capable of storing in a robust way some information about past states, were obtained by John Hopfield in a famous <a href="https://bi.snu.ac.kr/Courses/g-ai09-2/hopfield82.pdf">paper</a>.
At the time, the backpropagation algorithm had not been invented yet, and Hopfield's model was a proof of concept on single layer models with <strong>mutual connections</strong>, where each neuron could stimulate the others and even itself: remember that the perceptron and also feedforward models avoid any connection between neurons of the same layer.</p>
<p>The concept of <strong>self-connections</strong> was extremized in Hopfield's model in the sense that the neurons were unable to gather information from the external world: this limitation is a clear obstacle towards the usage of such ANNs for real world regression tasks.
But this idea was soon translated into the concept of <strong>internal state</strong> or <strong>internal memory</strong> of a network.
In 1986, a few months after backpropagation appeared, <a href="http://cseweb.ucsd.edu/~gary/PAPER-SUGGESTIONS/Jordan-TR-8604.pdf">Jordan</a> proposed a two-layer model introducing the concept of an internal state, which was later developd also by <a href="https://crl.ucsd.edu/~elman/Papers/fsit.pdf">Elman</a>.
The basic idea of this kind of networks is that a layer's output</p>
$$s_{t} = \sigma(x_{t} W^{(X)} + s_{t-1} W^{(S)})$$<p>can be <em>duplicated</em>, a copy being passed on to the next transformation while the second copy is <em>set aside</em> and reused as <em>information about the past</em> at next time step, when it will be combined with the <em>information about the present</em> coded by the input $x_{t+1}$.</p>
<p>What is not yet clear is how error signals could adjust the weights to improve the generation of the <em>memory states</em> $s_{1}, s_{2}, \dots s_{T}$.</p>

</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div>
<div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p><a id=bptt></a></p>
<h3 id="BackPropagation-Through-Time">BackPropagation Through Time<a class="anchor-link" href="#BackPropagation-Through-Time">&#182;</a></h3><p>The problem of with backpropagation in Recurrent Models is how to interpret the time dimension.</p>
<p>Analyzing the previous formula, we saw that one of the two copies of the internal memory $s_{t}$ is picked when it exits the computational node, and fed back to the same node at the next time step as input.</p>
<p>Our mental limit is that we give a computational node an immutable identity, and don't think that <em>the computational node will be a different one when it will process different inputs</em>: its functioning will be the same, but its state, and thus its global identity, will not.</p>
<p>We can thus see each computational node as uniquely determined only at a fixed timestep $t$, with its computed state $s_{t}$ exiting the node (red arrow in the figure below) at time $t$ and entering another node (blue arrow in the figure below) at time $t+1$.</p>
<p>&lt;img src='figures/rnn_unfolded.png', width=640, height=640&gt;&lt;/img&gt;</p>
<p>From the figure it emerges that a RNN can be thought at as a sequence of feedforward models (<em>snapshots</em> of the RNN model at different timesteps, or <strong>virtual copies</strong>) that passes their internal state as <em>batons</em> in a relay race.</p>
<p>This picture brings RNNs in the framework of standard backpropagation, but with an enormously larger amount of connections.
To understand how BackPropagation Through Time (BPTT for short) works, we will considerate how an error signal $\delta y_{t+3}$ might flow back to the hidden state of the first layer at time $t+1$, i.e. $h_{t+1}^{1}$.</p>
<p>&lt;img src='figures/rnn_errorpaths.png', width=480, height=480&gt;&lt;/img&gt;</p>
<p>Reverting the notation introduced in the previous lesson, where the symbol $s$ stood for <em>score</em> and the symbol $h$ stood for <em>hidden state</em>, the activation $h = \sigma(s)$, when talking of RNNs we will use the symbol $h_{t}^{l}$ to indicate the <em>scores</em> of hidden layer $l$ at timestep $t$, and $s_{t}^{l} = \sigma(h_{t}^{l})$ to denote its activation.</p>
<p>We see from the figure that the error between $y_{t+3}$ and $h_{t+1}^{1}$ can flow through three distinct paths:</p>
<ul>
<li>the <em>higher</em> path (red), due to the error that was induced by $h_{t+1}^{1}$ on the next feature extracted at the same timestep, $h_{t+1}^{2} \in H^2$, and then propagated through time to other features in the same feature space $H^2$, from $h_{t+1}^{2}$ to $h_{t+2}^{2}$, from $h_{t+2}^{2}$ to $h_{t+3}^{2}$, and finally from $h_{t+3}^{2}$ to $y_{t+3}$;</li>
<li>the <em>middle</em> path (orange), due to the error induced by $h_{t+1}^{1} \in H^1$ on the next latent representation $h_{t+2}^{1}$ in the same space $H^1$, which induced an error on a higher level feature $h_{t+2}^{2} \in H^2$, and from this passed on through time to $h_{t+3}^{2}$ and finally to $y_{t+3}$;</li>
<li>the <em>lower</em> path (violet), due to the error induced through time by $h_{t+1}^{1}$ to $h_{t+2}^{1}$ first and to $h_{t+1}^{3}$ then, and that from this propagated to the higher level features $h_{t+3}^{2}$ and $y_{t+3}$.</li>
</ul>
<p>Now, how can we translate these observations into a consistent computational procedure for BPTT?</p>
<p>We have two options.
The first one is a <strong>time unstack</strong> of error signals.</p>
<p>&lt;img src='figures/rnn_timeunstack.png', width=480, height=480&gt;&lt;/img&gt;</p>
<p>In this configuration, we reuse the similarity of a <em>relay race</em> which was used to explain forward propagation and imagine to run the race it in reverse mode.
We thing to each timestep <em>copy</em> of the network as an independent feedforward model, which we denote with $f_{t}(x_{t}, s_{t-1}, \theta)$.
This model takes as inputs the error signals $\delta y_{t}$ and $\delta s_{t}$, and consumes them to produce error signals $\delta \theta_{t}$ and $\delta s_{t-1}$.
This last error signal, $\delta s_{t-1}$, is passed back as a baton to the previous memeber of the relay, $f_{t-1}(x_{t-1}, s_{t-2}, \theta)$, and the process is repeated.</p>
<p>For example, a run of the BPTT algorithm on the previous graph would first call $f_{t+3}(x_{t+3}, s_{t+2}, \theta)$, pass $\delta y_{t+3}$ and $\delta s_{t+3}$ to it (the blue arrows on the right side of the picture) and produce $\delta \theta_{t+3}$ and $\delta s_{t+2}$.
This last signal will then be consumed by $f_{t+2}(x_{t+2}, s_{t+1}, \theta)$ together with $\delta y_{t+2}$ (the green arrows in the picture) and will produce $\delta \theta_{t+2}$.
And so on until the timestep $t = 1$ is reached in the <em>virtual network copy</em> $f_{1}(x_{1}, s_{0}, \theta)$.
The state $s_{0}$ represents the <em>virgin memory</em> of the model before processing any information, and is called <strong>initial state</strong>.
Notice that the parameters $\theta$ are shared by all these <em>virtual copies</em>, and thus the updates $\delta \theta_{t}, t=1,2,\dots T$ should be summed up to yield a global parameters update when the error signals reach the first virtual copy.</p>
<p>This interpretation of BPTT allows a straightforward implementation.
Notice that virtual copies of the model $f_{t}(x_{t}, s_{t-1}, \theta)$ differ only for the input operands $x_{t}$ and $s_{t-1}$, while the parameters $\theta$ remain unchanged operands.
If we add to <code>Node</code> classes an <code>history</code> list attribute and methods <code>pop_state</code> and <code>push_state</code>, we can thus store and retrieve virtual copies just pushing and popping operands from <code>Node</code>s' histories.
Parameter should be ignored during this stack operations, since they remain unchanged from copy to copy.</p>
<p>This implemention comes with an important drawback, though: as shown in the <em>time-vs-threads</em> graph above, BPTT invokes <code>backward</code> methods of each operation sequentially, and no <em>inter-node</em> parallelism is exploited (just <em>intra-node</em> parallelism, the one that can be used for example when backpropagatin errors through a linear transformation: $\delta x = \delta h W^T$).</p>
<p>From a parallelization perspective, the <strong>depth unstack</strong> of error signals is more appealing.</p>
<p>&lt;img src='figures/rnn_depthunstack.png', width=480, height=480&gt;&lt;/img&gt;</p>
<p>As we can see, computing the incoming error signals $\delta h_{t}^{2}$ (navy blue) and $\delta h_{t+1}^{1}$ (dark orange) can be done concurrently, since they both require to compute a single error signal vector $\delta h_{t+1}^{2}$ (dark cyan): as soon as this last signal is available, the former two can be computed comcurrently (provided the hardware architecture allows to do so).</p>
<p>This implementation is the one actually used in frameworks as Theano or TensorFlow, but is more complex and we will thus implement the first one, the time unstack.</p>

</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div>
<div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p><a id=statefulstateless></a></p>
<h3 id="Stateful-vs-stateless"><em>Stateful</em> vs <em>stateless</em><a class="anchor-link" href="#Stateful-vs-stateless">&#182;</a></h3><p>In computer science, <em>stateful</em> and <em>stateless</em> are properties of functions.
And since RNNs are a particular type of ANNs, which are in ultimate anlysis approximators of functions, the definition that we will now give can be extended to the models represented by RNNs.</p>
<p>A <strong>stateless</strong> function $f: X \to Y$ is a deterministic function.
Which is, when the input $x \in X$ is fixed, its transformed counterpart $y = f(x)$ will be fixed.
The behaviour of the function is thus stable and predictable.</p>
<p>A <strong>stateful</strong> function $g: X \to Y$ is instead a stochastic process.
Which is, even when presented the same input $x$, the map $g$ can produce different results, for example $y_1 = g(x)$ and $y_2 = g(x)$.
Mathematically, this can happen when $g$ is not a <em>perfect model</em> of the relation between $X$ and $Y$.
For example, $g$ could depend on a hidden random variable $s$, making it a map $g: X \times S \to Y$.
This is exactly the case for RNNs.</p>
<p>So, which is the advantage of stateful models versus stateless models?</p>
<p>Stateless models are really useful when the <em>stability</em> of the result is crucial.
Suppose you are building a model from the space $X$ of English books to the space $Y$ of German books, i.e. a language translation model.
Clearly, the way a book $x \in X$ is translated should not depend on other information other than $x$ itself.
Thus, to prevent <em>rubbish context</em> to influence the translation, the mapping $f$ should be able to focus only on the sequence of words that compose the text, and the initial state $s_{0}$ of the modelling RNN should be thus reset to a <em>zero state</em> whenever a new book is passed to it for translation.
Thus, stateless models can <em>discard unuseful context</em> by resetting their internal state whenever a brand new sequence is passed in.
But this is only feasible when the kind of sequence (that we remember is a data point $x \in X$) can be clearly and naturally separated from the others.</p>
<p>Stateful models are instead useful when there is some unknown <em>external noise</em> that influences the behaviour of the mapping $g: X \to Y$.
Suppose you are building a financial prediction model to link the cost of a good, quantified by $x \in X$, to the cost of another good $y \in Y$.
The dependence of $y$ from $x$ is not always neat; almost always, it will probably be just correlated, but not fully dependent.
As an example, if today wheat costs $x_{t}$ and bread costs $y_{t}$, it is likely that if tomorrow the wheat will cost $x_{t+1} &gt; x_{t}$ then also the bread will $y_{t+1} &gt; y_{t}$.
But maybe bakers have been really provident, saving some wheat today expecting tomorroy there will not be so much as today, and before increasing the bread's cost they can keep it the same for a few days: as a consequence, even if $x_{t+1} &gt; x_{t}$, we will have $y_{t+1} = y_{t}$.
A stateless model is uncapable of retaining this kind of <em>context</em> information, since at certain moments it's reset.</p>
<p>To this end, we need <code>Node</code>s to keep an <code>initial_state</code> that should be reset (if they are part of stateless models) or simply saved (if they are part of stateful models).
In fact, after the last virtual copy of the network has processed its inputs $s_{T-1}$ and $x_{T}$ and produced $s_{T}$, we cannot push them into <code>history</code> lists, since BPTT will pop and then discard them.
This information thus requires an appropriate attribute field to be stored.</p>

</div>
</div>
</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In&nbsp;[1]:</div>
<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="kn">import</span> <span class="nn">numpy</span> <span class="k">as</span> <span class="nn">np</span>


<span class="k">class</span> <span class="nc">Node</span><span class="p">(</span><span class="nb">object</span><span class="p">):</span>
    <span class="sd">&quot;&quot;&quot;Create an operation.</span>

<span class="sd">    Each operation takes an input consisting of one ore more operands (which</span>
<span class="sd">    are themselves states of other operations) and computes an output state.</span>
<span class="sd">    For BackPropagation, the operation should i) accumulate incoming errors,</span>
<span class="sd">    ii) then compute how to distribute the corresponding outgoing errors to</span>
<span class="sd">    input nodes; if a node contains trainable parameters, it should notify the</span>
<span class="sd">    framework that its state can be modified.</span>
<span class="sd">    Recurrent cells also require to store an initial internal memory state.</span>
<span class="sd">    For BackPropagation Through Time (BPTT), every operation should keep a</span>
<span class="sd">    history record of its state, a stack where to push states after each</span>
<span class="sd">    timestep of forward propagation and from which to pop them during BPTT.</span>

<span class="sd">    Attributes:</span>
<span class="sd">        inbound_nodes (:obj:`list` of :obj:`Node`): the operations whose</span>
<span class="sd">            states are required to perform the implemented operation.</span>
<span class="sd">        outbound_nodes (:obj:`list` of :obj:`Node`): the operations which</span>
<span class="sd">            require this operation&#39;s state for their computations.</span>
<span class="sd">        state (:obj:`ndarray`): the output of the operation.</span>
<span class="sd">        incoming_error (:obj:`ndarray`): the error signal collected from</span>
<span class="sd">            outbound nodes.</span>
<span class="sd">        outgoing_errors (:obj:`dict` of :obj:`ndarray`): the error signals</span>
<span class="sd">            to be communicated to inbound nodes.</span>
<span class="sd">        is_trainable (:obj:`Bool`): whether the node contains trainable</span>
<span class="sd">            parameters.</span>
<span class="sd">        initial_state (:obj:`None` or :obj:`ndarray`): the internal memory of</span>
<span class="sd">            the operation.</span>
<span class="sd">        history (:obj:`list` of :obj:`ndarray`): stack of subsequent states</span>
<span class="sd">            of the node.</span>

<span class="sd">    Methods:</span>
<span class="sd">        push_state: push the current state on self.history stack.</span>
<span class="sd">        pop_state: pop last state from self.history stack.</span>
<span class="sd">        forward: compute the output of the operation.</span>
<span class="sd">        backward: compute the outgoing error signals.</span>

<span class="sd">    &quot;&quot;&quot;</span>

    <span class="k">def</span> <span class="nf">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">inbound_nodes</span><span class="o">=</span><span class="nb">list</span><span class="p">()):</span>
        <span class="sd">&quot;&quot;&quot;Create a differentiable operation.</span>

<span class="sd">        Args:</span>
<span class="sd">            inbound_nodes (:obj:`list` of :obj:`Node`): the operations</span>
<span class="sd">                whose states are required to perform the implemented operation.</span>

<span class="sd">        &quot;&quot;&quot;</span>
        <span class="bp">self</span><span class="o">.</span><span class="n">inbound_nodes</span> <span class="o">=</span> <span class="n">inbound_nodes</span>
        <span class="c1"># communicate to inbound nodes that a node has been added on top of them</span>
        <span class="k">for</span> <span class="n">node</span> <span class="ow">in</span> <span class="bp">self</span><span class="o">.</span><span class="n">inbound_nodes</span><span class="p">:</span>
            <span class="n">node</span><span class="o">.</span><span class="n">outbound_nodes</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="bp">self</span><span class="p">)</span>
        <span class="bp">self</span><span class="o">.</span><span class="n">outbound_nodes</span> <span class="o">=</span> <span class="nb">list</span><span class="p">()</span>
        <span class="bp">self</span><span class="o">.</span><span class="n">state</span> <span class="o">=</span> <span class="kc">None</span>
        <span class="bp">self</span><span class="o">.</span><span class="n">incoming_error</span> <span class="o">=</span> <span class="kc">None</span>
        <span class="bp">self</span><span class="o">.</span><span class="n">outgoing_errors</span> <span class="o">=</span> <span class="nb">dict</span><span class="p">()</span>
        <span class="bp">self</span><span class="o">.</span><span class="n">is_trainable</span> <span class="o">=</span> <span class="kc">False</span>
        <span class="bp">self</span><span class="o">.</span><span class="n">initial_state</span> <span class="o">=</span> <span class="kc">None</span>
        <span class="bp">self</span><span class="o">.</span><span class="n">history</span> <span class="o">=</span> <span class="nb">list</span><span class="p">()</span>

    <span class="k">def</span> <span class="nf">push_state</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
        <span class="bp">self</span><span class="o">.</span><span class="n">history</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">state</span><span class="p">)</span>

    <span class="k">def</span> <span class="nf">pop_state</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
        <span class="bp">self</span><span class="o">.</span><span class="n">state</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">history</span><span class="o">.</span><span class="n">pop</span><span class="p">()</span>

    <span class="k">def</span> <span class="nf">forward</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
        <span class="k">raise</span> <span class="ne">NotImplementedError</span>

    <span class="k">def</span> <span class="nf">backward</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
        <span class="k">raise</span> <span class="ne">NotImplementedError</span>
</pre></div>

</div>
</div>
</div>

</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In&nbsp;[2]:</div>
<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="k">class</span> <span class="nc">RNNCell</span><span class="p">(</span><span class="n">Node</span><span class="p">):</span>
    <span class="k">def</span> <span class="nf">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">x</span><span class="p">,</span> <span class="n">w_xs</span><span class="p">,</span> <span class="n">w_ss</span><span class="p">,</span> <span class="n">initial_state</span><span class="p">):</span>
        <span class="n">Node</span><span class="o">.</span><span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">inbound_nodes</span><span class="o">=</span><span class="p">[</span><span class="n">x</span><span class="p">,</span> <span class="n">w_xs</span><span class="p">,</span> <span class="n">w_ss</span><span class="p">])</span>
        <span class="bp">self</span><span class="o">.</span><span class="n">initial_state</span> <span class="o">=</span> <span class="n">initial_state</span>
        <span class="bp">self</span><span class="o">.</span><span class="n">inbound_nodes</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="bp">self</span><span class="p">)</span>
        <span class="bp">self</span><span class="o">.</span><span class="n">outbound_nodes</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="bp">self</span><span class="p">)</span>
        <span class="bp">self</span><span class="o">.</span><span class="n">outgoing_errors</span><span class="p">[</span><span class="bp">self</span><span class="p">]</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">zeros_like</span><span class="p">(</span><span class="n">initial_state</span><span class="p">)</span>

    <span class="k">def</span> <span class="nf">forward</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
        <span class="n">x</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">inbound_nodes</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">state</span>
        <span class="n">w_xs</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">inbound_nodes</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span><span class="o">.</span><span class="n">state</span>
        <span class="n">w_ss</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">inbound_nodes</span><span class="p">[</span><span class="mi">2</span><span class="p">]</span><span class="o">.</span><span class="n">state</span>
        <span class="n">logits</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">dot</span><span class="p">(</span><span class="n">x</span><span class="p">,</span> <span class="n">w_xs</span><span class="p">)</span> <span class="o">+</span> <span class="n">np</span><span class="o">.</span><span class="n">dot</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">state</span><span class="p">,</span> <span class="n">w_ss</span><span class="p">)</span>
        <span class="bp">self</span><span class="o">.</span><span class="n">state</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">tanh</span><span class="p">(</span><span class="n">logits</span><span class="p">)</span>

    <span class="k">def</span> <span class="nf">backward</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
        <span class="bp">self</span><span class="o">.</span><span class="n">incoming_error</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">zeros_like</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">state</span><span class="p">)</span>
        <span class="k">for</span> <span class="n">node</span> <span class="ow">in</span> <span class="bp">self</span><span class="o">.</span><span class="n">outbound_nodes</span><span class="p">:</span>
            <span class="bp">self</span><span class="o">.</span><span class="n">incoming_error</span> <span class="o">+=</span> <span class="n">node</span><span class="o">.</span><span class="n">outgoing_errors</span><span class="p">[</span><span class="bp">self</span><span class="p">]</span>
        <span class="c1"># propagate error through tanh</span>
        <span class="n">error_tanh</span> <span class="o">=</span> <span class="p">(</span><span class="mi">1</span><span class="o">+</span><span class="bp">self</span><span class="o">.</span><span class="n">state</span><span class="p">)</span> <span class="o">*</span> <span class="p">(</span><span class="mi">1</span><span class="o">-</span><span class="bp">self</span><span class="o">.</span><span class="n">state</span><span class="p">)</span> <span class="o">*</span> <span class="bp">self</span><span class="o">.</span><span class="n">incoming_error</span>
        <span class="c1"># propagate error through addition</span>
        <span class="n">error_xwxs</span> <span class="o">=</span> <span class="n">error_tanh</span>
        <span class="n">error_swss</span> <span class="o">=</span> <span class="n">error_tanh</span>
        <span class="c1"># propagate error to x and w_xs</span>
        <span class="n">x</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">inbound_nodes</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">state</span>
        <span class="n">w_xs</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">inbound_nodes</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span><span class="o">.</span><span class="n">state</span>
        <span class="bp">self</span><span class="o">.</span><span class="n">outgoing_errors</span><span class="p">[</span><span class="bp">self</span><span class="o">.</span><span class="n">inbound_nodes</span><span class="p">[</span><span class="mi">0</span><span class="p">]]</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">dot</span><span class="p">(</span><span class="n">error_xwxs</span><span class="p">,</span> <span class="n">w_xs</span><span class="o">.</span><span class="n">T</span><span class="p">)</span>
        <span class="bp">self</span><span class="o">.</span><span class="n">outgoing_errors</span><span class="p">[</span><span class="bp">self</span><span class="o">.</span><span class="n">inbound_nodes</span><span class="p">[</span><span class="mi">1</span><span class="p">]]</span> <span class="o">=</span> <span class="n">x</span><span class="p">[</span><span class="kc">None</span><span class="p">,</span> <span class="p">:]</span><span class="o">.</span><span class="n">T</span> <span class="o">*</span> <span class="n">error_xwxs</span><span class="p">[</span><span class="n">i</span><span class="p">]</span>
        <span class="c1"># propagate error to self and w_ss</span>
        <span class="n">wss</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">inbound_nodes</span><span class="p">[</span><span class="mi">2</span><span class="p">]</span><span class="o">.</span><span class="n">state</span>
        <span class="bp">self</span><span class="o">.</span><span class="n">outgoing_errors</span><span class="p">[</span><span class="bp">self</span><span class="o">.</span><span class="n">inbound_nodes</span><span class="p">[</span><span class="mi">2</span><span class="p">]]</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">state</span><span class="p">[</span><span class="kc">None</span><span class="p">,</span> <span class="p">:]</span><span class="o">.</span><span class="n">T</span> <span class="o">*</span> <span class="n">error_swss</span><span class="p">[</span><span class="n">i</span><span class="p">]</span>
        <span class="bp">self</span><span class="o">.</span><span class="n">outgoing_errors</span><span class="p">[</span><span class="bp">self</span><span class="p">]</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">dot</span><span class="p">(</span><span class="n">error_swss</span><span class="p">,</span> <span class="n">wss</span><span class="o">.</span><span class="n">T</span><span class="p">)</span>
</pre></div>

</div>
</div>
</div>

</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div>
<div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p><a id=bloodpressure></a></p>
<h3 id="Predicting-Arterial-Blood-Pressure">Predicting Arterial Blood Pressure<a class="anchor-link" href="#Predicting-Arterial-Blood-Pressure">&#182;</a></h3><p><a href="https://physionet.org/mimic2/">MIMIC II databases</a> are collections of physiological data for biomedical research, which are in part publicly available.</p>
<p>Many physiological data can be naturally represented as sequential signals.
Some vital signals are easy to monitor; but other, important signals cannot be measured equally easily.
One example is Arterial Blood Pressure (ABP): its direct measurement requires cannulas to be inserted directly into a patient's veins, which is neither cheap nor desirable.
Other signals, such as electrocardiography (ECG) or photopletismography (PPG) require less invasive measurement techniques.
If we could tell $ABP(t)$ depending only on $ECG(t)$ and $PPG(t)$, i.e. if we could retrieve a model</p>
$$a(t) = f(e(t), p(t))$$<p>it would surely be a great deal.</p>

</div>
</div>
</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In&nbsp;[3]:</div>
<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="kn">import</span> <span class="nn">matplotlib.pyplot</span> <span class="k">as</span> <span class="nn">plt</span>
<span class="kn">import</span> <span class="nn">pandas</span> <span class="k">as</span> <span class="nn">pd</span>
<span class="kn">from</span> <span class="nn">sklearn.preprocessing</span> <span class="k">import</span> <span class="n">scale</span>

<span class="c1"># load data</span>
<span class="n">file</span> <span class="o">=</span> <span class="s1">&#39;./src/tests/data/blood_pressure.txt&#39;</span>
<span class="n">dataset</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">read_csv</span><span class="p">(</span><span class="n">file</span><span class="p">,</span> <span class="n">sep</span><span class="o">=</span><span class="s1">&#39;;&#39;</span><span class="p">)</span>
<span class="n">X</span> <span class="o">=</span> <span class="n">dataset</span><span class="o">.</span><span class="n">values</span><span class="p">[:,</span> <span class="p">(</span><span class="mi">3</span><span class="p">,</span> <span class="mi">1</span><span class="p">)]</span>
<span class="n">Y_hat</span> <span class="o">=</span> <span class="n">dataset</span><span class="o">.</span><span class="n">values</span><span class="p">[:,</span> <span class="mi">2</span><span class="p">][:,</span> <span class="kc">None</span><span class="p">]</span>

<span class="c1"># normalize data</span>
<span class="n">X</span> <span class="o">=</span> <span class="n">scale</span><span class="p">(</span><span class="n">X</span><span class="p">,</span> <span class="n">axis</span><span class="o">=</span><span class="mi">0</span><span class="p">)</span>
<span class="n">Y_hat</span> <span class="o">=</span> <span class="n">scale</span><span class="p">(</span><span class="n">Y_hat</span><span class="p">,</span> <span class="n">axis</span><span class="o">=</span><span class="mi">0</span><span class="p">)</span>

<span class="c1"># plot data</span>
<span class="n">timewindow</span> <span class="o">=</span> <span class="nb">range</span><span class="p">(</span><span class="mi">500</span><span class="p">,</span> <span class="mi">700</span><span class="p">)</span>
<span class="n">fig1</span> <span class="o">=</span> <span class="n">plt</span><span class="o">.</span><span class="n">figure</span><span class="p">()</span>
<span class="n">ax</span> <span class="o">=</span> <span class="n">fig1</span><span class="o">.</span><span class="n">gca</span><span class="p">()</span>
<span class="n">ax</span><span class="o">.</span><span class="n">plot</span><span class="p">(</span><span class="n">X</span><span class="p">[</span><span class="n">timewindow</span><span class="p">,</span> <span class="mi">0</span><span class="p">],</span> <span class="n">label</span><span class="o">=</span><span class="s1">&#39;ECG&#39;</span><span class="p">)</span>
<span class="n">ax</span><span class="o">.</span><span class="n">plot</span><span class="p">(</span><span class="n">X</span><span class="p">[</span><span class="n">timewindow</span><span class="p">,</span> <span class="mi">1</span><span class="p">],</span> <span class="n">label</span><span class="o">=</span><span class="s1">&#39;PPG&#39;</span><span class="p">)</span>
<span class="n">ax</span><span class="o">.</span><span class="n">plot</span><span class="p">(</span><span class="n">Y_hat</span><span class="p">[</span><span class="n">timewindow</span><span class="p">],</span> <span class="n">label</span><span class="o">=</span><span class="s1">&#39;ABP&#39;</span><span class="p">)</span>
<span class="n">ax</span><span class="o">.</span><span class="n">legend</span><span class="p">()</span>
<span class="n">plt</span><span class="o">.</span><span class="n">show</span><span class="p">()</span>
</pre></div>

</div>
</div>
</div>

<div class="output_wrapper">
<div class="output">


<div class="output_area">

<div class="prompt"></div>


<div class="output_png output_subarea ">
<img src="data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAYAAAAD8CAYAAAB+UHOxAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz
AAALEgAACxIB0t1+/AAAIABJREFUeJzsnWd4W+XZgO9Xy/IesZ3hHcfZexEyCARSCHuUXUYbSlmF
ssoolNnS0q+0FCibtrSQsPcOIYsEsndsx7Fjx3a8t7Wl8/04lmLHsi3ZGpZ87uvylVjnPed9bMnn
Oc8WkiShoKCgoDD0UAVbAAUFBQWF4KAoAAUFBYUhiqIAFBQUFIYoigJQUFBQGKIoCkBBQUFhiKIo
AAUFBYUhiqIAFBQUFIYoigJQUFBQGKIoCkBBQUFhiKIJtgC9kZycLGVnZwdbDAUFBYWQYdu2bXWS
JKV4snZQK4Ds7Gy2bt0abDEUFBQUQgYhRKmnaxUXkIKCgsIQRVEACgoKCkMURQEoKCgoDFEUBaCg
oKAwRFEUgIKCgsIQRVEACgoKCkMURQEoKCgoDFEGdR3AUKKqvYqvDn9Fq6WV9Nh0zs09F5VQ9LOC
goL/UBTAIOCNA2/w581/RuLYfObVZav548I/EqOLCaJkCgoK4YzyiBlkzHYzL+56kVnDZ/HlRV+y
++rd3Dv3XtaXr+eRTY8EWzwFBYUwRlEAQebLki9pNDdyw7QbSItJQwjBlROu5FfTfsWXh79kXfm6
YIvoF2x2B0caDMEWQ0FhSKMogCAiSRJv5r/J6PjRzB0xt8ux5ZOXMyZhDI//8DhmuzlIEvqPj3ZW
cupTa2k2WoMtioLCkEVRAEGkqKmI/fX7uXTcpQghuhzTqrXcN/c+jrYf5d3Cd4Mkof+oajFhsTmo
bQ0/5aagECooCiCIbKjYAMCpmae6PT535FzmjpjLK3tewWgzBlI0v9NutgHQZLAEWRIFhaGLogCC
yIaKDeQl5jE8eniPa26afhN1xrqwswIMFjsADe2KAhhK2B1S34sUAoaiAIJEu7Wd7TXbWThqYa/r
Zg2fxdSUqXxQ9EGAJAsMbS4LQIkBDBUsNgfz//Qtb285EmxRFDpQFECQ2Hx0MzaHjQVpC/pce87o
czjYeJCChoIASBYYDBZZATQoLqAhQ2F1K9UtZkob2oMtikIHigIIEt9Xfk+kJpKZqTP7XHtG9hlo
hIZPDn0SAMkCQ7tZdgE1DmEFsL58PbeuvpW7195Ng6kh2OL4nd3lzQCYrI4gS6LgRFEAQWJf3T6m
Jk9Fq9b2uTZBn8DC9IV8VvIZdoc9ANL5H2cQuHGIxgDWl6/npm9vYk/dHr478h2XfHIJh5oOBVss
v7KnogkAsy08PsPhgKIAgoDdYaeoqYixSWM9PueM7DOoM9aR35DvR8kCR7srCDz0YgCSJPHszmdJ
i0njq4u+4vVlr2OX7Nz87c3UG+uDLZ7fUCwA2F27m9f2vsbbBW9jspmCLY6iAIJBWWsZJruJsYme
KwBnodjmqs3+EiugOGMAQzENdM2RNeyv38+vpv4KnVrHxGETeWbJM9Qb67lz7Z1IUvhlypisdgqq
WgEw24amAthVu4uff/lz/rbtbzz2w2Pcv+F+HFJwfxeKAggChY2FAF4pgJSoFHLic8JGAThdQEMx
CPz6/tdJj0nnnNxzXK9NTp7Mb+f+lm3V21hbvjaI0vmH/KpWbB0poCbr0HMB1RhquHX1rQyPHs6q
n67ijll38E3pNzy/6/mgyqUogCBQ2FiIWqjJTcj16ry5I+ayvXo7Vkfou02cQeChlgZab6xnW/U2
zsk9B42qazPe88ecT0ZsBs/tfC7srIA95bL/PzU2YkhaAP878D+azc08s+QZhkcP59pJ13Ju7rm8
tPsl9tXvC5pcA1YAQogMIcR3Qoj9Qoh9Qojb3KwRQoh/CCGKhBC7hRB9p76EMYUNhWTHZROhjvDq
vDkj5mCwGdhfv99PkgUGu0PCaLWjErILyDGEioPWHFmDhOS2+lur0nLDtBvIb8hn9ZHVQZDOf5Q1
GNBrVeQkRw85C8BkM/H+wfdZkrnE9dAnhOCeufeQpE/ikY2PYHPYgiKbLywAG3CnJEkTgXnAzUKI
icetWQbkdXxdDwTX7gkyhY2FXgWAncwZMQeQawhCGaf/f0ScHocELaahYwWsKltFWkxaj+6/M3PO
ZFT0KN448EaAJfMvRqudaJ0GvVY95CyAL0q+oNnczOXjL+/yepwujvvm3seBhgNBS/EesAKQJOmo
JEnbO/7fChwA0o5bdh7wuiTzA5AghBg50L1DkRZLC5XtlV75/50k6ZPIjc9lZ+1OP0gWOJxtINIT
o4Ch0w6i1dLKj0d/5LTM07o1/3OiUWm4dPylbKna4ooVhQNGiwO9Vk2ERoV5iFkA7x58lzEJY5g9
fHa3Y0uzljImYQwr8lcExe3n0xiAECIbmAH8eNyhNKBz/Xc53ZXEkMCZ652XkNev86ekTGFv3d6Q
9hE720CkJ0YC0DhE4gBbq7ZidVhZnLG413UXjrmQCHUEK/NXBkgy/2Oy2YnQqoacBdBubWdv3V5O
zTzVrdIXQnD5+Ms50HCAXbW7Ai6fzxSAECIGeA/4jSRJLQO4zvVCiK1CiK21tbW+Em/QcLj5MAA5
8Tn9On9K8hQaTA1Utlf6UKrAYugIAKc5FcAQsQD21O1BLdRMTp7c67oEfQLLcpbxafGnYdMF1my1
E9lhAQylGMDu2t04JAczUmf0uObs0WcTq43lzfw3AyiZjE8UgBBCi3zzf0OSpPfdLKkAMjp9n97x
WjckSXpJkqTZkiTNTklJ8YV4g4rSllI0Kg2jYkb163znzWNP7R5fihVQ2i1dLYChkgq6t24veYl5
RGoi+1x7Zs6ZGG1GNlVuCoBk/sdotaPXqoecBbCzZicCwdSUqT2uidJGcdbos1hdthqDNbBT8nyR
BSSAV4EDkiQ91cOyj4GrO7KB5gHNkiQdHejeocjhlsNkxGZ0SwH0lLzEPHQqHXvqQlgBuFxAcgxg
KBSDOSQHe+v39vn072T2iNnE6eL4tuxbP0sWGExWB3qtashZADtqdpCXmEesLrbXdUuzlmK2m9lY
uTFAksn4wgJYAFwFLBFC7Oz4OlMIcYMQ4oaONZ8DxUAR8DJwkw/2DUlKW0rJisvq9/lalZYJwyaw
t26vD6UKLM42EKmxEWjVYki0gyhrKaPV0srU5J6fBDujVWlZnL6YteVrw6Luw9ThAhpKFoDdYWd3
3e5e3T9OZg6fSUJEAqvKVgVAsmP07zG0E5IkbQDcpzQcWyMBNw90r1DH7rBT1lLGwrTeZwD0xZTk
Kbxb+C42h63flkQwMXRYANERGuIjdTQbw98CcFpsnloAIE+K+6T4E7ZVb2PeyHn+Ei0gGK12Ijpi
AHaHhNXuQKsO7zrUg00Habe2Mz11ep9rNSoNi9MXs7psNVa71aMmkb4gvN+BQUaVoQqLwzIgCwBg
UvIkTHYTxc3FPpIssLR1UgBROjVGS/i7BPbU7SFSE8no+NEenzM/bT4R6gjWla/zo2SBwWx1oNfI
FgAMjX5ATit9WvI0j9aflnUardZWtlRt8adYXVAUQABxZgBlx2UP6DrjE8cDhGyeuLMOIEqnRq9V
DYnukPvq9zFx2ETUKrXH50RqIpmWMi2gNwR/YbTaidSpiNDKt5yhEAcobi5Gr9aTFutZxvu8kfPQ
qrRsOhq4wL+iAALI4ZbDAGTHZw/oOlnxWWhVWgobQlMBtFts6DQqtGo5L9wY5jcDSZIoaSphTMIY
r8+dM2IOBQ0FNJub/SBZ4DBZ7bIFoBk6FkBJcwnZ8dmohGe3Wb1Gz9SUqQFt+KgogABS2lJKtDaa
YfphA7qOVqUlNyE3ZC2AdrONmAg5dqHXqMP+abDOWEertbVftR9zR8xFQmJr1VY/SBYYJEmSFYBW
PaQsgJLmEq/f87kj5pLfkE+Lpd+lVF6hKIAAUtpSSmZsZo9tALxhbOLYkFUABrOdKJ38JKjXqTGF
+dOgM1bjjf/fyZTkKURqIkO6DbjF7sAhQaROTYTTAghzt5/RZqSyrdJrBTBnxBwckoNtVdv8JFlX
FAUQQCraKsiIzeh7oQeMTRxLrbE2JGfJtltsROucFkD494YpaS4B+qcAtGot01Omh7QCcMZ4IjSd
YgBhPhaytKUUCcnr93xayjQi1BEBe78VBRAgHJKDyrZK0mJ80wLJ2UwuFK2AdrOd6IgOC0Ab/i6g
4uZiorXRpEal9uv8OSPmUNRUFLJxAKeCj9R1igGEuQVQ3NQ/q0+n1jE9ZXrAAv+KAggQtYZarA6r
zxTAuKRxACEZCG632Ih2xgCGQBZQcXMxOXE5/Xb9OdsIhGrxnzPIr9eoh4wFUNxcjEqo+pXyfWrW
qYxJHIPd4f/fkaIAAoSzeVt/ewAdT5I+ieTIZAoaC3xyvUDSJQYwBLKASppLGJ3gvfvHycRhExGI
kG3/4VTweu3QsQBKmktIj0lHp9Z5fe7l4y/nT4v+5FXKcH9RFECAKG8tB/A4J9gTRsePdqWWhhJt
5mMWQGSYu4DaLG3UGGr63f0VIFYXS058TshaACaXC+hYDMA8BCyA/sR8Ao2iAAJEZVuHBRDtGwsA
ICsui7KWMp9dL1AYOgWBIzp6w4TyfIPecAaAB6IAQM4G2lO3JyR/T51dQK5K4DC2ACRJory1nMy4
zGCL0ieKAggQFW0VJEcmo9fofXbNrLgsmsxNIRcclKtCnS4g5xNheN4QnBZaTtzAFUCDqYGj7aHX
RNdpATh7AUF4xwDqTfWY7Cafxfv8iaIAAkRlW6XP/P9OnAGm0pZSn17X31jtErqORmBOn3C4uoGO
tB5BIEiPTR/QdSandMyBCME4gMsFpB0aFoDT3TvQ9zwQKAogQJS3lfv8icBpYoaSArA7JOwOydUJ
0nlDCNdMoLLWMkZGj+xXMLAzYxPGolPp2Fe3z0eSBY5jQWDVMQsgTBU+yH/rAOkxigJQAGwOG9Xt
1T5XABkxGaiEKqQUgNUu3wy0GjklMlInfwTDNRPoSMsRMuIGXvynVWsZnTCag00HfSBVYHHe7PVa
NVq1CrVKhK3LD6CiVR526GuL3x8oCiAA1BhqsEk2nysArVrLqOhRIaUALB0KYKi4gMpay8iM9U0w
MDchl6KmIp9cK5AYO7mAQK7+Dtf3G2QLICUyxafxPn+hKIAAUNHmvyeCrPiskFIA1o4nv+4uoPC7
ITSbm2kyN/lMAYxJGENVexVtljafXC9QdK4DgGOZX+FKRVtFSASAQVEAAcGpAPzhE8yKlRVAqKQH
Wu2ynE4FcKw7ZPjdEI60HgHwiQsIcLWTDjUrwJUFpHFafeFtAVS0Vvi03sefKAogAFS2VSIQjIge
4fNrZ8VlYbAZqDfV+/za/sAVA1DLMQCXBRCGaYHOGg1fuoAADjUd8sn1AoXJaidCo0Klkt/zCG34
doC1OqxUGapCIgAMigIICBVtFaRGpQ44E8QdzlRQ57SxwY4rBqDpGgMIx46gZa2yAvBVB9i0mDQi
NZEhaQE4FT3IlkA4vt8AVW1VOCSH4gJSOIY/fYJOBeC82Qx2rMcFgZ0FYeGYBXSk9QjDo4b7LBio
EipGx48OQQXgcBX8QXhbAEfaZLdfKNQAgI8UgBDiNSFEjRDCbbMSIcTJQohmIcTOjq/f+2LfUMGf
CmBk9Ei0Km3I9ASy2rrGAPRhHAMoaynzeTuA3ITckHMBGa12VwYQhPcMCH/G+/yBryyAfwNn9LFm
vSRJ0zu+HvXRvoMeq91KjaHGbznBapWajNiMkOkJZHHVAYR/GujhlsM+8/87GZMwhlpjbUi1/+jm
AgpjC6CitQKNStPv2Q+BxicKQJKkdUDojaYKAFXt/vcJZsZlhkwqaI9B4DCzABpNjTSZm3zeEdJ5
PWeTuVDAeJwCCGcLoLytnFHRowLSytkXBDIGMF8IsVsI8YUQYlIA9w0qFe2ySehPBZAdl01ZSxkO
afDfRI+PAYRrawDXHOABzAFwR3Z8NkDIuPxA7vtzfAwgXOsAKlpDpwYAAqcAtgOZkiRNBZ4BPuxp
oRDieiHEViHE1tra2gCJ5z+cZeH+zAvOjMvE4rBQ1V7ltz18xTELQP7oqVSCCI0q7NJAnQpgoG2g
jyctJg2NShNSFoDJNnQsgIq20KkBgAApAEmSWiRJauv4/+eAVgiR3MPalyRJmi1J0uyUlJRAiOdX
KtoqUAs1w6OG+22P7LhsIDSawlmOCwJDx1xgS3jdEEqaS4jURDIyeqRPr6tRaciMzQyZtF8Ao6Vr
EDhCqwrLGEC7tZ1Gc2PIBIAhQApACDFCdAxEFULM7dg3NCqXBkhFWwUjokegUWn8tocz0BgKCsDl
AtIcm48bjnOBi5uLyY7LRiV8/yeWHZdNSUsoWwDqsLQA/DH1z9/45K4khFgBnAwkCyHKgYcALYAk
SS8APwVuFELYACNwmRQqvQsGSEVbhd+7AqZGpRKpiQwpBdDNAggzF1BJUwnTUqf55do58Tmsq1iH
zWHz64OFr+heBxCeFkCopYCCjxSAJEmX93H8WeBZX+wVapS1lLEkc4lf9xBCkBkbGplAbhWAJrzm
AhttRirbK7kg/gK/XD87Phubw0ZFW4WrEHAwY7J0twDsDgmb3YFGHT61qK5BMCGkAMLntz8IaTY3
02hudPno/UlWXFZIVANb7G5iADp1WLmAnP55fw0Fd36eQiUQfLwLyNUAMMysgIq2CqK10cRHxAdb
FI9RFIAfcT6RO1P3/ElWXBblreVYHVa/7zUQnO2gdV0sgPDqDumvDCAnzuuGQiDYZndgtUuugj+g
01jI8HnP4djUv45wZ0igKAA/4szVDoSZnhWXhV2yU9lW6fe9BsLxE8GgIwYQRjeDwsZCNCqN3yy/
+Ih4kvRJIREIdj7lOye/Qafaj3CzAForQsr9A4oC8CuHmw+jFuqAfChCZUC8+yBweGUBFTYWkhuf
i1at9dse2XHZIWEBOJ/ydccF/TsfCwckSQq5GgBQFIBfKW0pJS0mza83AiehogCcMQCN6jgLIIyy
gAobCxmbONave+TE54RENbBzAJBO07UdNIRX+48aQw0mu8nnvZ/8jaIA/EhpS2lA/P8ACREJxOpi
B70CsNod6NSqLn7SyDByATWZmqgx1DAuaZxf98mOy6bB1DDom8JZnDEfTddWEADmMFL6zgQMRQEo
AOCQHJS1lgUsTU8IQXZc9uBXADaHqxGcEzkGEB5Pg4WNhQDkJeb5dR9nIHiwZwJZjmv+B+FpAfh6
/GegUBSAn6gx1GC0GQOSAuokFLqCWu0OVytoJxHa8MkCKmgsAGBcop8tgA7LctArgA4LIELjJgYQ
ThZASxkaofF56w9/oygAP+H0zwZSAWTFZXG0/Sgmmylge3qLxS51CQBDR2sAmwOHI/SLwwsbCxmm
H8awyGF+3cfZFG6wxwHcBf3D0QIoay0jPTY9JCqzO6MoAD9R2CC7ApyDvANBVuzgHw/pjAF05tgT
YejfEAoaCvweAIZjTeEGvQVg7x4DCEcL4EjrEZ/Nfg4kigLwEwcaDjA8arjfnwQ74+w97yxEGoxY
7Y4uNwOASNdYyNC+IRisBg42HmRScmDGXWTHZQ9+C8DWswVgDhMLQJIkv4z/DASKAvATB+oPMGHY
hIDuOTp+NGqhdlkfgxFLD0FgIORTQXfU7MAm2ZgzfE5A9suJz+FI65FBXf1tHgIWQL2pHoPNoFgA
CjIGq4GSlhImJk0M6L46tY6c+BxXJspgxGp3dI8BdNwQjCE+E2Bz1WY0Kg3TU6cHZL+c+BxsDpsr
A2Uw4q71R7jFAJzzuEMtBRQUBeAXChsLcUiOgFsAAGMTx7oyUQYjboPAYTIXeEvVFqYkTyFKGxWQ
/cYnjQcgvz4/IPv1h6EQA3DVACguIAWQ/f8AE5KCowCq2qsGbYGQ1dY9CByp67AAQjgG0GppZV/9
PuaMCIz7B+SYj1aldX3eBiPusoA0KoFKhL7Cd3K4+TAaoWFUtH/nfvgDRQH4gQP1B0jSJ5EalRrw
vZ0VqIPVDSTXAXSNAUS6LIDQVQDbq7fjkBzMHTE3YHtqVVrGJo7lQP0gVgA2ZyuIY7caIQR6rTps
LICCxgJyEnIC0vLF1ygKwA/srd/LhKQJQWkL60xBHNQK4HgLIAxiAJ8Vf0asNpZpKf6ZAtYTE4ZN
YH/DfgbrgD2zm0pgkOMA4WIBFDYW+r3wz18oCsDH1BpqOdh4kNkjZgdl/5TIFBIiEgatAnAXA3C2
Cg5VF1CtoZZvSr/h/Lzz0Wv0Ad17QtIEWi2trnGEgw1nEDhCre7yerhYAM7eT4Go/fAHigLwMRsr
NwKwMG1hUPYXQjAuadygdQv0VggWqgrgncJ3sEt2Lht3WcD3njhMzjQbrHEAi5v5DxA+FoDzQUux
ABQA2FCxgeTI5KB+IGamziS/IX9QBoJlF1D4xACq26t5q+AtFqYtDEoWSF5iHmqhHrwK300aKISP
BeBUAGOTFAtgyGN32NlYuZEFoxYEdSzcnBFzkJDYXr09aDL0hNwN1H0WUKgpAIPVwK9X/xqTzcRv
Zv0mKDJEqCMYkzCGvXV7g7J/X1jsDoQAtaoHC6C2AD69HT66BfI/C5KU/aegsYAkfRLJkcnBFqVf
+KRzkRDiNeBsoEaSpMlujgvgaeBMwABcK0lSv+5OVquV8vJyTKbB1/DMYrfw6JhHSdQncuCA909k
er2e9PR0tNqBZRNMS5lGhDqCzVWbOSXzlAFdy9dY7FK3bqDOebFGS2i5BJ7d+Sz5Dfk8e+qzQfUB
T0+dzqfFn2J32FGr1H2fEEAsbuY/gDwTIMF0BP7zMzC3giYCdvwXZl4NZ/4VNLogSewdgRj+4098
1bru38CzwOs9HF8G5HV8nQA83/Gv15SXlxMbG0t2dvagG75c2VaJ1iyn5nnbFVCSJOrr6ykvLycn
Z2DDxHVqHdNTprOlasuAruMP3MUAVCqBTqMKqRhAjaGGtwve5rwx53FS+klBlWV66nTeKniLg00H
XcVhgwWLm7oPgBi1jd/WPAA6C/zyOxiWC9/9ETY8BeY2uOgVGGTK7HisdiuHmg5x6bhLgy1Kv/GJ
C0iSpHVAQy9LzgNel2R+ABKEEP1qnG0ymRg2bNigu/nbHXaazc3E6+L71RJWCMGwYcN8ZtnMGTGH
gsYCmkxNPrmer3AXA4DQmwr26p5XsTvsXD/1+mCLwozUGYDci2iw4a75H8BZhg9Js1fCT1+D1PGg
1sJpD8HSR2Hf+/DZHTBIU1ud7KjZgdluZtbwWcEWpd8EKgaQBnRuWFLe8Vo3hBDXCyG2CiG21tbW
ur3YYLv5AzSZm3BIDpIik/p9DV/+XCeMlA2s1UdW++yavsBdHQDICiBU6gAMVgPvFr7LObnnDIoG
YKOiR5EamTooFYDFTcyHtlrObFrBRvUcyF3S9diC22DhHbDt37Dq4UCJ2S82VG5Ao9K4/tZCkUEX
BJYk6SVJkmZLkjQ7JSUl2OK4Ra1WM336dNfXE088QYOpAQ0aHnngEfLy8pg5cyYnnngiX3zxBQBt
bW3ceOON5ObmMnPmTGbNmsXLL7/sNxmnpUxjQtIEXtr90qDpFilJElY3dQAgB4JDxQW0vWY7FoeF
M7LPCLYogPzgMD11OjtrdgZblG5Y7VJ3C2DD39A5TDyjvtr9Saf+HmYvh+//Drve8r+Q/eT7iu+Z
mTqTaG10sEXpN4FSABVA50el9I7XQpLIyEh27tzp+rrlzluw2C288OcXOHr0KHv37mX79u18+OGH
tLa2AnDdddeRmJjIwYMH2b59O19++SUNDb15zQaGEIJbZtxCRVsFHxV95Ld9vMFq794WwIleGzoK
INBdPz1hRuoMjrYfpbKtMtiidKFb+29DA2z7N7sTT6PQ3oMXWAg48y+QMQ8+vwuaBt+Ao+r2agob
C1mQtiDYogyIQM0v+xi4RQixEjn42yxJ0tEA7e13GkwNWE1WXn/tdUpKSoiIiABg+PDhXHLJJRw6
dIjNmzfz5ptvolLJN7+UlBTuuecev8q1KG0RU5On8tgPj7G6bLVrXqlapeaivItcfYMChbMxmLug
YGQIzQXecjSwXT89YXHGYv5v6//x3/3/5Z65/v1ceYPF7kCn6RTM3fIKWNvZPPJnmBt6yfpSqeHC
F+H5hfD2NXDNJxAR43+BPcRZ8LlgVGgrAJ9YAEKIFcAmYJwQolwIsVwIcYMQ4oaOJZ8DxUAR8DJw
ky/2DRZGo9Hl/pk2fRrvvPUOjRWNZGZmEhcX1239vn37mDZtmuvmHyiEEDy95Gl+PunnlDSXsKps
FavKVvHBwQ+48vMr+aw4sHnX1h76wkCHCygEYgBtljb2N+wPaNdPT8iIzeDc3HN5u+Btqturgy2O
CzkLqOP9tlngxxcg73Sa48b2rfATs2UlcHQXrLwCrIMn9fuT4k9Ii0kL6RRQ8JEFIEnS5X0cl4Cb
fbFXZx75ZB/7K1t8es2Jo+J46JzeR/o5XUAg94GpMdRgOuL5h/MPf/gD77zzDjU1NVRW+tdkT45M
5jezftOlUKnOWMdda+/i/g33kxOf42on4G+OtQVwHwRuMgyOWEVvbK+Ru36eMGLwBf6un3o9nxz6
hOd2PsejCx4NtjjAcVlApRvAUA+zrkFfocbmkLDZHWjcWIQuxp8F5z0HH94A7y2Hi/8D6uAOXj/Y
eJAtVVu4fdbtgzIhxRsGXRA41Gi3tqPX6JkwbgJlZWW0tHRXSBMnTmTXrl04HPIN8He/+x07d+50
uzYQJEcm848l/yBJn8TDGx/G5rAFZF9nDMBdEDhUYgDbqrehVWmZlhrYrp+ekB6bzlWTruKDog94
K39wBE+7ZAHlfwbaKMhdcmwusM2D4r/pl8OyJyH/Uzk9NMiszF9JhDqCC8dcGGxRBkxwVekA6etJ
3d84JAcGm4EkfRJRUVEsX76c2267jRdffBGdTkdtbS1r1qzh4osvZvbs2TzwwAM89thjqNVqTCZT
UFv4xuniuG/ufdy59k7eLXyXy8b7v5FZT31hQFYAphBwAR1qOkR2fDYR6ohgi+KW22bcxqGmQzyx
+Ql21O5CIxLPAAAgAElEQVTgoryLmD18dtCeVK12BzF6jZzTn/+5nPapjew0FcxBtCe/yhN+Ba1H
YcPf5GtMOt+/gvdAraGWT4o/4cycM0nQJwRFBl+iWAD9oHMM4MLFF/Lkw08C8Pjjj5OSksLEiROZ
PHkyZ599tism8Morr1BfX8+YMWOYPXs2S5cu5cknnwzmj8HSrKVMTZnK/w78D4fk/zYM7qZDOYnU
qjF58jQYZIqbi8mJG1iltj9Rq9Q8edKTXDruUtYdWccvvvoF5354LgUNwRkTanZaAJU7oLVSdunQ
eS6wF0r/lN/BqJnwyW3QEpwckj9t/hN2h53lU5YHZX9fE9IWQLCw2+UPbY2hhlpDrav8XqfT8eST
T7q9scfFxfHiiy8GVM6+EEJwxfgruHf9vWyq3OT3lDZLiAeBzXYzFW0VnDX6rGCL0ivR2mjuO+E+
fjPrN3xT+g1Pb3+am769iRVnrQj4lDpXDKDgcxAqyDsdoIsF4DFqLVz4Mjx/Inz7KFzwvD9E7pG1
R9bydenX3DrjVrLisgK6t79QLIAB4PT/D7YGXN7wk6yfMEw/jDfz3/T7Xq4YQC91AIN1shVAaUsp
DskxqC2AzkRqIjk391z+eeo/abW0cvt3t2N3BFbJOpvBcfAbSJ8L0cOAfloAAMlj4IQbYNcKqAxs
4dt/D/yX9Jh0rp10bUD39SeKAugnkiRhtBlDugoQQKvWcvG4i1lfvp4jLUf6PmEA9F4H0I8nwgBT
3FwMyMPYQ4lxSeN46MSH2F23OyCKvjNWm0Si1AxHd8KY01yv98sCcLLoTohKgq8fCFi/oGZzM1ur
tnJGzhkhOfu3JxQF0E+sDiuSJA3aYKA3XDz2YtRCzcqClX7dxxkEdh8D6BgLOYjdQCXNJQhESJr/
Z+acycK0hTyz4xmOtgXOf26xO5hg2Cp/M+ZU1+v9tgAAIhPg5Pvg8Hoo+MIXYvbJ2vK12CU7p2ae
2vfiEEJRAP3EbDcDcuvlUCc1KpXTsk7jg6IPMFgNftunpwHhcGwozGBOBS1pKmFUzCgiNZHBFsVr
hBA8MO8BLHaL3xV9Z6w2B+PaNkPUMBh5rHVGxEAtvlk/h+Sx8M2DYPd//ciq0lUMjxrOpGHBzTz0
NYoC6CcWuwUgLCwAgMvHX06rpZXPSvxXHdybBRAKc4FLWkrIiQ8N/7870mLSmD9qPp8VfxaQrC8A
i91GbsuPcupmp0r4AVkAIBeD/eRxqC+S4wF+xGgzsrFyI0syl4R84dfxKAqgn5jtZlRChVqEbgC4
MzNSZzAucRwr8lf4LRDbWzM4ZwxgsLqAHJKDw82HQ1oBAJyTew7VhuqADAuSJIk8RwnRtsYu/n8Y
YAzASd5PIHUibP3XQMTsk/yGfMx2M/NHzffrPsFAUQD9QK1Wc9qJp3HewvO45JJLMBgMrtenT5/O
5MmTufjii12vV1dXc8UVVzB69GhmzZrFiSeeyAcffBDMH6EbQgiumHAFBxsPsq16m1/26K0OQN95
MHxDMby2DB4fDn/Ohh9fDPpwkBpDDSa7iey47KDKMVBOyTiFaG00nxz6xO972RwSJ4ld8jfH9f0f
sAUActfQWddC5Xa5X5CfKGyQB7+PSwxs88RAoCiAfhAZGcmHaz9k9ebV6HQ6XnjhBdfrO3fuZO/e
va7XJUni/PPP56STTqK4uJht27axcuVKysvLg/xTdGdZzjLidHF+yxTpqw4AJGLz34YXFkHNPrkn
/Mhp8MVv4c1LoK3GL3J5QrVBbrA2InpE0GTwBXqNnp9k/YSvS7+m1dLq172sdgeL1bupjRkPMV3r
D3xiAQBMvRQ0enmAjJ8oaCwgVhcb8u+9OxQF0E+sDis6tY5FixZRVFTU7bjz9dWrZSVxww03uI5l
ZWXx61//OpDiekSkJpIL8y5kddlqylt9r6B6TQPVqHhC8wrjfrhHDhbeuBHO+CNc9SGc+X9Qsg6e
nw/V+30ulyc4O2wOjxoelP19ySXjLsFoM/LxoY/9uo+1vZlZopDK5O6uk4iOrC/zQGM+kQkw6QLY
/Q5YjQO7Vg84B7+Hm/8fFAUwINSSmi+++IIpU6Z0ed1ms7le37dvHzNnzgyShN7zswk/Qy3UvLT7
JZ9fu7cgcErxe1yu+Y6Decvhmo8hPl0+IATM/SVcvwYQ8N51QWkL7LQAwkEBTE6ezNTkqazMX+nX
YLBUshaNcFCdurDbMb3Gh3UfUy8FSysc/Hrg1zoOh+TgYOPBkG/73BOh3Qrii3uhao9vrzliCiz7
U69LjEYjF518ETq1jsUnLWb58uWu16dPl1PdFi1axPLly13uISc333wzGzZsQKfTsWWL/wNx3jI8
ejiXjLuEFfkruG7KdWTGZfrs2j1WAjeWkrL+QTbZJ1Kedyt57iqrUyfA+f+EN34Kqx+D0//gM7k8
oaq9Cr1aT3xEfED39ReXjb+M+zfczw+VPzA/zT/BTW3xalqlSJqHzeh+TC0QYoAxACc5J0F0Kux5
ByaeN/DrdaKitQKDzRCW/n9QLIB+oY/U896a99ixYwfPPPMMOp1cC9B5VKTz9UmTJrF9+3bXuc89
9xzffvstPQ28Hwwsn7IcrUrL09uf9ul1e4wBfP0AAok7rTdg6K0zdd5SmHMdbHoWitf4VLa+qDZU
Mzx6eNi4AU7PPp34iHg+LPrQPxtIEhGlq9nomIRW1z1VWgiBXqP2jQJQqWHyRVD4NRibBn69ThQ2
ygFgxQIYjPTxpO5P1Cq1Rz2AlixZwv3338/zzz/PjTfeCODKDhqsJEcmc92U63h257OsK1/HSekn
+eS6ZpsDIY6LAZRuhAMfYzvpPiq/Tu67DmDpY1C8Fj64EW7aCJGJPpGtL6rbq8PC/eNEp9ZxRvYZ
fFj0IW2WNmJ0Ph63WFeIprWCtY4zWNDDwJcIrcp3rT+m/BR+fF6eOTDjSt9cE1kBCARjEsf47JqD
CcUC6CdalWf9QIQQfPjhh6xdu5acnBzmzp3LNddcw5///Gc/SzgwfjH5F4xJGMNjPzxGs7nZJ9c0
WmxEatXHnqIlCb5+EGJHoV7w6441fSgAXRRc+BK0VcN3T/hELk+oMlSFXRbI2aPPxmw3803pN76/
eNG3AKxzTHVb9wH4zgIASJsF8Rny0BgfUtRUREZsRkhWf3uCogD6wa7yXW4VQFtbm9v1I0eOZOXK
lZSUlLB582a+++47Lr30Un+LOSC0ai2Pzn+UemM9d665E6tj4OX2RqvdVfAFyG6ciq2w+LeoIqLR
ezoYPm0mzLpGHjBeWzhgufrC7rBTa6gNKwsAYFrKNDJjM/m02Lc3TQCKVmGKz6VcSnGb9gs+tgCE
gHFnwqHVYGn3zTWRYz+jYkb57HqDDUUB9AOr3eqxBRDKTEmZwsPzH+bHqh+5Z909A+4TZLQ4XPnf
AHz/d4gZAdOvALwcC3ny/fJ4wVUPDUgmT6g31WOX7GGnAIQQnDfmPDZXbSa/Id93F7YaofR7GkfJ
rsOAWAAgD5uxmeDQdz675NH2o4yMHumz6w02fKIAhBBnCCEKhBBFQoh73Rw/WQjRLITY2fH1e1/s
GwzsDjsOyRFWLWF749zcc7lr9l2sKl3FVV9cRb2xvt/XMlntrqZvVO6QLYB5N4JGDhJGar0YChOT
AvNvkQeN1B/qt0ye4KoBiA4vBQByNlCsLpZ/7vyn7y5avhVsJhpSTwTc132Ajy0AgKz5oE+Q4wA+
wGq3UmesUxRAbwgh1MBzwDJgInC5EGKim6XrJUma3vH16ED3DRZOV8hQsACcXDPpGv552j8paynj
tu9uc3VC9RaDxUaUUwH8+BLoYmD2z13HI70dDD/zGhBq2P56v+TxlCpDFRD6VcDuiNPFcc3Ea/ju
yHfsq9vnm4uW/QBAXZKc/hkwC0CthbGnQ+EX4IPBN9WGaiSksHzfnfjCApgLFEmSVCxJkgVYCfg2
GXcQMRQVAMDCtIU8segJdtXu4sHvH+xXwzij1S67gIyNsO99mHIx6I/l1Udo1ZisXjwRxo2Ecctg
5xtgs3gtj6eEUxWwO66ccCWJEYn8cfMffTMxrGwTpE7EqI4F3Bf+gR8sAJAbxBkbZQtzgBxtl+cm
KAqgd9KAzqOkyjteO575QojdQogvhBAh21Q73BVAQVUrD320F7uj+w3+tKzTuG3mbXxR8gXP7/J+
HqvR6pCDwLvfln21s67tcjxCo8Js8/IGNOvn0F4L+f5rblZtqCZCHUFCRILf9ggmMboY7p5zN7tr
d/N24dsDu5jDDuVbIHOe6+bekwUQofFS4XtC7hJAQNGqAV+qqj18LT8ngQoCbwcyJUmaCjwD9Fh9
IoS4XgixVQixdTAWS1k7hk9oVKFdQtET//d1Af/ZVEpZg/uA7/LJyzl/zPk8v+t51pev9+raJoud
SI1Kbtw1agaMmt7luL4/T4S5p0BSLmx8xm8dQ501AOFSBOaOs0efzYkjT+Tp7U+7bnz9omY/mFsg
Y96x9t+9WgA+bv8dlSSnhCoKwCN8oQAqgIxO36d3vOZCkqQWSZLaOv7/OaAVQiS7u5gkSS9JkjRb
kqTZKSkpPhDPt1gdcgbQRx99hBCC/Hw5e+Lw4cNERkYyffp0pk2bxvz58ykoKABgzZo1xMfHM336
dCZMmMAjjzwSzB+hR8obDXx7QHZ3HOlBAQgheHDeg+TG5/LYD495lRlksNqYaM+XbxLHPf2D/ETo
dXMwlRoW3Cqb/CVrvTvXQ6oMVWEZAO6MEIIHT3wQu8POH378Q/9nQnT4/8mch6UPC0CvUWP2tQUA
8uyBim1gaBjQZaraq0iMSAzbGgDwjQLYAuQJIXKEEDrgMqBLm0EhxAjR8fgkhJjbsW//00mCiNVh
RavWsmLFChYuXMiKFcemEeXm5rJz50527drFNddcwx//+EfXsUWLFrFz5062bt3K//73vy7tIYLF
8W6eN34sw/lSTxYAyFWkD81/iKPtR3l257Me72e0OFjQ8qkc/J38027HZRdQP24I0y6X00k3/M37
cz0g3KqAeyIjNoObpt/EmiNr+OrwV/27SNkPEDsSEjJ7nf8AfrIAQFYAkmPA7UKOth8N66d/8IEC
kCTJBtwCfAUcAN6WJGmfEOIGIYSzB/JPgb1CiF3AP4DLJH+NnfIzVocVi8HChg0bePXVV1m50v18
1ZaWFhITu7cpiI6OZtasWW5bSAeSVzeUsPgv32GwyM13HA6Jd7Ye4bQJw9GpVRxp7P3JfkbqDC4d
dylvHHiDvXV7PdpTZ21matO3cvA3onvrAb22n1khmgg5nbR4jU+Cf51xSA5qDDVDQgEAXDXxKqYm
T+X3G3/PgfoD3l+gYhukzwYhPLIAfB4DALlQUJ/gqkbuL4oC8BBJkj6XJGmsJEm5kiT9oeO1FyRJ
eqHj/89KkjRJkqRpkiTNkyRpoy/2DTSSJGFz2Fj1xSrOOOMMxo4dy7Bhw9i2TZ6gdejQIaZPn05u
bi5PPfUUd9xxR7dr1NfX88MPPzBpUnDj4DuPNFHeaOTNH8sAOFTbRl2bhdMnDSctMZLyhr57q982
8zaS9ck8vPHhPiuFJUliqX09WsnSJfWzM/22AABm/wIi4mHD3/t3fg80mBqwSbawuhHkV7WwttB9
fE2j0vD3U/5OnC6OW1bf4l0bEEMDNJbAKLn9eW8DgMCPFoBKLceGilYNKC5U3V4dVu+7O0I6kvnn
zX/2bQUjMD5pPPfMvcftMbtkR5IkPnrnI+6+424ALrvsMlasWMEtt9zicgEBvPXWW1x//fV8+eWX
AKxfv54ZM2agUqm49957g64Ayjue8F9YW8zP5mWxrbQRgFlZiWQkRfXqAnISq4vlvhPu4/Y1t/N+
4ftcOr7n9hYWu4PTxY80ROWQNHKa2zUDSgvUx8GcX8gKoP4QDMvt33WOwxkIDBcLYEdZI1e9uhmL
3cHO3y8lStf9FpASlcLTS57mys+u5K9b/8qjCzws23FaX2kdCsDW8wAgkC0Aq13C7pBQq3wcYB9z
Guz7AKr3wYjJXp/eZmmj1doa1kVgoLSC8Aqbw0ZzYzMb1m7guuuuIzs7m7/85S+8/fbb3YJm5557
LuvWrXN9v2jRInbs2MG2bdu6TAcLFuWNRsakxlDXZuatLUfYXtZIYpSWnORoMhIj+3QBOTk181Rm
pM7gpT0v9VogZm6uZ64qnyOpS3pcM+DCoBM6qorX+K5LbDhVATcZLFzz2mZUQr45f1/Ucxhu0rBJ
XD3paj4o+oDNRzd7tkFlR1xrpJzdZbU7Ovr+92wBAP6xAnJPlf891D830FDIAIIQtwB6elL3FzaH
ja8/+ZrLrryM115+zfX64sWLOXLkSJe1GzZsIDfXN0+hvsZktVPbaubqeVl8m1/DfzYdBuSnfyEE
mUlRNBmstJisxOl7r3cQQnDz9Ju57uvreK/wPa6YcIXbdY7Cr9AIB1WjTsP9878PCoNih8O8m2DD
UzDvBjkdcICEUxXw90X1tJhsvPnLE7j+9W2szq+hrs3MF3ur+M/P53S7Ud847Ua+OfwNj/7wKO+e
8y56jb73DSp2wLAx8phGZCXT09M/gN41GN5BlG5gP1s34kZC6iTZDbTgNq9PrzPVAXJr9HBGsQC8
wOaw8fn7n3PB+Rd0ef2iiy7iiSeecMUApk2bxv33388rr7wSJEl7p6JJ9u+nJ0Vy9YlZFNe2U1zb
zswsOWidkRQF9JwKejxzR8xl9vDZvLj7xR59xtqDn1MlJWJMnuL2OMgWgN0hYbMPQAksvB2iU+Cr
B3xSF1DdXo1WpSUxIjBzB/zJxkN1xERomJudxKK8ZL7ZX8Xjn+5nXWGtW5dfpCaSB098kNKWUs9G
hFZud/n/ocMC6CEADHLlN/jJAgAYcyqUbgJTi9enOnteDYsc5mupBhWKAvACm2TjXx/+izOXndnl
9VtvvZUvvvgCo9HoSgPdsmULJ5xwAgAnn3wyn37qh5a7/aS8sUMBJEZx5pSRJEXLj1+zMjsUQKJ3
CkAIwb1z76XZ3Mxft/61+wKLAX3ZGlbZZ6LX9WxROF0CpoFYAfo4WPIglG2EzS/3/zodVBmqwqYI
bFNxPXOyE9GoVSwZn0pdm8XVe2nL4Ua358wfNZ9zc8/lX3v/RWlLac8Xb6mE1qNdrC5zXxaA9pgF
4BfGLQOHtV9uIJcC0CsKQKEDm8OGSqhQidD+tTkDwOmJkei1aq6al0VshIap6bLpnumyAPrOBHIy
Lmkc10661r3PuPBL1DYDnznmHWsG54YI56DwgTYIm3k1jFkK3zwItQUDulR1e3VY+P+rW0wU17Yz
P1d2aZwyPpUIjYpbThlDfKSWLSU9F03dPut2NCoNL+x6occ1VHT4/9OOWQDtFjsxET17mV3vt78s
gIwTIGpYv7qD1pvq0ag0xOni/CDY4CG072QBxuawoVapQ/5psLzRiFYtSI2Vfbq3nprHmrtPdrVq
jo/SEqvXeJQJ1Jkbpt1ARmwGj2x6BJPNdOzAnncxRw7nR8eEY+2g3aD3hQUA8nCQ856T5wV8dufA
UgEN4ZEKuOmQ/ER7Yq78RJscE8GGe5Zw+9KxzM5KZEtpzwogOTKZy8dfzucln1PcXOx+UeV2UGlg
xDEXX7vZRlSEB++3vywAlRrGLpNnBXvZLLDB1ECSPink/9b7QlEAXmBz2MKiB1B5o5FRCZGu1Du1
SjAspuvg7sykKI8zgZzoNXp+f+LvKWstO+YzNjbCwa85mnEmDlRdJ4Idh88sAJADwqfcD4fXyzMD
+oFDcsjD4EM8BdRqd/De9nLi9BomjDz2RJsSG4EQgtnZSRTXtlPf1nMW17WTryVCHcHLu3twq1Vs
h9QJoD3WNqHdbCPaTZqpE5++3z0x/iwwN0PpBq9OqzfWh737B0JUAQSriNjmsKEV/usCGqif60iD
weXn74mMRM9qAY5n3sh5Lp9xYWMh7P8YHFZKRy4D6DoR7DgiNM60QB89Ec66FpLHynOHraY+lx9P
g6kBm8MW0grA7pC4/a2drD9Yx12nj3Obbz8nW479bC11HwcASNIncWHehXx5+EvqjHVdD0qSXAPQ
KQAM0G6x9eoC8pnF1xu5p4AmEgq+8Oq0elN92AeAIQQVgF6vp76+PihKwCbJLiB/IEkS9fX16PV9
pNr5gPJGI+mJvTe4yhwWRXmjEYebttB9cffsu4nVxfLIxkew7/8IErOpih4P0IcLSD7msyEhai2c
8QQ0HILVj3l9erUh9GsAVm4p49PdR7nnjPFcfWK22zVT0uPRqVVs70UBAFw27jJsDhvvFr7b9UBD
MZiauvj/AdrNdqI8iQH40wLQRkLOIq+7gw4VCyDk/Bnp6emUl5cT6FbRkiRxtP0oLboWmnVelMd7
gV6vJz093S/XdtJstFLXZiYtoXcFkJEYicXmoKbVzIh475RSgj6B3879Lfetv49XG1u5fsKVGDv8
vFGBtABArgidvRw2PScPCxm92ONTnUVgoRoDaDJY+L+vCpg3OokbFo/ucV2ERs3YETHsq+w9XTI7
PpsFoxbwTsE7LJ+y/NhMDFcFcNe6i3azjRhPYgD+tABA/gwc/FpWVEk9/x6cSJIkxwAik/wr1yAg
5BSAVqslJycn4PtWtVdx6buX8tCJD/HTsd07WYYKz64+iBByFkhvuGoBGg1eKwCAs3LOYt3eN3mG
PeQMG4mhTX7K680COJYX7uMbwk8el1tFf3gj3Pg9RHqW0x/qbSCe/vYgzUYrD587qc9g5qSR8Xy9
vwpJknpde8WEK7j525v59NCnXJDXUQ9TsU12s6RM6LJ2UMQAQFYAIDeHm9u3Ami1tmJ1WIeEBRBy
LqBg4fR7hnJlYFFNG//6/jCXzs5gclp8r2u9LQY7HiEEj5kjmGax89sDr7Kz8XNAcj3luyPCVRnq
4xuCLgoufAnaquGzuzw+rdpQjUalIUkfek+CJqudd7eWc970NMaP6DuVcXJaHI0GK0ebe4+VLEpb
xISkCby4+8VjDQDLt8LIqaA+drN3OCTaLX24gAJlASSNhsRsj7uDDpUiMFAUgMeEgwL453dFRGrV
3H36uD7XpiVEIkTvcwF6xWYhomgVzyUvZN7IeWxqeZno4at6fbrUa/3gAnKSNgsW3wt734Xd73h0
ijMDKBTrPr7ZX02r2cbFszxzKU4cJT8Q7K3o3b0phOCWGbdQ0VbBx0Ufg9Uou4Ay53VZ5yww680F
FDALQAi5N1DJOo+SAYZKERgoCsBjQl0BWO0OVh2o5ieTRnRL+XSHXqtmeKzeq2KwLpRuAHML8RMv
4LlTnyNTezKqpG/5qOijHk9x3hB8bgE4WXg7pM+VawOajvS5PJQHwby/vZxR8XrmjfbsJjZhZCxC
0GccAGQrYEryFF7b+xqOim1ytW3miV3WtJvlORPuuo068avCP54J54C1HfL7rsivNykWgMJxOBVA
qD4VbClpoMVk4yeTPL+hZSRF9tsFRP5nciHW6JNRCRV56mtQm8fy0MaHeH3f626zuCL8fUNQa+DC
F0Gyw2fdZzUcT1V7/0dBmm12Xlx7iO8Kavp1/kCobTWz7mAd589IQ+Vhm+UonYbclL4DwSBbAT+b
8DPKWsvYWPCe/GLGCV3WtHUogN7SQHVqFUIEwAIAyFksu4G2/bvPpU4LIBRdf96iKAAPqTPWkRCR
gFbtvzoAf/L1/moiNCoW5XluwWT0oxgMAIcD8j+Xm3F1FAZZbIKktus5OeNk/rL1L1zx2RV8VPRR
F0UQEJdA0mhYfI+cFdKLT1iSJLkKOMr7DKDyRgPnPfs9T3yRzz3v7nb1xW8z23jq6wJaTb0Pzxko
Ww43YHdInD7JO9knjYpjX6VnGW5Ls5aSHJnMiupNkDJeHsbeCYNFfg+je1EAQggiNCr/xwAAVCqY
eY1cGFh3sNelDaYGVELlUQPA/2w8THFtm6+kDDiKAvCQemN9yLp/JEnim/3VLMpL7tUkP56MxCiq
Wkze92o5ugNaK2HcWa6XjBY70dpo/nby33hw3oMYbUYe+P4B7lhzB20W+Q8oYC6BE34FCVnw9QPg
cP+zNZobsTqsHlsAVruDvRXNOBwSd72zi/JGIzefkktNq5mPd1UCsHJzGf9YXcTKzX27nwZCYXUr
QsDY4bFenTd5VDxHm000tPfdNkGr1nJx3k9Z72jjyKip3Y47LYDoXrK+QFb6AbEAAGb8TG5Xsf0/
vS6rN9WTEJHQZ81PZZORhz7ex4rNZb6UMqAoCsBDao21IesTLKppo6LJyGkTvHNnZCZFIUnHuod6
TP7nINQw9nTXS0arnUid3EfpknGX8MF5H3D37Lv57sh3XPH5FRQ3FwfOJaCJgKWPQs1++P5pt0tc
NQAeWgD/+r6Es5/ZwDnPbuCH4gZ+d9YE7vrJOMYNj+XldcU4HBIrt8g3/hVbyvxayHiwuo2MxKhe
U27dMWmUnC3kqRXw06SpqIGVEd0VtjMG0JsFALLS91svoOOJSYW802HPuz0qfugoAvPgb33LYbl/
krs4WWWT0aUEvSW/qoUdZY39KsL0FkUBeEidsS5kLQDnuMe5Od75NLOTowE4XNfu3YZF33R0Yjy2
n9Fi79IHSAjB1ZOu5uWfvEyzuZnLP72cVWWrBjYX2BsmngcTz4fv/gCVO7sddtUAeGgBfL2vmmHR
Og5Wt7FgzDAum5OBEILrTxpNQXUrN76xjaKaNk7Ikfvu9NZ2YaAUVrcydniM1+dN7FAAeys865+f
WlvEae0GPmjah8Ha1VXY7oELCDosAH91A3XHlIvkttWlPY8lrzd5VgW8taOFtrtMuYtf2MRtK3b0
S8RX1pew/D9bCUQfOkUBeIAkSbILSO87BfDetnJeWd9DZ0Ufs6302LhHbxjdsb7EGwXQVgNHd8n+
/04YrXa3jeDmjJjDW2e/xZiEMdyx5g50w9b4LwuoM0LA2X+Th8e8fZXcz74TzjYQnlQBN7Zb2F7W
yMKGC/8AACAASURBVJXzsthwzym8es2x6VoXzEjjsjkZfLWvmmidmmevmElMhIY3f/SP28Bic1BS
106el+4fgIQoHemJkR5bAFRs5XKzoNXazmclXVsuH7MAerdCAmoBgNwdVBsNe3pOBa431nsUAD5m
ARi6WHQ1rSYqmox8m1/DnnLvuwZsK21kZmZiQDqR+kQBCCHOEEIUCCGKhBD3ujkuhBD/6Di+Wwgx
0911Bivt1nZMdpPPLIDXNx3mznd28fhnB9hyuAGr3UFxbZurT7+v2VbW6Br36A2J0ToSorQUe6MA
Dq2W/3VWX3bgdAG5Y0T0CP51xr84M+dMpMTPKTau90rOfhOVBJevAEMjvH6+3Lm0g2pDNRrhWRHY
2sJaHBIsGZ9Kapy+S8M7lUrwxwumcMspY7h32XhSYiO4dE4GH+2s8PxG6wWH69uxOaR+WQAgu4H2
e5AJBED5NmakTGd80nj+vfffXWZCe+oCCrgFoIuSO4Tu/6jHFtENpoY+XUDNBisF1a0Mi9bRarbR
bDwW2D9wtNX1/2dWdw04m232Xh9w6trMlNS1Mzs7MBPoBqwAhBBq4DlgGTARuFwIMfG4ZcuAvI6v
64HnB7pvIHGlgHoYAzDb7BxtNnb7OtJg4Hcf7OH3H+3jtAmpjIzXc//7e1j29HqW/HUtC//8HS+s
PQTIQUVf+Ikb2i1dxj16S/awaO9cQEWr5KfqEV0Dg0aLvddOoDq1jscXPI7GMoZd5pf4tDhAE9RG
zZCVQMMh+PQO1+yAqvYqUqJS+iwCq2kx8dW+KpJjdEztobpapRLcdfo4rupoxnbrkjwSonQ8/PG+
bp+R/lg/FpsDe4e/uLBavvnkpXpvAYAcCC6ua+/bf21qgdp8RMYc7ph1B2WtZby460XX4Xaz/HP0
1vsJgmABAEy5WG5e52ZSmMFqwGgz9ukC2lbWgCTBOdNGAV3jAAeOygr02vnZfL2/msc+3U95o4Ev
9x5lwZ9Wc/1/t/V83Q7X4Ox+/r16iy96Ac0FiiRJKgYQQqwEzgP2d1pzHvC6JN/RfhBCJAghRkqS
dNQH+/sdb4rAjjQYuPzlH3oMnAoB1y3M4e4zxvH1vmp+vWIHI+P1/OGCyXy1r5q/rypkZLyeBz/c
y31nTuDyuZkDkn1HmfyBco579JbRydFsKq73bLHDIVsAY5bKaXedMFrsvU4DAzmzJKntl5iSXuO+
9fdR3lrODdNu6JfcXpGzCE6+F1Y/Lo8RnHoJFW0VpMWk9XiK3SHx23d38972cgB+Oivd45z7+Cgt
d58+jvve38OJT6zuciw1NoI3fzmPMamePcFbbA7OeWYDk9LieOqS6RRWt6ESeHz+8UxKk+MAB462
MCe7F+uncgcgQdpsThx1oqsF+LKcZeQl5tFusaHXqtD0MhISZAvAYOlfsLTf5J4CkUmyG2jcsi6H
PC0C23q4EY1KcM60Ufx742HKGgxMSZcfAA4cbWFkvJ57l43H7pB4dUMJr24oASBWr2FdYS35VS1u
W3RsK21Ep1b12arFV/hCAaQBnfPayoETPFiTBoSGAjB5pgCKa9u4+rXNtBitPHreJLfzUCeOinON
Xjx76kjiI7VMy0ggPlLL4rEpnPbUWm5bKQclv95XNWAFsK1U/qA69/SWnORo3t9RIQdx+8oqOboT
DPXd3D+SJPUYAzieSE0sGdKdDB/9Ic/vep6FaQuZnDy5X7J7xYLb4eA3cq+gzBMpby1nQdoCt0tN
Vjv3f7CH97dXcO38bCaMjGXJeO8yrC6bk0FKTAR1nYaw2BwSf191kCte/oH//GJul+Etx/Pl3qPE
RWrZXd5MQXUrJfXtPHT2JA5Wt5KZFNWrtdUbkzq1hOhVAVRslf/taAF91+y7WF++noc3PczrZ7xO
m7n3WQBO9FoVDe0BtgDUWph0PuxaCeY2iDimLD1tA7GnoplxI2JdrrYjjQbe317O5LR4DhxtYcLI
OPRaNY+dP5kLZqZRWNVKVISGeTlJLHryO/67qZSTx6WiUQtOGXesMePWww1MSY/v9/vnLYOuG6gQ
4npkNxGZmQO7+fkK54fCnQKoaTGxYvMRmo1W3txcil6r5n/XneDRDVcIwUljU1zfpydGcd+yCby/
o4KUGB0/ljTgcEioVIKqZhObDzdwztSRHvvyHQ6JL/dWMSMzweuUQCeuTKD69l5vSEBHYZWQn7A6
0WKy4ZAgLtKDG4JGjcUqeGDeA2yt2srDGx9mxdkrjrUe9hdqDVzwIrywEOMH11OrqiUjNgOQg+BF
NW0snTicgqpWbn5zO0U1bdyxdCy3nprXr+2EEJw2sbvSmJuTxJWv/Mi5z27gnjPGc92i7t0rq5pN
3PLmDmwOCa1aMG54LAXVrbyyoZjvCmo4a8qofskEsgWSFK2joKq194Xl2yAp15XplahP5O45d3P/
hvt5q+AtDOYJHtWcBDwG4GTKxbD1NXlQzNSLXS87LYDeWkFLksTu8mbOnDKCWL2WxCgt6w/W8n1R
PWNSYyipa2dpp/d2ZmYiMztZ4OdMG8Wbm8t448cyIjQqvr1zMemJUZisdvZWtPDzBdm+/3l7wBdB
4Aogo9P36R2vebsGAEmSXpIkabYkSbNTUlLcLQk4dcY6NEJDfER3s+wfqw/yt1WF/GtjCQvHJPPV
b07q99M2wDXzs/no5gUsmzySVpONwppWKpqMXPziRm5dscPlI/SEDUV1FNe1c+UJWf2WJ8ebTKCi
VTBqOkR3VZTOdhJ9TSEDuR2E2eYgVhfL/SfcT0FjAf/d/1/vBe8PSTmw7M9UVspD7dNj5EZq97y7
m1++vpVHP9nPZS9tosVo5T+/mNvvm39vjB0ey1e/OYnFY1N5/LMDbHYzrP3NzWXYJYnL5mSQGqvn
xatmkZcawzOriwC4fWn/5RJCMDo5uvfAvyTBkR8hY26Xl88efTbzR83nqW1PUWU+2GcAGOT3O+Ax
AICMeRCX3i0bqMEk/757swCONBhpNlqZkib/nWcmRfF9UT1CyDU3dofU68PSdYtyGBmn56aTcxEC
/vDZAUC2Kix2B7MC5P8H3yiALUCeECLn/9s77/CoyuyPf97JpE4aaaRXSggQeqSjgBSl22AV7IiU
VX8q4uq6sq5lbYuu2BtYQRFQQKWugPROgEAoCSQhIT0hvdzfH+8khGQmdTKTcj/PkyeTm3vnPblz
c89933PO9wghbIDpwM/V9vkZmKXPBhoIZDfb+n9pEfz+HFwwXSZJWkEabvZuNQKCBcVlrD2cxNQ+
flx49VY+vXcAHZ1N09GrImd/+5lUZn66l6y8EnQ2VpXFRFVRFIUF3x3mvi/2XZdJtHx3PB6ONozv
2fiGJvV2AAVZkLC/xvIPVHEAbnU7ALsqT4SjgkYxKnAU7x95n0s5zVs9W0nvu0kIHgyAf3ER51Kv
si8uA29nOz7/8wK2Wit+mDOIEV2a7+HETWfDuzN64+tixz9+PkFZuUJ+cSm3f7CLp384ynf7LnJj
F09euy2SPxeNJNhDx7S+0lk9OqIT/vVwtLUR6qnjfGotn3f6OchPq6EAKoTg1WGv4mHvwWnlHWzt
6s5ykjMACzgAjQZ6TJOB4PxrTrY+S0DHErMAiNSv+fvrr+sJkb6VM/raHEC4tzO7nh3FwnHhzLux
E79GJ3MwPqOyrqBVOQBFUUqB+cDvwClgpaIoJ4QQc4QQFRG8DcB54CzwCTC3qeMapbxUTutWz5E3
JROQVpCGp738YBVF4b2tsUxZ+idLt50lt6iU6QMC6niHhuPfwZ6Ozra8ufEM59Py+GhmPyb19mPd
sSRyCktIv1rEfV/sY9GqY3y99yK/HE1iZ2wa45bsYNPJFHbGprElJoXpAwIrNXYag85Wi5eTLefq
0ju58IcUWTPkADLr7wCqPxE+G/Us1hprnv/zeUrKmldDBwAhSOg+EQD/zS/xy65jaDWC1fMG84+J
Eax8ZBBB7g2rp2gMDjZa/nZrN05dzuGN30/z5u9nOBCfyU+HE0nNLWLW4ODr9r9nYCB/uyWcR2rp
/FVfQjwcSbtaRI4xzaKLu+X3agqgIAXUlo5aShnFJNouJa+k9gcHO2uN+aQgqtPzDnm/OLmmclN6
QTrONs61an4dT8jGxkpTKbURqL+u7x0UxOu3RfLPyd0ra2jq4sFhIdhqNfxy9DIH4zMI9dTVS63X
VJgkBqAoygbkTb7qtg+rvFaAeaYYq05sdDDtE/jsZtjwFNz2aZPfMr0gHS8HLxRF4ekfj/HjwQRs
tBqOXMoi1EPX4Arb+iCEoH+wG+uPXWbmwCAGd/JAZ6vlu30XmffNIeLS80jJLqK4rBz2XyIqxI23
7ujFvG8PMfurA1gJQRcvJ5OsJ/YOcGXv+Yzau0XFbgJbF/DrX+NXlzIKcLG3xsW+7nX86mvCHXUd
eX7g8yzasYiX9rzE4sGLm71A5lJxBg5WtrhmXWJs2jwudPkPPi723D/EvJ3obu3pw47+aZWpwfcM
DOTO/gHsu5DBiM7Xz0Cc7KyZPTzMJOOGeupnfal59AowsJx5cY/MovHoYvD4MNcwXHIeJMvlAx7d
/CiPRD7CIN9BBlNqLTYDAPDuCR5dpTRE/weA2pvBbzyRzPm0PA5dzCTcxwkbfQMjuRRnW1lrY6z3
siEcbLQM6+zBppMp5BeXXhc7MAdtsxLYvx+MWCjX985ta/LbVchAnEjK4ceDCcweHsqW/xvB0E4e
PDa6c7PdkKb29mNQqDvPjJcN1SP9XRjbvSNnUnLR2WhZ8chA3vtLH3oHuPL6bZEEuDmwYvYgpvbx
Y2x3b358dJBJniZGdPUkMauAc8aWBcrL4Mxv0GnkdV2hKriYkU+AW+09iCswlBd+a+itPBL5CKvP
rmbN2TVGjjQdCbkJ+DsH8VvE63RS4vln/suy8YmZEULw2m09WTQ+nMFh7jwzLpxIf1ceGhZa75TT
xlDx9Ho+zcis7+JuufxTy3VfmteZSJvZxGXHMWfzHBZsXUBOcc0CMztrDcVl1+oYzIoQ0PN2iP+z
sj+EoWbw5eUKT648yuyvDvLarzHsj8ukZ5U0zSB3HfcPCWn0fWBMhDeJWQVk5pfQP8i8EtRt0wEA
DHm8TsXH+lBWXlb5VLDqkHzyn3djJwLcHPj6oRuY3Nt4rnhTGR3Rke9mD6xMpxNC8NHM/uz922h+
e3w4fQI7MCHSlzXzhlRm69jbWPH2nb1ZendfnOxMkzkzXP+0uf1MquEdEg5AXiqETzD460uZ+fUK
AINxdci5vefSr2M/3jzwZmVdRnORkJuAl50vTx/pyOdei3C5sh9W3gvmWIKqhhCCOSPC+PbhgSb7
POsi0N0BjZAzgBpcTZVFc9XW/2vsVlRKZ92NbL5jM88MeIZdibuYvm46ZzLPXLdfxfJksaVmAZF3
AgIOfw3IIHD16u+DFzNZdSiBB4eG8Mms/nTzceaWnj4mM2FkN69KX9rPTBXAFbRdB2BtBzcvhpRo
ONz4LJLMokzKlXI62Lrz85Ekbu7WEReH1tkToLEEuDkQ6qnjD2MO4PR60FhD55tr/Kq8XCEho6By
nbQujInBaYSGFwa9QEFpAa/sfaVZ1DRf/PkE93+5j4SrCVy8Yk9JmcK46fNgwtsQ+7uMKzXhYaK1
YKu1IsDNgXOGAv8X9SJqAcYdgKIo5OnrAGysbLgn4h6+GPcFhaWF3LPhHrYnbK/ct0IC3Cz6T4bo
EAxhI+U9oqy08mHvjzOpDHltKwfjM9kRm4ZGwF9HdebmiI78+tgwhnQynS6Yh6Mt/QI74KazqXfs
wFS0XQcAUu0xaChs/DtkNU58q0IWODXTjvS8Yqb1bb4n/pbMiC6e7DmfbvgfNWa9rKa1q5kmeyVX
xin86+sArOWasKEbfKhLKAv6LGBT/CY+PvZxg/+Guth4Ipn/nT1HUVkRZxJteHh4iAz49n8ARr8o
+wlveKpSLqItE+JhJBPowg4ppubbx+ixRaXllCvgUEUIrrdXb1ZOXEmwczBP/fEUMRkxAJXV4fmW
cgAA/e6DnESKz/xGbnEu+fn2zF5+gMSsAr7aHcefZ9OI9HetVwyrsbw8tSfv/aWPWQTgqtK2HYAQ
MOV9+Q/byKe35HwpC7zvbDkejjbXFW61J4Z39qSotLxmHUJaLKSfha63GDyuIgOoITMAMN4U5r7u
9zExdCLvHXmP3+J+q6f1dZN+tYik7EKsbGULR1etL/Nu6nRth6FPyGXFA5/D1pdMNm5LJdTDkbi0
vJqa9HE75PKP1sboscbaQXrYe7B01FKcbZyZv2U+qfmp2OuLxQrMLQdRla7jwbEjGfukRNnecyV4
OdsyIdKH304kc+RSFkNN+MRv0ARvJwaHmV9uvm07AIAOQXDL6zLQs+vdBh9eMQPYG1vG9AGBWNeh
bdJW6R/cAY2AvdV1gWI3ye9Vmr9U5WJ6RRFY/YLAdTkAIQQvDn6Rvl59eX7n8xxLPVav962Lil64
N0XKdf5Fo0bWrGQd/aJsK7jjLThtOufTEgnz0lFQUkZiVpXg99UrkBojZ3u1kJUvVTYNPTF7Oniy
dNRScopzWLB1AdZW8nxXiMdZBCtrGPxX0hNlAWByhpYRXTx5YGgIhSUyQD20Aa1UWxPt427WawZ0
mwRbX5Za9Q0gJT8FDVpEuY6/3NAypCksgZOdNT38XNhTrTK1MGYjidoAVp3TsPtcOlOW/snIt/5X
+fXKhlMIAX71dAAVGii15YbbWNmw5KYleNp7smDrApKuJhndt75E66WZHZ1S8Nb5cFvv8Jo7CQG3
vAEde8DaefKG2EapyHI5nlilmCtOX1wZPLzWY+PSap/1dXXryuvDX+dk+km+j3sdKCfPkjMAgKiH
SXOREhqF+baEezvTJ8CVEA8d9tZW9AlsfHV/S6Z9OAAhYOI7Urdkw8I6d88tLOGhZfs5npBNQu5l
ykucubmbD76u9buJtVVuCHHjyKWsa3GAkgI0F//kt8IePPnDUWZ8soeMvGIifJwrvwaFufPUmK71
LkarawZQQQe7DiwdvZSSshLmb51f2Ve4sZxIzCHQzYGz2TFEuHUzvqPWVtaZFOXCJyMhfneTxm2p
dPV2wtpKcKxqQ5MLO8DWGXx61XpsXLqMHQTXUjB3Y8CNPNn/SQ6lb8fGcyP5Zp4BlJSV8+LPJ1ip
r6xfuuMSO9xvBOBWJZpuPk5ytjmpOy9OimhSMWVLpsWJwTUbDm4w7En4daH8pw2qWcVYwZojSWw+
dYXL2YUUesRRWuLM7DFNr7Bs7USFuPPJjgscvZTFDaHunNy9gQilmA6R43jBN4K0q0XMH9mpQY3n
q1MxA6hPVkioSyhv3fgWczfP5Z4N97DkpiUEuwQ3aLyMvGI0Qs4Awn2t2ZMTz62ht9Z+UMcIuH8D
rHoIlk+Ch7aAT83G6K0ZW60V4d7OHE+sUk1/YTsEDTZY61GVuPQ8XOyt6aAzHicAmBUxi2MpsWxk
LccyxjIaw2nEpqa0rJzHvz/C+uOXcdPZMKKrJ+9sjsXeUwsdYCHrcHJ9GaBZJT9aAu1jBlBBn3tk
BeOfS2rdbcX+izjaajmRlENyXgr+jj7Xqfm1V6KC3RAC9l7IoLCkjFM711CEDbdMuJ0HhoawcFx4
k27+UNUB1C8vfJDvID64+QPSC9OZsX4GWy9urfOY7IISCkvKyCsqZdJ7Oxn2+jbi0/Pxcs9AQSHC
vXo/IwP494eHt8rr6afZUFJYL3tbEz39XTiWkC0zsnKSIOMcSvBQruTW/rfGpeVX1qXUhhCCBb2e
orzIg9UX36Gg1DzFdu9uPcv645e5tacPGXnFLPjuMMVl5RSSjbbMDl+Rg9OB98xii6VpXw7ARgc3
zJFVqyknDe4SnZhNdGIOT4/tSlRIB4R1NiPCTK/62BpxcbCmh68LX++J59mfjtOj8BB53gOwc2hc
8xFDVPSQrbMjVRUG+gxkxYQVBDoH8ti2x/jv4f9SZiTjK7ughHFLtjPmP9tZuOoYiVkFBLnLtWp7
R5nx1a22JaCqOLjBlKWQegrWPSEb4rQhIv1cyC0sJT49v1Jc8e3YjkS9vIXfTyQbPe5CWh4h7vXL
+nK1d6AweSpZJcl8ePTDug9oIqcu5/D+trNM7ePHuzP64Odqz74LGYR7O2FlfZWiEhf26EbB7qWV
1cFtmfblAACiHgZrB/jzHYO//mp3PLZaDVN6+/H2jE4IUUZIh8brq7c13rgjEq1GsOPwSbpqEnDr
XlP8rSk42crMkYY4AABfR1+Wj1/O1E5T+fjYx8zbMo/soppqlK9uOEVKTiF5RaWsP3aZewcFs3be
UDb8dRj5Ih4Pew88HRow7e80Gm58Fo5+K6vO21CNQEWHq5fWneSXtSvIRsd7p+zxcLTl+TXRLN8d
x6T3dlaqvYJcukvKLqi3YJ6DjZay/DC66kax7MSyyvqA5uIfa0/g6mDNCxMisNII7ugvVVTvGxyM
ky4fpdSZo10fkztv+Wez2tISaH8OwMFNFn4c/6FGcdjJpBx+OHiJGVGBuDhYk1MiJQe8HRovp9zW
CPd2Zs28IbzUW18PEDLCpO9fMQPIa6ADALC1smXx4MW8MOgF9ibv5a51d7EjYUflbOC36Mt8v/8S
Dw8PZe38ITx5cxcWjuuKlUYQ7Kll28VtDPAe0HCjRzwjZ5Z7lsL2Nxp+fAulS0cpeLYl5go3iBNc
cu7HJ7Oi+PL+AWTkFfPC2hMcS8hm7ZFrrT0SMvNRlGsy4nVho9VgbSWIdLgbF1sXXtz1otHZW1Mp
LSvn0MVM7ugfUBmfuH9wCE+M7sKUPn5ora+ilDrhF9wZBs2D4ytl45s2TPtzACA/XCFg57VYgKIo
vPjzCVwdbHhitFQ5rKgB8NapDqAqXs52jNfFgo1TnRkhDcXRTsYQchvhAECuK9/R5Q6WjVtGcWkp
c7fMZeKaiXxx4A8WfHeY3gGuPDG6C/4dHFgwqnNlzGL9hfXkluQyI3xGYwaFsa9Cr7/Atpdh70d1
H9MKsLbS8K8pPfhsshdeZcn0GDKB0REd6eHnwlt39OLlqT3o6efC1phr6bAX9Cmg9YkBVGBvbUVZ
iT0LByzkRPoJfo371eR/C0BCZgGl5cp1cgsuDtY8NroztloN+eWZ9PYNkC0ahz4BOk/Y+FybmtVV
p306ABd/6He/rOrUr23+fDSJfXEZPD22a6XWT0q+dAAddeaVaG0VxO2oV0ZIQ6lcAipsWl64iyaM
hOjHCFPmUFxWylvRj+EXcIxlD0TV6LeqKArfxXxHuFs4vT17N25AjQYm/VcK4v26EI581yT7Wwp3
9g9glI0+XhZ8rQBsSh8/7r4hiJHhXhy+lEVGniz+itPrB4U0oGeCzlZLfnEZ40PG06VDFz48+iGl
5aavC6hoalQhd12VrKIsSstLmdA9XHYys3WCm56Tyqenqve3aju0TwcAUijOLRRWzyEv8wqvboih
h58zd/a/1twlOS8ZrdDWUAds9+QkSfmHkNoLghqDnbUGK43galHTlDdfWneS4hINR2KC0Vx+AiU/
lHT7b3jr0Etcvnp9M7p159cRmxnLjPAZTdNisdLCbZ/JZbG1c2Hbq1Bm4QInU3B6A7gGglfN4Pio
bl4oCvzvtJwFnEu9SgcH6wYJJjrYWJFfXIZGaJjbey7xOfGsO7/OJKYrisJL606yMzat0gEYqk9I
LZBChx4OVSp++8wErwjYvLjNigC2Wwfw/ZF0tnR/GfJSyf5sCjk5WSye1AOrKjrrSXlJdNR1NNjI
ol1z/n/yezM4ACEEjrbaJkkD/BadzOZTV3hqTBdCPXScSSpnVuhLPNTzIdacXcPYVWOZs3kO686v
Y+3Ztfxj1z/o17EfE0MnNv0PsLaDGd9B5F3wx2vwxXjIjGv6+1qKoquyp0b4BIP6/z18XfBwtGX9
scskZRWw9khSg5UyHWy0lZXAIwNG0t29O+8eepfc4joa0xuhrFzhtV9j2HU2jdWHE/ls5wW+3HWB
C2l5ONtpcTNQn5CWL+N9FZ3/AOnQRzwj5a/b6CygXd7ZSsvKeWXDKR7dCtFDluCVe4o17kvp53t9
pW98TjzBzsGWMbIlc3YzOHaUHZWaAUdbLbmNXALaGZvGX78/THdfZ2YPD+ONO3oxpbcvC0Z25bG+
j7Fh2gZmR84mNjOWZ3c8y/N/Po+PzoclNy6ptQ1gg7DRwdQP5Wwg9TR8MBSOrjDNe5ubc1uhrMio
2J9GI7j7hkC2xFzhtg92Ua4oPDPOgIxGLVTMAEA+APx94N9JL0xnycHa63VAJgtUV45deeASH/5x
jvu+3M9L6+Ty1b4LGZxLvUqIp6PBWV7FDOA6BwDQbSK4hcl4YRuMBbRLB3AwPpOcwlKKy8qZvNmV
vzOXLnkHYdWDlU0/FEUhLjuOIOcgC1vbwigvk0+EYSNr7QjVFBxttQ1aAqpoIZmVX8wjXx0g1EPH
Vw/egI1WQ7+gDiyZ3qcy2Ovv5M/8PvPZeNtGVkxYwTe3fMMPE3/A1a4ZtF563g6P7gTvHrB6tqwc
Lqy7UXqLImY92Hcw2P+3gsdHd+buGwK5nF3InBFh9er9XBUZA7jm8Lt7dOfubnez8sxKDqUcMnrc
lZxC+r60iU0nUyq3ZeeX8Mbvp+kb6EqYpyOZ+SXMGhRETmEp++MyjNYnVDQZ8rCvNnvRWMGQx+Dy
kWsz3zZEu3QAW09fwdpKMP+mTpSVK4SMegDGvwEx6+DzsZAWS1pBGvml+aoDqE7SESjIMNj83VQ4
2mnrXQeQmVdM/5c289nOC6w8cIm84jL+c1dvg9P8qlhprIhwjyDSMxIH64bdsBqEayDctx5ueh6i
f4IPhsCu/0LO5bqPtTSlRXDmV+gyvtZgvxCClyb3YOUjg1gwspPR/Yxhb2NVQwtofu/5+Op8Wbx7
McVlxQaPO3Qxk6LScnadu6ZQ++nO82TlF/PSlB6seGQgv8wfyuzhUsalpEwhxMNw0WJaQRo6cuLr
4QAAGftJREFUa53ha6HXdHD0rlNBoDXSLh3AtpgrRIW48cTNXfj24Rt4cGgo3DAb7lgmg5vv9Sdu
xV0ABNu1TRnYRnN2MyAg9KZmG8LRVlvvLKAtMVfILSrlrY2n+fLPOKKC3ejm49xstjUKjRWMeBoe
+B0cvWTB2NvdYPlkyDhvaeuMc3aLnLH0uK3OXTUaQVSIG9pGyKXrqiwBVeBg7cDzA5/nfPZ5Pov+
zOBxFUqlxxKu6RVtOH6ZgaHudPd1wdnOmp7+Lvh3cMBPL+QY7GHY2acWpNZc/qlAawuD5soZQNLh
Bv51LZsmOQAhhJsQYpMQIlb/3aBgjhAiTghxXAhxRAhxoCljNpVLGfmcSbnKTV29sNIIBod5XAv8
dp8C8/bByOeJL5aFTsE/zoYTzd+IvNVwdpPsBqVzr3vfRlLXDKCotIzv910kK7+YTSeTcdPZUFqu
kJRdyKzBLXjGFjBA6gfNPyiDi5ePwrLJMquqJXL8B3Bwh1DTFvtVp2oQuCrD/IcxPmQ8nxz7hPNZ
NR3l8UTZw+Hk5RxKy8o5n3qVc6l5jImombZ9Q6jM5As1MgNIzU+tufxTlX73g60L7PxPff6kVkNT
ZwCLgC2KonQGtuh/NsZNiqL0VhSlfxPHbBLvbInFSiMYE2GkuMvJG4Y/TXzfu7HRWOPtEgw/3Cv1
34uaJjnc6slNlg3gjQQETYWTrXEHkJlXzMxP97Hop+P838qjbD+Txq09fXh8dGfCvZ0Y270VFO15
dIKbnoWZq6EgEz4fJ89rS6LoKpz+FbpPlQ1TmpGKILChNqALByzEXmvP4t2LKVeuaS0pikJ0YjbO
dloKS8o5m3q1MhYw2oADmBjpS4iHjjAvw/UJiVcT8XWsRfLFzhn63w+nfoHsROP7tTKa6gAmA8v0
r5cBU5r4fiZDURSyC64PJP5xJpUfDyYwZ0QogXWIVcXlxBPoHITmwU0w7Ck4/A18NAwS23ZpeK2c
/hVQILx5HYCuliWgv6+N5khCFuO6e7M15goFJWXcHNGRuTd24rfHh7eujm2+fWDWGpld8tkYOPq9
pS26xukNUFoAPe9o9qF0tlrKyhWKy2qK6XnYe/BU/6c4dOUQq2JXVW5Pyi4kI6+Y2/pJLZ9jCdls
OplChI8z/h1q/m/fFO7FtqduNKhWW1RWxJX8K/g7+dduaL/7QCmHw1837A9swTT1v6WjoigV0axk
wFjJrAJsFkIcFELMru0NhRCzhRAHhBAHUlNTG2VUWbnCkyuP0v9fm9ioVy08l3qVJ1ceJcxTx4KR
dat7xuXEEeISIp9+Rv1dBvJKi+U/6uFvGmVXqydmPXQIlsUxzYijrZa84rIa/WiLSsvYFnOFO/r5
895f+hDp74KznZaBoc23HNXs+PeHOTsgeAismQunTFMA1WSO/wAuAeAf1exD2esrs401hZnSaQpR
3lH858B/SM2X94Tj+kY1E3v54mSrZfnuOA5ezGRM94ZX7SdeTURBIcApoPYd3UJk9tuh5W2mMKxO
ByCE2CyEiDbwNbnqfoqcvxlLlB2qKEpvYDwwTwhhtIJIUZSPFUXpryhKf0/PhjdjKC9XWLTqGD8d
TsTD0ZZ53x5i4Y9HmfHxHkDho5n9akgBVKe0vJSE3ITrM4CCh8iUvuBh8PN8iF5l/A3aIkW5cOEP
owVBpsRJrwdUfV14/4VM8orLGBnuhdZKwxf3DeCHOYOx0baip35D2LvC9O/kjGDlTNj6r8p0ZIuQ
lyYDwD1ukxIXzUylAKCRtpBCCF4Y9AJFZUW8uu9VQMq2W2kEET7O9PBzIToxh0h/Vx4cGtLg8RNy
EwDwd6xjBgByFpCTALEbGzxOS6TOT1dRlNGKovQw8LUWSBFC+ADovxtskqooSqL++xVgNdBsjxW5
haUcuZTFY6M689vjwxkU5sG206m46Wz45qGBdPJyqvM94nPiKVVKaxaB2XeA6d/Kp6IfH5Apfe0l
QHx2M5QVQ3gd3bJMgKOtdADV4wBbYlKw1WoYHCaDde6OtnT1rvvzbBXYOsrloF4zpKLo5+Mg/Zxl
bDm5BpQysyz/AJXLMgXFxp+qg5yDmNNrDpviN7Ht4jaOJ2bT2csRO2srJvX2ZVhnD5bfH4WTXcPj
FZUOoK4lIJDxL2c/2S+gDdBUJa+fgXuB1/Tf11bfQQihAzSKouTqX48Bmk1o28XBmjXzhuBgY4UQ
guUPNNzXHEiWAbm+Xn1r/tLGAe5ZBUe+hUPLZIA49h4Y/5oUkGqrxKyXGSEBNzT7UBWKoFcLS8Hl
2vZtMVcYFOaOvU3b7M+KrRNMeV/WWKx7HN4bAJ1GQYcQsHOBiMmyqKy5Ob4KPMOhY/fmHwsZBAbI
q8UBANzX4z42XNjAv/b+i4ykxxndNRCAGVGBzIgKbPT4CVcTsNfa425Xj6VEK2upJvz73+DSfpnZ
1Ypp6vzuNeBmIUQsMFr/M0IIXyHEBv0+HYGdQoijwD5gvaIovzVx3FrR2WqbJOq1N3kv3jpv408E
to6ybmD2/2D407IZyIfDIGaDZafuzUVpMZzZKAuCNM1/862YAVSVhI5PzyMuPZ+R4V7NPr7F6TEN
Ht0NgxdA2hk4tgJ2vAUfDoH3B8GOt5uvW1Xycbi4SxY/NfNSXwUVM4D8Oor/rDXWLB68mLSCdApd
l9PdzzSd6C7lXsLfyb/+94y+94Kda5tICW2SA1AUJV1RlFGKonTWLxVl6LcnKYpyi/71eUVReum/
uiuK8rIpDK/FJl748wW2J2xv1PHlSjkHkg8Q5R1V9wVhZQ0jn5cB4vIy+H4GvB3RcgJ5piJ+JxRl
m2X5B6osAVXJBDqZJHO+201vZhc/qVj72FFYFA9PxcItb4KNI2xZDEt6yF7EppaW2LlE9nnod79p
37cWKmIA1YvBKigpK+eXo0lcLSol0jOSyQHz0DrFEF3wrUnGT8hNqN/6fwW2jjBwLpxeD2d+N4kN
lqKVR89qklOcw6mMU8zfMp9/7/s335z6hsNXDhvMMTbE2ayzZBZlNqwzVNBgWHAQZqyQdQQr7oZv
p8uc4bYgIBWzQbbRDGu+6t+qVCwBVe0KdjolFyEgzNN0/YdbFTp32c70oU3w1yMw5HE4/qMUmovf
bZoxMi7AiZ+g/30yMG0mri0BGZ4BfLz9PAu+O8y4Jds5EJeBc/FwSjKG8HvCSlaeXtmksRVFIfFq
Yv3W/6sy9HHo2EPWB101GPpsFbQ5B+Bi68Ly8cuZGDaRr099zWv7XmPWr7OYsnYKy04sI7Mws9bj
9yfvB2h4a0CtDXQdBw9tgRGLIOkQrLgHvrm9VV8glBTIoGDYSLC2r3t/E2BoCSg25SqBbg5td/2/
IbiFyNnBgxtlls6Xt8DWl5vWe6A4H1bPAY21fLo1I5VLQAZmAJezC3hv61miQtzQCMFDyw/w57k0
gsR0hvkN45W9r7D+/PpGj51emE5BaUHDZgAg5SGmfQyFOfDJKLi4p9E2WJI25wAA7LX2vDz0ZXbN
2MW2O7exePBiHK0defPAm4z6YRRP//E0ey7vua6yEKCsvIx159bh7+iPn6Nf4wbX2sgqzydOSoG5
uJ1y3ba1ThWPfAt5qbLnrZkw1BXsTEouneuRwdWu8O8Pc3ZC5HTY/jp8MU6mcDaU8jJYOQsS9sG0
j8C5lorYZkBnxAGkXy3iyZVHKVcU3rqjFx/P6kduYSmHL2bR068Db4x4g16evVi0YxGv73+dkvKG
x98qMoDqrAEwRMfucvlXo4Evb4XYTQ1/DwvTJh1ABU42TnjYezCt8zS+ufUbVk1axZ1d72RX0i4e
3vgwM3+dSXJecuX+K06vIDo9mrm9TfAEZKW9Fih28oZv75QtKFsTZaWw613w6wfBQ802bMWacEUa
aHFpORfS8ujSsZ0u/9SGrRNM/QBu/xySo+HraQ2PC+z6r9R4uuVNKf1gZipmdVWDwGdSchm7ZDsH
4jL55+TuBLg5EO7tzMyBsjYn0t8FnbWOT8d+yt3d7uark1/x8MaHySjMaNDYsVmxAAS7BDfO+IAB
MPsPWRy5YiZc3Nu497EQbdoBVKdLhy4silrE1ju3snjwYs5mnuWudXfxybFP+CL6C9459A6DfQcz
IXSC6Qb16iaXhTqPhXX/J9dtWwsx62Q3q6FPmC0jBEBrpcHOWlPpAOLS8ygtV+jSUZ0BGKXHbXDn
ckg5AV9OgLTY+h2XdEQWnkVMhv4PNK+NRrDRarC2EpVpoKVl5Ty58iiKAj8vGMJdA66leD5xcxdm
RAUyTq/5ZK2xZlHUIl4Z+grRadEs2LqAwtLCeo8dnRaNq61rw5eAqmLvCvf8JB/0fnwACrLqPqaF
0K4cQAW2VrZM6zyN7279jk6unXj38Lu8ffBtwt3CeXHQi03rC2sIazu4c5kMFq9+RKZUtgYOfA4u
gc0u/mYIR1vrSgdwJkW2BuyszgBqp8sYWVGcnQAfDYeDX9aehJBxAb69C3SeMGGJWZ18dRxstBTo
g8Cf7bzA8cRsFk/uTrj39dLeLvbWvDqtJ17Odtdtnxg2kdeGvcax1GM8t/M5oz0EqnM87Tg9PHo0
/X/e0VN2gMu9DOufbDXJH+3SAVQQ6hrKZ2M/Y8O0Daybuo5l45fh4+jTPINZ28OM72XmwMqZLV9K
Iv2clH7oN8ssuf/VcbK7Jgh3JuUqmvacAdQQuoyBR3eB/wD45TGZiJBvYFkk6QgsnyTbPc78CRzc
zG9rFZzttWTkyzX8r/fGM7STB7f2bNj/4uig0TzV/yk2xm/k/t/ur9QNMkZ+ST7nss7R08NErU39
+8GNiyD6R1nN3Qpo1w6gggCnAPN0/rJzllNF70g5VVw9R2rstEQOLQNhBb3vscjwjrZasvRqrjGX
cwhy19Wp4aSix9kHZq6BMf+SyQfvD5JFSwkH4MIO2PA0fDpaxnju+UkuU1qYIDcdFzPyKS4tJzGz
gL6Bro16Kr+3+728NeItYrNieXTzo+SX5Bvd92T6ScqVcnp4mLC6ethTUs5j28tw4AvTvW8zoToA
c6Nzh/t/lQ1Bjq2AD4e2PC344nwpedt1vLyZWIBIfxf2XUgnJaeQHbFprVvx0xJoNLKS+OEtUsF1
84vw6ShYNkEuDfW8HR79E/wMyJ1YgCB3B+LT80jIzKdcgSB3w7r99WFM8BjevvFtYrNieWb7M5QZ
Ue48nnYcwLQOQKOBSe9B2Cj4bRFciTHdezcDqgOwBFZauOlv0hGUl0uJ6T/eaDkSs0e+gfx0GDTf
YiZM6+tPYUk5j39/hIKSMm7v18i03PaOTy948HdZPDZjBdz9o6wqnvqhxZd9qhLsriMrv4Sj+vaO
xlo31pehfkNZFLWI/yX8j7cPvm1wn+Npx/Fz9MPNzsTnwUoLUz4AGx389FCLlodRHYAlCRwoJaZ7
TINt/5K5xBf3WDaAVFYCf74rRd+CBlnMjL6BroR46Nh9Pp1gd4f2IwHRXLiFyELFzjebtcq3vgTp
GzRtP5Om/7nxM4AKZoTPYEb4DJafXM4PZ3647nc5xTnsSdpjWPDRFDh1lIH15OOw33BP45aA6gAs
jZ0L3PYpTPtEpvB9Phbe6QVbXoLU0+a358i3kH1Rpn5aECEE0/rIp/5pfRsg1KXSKgn2kDf8HbGp
ONpqcdfZmOR9Fw5YyFC/obyy5xX2XL5WrfvVya/ILcllVvdZJhnHIN0mQsgI+OM12fqzBdJUOWgV
UxF5p1xzj1kPx1bCzrdhx5syYBx5J3h0ubavVwS4NqJysS4y4+D35yBwsKxbsDDTowKJScllelQz
/K0qLYpANzkDSLtaTISPs8kcvlaj5Y3hbzDz15n8detfmRQ2iQj3CL46+RU3B91MuFu4ScYxiBAw
9mWpFLz9Tfm6hSHqK5JmCfr3768cONDCAqTmIjdFCnMdWwFJh2v+PmiIbNgRMdk0a7nlZXIJKuWE
lBfoYIasKBWVKgx8ZQvJOYXc0tOb9+/uZ9L3Ts5L5p1D77AxbiPF5cVoNVpWTlhJ5w51t4dtMmvn
wdEVMH8fuIU2+3BCiIOKovSv176qA2gFZMbJoCzIG/X5P+D4SqkVr7GWa7tj/iWzPRrLjrdgyz9h
6kdSC15Fxczc9dFu9l7I4NEbw3hmXPM8meeV5JFdlI2D1gFXOzPFQnIuw3/7ykY/d33V7MM1xAGo
MYDWQIdgqcfj1w8ComDE0zBvn9QgiZotHcIHQ+VTRmNIPAjbXpE6MJF3mdR0FZX6EqwP/Aa7Ny0D
qDZ01jp8HX3Nd/MHmUo95HE49TMkHDTfuPVAdQCtFSHAtzeMe0Xmc3v3gNWzYdVDDRMDSzsL39wJ
Tj5w69sWlQNQad8E6VM/TZEB1OIYNFcmfOw0nJJqKVQH0BZwDZSytDc9D9E/ydnA/k/h6PeQk2T8
uPjdsGyifD1zdYvKC1dpfwwO86CzlyPdfJzr3rm1YeskZ+sx6yH1jKWtqUSNAbQ1Lu2XxSeZcfoN
AkKGQY/b5VM+QEk+nN4Ax3+QzuOur8HbRHooKioqhslLg//0kEutUz9otmEaEgNQ00DbGgEDYP4B
yE2Gohw4+bMMGP/y1+v3s3WBAQ/DqL/LpxMVFZXmRecBAx6E3Uth0Dy5bGthmjQDEELcAbwIdAOi
FEUx+LguhBgHvANYAZ8qivJafd5fnQGYCEWB1Bip8QNynd8rQspUq6iomI+CTHi3j5TomLmmWWJu
5swCigamAdtrMcYKWAqMByKAGUKIiCaOq9IQhJCKj/795JdfX/Xmr6JiCew7SCHI8/+Dk2stbU3T
HICiKKcURalLryAKOKsoynlFUYqB74HJTRlXRUVFpdUy4CHw7QvrHpc1AhbEHFlAfsClKj8n6Lep
qKiotD+srKX2V2kRfP8XyLpU9zHNRJ1BYCHEZsDbwK+eUxTF5HMYIcRsYDZAYGBgHXurqKiotEI8
OsG0j2H1o/D+QHCp1pPY3g0e+LXZzajTASiKMrqJYyQCVdW8/PXbjI33MfAxyCBwE8dWUVFRaZl0
myhbxO54s2ZnQDsXs5hgjjTQ/UBnIUQI8sY/HfiLGcZVUVFRadm4hcDkpRYbvkkxACHEVCFEAjAI
WC+E+F2/3VcIsQFAUZRSYD7wO3AKWKkoyommma2ioqKi0lSaNANQFGU1sNrA9iTglio/bwA2NGUs
FRUVFRXTomoBqaioqLRTVAegoqKi0k5RHYCKiopKO0V1ACoqKirtFNUBqKioqLRTVAegoqKi0k5p
0Q1hhBCpQHwjD/cA0kxojqlQ7Wo4LdU21a6GodrVcBpjW5CiKJ712bFFO4CmIIQ4UF9NbHOi2tVw
Wqptql0NQ7Wr4TS3beoSkIqKiko7RXUAKioqKu2UtuwAPra0AUZQ7Wo4LdU21a6GodrVcJrVtjYb
A1BRUVFRqZ22PANQUVFRUamFNucAhBDjhBCnhRBnhRCLLGhHgBBimxDipBDihBDiMf32F4UQiUKI
I/qvW+p6r2ayL04IcVxvwwH9NjchxCYhRKz+ewcz29S1ynk5IoTIEUI8bolzJoT4XAhxRQgRXWWb
0fMjhHhWf82dFkKMtYBtbwghYoQQx4QQq4UQrvrtwUKIgirn7kMz22X0szPXOTNi14oqNsUJIY7o
t5vzfBm7R5jvOlMUpc18AVbAOSAUsAGOAhEWssUH6Kt/7QScASKAF4GnWsC5igM8qm17HVikf70I
+LeFP8tkIMgS5wwYDvQFous6P/rP9ShgC4Tor0ErM9s2BtDqX/+7im3BVfezwDkz+NmZ85wZsqva
798CXrDA+TJ2jzDbddbWZgBRwFlFUc4rilIMfA9MtoQhiqJcVhTlkP51LrIZjp8lbGkAk4Fl+tfL
gCkWtGUUcE5RlMYWAjYJRVG2AxnVNhs7P5OB7xVFKVIU5QJwFnktms02RVE2KrL5EsAeZOtVs2Lk
nBnDbOesNruEEAK4E/iuOcaujVruEWa7ztqaA/ADLlX5OYEWcNMVQgQDfYC9+k0L9FP1z829zFIF
BdgshDgohJit39ZRUZTL+tfJQEfLmAbI1qFV/ylbwjkzdn5a2nX3AFC1o3iIfjnjDyHEMAvYY+iz
aynnbBiQoihKbJVtZj9f1e4RZrvO2poDaHEIIRyBVcDjiqLkAB8gl6h6A5eR009LMFRRlN7AeGCe
EGJ41V8qcs5pkRQxIYQNMAn4Qb+ppZyzSix5fmpDCPEcUAp8o990GQjUf9b/B3wrhHA2o0kt7rOr
xgyuf9Aw+/kycI+opLmvs7bmABKBgCo/++u3WQQhhDXyg/1GUZSfABRFSVEUpUxRlHLgE5pxqaA2
FEVJ1H+/gmzrGQWkCCF89Lb7AFcsYRvSKR1SFCVFb2OLOGcYPz8t4roTQtwHTADu1t840C8XpOtf
H0SuG3cxl021fHYWP2dCCC0wDVhRsc3c58vQPQIzXmdtzQHsBzoLIUL0T5HTgZ8tYYh+bfEz4JSi
KG9X2e5TZbepQHT1Y81gm04I4VTxGhlAjEaeq3v1u90LrDW3bXqueyprCedMj7Hz8zMwXQhhK4QI
AToD+8xpmBBiHLAQmKQoSn6V7Z5CCCv961C9befNaJexz87i5wwYDcQoipJQscGc58vYPQJzXmfm
iHab8wvZjP4M0nM/Z0E7hiKnbseAI/qvW4CvgOP67T8DPhawLRSZTXAUOFFxngB3YAsQC2wG3Cxg
mw5IB1yqbDP7OUM6oMtACXKt9cHazg/wnP6aOw2Mt4BtZ5HrwxXX2of6fW/Tf8ZHgEPARDPbZfSz
M9c5M2SXfvuXwJxq+5rzfBm7R5jtOlMrgVVUVFTaKW1tCUhFRUVFpZ6oDkBFRUWlnaI6ABUVFZV2
iuoAVFRUVNopqgNQUVFRaaeoDkBFRUWlnaI6ABUVFZV2iuoAVFRUVNop/w+AradVLDh1wgAAAABJ
RU5ErkJggg==
"
>
</div>

</div>

</div>
</div>

</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div>
<div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>From the figure, there seems to be a high correlation between the ABP (green signal) and ECG/PPG (blue/green respectively).</p>
<p>Our next step, is to define the training samples for the model.
A natural choice could be setting a parameter <code>time_steps</code> that define a splitting length for the sequential signals</p>
$$x(t) = \begin{pmatrix} x_1(t) \\ x_2(t) \end{pmatrix}$$<p>and $y(t)$, where $x_1$ represents ECG, $x_2$ represents PPG and $y$ stands for the ABP signal.</p>

</div>
</div>
</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In&nbsp;[4]:</div>
<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="n">time_steps</span> <span class="o">=</span> <span class="mi">10</span>

<span class="c1"># remove samples in order to have a multiple of `time_steps` of them</span>
<span class="n">X</span> <span class="o">=</span> <span class="n">X</span><span class="p">[</span><span class="nb">len</span><span class="p">(</span><span class="n">X</span><span class="p">)</span><span class="o">%</span><span class="k">time_steps</span>:, :]
<span class="n">Y_hat</span> <span class="o">=</span> <span class="n">Y_hat</span><span class="p">[</span><span class="nb">len</span><span class="p">(</span><span class="n">Y_hat</span><span class="p">)</span><span class="o">%</span><span class="k">time_steps</span>:, :]

<span class="c1"># reshape measurements to make sequential samples</span>
<span class="n">X</span> <span class="o">=</span> <span class="n">X</span><span class="o">.</span><span class="n">reshape</span><span class="p">((</span><span class="n">X</span><span class="o">.</span><span class="n">shape</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">//</span><span class="n">time_steps</span><span class="p">,</span> <span class="n">time_steps</span><span class="p">,</span> <span class="n">X</span><span class="o">.</span><span class="n">shape</span><span class="p">[</span><span class="mi">1</span><span class="p">]))</span>
<span class="n">Y_hat</span> <span class="o">=</span> <span class="n">Y_hat</span><span class="o">.</span><span class="n">reshape</span><span class="p">((</span><span class="n">Y_hat</span><span class="o">.</span><span class="n">shape</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">//</span><span class="n">time_steps</span><span class="p">,</span> <span class="n">time_steps</span><span class="p">,</span> <span class="n">Y_hat</span><span class="o">.</span><span class="n">shape</span><span class="p">[</span><span class="mi">1</span><span class="p">]))</span>

<span class="c1"># make batches</span>
<span class="n">samples</span> <span class="o">=</span> <span class="nb">list</span><span class="p">(</span><span class="nb">zip</span><span class="p">(</span><span class="n">X</span><span class="p">,</span> <span class="n">Y_hat</span><span class="p">))</span>
<span class="n">val</span> <span class="o">=</span> <span class="mf">0.10</span>
<span class="n">num_val_samples</span> <span class="o">=</span> <span class="nb">int</span><span class="p">(</span><span class="n">val</span> <span class="o">*</span> <span class="nb">len</span><span class="p">(</span><span class="n">samples</span><span class="p">))</span>
<span class="n">train_samples</span> <span class="o">=</span> <span class="n">samples</span><span class="p">[:</span><span class="n">num_val_samples</span><span class="p">]</span>
<span class="n">valid_samples</span> <span class="o">=</span> <span class="n">samples</span><span class="p">[</span><span class="n">num_val_samples</span><span class="p">:]</span>
</pre></div>

</div>
</div>
</div>

</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div>
<div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>The global sequence is splitted in subsequences of length <code>time_steps = 10</code>, which are then grouped in two sets: a training set and a validation set, this last one containing abount 10% of the total amount of subsequences.</p>
<p>Now that the data is prepared, we should build up and train the model.</p>

</div>
</div>
</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In&nbsp;[5]:</div>
<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="kn">import</span> <span class="nn">sys</span>
<span class="n">sys</span><span class="o">.</span><span class="n">path</span><span class="o">.</span><span class="n">insert</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span> <span class="s1">&#39;./src&#39;</span><span class="p">)</span>

<span class="kn">from</span> <span class="nn">network_management</span> <span class="k">import</span> <span class="o">*</span>
<span class="kn">from</span> <span class="nn">nodes</span> <span class="k">import</span> <span class="n">Placeholder</span><span class="p">,</span> <span class="n">Variable</span><span class="p">,</span> <span class="n">Linear</span><span class="p">,</span> <span class="n">RNNCell</span><span class="p">,</span> <span class="n">MSE</span>
<span class="kn">from</span> <span class="nn">trainers</span> <span class="k">import</span> <span class="n">SGD</span>
</pre></div>

</div>
</div>
</div>

</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In&nbsp;[6]:</div>
<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="k">class</span> <span class="nc">RecurrentModel</span><span class="p">():</span>
    <span class="k">def</span> <span class="nf">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
        <span class="bp">self</span><span class="o">.</span><span class="vm">__name__</span> <span class="o">=</span> <span class="s1">&#39;BloodPressure&#39;</span>
        <span class="bp">self</span><span class="o">.</span><span class="n">learning_rate</span> <span class="o">=</span> <span class="mf">0.0001</span>
        <span class="bp">self</span><span class="o">.</span><span class="n">num_epochs</span> <span class="o">=</span> <span class="mi">10</span>
        <span class="bp">self</span><span class="o">.</span><span class="n">input_nodes</span> <span class="o">=</span> <span class="nb">list</span><span class="p">()</span>
        <span class="bp">self</span><span class="o">.</span><span class="n">inference_graph</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">_build_inference_graph</span><span class="p">()</span>
        <span class="bp">self</span><span class="o">.</span><span class="n">training_graph</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">_build_training_graph</span><span class="p">()</span>
        <span class="bp">self</span><span class="o">.</span><span class="n">time_steps</span> <span class="o">=</span> <span class="mi">10</span>

    <span class="k">def</span> <span class="nf">_build_inference_graph</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
        <span class="c1"># input layer</span>
        <span class="bp">self</span><span class="o">.</span><span class="n">X</span> <span class="o">=</span> <span class="n">Placeholder</span><span class="p">()</span>
        <span class="c1"># RNN cell (hidden layer)</span>
        <span class="c1"># batch size is required here to setup proper initial state for RNN cell</span>
        <span class="n">num_hidden</span> <span class="o">=</span> <span class="mi">32</span>
        <span class="n">W_XS</span> <span class="o">=</span> <span class="n">Variable</span><span class="p">(</span><span class="n">np</span><span class="o">.</span><span class="n">random</span><span class="o">.</span><span class="n">randn</span><span class="p">(</span><span class="mi">2</span><span class="p">,</span> <span class="n">num_hidden</span><span class="p">))</span>
        <span class="n">W_SS</span> <span class="o">=</span> <span class="n">Variable</span><span class="p">(</span><span class="n">np</span><span class="o">.</span><span class="n">random</span><span class="o">.</span><span class="n">randn</span><span class="p">(</span><span class="n">num_hidden</span><span class="p">,</span> <span class="n">num_hidden</span><span class="p">))</span>
        <span class="n">initial_hidden_state</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">zeros</span><span class="p">((</span><span class="mi">1</span><span class="p">,</span> <span class="n">num_hidden</span><span class="p">))</span>
        <span class="bp">self</span><span class="o">.</span><span class="n">RNN</span> <span class="o">=</span> <span class="n">RNNCell</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">X</span><span class="p">,</span> <span class="n">W_XS</span><span class="p">,</span> <span class="n">W_SS</span><span class="p">,</span> <span class="n">initial_hidden_state</span><span class="p">)</span>
        <span class="c1"># output layer</span>
        <span class="n">W_SY</span> <span class="o">=</span> <span class="n">Variable</span><span class="p">(</span><span class="n">np</span><span class="o">.</span><span class="n">random</span><span class="o">.</span><span class="n">randn</span><span class="p">(</span><span class="n">num_hidden</span><span class="p">,</span> <span class="mi">1</span><span class="p">))</span>
        <span class="bp">self</span><span class="o">.</span><span class="n">y</span> <span class="o">=</span> <span class="n">Linear</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">RNN</span><span class="p">,</span> <span class="n">W_SY</span><span class="p">)</span>
        <span class="c1"># build inference graph</span>
        <span class="bp">self</span><span class="o">.</span><span class="n">input_nodes</span><span class="o">.</span><span class="n">extend</span><span class="p">([</span><span class="bp">self</span><span class="o">.</span><span class="n">X</span><span class="p">,</span> <span class="n">W_XS</span><span class="p">,</span> <span class="n">W_SS</span><span class="p">,</span> <span class="n">W_SY</span><span class="p">])</span>
        <span class="n">graph</span> <span class="o">=</span> <span class="n">get_graph_flow</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">input_nodes</span><span class="p">)</span>

        <span class="k">return</span> <span class="n">graph</span>

    <span class="k">def</span> <span class="nf">_build_training_graph</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
        <span class="bp">self</span><span class="o">.</span><span class="n">Y_hat</span> <span class="o">=</span> <span class="n">Placeholder</span><span class="p">()</span>
        <span class="bp">self</span><span class="o">.</span><span class="n">loss</span> <span class="o">=</span> <span class="n">MSE</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">Y_hat</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">y</span><span class="p">)</span>
        <span class="bp">self</span><span class="o">.</span><span class="n">input_nodes</span><span class="o">.</span><span class="n">extend</span><span class="p">([</span><span class="bp">self</span><span class="o">.</span><span class="n">Y_hat</span><span class="p">])</span>
        <span class="c1"># add Trainer</span>
        <span class="n">parameters</span> <span class="o">=</span> <span class="n">get_parameters_nodes</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">input_nodes</span><span class="p">)</span>
        <span class="c1"># build training graph</span>
        <span class="bp">self</span><span class="o">.</span><span class="n">trainer</span> <span class="o">=</span> <span class="n">SGD</span><span class="p">(</span><span class="n">parameters</span><span class="p">,</span> <span class="n">learning_rate</span><span class="o">=</span><span class="bp">self</span><span class="o">.</span><span class="n">learning_rate</span><span class="p">)</span>
        <span class="n">graph</span> <span class="o">=</span> <span class="n">get_graph_flow</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">input_nodes</span><span class="p">)</span>

        <span class="k">return</span> <span class="n">graph</span>
    
    <span class="k">def</span> <span class="nf">_load_data</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
        <span class="c1"># load data</span>
        <span class="n">file</span> <span class="o">=</span> <span class="s1">&#39;./src/tests/data/blood_pressure.txt&#39;</span>
        <span class="n">dataset</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">read_csv</span><span class="p">(</span><span class="n">file</span><span class="p">,</span> <span class="n">sep</span><span class="o">=</span><span class="s1">&#39;;&#39;</span><span class="p">)</span>
        <span class="n">X</span> <span class="o">=</span> <span class="n">dataset</span><span class="o">.</span><span class="n">values</span><span class="p">[:,</span> <span class="p">(</span><span class="mi">3</span><span class="p">,</span> <span class="mi">1</span><span class="p">)]</span>
        <span class="n">Y_hat</span> <span class="o">=</span> <span class="n">dataset</span><span class="o">.</span><span class="n">values</span><span class="p">[:,</span> <span class="mi">2</span><span class="p">][:,</span> <span class="kc">None</span><span class="p">]</span>
        <span class="c1"># normalize data</span>
        <span class="n">X</span> <span class="o">=</span> <span class="n">scale</span><span class="p">(</span><span class="n">X</span><span class="p">,</span> <span class="n">axis</span><span class="o">=</span><span class="mi">0</span><span class="p">)</span>
        <span class="n">Y_hat</span> <span class="o">=</span> <span class="n">scale</span><span class="p">(</span><span class="n">Y_hat</span><span class="p">,</span> <span class="n">axis</span><span class="o">=</span><span class="mi">0</span><span class="p">)</span>
        <span class="c1"># create sample sequences</span>
        <span class="n">X</span> <span class="o">=</span> <span class="n">X</span><span class="p">[</span><span class="nb">len</span><span class="p">(</span><span class="n">X</span><span class="p">)</span><span class="o">%</span><span class="k">self</span>.time_steps:, :]
        <span class="n">Y_hat</span> <span class="o">=</span> <span class="n">Y_hat</span><span class="p">[</span><span class="nb">len</span><span class="p">(</span><span class="n">Y_hat</span><span class="p">)</span><span class="o">%</span><span class="k">self</span>.time_steps:, :]
        <span class="n">X</span> <span class="o">=</span> <span class="n">X</span><span class="o">.</span><span class="n">reshape</span><span class="p">((</span><span class="n">X</span><span class="o">.</span><span class="n">shape</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">//</span><span class="bp">self</span><span class="o">.</span><span class="n">time_steps</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">time_steps</span><span class="p">,</span> <span class="n">X</span><span class="o">.</span><span class="n">shape</span><span class="p">[</span><span class="mi">1</span><span class="p">]))</span>
        <span class="n">Y_hat</span> <span class="o">=</span> <span class="n">Y_hat</span><span class="o">.</span><span class="n">reshape</span><span class="p">((</span><span class="n">Y_hat</span><span class="o">.</span><span class="n">shape</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">//</span><span class="bp">self</span><span class="o">.</span><span class="n">time_steps</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">time_steps</span><span class="p">,</span> <span class="n">Y_hat</span><span class="o">.</span><span class="n">shape</span><span class="p">[</span><span class="mi">1</span><span class="p">]))</span>
        <span class="n">samples</span> <span class="o">=</span> <span class="nb">list</span><span class="p">(</span><span class="nb">zip</span><span class="p">(</span><span class="n">X</span><span class="p">,</span> <span class="n">Y_hat</span><span class="p">))</span>
        <span class="n">val</span> <span class="o">=</span> <span class="mf">0.10</span>
        <span class="n">num_val_samples</span> <span class="o">=</span> <span class="nb">int</span><span class="p">(</span><span class="n">val</span> <span class="o">*</span> <span class="nb">len</span><span class="p">(</span><span class="n">samples</span><span class="p">))</span>
        <span class="n">train_samples</span> <span class="o">=</span> <span class="n">samples</span><span class="p">[:</span><span class="n">num_val_samples</span><span class="p">]</span>
        <span class="n">valid_samples</span> <span class="o">=</span> <span class="n">samples</span><span class="p">[</span><span class="n">num_val_samples</span><span class="p">:]</span>
        
        <span class="k">return</span> <span class="n">train_samples</span><span class="p">,</span> <span class="n">valid_samples</span>

    <span class="k">def</span> <span class="nf">train</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
        <span class="n">train_samples</span><span class="p">,</span> <span class="n">valid_samples</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">_load_data</span><span class="p">()</span>
        <span class="c1"># training/validation statistics</span>
        <span class="n">tr_errors</span> <span class="o">=</span> <span class="nb">list</span><span class="p">()</span>
        <span class="n">val_errors</span> <span class="o">=</span> <span class="nb">list</span><span class="p">()</span>
        <span class="k">for</span> <span class="n">i_epoch</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">num_epochs</span><span class="p">):</span>
            <span class="c1"># training</span>
            <span class="n">train_error</span> <span class="o">=</span> <span class="mf">0.0</span>
            <span class="k">for</span> <span class="n">sample</span> <span class="ow">in</span> <span class="n">train_samples</span><span class="p">:</span>
                <span class="n">x</span><span class="p">,</span> <span class="n">y_hat</span> <span class="o">=</span> <span class="n">sample</span>
                <span class="c1"># forward pass</span>
                <span class="n">sample_error</span> <span class="o">=</span> <span class="mf">0.0</span>
                <span class="n">load_initial_states</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">training_graph</span><span class="p">)</span>
                <span class="k">for</span> <span class="n">t</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">time_steps</span><span class="p">):</span>
                    <span class="bp">self</span><span class="o">.</span><span class="n">X</span><span class="o">.</span><span class="n">forward</span><span class="p">(</span><span class="n">value</span><span class="o">=</span><span class="n">x</span><span class="p">[</span><span class="n">t</span><span class="p">][</span><span class="kc">None</span><span class="p">,</span> <span class="p">:])</span>
                    <span class="bp">self</span><span class="o">.</span><span class="n">Y_hat</span><span class="o">.</span><span class="n">forward</span><span class="p">(</span><span class="n">value</span><span class="o">=</span><span class="n">y_hat</span><span class="p">[</span><span class="n">t</span><span class="p">][</span><span class="kc">None</span><span class="p">,</span> <span class="p">:])</span>
                    <span class="n">forward_prop</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">training_graph</span><span class="p">)</span>
                    <span class="n">push_graph_state</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">training_graph</span><span class="p">)</span>
                    <span class="n">sample_error</span> <span class="o">+=</span> <span class="bp">self</span><span class="o">.</span><span class="n">loss</span><span class="o">.</span><span class="n">state</span>
                <span class="n">save_last_states</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">training_graph</span><span class="p">)</span>
                <span class="n">train_error</span> <span class="o">+=</span> <span class="n">sample_error</span>
                <span class="c1"># backward pass</span>
                <span class="c1"># compute and accumulate gradients over each unfolded timestep</span>
                <span class="k">for</span> <span class="n">t</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">time_steps</span><span class="p">)[::</span><span class="o">-</span><span class="mi">1</span><span class="p">]:</span>
                    <span class="n">pop_graph_state</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">training_graph</span><span class="p">)</span>
                    <span class="n">backward_prop</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">training_graph</span><span class="p">)</span>
                    <span class="bp">self</span><span class="o">.</span><span class="n">trainer</span><span class="o">.</span><span class="n">update_gradients</span><span class="p">()</span>
                <span class="c1"># apply corrections</span>
                <span class="bp">self</span><span class="o">.</span><span class="n">trainer</span><span class="o">.</span><span class="n">apply_gradients</span><span class="p">()</span>
            <span class="c1"># store average loss on training batches</span>
            <span class="n">tr_errors</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">train_error</span><span class="o">/</span><span class="p">(</span><span class="nb">len</span><span class="p">(</span><span class="n">train_samples</span><span class="p">)</span><span class="o">*</span><span class="bp">self</span><span class="o">.</span><span class="n">time_steps</span><span class="p">))</span>
            <span class="c1"># validation</span>
            <span class="n">valid_error</span> <span class="o">=</span> <span class="mf">0.0</span>
            <span class="k">for</span> <span class="n">sample</span> <span class="ow">in</span> <span class="n">valid_samples</span><span class="p">:</span>
                <span class="n">x</span><span class="p">,</span> <span class="n">y_hat</span> <span class="o">=</span> <span class="n">sample</span>
                <span class="n">sample_error</span> <span class="o">=</span> <span class="mf">0.0</span>
                <span class="n">load_initial_states</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">training_graph</span><span class="p">)</span>
                <span class="k">for</span> <span class="n">t</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">time_steps</span><span class="p">):</span>
                    <span class="bp">self</span><span class="o">.</span><span class="n">X</span><span class="o">.</span><span class="n">forward</span><span class="p">(</span><span class="n">value</span><span class="o">=</span><span class="n">x</span><span class="p">[</span><span class="n">t</span><span class="p">][</span><span class="kc">None</span><span class="p">,</span> <span class="p">:])</span>
                    <span class="bp">self</span><span class="o">.</span><span class="n">Y_hat</span><span class="o">.</span><span class="n">forward</span><span class="p">(</span><span class="n">value</span><span class="o">=</span><span class="n">y_hat</span><span class="p">[</span><span class="n">t</span><span class="p">][</span><span class="kc">None</span><span class="p">,</span> <span class="p">:])</span>
                    <span class="n">forward_prop</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">training_graph</span><span class="p">)</span>
                    <span class="n">sample_error</span> <span class="o">+=</span> <span class="bp">self</span><span class="o">.</span><span class="n">loss</span><span class="o">.</span><span class="n">state</span>
                <span class="n">valid_error</span> <span class="o">+=</span> <span class="n">sample_error</span>
            <span class="c1"># store average loss on validation batches</span>
            <span class="n">val_errors</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">valid_error</span><span class="o">/</span><span class="p">(</span><span class="nb">len</span><span class="p">(</span><span class="n">valid_samples</span><span class="p">)</span><span class="o">*</span><span class="bp">self</span><span class="o">.</span><span class="n">time_steps</span><span class="p">))</span>
            <span class="nb">print</span><span class="p">(</span><span class="s2">&quot;Epoch </span><span class="si">{:2d}</span><span class="s2"> - Loss: </span><span class="si">{:6.2f}</span><span class="s2">&quot;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">i_epoch</span><span class="o">+</span><span class="mi">1</span><span class="p">,</span> <span class="n">val_errors</span><span class="p">[</span><span class="o">-</span><span class="mi">1</span><span class="p">]))</span>
        <span class="n">plt</span><span class="o">.</span><span class="n">plot</span><span class="p">(</span><span class="nb">range</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">num_epochs</span><span class="p">),</span> <span class="n">tr_errors</span><span class="p">,</span> <span class="n">label</span><span class="o">=</span><span class="s1">&#39;Training&#39;</span><span class="p">)</span>
        <span class="n">plt</span><span class="o">.</span><span class="n">plot</span><span class="p">(</span><span class="nb">range</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">num_epochs</span><span class="p">),</span> <span class="n">val_errors</span><span class="p">,</span> <span class="n">label</span><span class="o">=</span><span class="s1">&#39;Validation&#39;</span><span class="p">)</span>
        <span class="n">plt</span><span class="o">.</span><span class="n">legend</span><span class="p">()</span>
        <span class="n">plt</span><span class="o">.</span><span class="n">show</span><span class="p">()</span>

    <span class="k">def</span> <span class="nf">infer</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">x</span><span class="p">):</span>
        <span class="c1"># set RNNCell state to inference mode: only a single state is mantained</span>
        <span class="k">if</span> <span class="bp">self</span><span class="o">.</span><span class="n">RNN</span><span class="o">.</span><span class="n">state</span><span class="o">.</span><span class="n">shape</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="ow">is</span> <span class="ow">not</span> <span class="mi">1</span><span class="p">:</span>
            <span class="bp">self</span><span class="o">.</span><span class="n">RNN</span><span class="o">.</span><span class="n">state</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">RNN</span><span class="o">.</span><span class="n">state</span><span class="p">[</span><span class="mi">0</span><span class="p">,</span> <span class="kc">None</span><span class="p">,</span> <span class="p">:]</span>
        <span class="bp">self</span><span class="o">.</span><span class="n">X</span><span class="o">.</span><span class="n">forward</span><span class="p">(</span><span class="n">value</span><span class="o">=</span><span class="n">x</span><span class="p">)</span>
        <span class="n">forward_prop</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">inference_graph</span><span class="p">)</span>

        <span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">y</span><span class="o">.</span><span class="n">state</span>
</pre></div>

</div>
</div>
</div>

</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In&nbsp;[7]:</div>
<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="n">rec_model</span> <span class="o">=</span> <span class="n">RecurrentModel</span><span class="p">()</span>
<span class="n">rec_model</span><span class="o">.</span><span class="n">train</span><span class="p">()</span>
</pre></div>

</div>
</div>
</div>

<div class="output_wrapper">
<div class="output">


<div class="output_area">

<div class="prompt"></div>


<div class="output_subarea output_stream output_stdout output_text">
<pre>Epoch  1 - Loss:   1.84
Epoch  2 - Loss:   1.41
Epoch  3 - Loss:   0.83
Epoch  4 - Loss:   0.62
Epoch  5 - Loss:   0.62
Epoch  6 - Loss:   0.68
Epoch  7 - Loss:   0.65
Epoch  8 - Loss:   0.63
Epoch  9 - Loss:   0.62
Epoch 10 - Loss:   0.60
</pre>
</div>
</div>

<div class="output_area">

<div class="prompt"></div>


<div class="output_png output_subarea ">
<img src="data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAXcAAAD8CAYAAACMwORRAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz
AAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xd8VGXa//HPNSW9Q0iAgAFUIKElRERQigiCq/KorAv2
3VVW7Nt8/LmubR/38dniYltXXRu7qKuiWEEsuMJaICAtBKRLKCEkkELqZO7fHzOEBIJpMzmZmev9
es1rzsyccjnq9z655z73EWMMSimlgovN6gKUUkr5noa7UkoFIQ13pZQKQhruSikVhDTclVIqCGm4
K6VUENJwV0qpIKThrpRSQUjDXSmlgpDDqgN3797dpKenW3V4pZQKSKtWrTpojEluaT3Lwj09PZ3c
3FyrDq+UUgFJRHa1Zj3tllFKqSCk4a6UUkFIw10ppYKQZX3uSqngUVdXR0FBAdXV1VaXEjQiIiJI
S0vD6XS2a3sNd6VUhxUUFBAbG0t6ejoiYnU5Ac8YQ3FxMQUFBfTr169d+9BuGaVUh1VXV9OtWzcN
dh8REbp169ahv4Q03JVSPqHB7lsd/T4DLty3FJbzu/c2UuOqt7oUpZTqsgIu3AsOVfHc8h18ua3Y
6lKUUl1EcXExI0aMYMSIEaSmptK7d++G17W1ta3ax49//GM2b978ves8+eSTzJ8/3xcl+13A/aB6
1oBuRIfZWbKxkAkDe1hdjlKqC+jWrRtr1qwB4P777ycmJoZf/epXTdYxxmCMwWZr/pz2hRdeaPE4
N998c8eL7SQBd+Ye4bQzYWAPPt5YiNttrC5HKdWFbd26lYyMDK688koyMzPZt28fs2fPJicnh8zM
TB588MGGdc8++2zWrFmDy+UiISGBu+66i+HDh3PWWWdx4MABAO655x7mzp3bsP5dd93FqFGjGDhw
IF988QUAR44c4bLLLiMjI4MZM2aQk5PT0PB0poA7cweYnJHC++v3sbbgMFl9E60uRynVyAPv5rFx
b5lP95nRK477Lsps17abNm1i3rx55OTkAPDwww+TlJSEy+Vi4sSJzJgxg4yMjCbblJaWMn78eB5+
+GF+8Ytf8Pzzz3PXXXedsG9jDCtWrOCdd97hwQcfZPHixTz++OOkpqayYMEC1q5dS3Z2drvq7qiA
O3MHmDiwBw6bsGRjodWlKKW6uAEDBjQEO8Arr7xCdnY22dnZ5Ofns3HjxhO2iYyMZNq0aQCMHDmS
nTt3NrvvSy+99IR1li9fzsyZMwEYPnw4mZnta5Q6KiDP3OOjnIzu342PNhby31MHWV2OUqqR9p5h
+0t0dHTD8pYtW3j00UdZsWIFCQkJXHXVVc2OJQ8LC2tYttvtuFyuZvcdHh7e4jpWCcgzd/B0zWw9
UMG2ogqrS1FKBYiysjJiY2OJi4tj3759fPjhhz4/xtixY3nttdcAWL9+fbN/GXSGgA53gI+0a0Yp
1UrZ2dlkZGQwaNAgrrnmGsaOHevzY9x6663s2bOHjIwMHnjgATIyMoiPj/f5cVoixlgz4iQnJ8d0
9GYdFz2+HKddePMm3/8LUkq1Xn5+PoMHD7a6jC7B5XLhcrmIiIhgy5YtTJkyhS1btuBwtL0XvLnv
VURWGWNyTrJJg4Dscz9qckYKf/n4Ww6UV9MjNsLqcpRSioqKCiZNmoTL5cIYw9NPP92uYO+ogA73
KZkpPPLRt3ySf4BZo/paXY5SSpGQkMCqVausLiNw+9wBBqbE0jcpiiV5+60uRSmlupSADncRYUpG
Cv/ZVkxFTdcahqSUUlYK6HAHT797rcvN598WWV2KUkp1GQEf7iNPSSQpOky7ZpRSqpGAD3eH3cak
QT34dNMB6urdVpejlLLAxIkTT7ggae7cucyZM+ek28TExACwd+9eZsyY0ew6EyZMoKUh23PnzqWy
srLh9QUXXMDhw4dbW7rfBHy4g6drpqzaxYodJVaXopSywKxZs3j11VebvPfqq68ya9asFrft1asX
b7zxRruPfXy4f/DBByQkJLR7f74SFOF+zmnJRDht2jWjVIiaMWMG77//fsONOXbu3MnevXvJyspi
0qRJZGdnM3ToUN5+++0Ttt25cydDhgwBoKqqipkzZzJ48GAuueQSqqqqGtabM2dOw1TB9913HwCP
PfYYe/fuZeLEiUycOBGA9PR0Dh48CMAjjzzCkCFDGDJkSMNUwTt37mTw4MHccMMNZGZmMmXKlCbH
8ZWAHud+VGSYnXGnJfPRxkLuvzhT7+WolJUW3QX71/t2n6lDYdrDJ/04KSmJUaNGsWjRIqZPn86r
r77K5ZdfTmRkJG+99RZxcXEcPHiQ0aNHc/HFF580I5566imioqLIz89n3bp1Tabrfeihh0hKSqK+
vp5Jkyaxbt06brvtNh555BGWLl1K9+7dm+xr1apVvPDCC3z99dcYYzjzzDMZP348iYmJbNmyhVde
eYVnn32Wyy+/nAULFnDVVVf55rvyCoozd/B0zewtrSbPx/NIK6UCQ+OumaNdMsYY7r77boYNG8Z5
553Hnj17KCw8+XxUn3/+eUPIDhs2jGHDhjV89tprr5GdnU1WVhZ5eXktTgi2fPlyLrnkEqKjo4mJ
ieHSSy9l2bJlAPTr148RI0YA3z+lcEcExZk7wKTBKdgEluTtZ0jvzp+kRynl9T1n2P40ffp0fv7z
n7N69WoqKysZOXIkL774IkVFRaxatQqn00l6enqzU/y2ZMeOHfzpT39i5cqVJCYmct1117VrP0cd
nSoYPNMF+6NbJmjO3JOiwzgjPUlv4KFUiIqJiWHixIn85Cc/afghtbS0lB49euB0Olm6dCm7du36
3n2MGzeOl19+GYANGzawbt06wDNVcHR0NPHx8RQWFrJo0aKGbWJjYykvLz9hX+eccw4LFy6ksrKS
I0eO8NZbb3HOOef46h+3RUET7gBTMlPZtL+c74orW15ZKRV0Zs2axdq1axvC/corryQ3N5ehQ4cy
b948Bg36/pv7zJkzh4qKCgYPHsy9997LyJEjAc8dlbKyshg0aBBXXHFFk6mCZ8+ezdSpUxt+UD0q
Ozub6667jlGjRnHmmWdy/fXXk5WV5eN/4pML6Cl/j7e7pJJz/rCUe34wmOvP6e/TfSulTk6n/PWP
jkz5G1Rn7n2SohiUGqtdM0qpkNdiuItIHxFZKiIbRSRPRG5vZp0JIlIqImu8j3v9U27LpmSmkruz
hJIjtVaVoJRSlmvNmbsL+KUxJgMYDdwsIhnNrLfMGDPC+3jQp1W2wZSMFNwGPsnXs3elOpNVXbzB
qqPfZ4vhbozZZ4xZ7V0uB/KB3h06qh9l9oqjV3yEds0o1YkiIiIoLi7WgPcRYwzFxcVERLT/DnNt
GucuIulAFvB1Mx+PEZF1wB7gV8aYvHZX1QEiwpTMVF5d+R1VtfVEhtmtKEOpkJKWlkZBQQFFRTr1
tq9ERESQlpbW7u1bHe4iEgMsAO4wxhx/GehqoK8xpkJELgAWAqc1s4/ZwGyAvn39d1u8yRkpvPjF
Tj7fUsT5mal+O45SysPpdNKvXz+ry1CNtGq0jIg48QT7fGPMm8d/bowpM8ZUeJc/AJwi0r2Z9Z4x
xuQYY3KSk5M7WPrJjeqXRFyEg4+0a0YpFaJaM1pGgOeAfGPMIydZJ9W7HiIyyrvfYl8W2hZOu41J
g1P4JL8Ql87xrpQKQa05cx8LXA2c22io4wUicqOI3OhdZwawQUTWAo8BM43Fv6xMyUjhUGUdubsO
WVmGUkpZosU+d2PMcuB759A1xjwBPOGronxh3OnJhDlsfLSxkNH9u1ldjlJKdaqgukK1sehwB2ef
2p0lG/fr8CylVMgJ2nAHT9fM7pIqNu0/ccY2pZQKZkEd7pMGpyCCjppRSoWcoA735NhwsvsmsmSj
3ltVKRVagjrcwdM1s2FPGXsO+/5OJ0op1VUFf7h7r1D9KE/P3pVSoSPow71f92hO7RHDRzpLpFIq
hAR9uIOna+ar7SWUVtZZXYpSSnWK0Aj3zFTq3YZPN+vZu1IqNIREuA/rHU+P2HAdEqmUChkhEe42
mzA5I4XPNhdRXVdvdTlKKeV3IRHu4Omaqayt54ttB60uRSml/C5kwn10/yRiwh0sydOuGaVU8AuZ
cA932JkwMJmP8wupd+tEYkqp4BYy4Q6erpmDFbWs2a1zvCulgltIhfuEgck47aJdM0qpoBdS4R4X
4WR0/24s2Vioc7wrpYJaSIU7eLpmdhw8wraiCqtLUUopvwm5cJ88OAWAD7VrRikVxEIu3FPjIxie
Fs8SvVpVKRXEQi7cwdM1s3b3YQrLqq0uRSml/CI0wz3D0zWjc80opYJVSIb7qT1iSO8WpV0zSqmg
FZLhLiJMyUzly20HKa/WOd6VUsEnJMMdPF0zdfWGzzYXWV2KUkr5XMiGe1bfRLrHhGnXjFIqKIVs
uNttwqRBKXy26QC1LrfV5SillE+FbLgDTMlMobzGxVfbi60uRSmlfCqkw33sqd2JCrOzZON+q0tR
SimfCulwj3DaGXdaMh9tLMStc7wrpYJISIc7eLpmCstqWL+n1OpSlFLKZ0I+3M8d1AO7TbRrRikV
VEI+3BOiwhiVnqQ38FBKBZWQD3fwdM1sOVDBjoNHrC5FKaV8QsMdmNwwkZh2zSilgoOGO5CWGEVm
rzjtmlFKBY0Ww11E+ojIUhHZKCJ5InJ7M+uIiDwmIltFZJ2IZPunXP+ZnJHCqu8OUVReY3UpSinV
Ya05c3cBvzTGZACjgZtFJOO4daYBp3kfs4GnfFplJ5iSkYox8OkmPXtXSgW+FsPdGLPPGLPau1wO
5AO9j1ttOjDPeHwFJIhIT59X60eDe8aSlhipXTNKqaDQpj53EUkHsoCvj/uoN7C70esCTmwAEJHZ
IpIrIrlFRV1rql0RYXJGCsu2HuRIjcvqcpRSqkNaHe4iEgMsAO4wxpS152DGmGeMMTnGmJzk5OT2
7MKvpmSkUutys2xL12p4lFKqrVoV7iLixBPs840xbzazyh6gT6PXad73AsoZ6YkkRDm1a0YpFfBa
M1pGgOeAfGPMIydZ7R3gGu+omdFAqTFmnw/r7BQOu41zB/Xgk00HqKvXOd6VUoGrNWfuY4GrgXNF
ZI33cYGI3CgiN3rX+QDYDmwFngVu8k+5/jclI5XSqjpW7iixuhSllGo3R0srGGOWA9LCOga42VdF
WWnc6d0Jd9hYsrGQMad2t7ocpZRqF71C9ThRYQ7O8c7x7mmzlFIq8Gi4N2NKRgp7DleRt7ddg4KU
UspyGu7NmDS4BzaBjzbqqBmlVGDScG9Gt5hwck5JYomGu1IqQGm4n8TkjBTy95Wxu6TS6lKUUqrN
NNxP4ugc73r2rpQKRBruJ5HePZqBKbF6Aw+lVEAKvHB310P+e51yqCmZKazYUcKhI7WdcjyllPKV
wAv3b/4B/7oSvn7G74eanJGC28Anmw74/VhKKeVLgRfuWVfD6dNg0Z2w6X2/Hmpo73hS4yK0a0Yp
FXACL9xtdpjxHPTKgjd+CgW5fjuUiDAlM4V/f1tEVW29346jlFK+FnjhDhAWDVe8BrEp8PLlULzN
b4eanJFCdZ2b5VsP+u0YSinla4EZ7gAxyXDlAjBumD8DjhT75TBn9utGbISDJXnaNaOUChyBG+4A
3U+FWa9C6R54ZSbUVfn8EGGOY3O817t1IjGlVGAI7HAH6DsaLnsWClbCgus9QyV9bHJGCiVHalm1
65DP962UUv4Q+OEOkDEdzv89bHoPPvyNz3c//vRkwuw27ZpRSgWM4Ah3gLNugtE3wddPwZdP+nTX
sRFOxpzajY/ydY53pVRgCJ5wB5jyPzD4Is/Ze95C3+46I5VdxZV8W1jh0/0qpZQ/BFe42+xw6bPQ
ZxS8ORu++8pnuz5vcA8A7ZpRSgWE4Ap3AGckzHwF4tM8I2gObvHJbnvERZDVN4GP8nWWSKVU1xd8
4Q4Q3Q2uegPEDv+8DCp8MzfMlIxU1hWUsvew74dcKqWULwVnuAMk9fdcxVpxwHMVa+2RDu/y6Bzv
H+vZu1KqiwvecAdIGwkznod9a+GNn0C9q0O7O7VHDP2To1mSp+GulOragjvcAQZdANP+AN8u9swk
2cGhjFMyUvlqezGlVXU+KlAppXwv+MMdYNQNMOY2yH0O/jO3Q7uanJGCy234bLPO8a6U6rpCI9wB
znsAhlwGH98P699o926y+iTQPSZcu2aUUl1a6IS7zQb/9RScMhYWzoGdy9u5G2FyRgqfbT5AjUvn
eFdKdU2hE+4AjnD40T8hMR1evQIObGrXbqZkpnCktp4vtvlnmmGllOqo0Ap3gKgkuPINcER45oEv
b/sVp2MGdCM6zK5dM0qpLiv0wh0g8RTPGPjKEpj/Q6gpb9Pm4Q47Ewb2YEnefg5X1vqpSKWUar/Q
DHeAXiPg8pegMA9evw7q2za08fpz+lFe7WLOP1dTV+/2T41KKdVOoRvuAKdNhgsfga0fw/u/aNMY
+Ky+iTx82VC+3F7Mbxdu0KmAlVJdisPqAiw38jo4vBuW/Qni+8L4X7d600uz09hedIQnlm5lQHIM
N4zr7786lVKqDTTcAc69B0oLYOn/eGaTHDGr1Zv+YvLpbD9Ywe8X5ZPePbph/hmllLJSaHfLHCUC
Fz8O/cbBO7fAtqWt3tRmE/78wxEM6x3P7a9+Q97eUj8WqpRSrdNiuIvI8yJyQEQ2nOTzCSJSKiJr
vI97fV9mJ3CEecbAdz8d/nU17G/2H7dZkWF2nr0mh/hIJ9e/lMuBsmo/FqqUUi1rzZn7i8DUFtZZ
ZowZ4X082PGyLBIRD1e+DuExniGSpXtavWmPuAj+fm0OpVV13DAvl6pavXpVKWWdFsPdGPM5UNIJ
tXQN8WmegK8p9wR8deu7WTJ7xfPozCzW7Snll6+vwe3WETRKKWv4qs99jIisE5FFIpLpo31aJ3Uo
/GgeHNwMr10DrtZfqDQ5I4W7pw3mg/X7+cvH3/qxSKWUOjlfhPtqoK8xZhjwOLDwZCuKyGwRyRWR
3KKiIh8c2o8GnAsXPQbbP4N3b2vTGPjrz+nHzDP68PinW3lzdYH/alRKqZPocLgbY8qMMRXe5Q8A
p4h0P8m6zxhjcowxOcnJyR09tP9lXQkT7oa1r8DS37d6MxHhwelDOKt/N+5asJ6VO0OnV0sp1TV0
ONxFJFVExLs8yrvP4JkucfydkHUVfP4HWD2v1ZuFOWw8dVU2vRMj+dk/VvFdcaUfi1RKqaZaMxTy
FeBLYKCIFIjIT0XkRhG50bvKDGCDiKwFHgNmmmC6Fl8ELpwLAybBu3fAlo9bvWlCVBjPXZtDvdvw
k5dWUlatt+ZTSnUOsSqHc3JyTG5uriXHbpeacnhhGpTsgB9/AD2Ht3rTL7Yd5JrnVnDWgG68cN0Z
OOx67ZhSqn1EZJUxJqel9TRlWis8Fq54HSISPEMkD3/X6k3HDOjOQ5cMYdmWgzz43kY/FqmUUh4a
7m0R1xOuegPqqj0BX3Wo1Zv+6Iy+/Gxcf+Z9uYuXvtjpvxqVUgoN97brMRhm/hOKt3mmKXC3/krU
O6cOYnJGCg+8m8fSzQf8WKRSKtRpuLdHv3Fw0VzYuQy+eqrVm9ltwtwfjWBQahy3vvwNm/e37Q5Q
SinVWhru7TXiSjh9Gnz6O89ZfCtFhzt47rocosLs/OTFlRysqPFjkUqpUKXh3l4icOFfwBEOb98C
7tbfaq9nfCR/vzaH4iM1zJ6XS3WdTjKmlPItDfeOiOsJ5/8vfPcFrHy2TZsOS0vgkctHsPq7w/z3
gnV6mz6llE9puHfUiCs8Fzh9fD8c2tmmTS8Y2pNfnz+Qt9fs5fFPt/qlPKVUaNJw7ygRuPgxEDu8
c2ubJhgDuGnCAC7N7s0jH33Lu2v3+qlIpVSo0XD3hfg0mPI72PE5rHqhTZuKCP976VDOSE/kV6+v
5ZvvWj92XimlTkbD3VdGXgf9xsOSe+Hw7jZtGu6w8/TVOaTERXDDvFwKDukkY0qpjtFw95Wj3TPG
De/e3ubumaToMJ6/Local5vrX8qlosblp0KVUqFAw92XEtPhvPth2yewZn6bNz+1RyxPXpHNlgMV
3PbKN9TrbfqUUu2k4e5rZ1wPp4yFxXdDWdt/IB13ejL3X5zJp5sO8PsP8v1QoFIqFGi4+5rNBhc/
DvW18N7P29w9A3D16FO4bkw6zy3fwfyvd/mhSKVUsNNw94duA+Dce+DbxbD+9Xbt4p4fDGbCwGTu
fTuP5VsO+rhApVSw03D3l9FzIG0ULLoTygvbvLnDbuPxWVkMSI5mzvxVbD1Q4YcilVLBSsPdX2x2
mP4k1FbCB79sV/dMbIST5649g3CHjZ++tJKSI7V+KFQpFYw03P0p+XSYcBfkvwsbF7ZrF32Sonj6
6hz2lVZz4z9WUePSScaUUi3TcPe3MbdBryx4/1dwpH195yNPSeSPM4axYmcJd7+5QScZU0q1SMPd
3+wOmP5XqC719L+30/QRvbl90mksWF3A3/693YcFKqWCkYZ7Z0jJgPF3woYFkP9eu3dzx3mncdHw
Xvzf4k0s3rDPhwUqpYKNhntnOfvnkDIU3v8FVJa0axciwh9nDGNEnwTu+Nca1heU+rhIpVSw0HDv
LHYn/NeTUFkMH97d7t1EOO08c81IukWHc/28lewvrfZhkUqpYKHh3pl6Dvecwa99Bb79sN276REb
wd+vzaGi2sVPX1pJZa1OMqaUakrDvbON+zUkD4Z37/D8yNpOg3vG8fgVWeTvK2P2vFV6o22lVBMa
7p3NEe7pnqnYD0vu6dCuzh2UwsOXeoZITp37OZ9uavuVsEqp4KThboXeI2HMrbB6Hmz7tEO7uvyM
Prxzy1i6x4Tzkxdz+c1b67WbRiml4W6ZCXdDt9PgndugprxDuxqUGsfbt4xl9rj+vLziOy58bDlr
dx/2UaFKqUCk4W4VZ4Rn7pnSAvj4/g7vLtxh5+4LBjP/+jOprqvn0qe+4LFPtuCqd3e8VqVUwNFw
t1LfMz2zR678O+xY5pNdjhnQnUV3jOPCYT155KNvufzpL9lVfMQn+1ZKBQ4Nd6ud+1tI7Afv3AK1
vgnh+Egnj87M4tGZI9hyoIILHl3Gayt365w0SoUQDXerhUXB9Cfg0E745Hc+3fX0Eb1ZfMc4hqbF
c+eCdfzsH6so1iGTSoUEDfeuIP1sz71Xv/4bfPeVT3fdOyGSl68fzW8uGMxnm4s4f+4ylm4+4NNj
KKW6Hg33ruK8ByC+D7x9M9RV+XTXNptww7j+LLx5LEnRTn78wkp+u3ADVbU6N7xSwUrDvasIj4GL
H4PirbD09345REavON655Wx+enY//vHVLn7w+DKdfEypINViuIvI8yJyQEQ2nORzEZHHRGSriKwT
kWzflxkiBkyE7GvhyyegYJVfDhHhtPPbCzOYf/2ZVNbUc8lf/8OTS7dS79YfW5UKJq05c38RmPo9
n08DTvM+ZgNPdbysEDbldxDbE96+CVz++/Fz7KndWXzHOZw/JJU/friZHz39JbtLKv12PKVU52ox
3I0xnwPfNwH5dGCe8fgKSBCRnr4qMORExMNFj0LRJvj3H/x6qISoMJ6YlcVffjSczfvLmTr3c17P
1SGTSgUDX/S59wZ2N3pd4H3vBCIyW0RyRSS3qKjIB4cOUqdNhuFXwPK/wN41fj2UiHBJVhqL7jiH
zN7x/PqNdcz552oOHan163GVUv7VqT+oGmOeMcbkGGNykpOTO/PQgWfq7yG6u2f0jMv/QZuWGMUr
N4zmrmmD+GRTIefP/Zx/f6sNsFKByhfhvgfo0+h1mvc91RGRiXDhX6Bwg+cMvhPYbcKN4wfw1k1j
iY90cu3zK7j/nTyq63TIpFKBxhfh/g5wjXfUzGig1Bijd2/2hUE/gCEz4PM/QmFepx12SO943r31
bH48Np0Xv9jJhY8vZ8MeHTKpVCBpzVDIV4AvgYEiUiAiPxWRG0XkRu8qHwDbga3As8BNfqs2FE37
A0QmwMKboL7z5mmPcNq576JM/vHTUZRX13HJX//DXz/TIZNKBQqxamRETk6Oyc3NteTYASdvIbx+
LUy6D875Racf/tCRWn6zcD0frN/PqPQk/nz5cPokRXV6HUopEJFVxpicltbTK1QDQeZ/weCL4bP/
haLNnX74xOgwnrwimz//cDgb95Ux7dFlLFhVoEMmlerCNNwDxQ/+DGExntEz7s7/gVNEuGxkGotu
P4dBqbH88vW13PLyNxyu1CGTSnVFGu6BIqaHp/+9YCV8Zd1FwH2SovjXz87i1+cP5MO8/Zw/93OW
bdEhk0p1NRrugWToDBh4AXz6OyjeZlkZdptw88RTWXjzWGLCHVz93AoeeFeHTCrVlegPqoGmbB/8
9UzokQnXvQ82a9vnqtp6Hl6Uz0tf7iIm3MG5g3owdUgqEwYmExXmsLQ2pYJRa39Q1XAPRN/M90ws
Nu0PcObPrK4GgBU7SliwqoAlG/dzqLKOcIeN8acnM21oKucOSiE+0ml1iUoFBQ33YGYMzJ8Bu76A
m76ExHSrK2rgqnezYmcJH27Yz+K8/RSW1eC0C2MGdGfakFQmZ6TQLSbc6jKVClga7sGutACeHA29
s+Cad0DE6opO4HYbvtl9mA/z9rNowz52l1RhExjVL4lpQ3pyfmYqqfERVpepVEDRcA8FuS/Ae3fA
1Idh9Byrq/lexhg27itj8Yb9LNqwn60HKgDI6pvA1MxUpg3pSd9uemGUUi3RcA8FxsA/L4Vtn8Kp
58GU/4Eeg62uqlW2HihnsbfrZsOeMgAG94xj2pBUpg1J5dQeMUgX/GtEKatpuIcKVw2seMYzuVhN
OWRfAxN/4xkXHyB2l1R6u272s2rXIQD6J0czbUgqUzN7MqR3nAa9Ul4a7qGmsgT+/X+w8u/giICz
fw5n3QzOSKsra5PCsmqW5HnO6L/aXkK929A7IZKp3jP67L6J2Gwa9Cp0abiHqoNb4eP7YNN7EJcG
k+6FoT+0fDx8e5QcqeXj/EIWb9jP8i0Hqa13kxwbzvmZKUzN7MmZ/ZNw2gPvn0upjtBwD3U7l8OH
d8O+tdAjpUYVAAALRElEQVQrC6Y8BOljra6q3cqr6/h00wEWb9jPZ5uLqKqrJyHKyeTBKUwdksrZ
p3Un3GG3ukyl/E7DXYHbDetfg08ehLI9MOhCmPwgdBtgdWUdUlVbz7+/LeLDvP18nF9IebWLmHAH
Ewf1YGpmKkN7x9MzIULP6lVQ0nBXx9RWwldPwrK/QH0NnHEDjL8TopKsrqzDal1u/rPtIB9u2M+S
jYWUeG/sbRPoGR9JWmIkfZKiPM+JUQ2vU+IisGvfvQpAGu7qROWFsPQh+OYfEB4L4+6EUTeAIziu
GHXVu1mz+zDbDx6hoKSSgkNV7D5Uye6SKgrLq2n8n7rTLvRKaBr4aYmRpCVG0ScxkuTYcB2ho7ok
DXd1coUbYck9sO0Tz9QFkx/03AwkiMOsxlXP3sPV7G4U+gWHqryvKzlY0XRe+nCH7VjYJx0N/WMN
QWKUU8NfWULDXbVsy8eekC/Kh75neX50TRtpdVWWqKqtp+BQ47P9po3A4cq6JutHh9mbBH+T7p+k
KOIidKI05R8a7qp16l2ebpqlD8GRIhgyA867DxL6Wl1Zl1JWXUdBSRUFhyrZfcj7fPR1SSVHapvO
ZR8b4SAlLoIeseEkx4Y3eo5o8jo+Uv8CUG2j4a7apqYcls+FL5/wTGsweo7nZtwR8VZX1uUZYzhc
WXfCWX9ReQ1FFTUcKK/mQFkNNS73CduG2W0kx4bTvUkDcGJD0D0mnDCHjv5RGu6qvUoLPEMn1/0L
orrDxP8H2deBXW+80RHGGMprXBSV13CgzBv6ZdUUVdRQ5H1dVF7DgfKahhE/x0uMcpJ8kr8Aji1H
EBfh0L8GgpiGu+qYPas9/fG7/gPdB8KU38FpU4L6R9d2qa2Eku1QvBVKtnluf1i8zdNIhkVBZBJE
JnoeUUkQmXDs9fGfhcWACHX1bg4eDfuGhqCGoorqhtdHG4LaZv4aCHfYGgK/W3QYTrsNh92G0ybe
Ze+zTXDYbYTZPc8Ou+C0HfvcaRccTV6ffB2ndx/O4/bd+LU2OL6h4a46zhjY9D58dK8nuPqNh/Mf
gtShVlfWuVy1cGinN7y3egN8qyfUy/Y0XTcm1XORWHwfcFV55vypOgxVhzyPuiMnP47NcfLgb6ZR
MJEJlEkcRTUODlTUerqBvKHvea6m5Egdrno3Lreh1uXG5XbjqjfUed9z1Rtq609sIHzNbhNiwh3E
RjiIi3ASG+EgNsJJXIT3vchj7zX97NhzhNOmDQQa7sqXXLWQ+5xnYrKqw5B1JUy8B+J6Wl2Z77jr
oXS3N7wbn4lvhcPfgWkUgJFJngDvdiokDfAuD4Ck/p7rB75PXTVUNwr7ypJjy1WHoOq415UdaBSO
PiLiIDzO8/tJw/Kx94zYqHcbXG5v6Ncb6txu6uoNrnrvc6NGoeF997HPPY3F0c9Mw7Jnf26q69yU
V9dRXu2irNpFmXe5vNGzu4UoctikUSPgIDbceULD0NBYRDgbNRTezyIdQTFFhYa78r2qQ/D5n+Dr
p8HuhLG3w5hbISza6spaxxgo39fozLtRN8qhHVDfqK87LMYb2N4Qbwjz/tZc2euq8f4FcHz4n6xx
OOz57PsahaPCYk4I/CaNQET8sfdPeC8OwmI7PDGdMYYjtfVNwr6s2kVZ1dHXrkaNw/ENg2e9iloX
LcWZ3SaEO2zeh52wo8tOz+twh+3Ye97X4U4bYXa7d53jtnPYCHeeZLujy8dt19ErozXclf+UbIeP
74eNb0NsTzj3Hhg+C2xd4KzIGE+oNT7zPhrgJdubhp093BPWR8+8G5+Jx6QEx+8LrlqoKYPqUu9z
2bHXDctHP2/uvbKmjV6zxPMXS+PAP74RCI8BZ5RnOmpnpOfhiGy03Oj9o585wtv078DtNlTUHmsI
yqpcJzQWVbX11LjqqXG5qXW5qXG5Pa/rji0fe99NTV3TdX3RheWwCTeOH8Cvzh/Yru013JX/7foS
lvwG9qzy9MOfPtXTvWHqPc8Ny67mXzdez+1q9Jn7uO1cnm6RJtt512vY7uijDuoqj9Uods9VuI3P
vI+eicelBeRUyJ2urvq4BqG0hUbiuMakutTz76jNpFHwR4EzolGjENGosfB+1tBYRHxPQxLleQ6L
Ame0dzm61ScmbrfnN4qaOjc19U0bhSYNhrdROPae93WdZ/mMfklMHNi+G+pouKvO4XZD3pue4ZOH
d3n6f8XuebbZQWzHlhs+sx/32nbidjZ7o3VPtl3j9RodI673sTBP6OvpQlLWMQZc1VBXdey5YbnS
03i4qo69X1flfe393FV93PLx2xy3fXsaEnuYJ/jDoo81As0uR3sbhsbLUY0ajehmlqN8OpS4teGu
g5dVx9hsMHQGDLksOLoxlO+JHDuD9jdjoL6umcahceNxxPNc632uq2y0fMQzvPXocnWZZ8K94983
beyeOdp4OKM8DcLIH8OYW/zzHXhpuCvf0GBXXYEIOMI8D39dXW2M5wfuukrv42hD4X1dW/k9y95G
JCbFP7U1ouGulFJtIeLt148Auu49EfTXJKWUCkIa7kopFYQ03JVSKgi1KtxFZKqIbBaRrSJyVzOf
TxCRUhFZ433c6/tSlVJKtVaLP6iKiB14EpgMFAArReQdY8zG41ZdZoy50A81KqWUaqPWnLmPArYa
Y7YbY2qBV4Hp/i1LKaVUR7Qm3HsDuxu9LvC+d7wxIrJORBaJSKZPqlNKKdUuvhrnvhroa4ypEJEL
gIXAacevJCKzgdkAffvqPTqVUspfWhPue4A+jV6ned9rYIwpa7T8gYj8VUS6G2MOHrfeM8AzACJS
JCK72ll3d+Bgi2uFDv0+mtLv4xj9LpoKhu/jlNas1JpwXwmcJiL98IT6TOCKxiuISCpQaIwxIjIK
T3dP8fft1BiT3JoCmyMiua2ZOCdU6PfRlH4fx+h30VQofR8thrsxxiUitwAfAnbgeWNMnojc6P38
b8AMYI6IuIAqYKaxarpJpZRSretzN8Z8AHxw3Ht/a7T8BPCEb0tTSinVXoF6heozVhfQxej30ZR+
H8fod9FUyHwflt2sQymllP8E6pm7Ukqp7xFw4d7SPDehRET6iMhSEdkoInkicrvVNVlNROwi8o2I
vGd1LVYTkQQReUNENolIvoicZXVNVhGRn3v/H9kgIq+ISITVNflbQIV7o3lupgEZwCwRybC2Kku5
gF8aYzKA0cDNIf59ANwO5FtdRBfxKLDYGDMIGE6Ifi8i0hu4DcgxxgzBM+pvprVV+V9AhTs6z00T
xph9xpjV3uVyPP/zNjc1REgQkTTgB8Dfra7FaiISD4wDngMwxtQaYw5bW5WlHECkiDiAKGCvxfX4
XaCFe2vnuQk5IpIOZAFfW1uJpeYCdwJtvHtxUOoHFAEveLup/i4i0VYXZQVjzB7gT8B3wD6g1Biz
xNqq/C/Qwl01Q0RigAXAHY2nggglInIhcMAYs8rqWroIB5ANPGWMyQKOACH5G5WIJOL5C78f0AuI
FpGrrK3K/wIt3Fuc5ybUiIgTT7DPN8a8aXU9FhoLXCwiO/F0150rIv+0tiRLFQAFxpijf8m9gSfs
Q9F5wA5jTJExpg54ExhjcU1+F2jh3jDPjYiE4flR5B2La7KMiAiePtV8Y8wjVtdjJWPM/zPGpBlj
0vH8d/GpMSboz85OxhizH9gtIgO9b00Cjr/BTqj4DhgtIlHe/2cmEQI/Lvtqyt9OcbJ5biwuy0pj
gauB9SKyxvve3d7pIpS6FZjvPRHaDvzY4nosYYz5WkTewDM1uQv4hhC4UlWvUFVKqSAUaN0ySiml
WkHDXSmlgpCGu1JKBSENd6WUCkIa7kopFYQ03JVSKghpuCulVBDScFdKqSD0/wHKmqOxgWedoAAA
AABJRU5ErkJggg==
"
>
</div>

</div>

</div>
</div>

</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div>
<div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p><a id=explodevanish></a></p>
<h3 id="The-exploding/vanishing-gradient-problem">The exploding/vanishing gradient problem<a class="anchor-link" href="#The-exploding/vanishing-gradient-problem">&#182;</a></h3><p>The main problem with RNNs is their extremely high number of operations. The time dimension gives depth to their computational graph by stacking virtual copies one upon another, replicating a feedforward graph over and over while joining each copy to the preceding one.</p>
<p>Training such models require a lot of transformations of error signals.
We will focus on the two most used building blocks of ANNs: linear transformations and sigmoid activations.</p>
<p>We verified that backpropagating through a linear transformation $xW = s$ maps the incoming error signal $\delta s$ into</p>
$$\delta x = \delta s W^T$$<p>This operation is itself a linear transformation, which using the polar decomposition can be decomposed into rigid rotations and homotecies: if these homotecies are such that the maximum dilation</p>
$$\|W^T\| = \sup_{\|s\|=1} {\|s W^T\|}$$<p>is large, the outgoing error signal $\delta x$ might have a greater norm that the incoming one.
Imagine now to replace $s$ with $s_{t}$ and $x$ with $s_{t-1}$.
RNNs do these kind of manipulations over and over when moving <em>horizontally</em> through time (i.e. not changing the feature space of the internal memory, but only going back through time through preceding representations in that space): if $\|W^T\|$ is large, eventually there will be some $\delta s_{t-k}$ which norm has exploded!</p>
<p>This is known as the <strong>exploding gradient</strong> problem.
This problem can be effectively tackled by the so-called <em>gradient clipping</em>.
Once BPTT has terminated computing the updated $\delta \theta_{t}$, these vectors are rescaled to have a maximum norm</p>
$$\delta \theta_{t}' =
\begin{cases} \delta \theta_{t} \frac{K}{\|\delta \theta_{t}\|}, &amp;\text{if } \|\delta \theta_{t}\| &gt; K \\
              \delta \theta_{t}, &amp;\text{if } \|\delta \theta_{t}\| \leq K
\end{cases}$$<p>which is, they are cut if they are longer than a fixed <em>clipping threshold</em> $K$.</p>
<p>The second problem arises for sigmoid and hyperbolic tangent transformations.
When we implemented the <code>backward</code> methods of such operations, we pointed out that the maximum value of the sigmoid derivative is $\frac{1}{4}$, while that of the hyperbolic tangent is $\frac{1}{2}$.
This implies that filtering an incoming error signal $\delta h$ back through a sigmoid layer yields an outgoing error signal with norm</p>
$$\|\delta s\| \leq \frac{1}{4}\|\delta h\|$$<p>As for linear transformations, this operations in a RNN are traversed bacward many times, and can quickly drive error signals to zero: this problem is known as <strong>vanishing gradient</strong>.
A solution analogous to the gradient clipping (we could call it <em>gradient blowing</em>) is not feasible since this must be performed <em>at the end</em> of BPTT, but concrete numerical implementations on computers introduce numerical approximations of small numbers to zero.
Thus, before inflating the gradient, if would have been turned to zero; and inflating zero yields zero.</p>
<p>To overcome the vanishing gradient problem, in 1997 Sepp Hochreiter and Jurgen Schmidhuber introduced a system of <strong>gates</strong> that instruct a RNN cell called a <a href="http://www.bioinf.jku.at/publications/older/2604.pdf">Long Short-Term Memory</a> (LSTM) on which information about the past to forget, which information about the present to retain, and which combined information to unleash towards higher level abstractions.</p>
<p>In 2014, a cell called <a href="https://arxiv.org/pdf/1406.1078.pdf">Gated Recurrent Unit</a> (GRU) was proposed, which has shown similar performance to that of LSTMs, but with a simpler architecture.</p>
<p>LSTMs and GRUs are now the basic building blocks of Recurrent Neural Networks.</p>

</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div>
<div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p><a id=overfitregularize></a></p>
<h3 id="Overfitting-and-regularization">Overfitting and regularization<a class="anchor-link" href="#Overfitting-and-regularization">&#182;</a></h3><p>The problem of statistical learning is framed as the search for the best approximation of an unknown map</p>
$$f: X \to Y$$<p>in a constrained space $\mathcal{F}_{\Theta} \subset \mathcal{F}$, and with only an approximate knowledge of the probability measure $\mathbb{P}(dx)$ on the input space $X$.</p>
<p>The approximation of the measure $\mathbb{P}(dx)$ is the empirical distribution dependent on the dataset $\mathcal{D} = \{x_1, x_2, \dots x_D\} \subset X^D$ (corresponding to $D$ samples from $X$):</p>
$$\tilde{\mathbb{P}}(x) = \begin{cases} \frac{1}{D}\sum_{i=1}^{D}\mathbb{1}_x(x_i), &amp;\text{if } x \in \mathcal{D} \\
                                        0, &amp;\text{if } x \notin \mathcal{D} \end{cases}$$<p>The <em>retrievability</em> of $f$ is thus strongly influenced by the dataset $\mathcal{D}$: if it is unbalanced, it will be difficult for the model $\mathcal{F}_{\Theta}$ to discover a good approximation $f_{\theta} \approx f$.</p>
<p>The <strong>generalization capability</strong>, or simply <strong>generalization</strong> of an approximation $f_{\theta}$ is defined in terms of the <em>residual loss over $X$</em></p>
$$\mathcal{G}(\theta) = \int_{X \setminus \mathcal{D}} l(\hat{y}, f_{\theta}(x)) \mathbb{P}(dx)$$<p>where with abuse of notation we denoted with $\mathcal{D}$ the set of unique instances $\{x \, | \, x \in \mathcal{D}\}$.
Of course, this definition is only of theoretical interest, since it assumes perfect knowledge of the measure $\mathbb{P}$, which we cannot access.</p>
<h4 id="Batch-processing">Batch processing<a class="anchor-link" href="#Batch-processing">&#182;</a></h4><p>One first improvement of the backpropagation algorithm is a better approximation of the estimate of the loss functional $L(\theta)$.
During the last lesson, we observed that naive backpropagation relied on the <em>extreme estimate</em></p>
$$L(\theta) \approx l(\hat{y}_i, f_{\theta}(x_i))$$<p>where $(x_i, \hat{y}_i)$ is the sample that is passed to the algorithm.
The algorithm then:</p>
<ol>
<li>computes error signals $\delta \theta_{i}$ based on this estimate,</li>
<li>translates them into a parameters update $\Delta \theta_{i}$</li>
<li>and finally applies them to get a new approximation in the model $f_{\theta_{i+1}} = f_{\theta_{i}+\Delta \theta_{i}}$.</li>
</ol>
<p><strong>Batch backpropagation</strong> (which is different from batch processing that we will explain in a few moments) modifies the algorithm between step 1 and step 2, replacing step 1 with a loop over a minibatch</p>
$$\mathcal{B} = \{x_j \sim \mathcal{U}(\mathcal{D})\}_{j=1,2,\dots,B}$$<p>and averaging over the respective error signal</p>
$$\delta \theta_{i} = \frac{1}{B}\sum_{j=1}^{B} \delta \theta_{ij}$$<p>before trasforming this <em>batch error signal</em> into an update $\Delta \theta_{i}$.
This procedure can be thought as creating $B$ <em>virtual copies</em> of the computational graph representing the model, each processing a different sample $x_j \in \mathcal{B}$.</p>
<p>&lt;img src='figures/batch_forward.png', width=240, height=240&gt;&lt;/img&gt;</p>
<p>Each copy then computes its own error signal $\delta \theta_{ij} = \frac{1}{B}\delta \theta_{j}$, which is just a fraction of the error signal it would have been in simple <em>online sample processing</em>, and communicates it to the parameters, which are <em>shared</em> between all the virtual copies.</p>
<p>&lt;img src='figures/batch_backward.png', width=240, height=240&gt;&lt;/img&gt;</p>
<p>Up to now, this process results more stable than simple <em>online processing</em>, but comes at the cost of doing $B$ forward <strong>alternated</strong> with $B$ backward passes through the graph.
But it is the case that all the $B$ samples can be processed concurrently forward and concurrently backward: this optimization is known as <strong>batch processing</strong>.</p>
<p>We show the example of linear transformations $s = xW$, where $W$ represents a map from $\mathbb{R}^n$ to $\mathbb{R}^m$.</p>
<p>In forward batch processing, the input $x$ is a <strong>stack</strong> of $B$ $n$-dimensional vectors, and multiplying it by $W$ yields a $B$-by-$m$ matrix, a stack of $B$ vectors in $\mathbb{R}^m$.</p>
<p>The backward batch processing procedure is a bit trickier.
As for error signal directed to $\mathbb{R}^n$, i.e. to $x$, it suffices to remember that each $y_{b}$ depends only on $x_{b}$, and we can simply imagine to apply the same procedure as for online backpropagation to each vector independently from the other vectors in the batch:</p>
$$\delta x_{(b)} = \delta s_{(b)} W^T$$<p>&lt;img src='figures/delta_x_batch.png', width=640, height=640&gt;&lt;/img&gt;</p>
<p>When it comes to computing corrections to the shared weights, things are slightly more complicated.
In this case, rigorously computing the tensor derivative shows that the part of error signal directed to $W$ to to the $b$-th sample is</p>
$$\delta W_{b} = T_{b} \odot U_{b}$$<p>where $T_{b}$ has the same shape of $W$ and all the rows equal to $\delta s_{(b)}$, and $U_{b}$ has the same shape of $W$ and all the columns equal to $x_{(b)}^T$.
The error signal directed to $W$ is thus:</p>
$$\delta W = \sum_{b=1}^{B} \delta W_{b}$$<p>&lt;img src='figures/delta_w_batch.png', width=640, height=640&gt;&lt;/img&gt;</p>
<h4 id="Dropout">Dropout<a class="anchor-link" href="#Dropout">&#182;</a></h4><p><a href="https://www.cs.toronto.edu/~hinton/absps/JMLRdropout.pdf">Dropout</a> is a relatively recent technique, developed in 2014 by Geoffry Hinton and his students.
It tackles the regularization problem by preventing <strong>overfitting</strong></p>
<p>Overfitting is an issue of overparametrized models.
When the parameters search space $\Theta$ is too big, the learning procedure can end up selecting a $\theta$ that is perfect for the given dataset; too much perfect.
In fact, the model can capture not only the <em>essence</em> of the map $f: X \to Y$, but also accidental noise in the data.
To summarize: it will fit the training data perfectly, but will have poor performance on unseen instances.</p>
<p>Before each pass of the backpropagation algorithm, dropout randomly selects some neurons from the global network topology and <em>hides</em> them from the forward and backward passes, thus simulating an <em>under-parametrized</em> submodel.</p>
<p>&lt;img src='figures/dropout.png', width=360, height=360&gt;&lt;/img&gt;</p>
<p>This procedure produces an ANN which is in fact an <strong>ensemble</strong> of weaker models to guarantee that none of them overfits the dataset, and thus each will ultimately be able to provide a not-too-complicated answer to the learning query.</p>
<h4 id="Batch-normalization">Batch normalization<a class="anchor-link" href="#Batch-normalization">&#182;</a></h4><p>The last regularization technique we will talk about in this lesson is <a href="https://arxiv.org/pdf/1502.03167.pdf">batch normalization</a>.
As for dropout, batch normalization (abbreviated in <strong>batchnorm</strong>) is a recent technique, developed by Sergey Ioffe and Christian Szegedy in 2015.</p>
<p>Differently from batch processing and dropout, which address more statistically involved issues, batchnorm is concerned with an analytical probelm of ANNs.
To understand what batchnorm does and why, let's first imagine a sequence of two linear transformations interleaved with ReLUs</p>
$$\phi(x) = r(r(xW_1)W_2)$$<p>where we denoted ReLUs with the symbol $r$.
Imagine that, unluckily, all the components of $W_2$ that multiply non-zero elements exiting from the first ReLU are negative: this implies that the second ReLU will shut down every signal directed to following transformations, since every product $r(xW_1)W_2$ will be lesser or equal to zero!
Another issue could be that all the components of $W_2$ that multiply non-zero elements exiting from the first ReLU are positive: this way, all entering signals will be <em>exhalted</em> and passed to the subsequent transfomration.
This could happen again and again, leading to a <strong>numerical drift</strong> which will ultimately harm the network training.</p>
<p>As shown in the picture below, it might become difficult for the network to identify weight vectors substantially different from one another (the purple and the cyan one) when the data points all have high-magnitude and concordant values.</p>
<p>&lt;img src='figures/batch_normalization.png', width=480, height=480&gt;&lt;/img&gt;</p>
<p>To prevent this drift, batch normalization keeps trach of the batch means and variance during batch processing (thus, batchnorm is run in combination with batch processing), in the hope to keep the data points centered around the coordinate system origin and not too scatterd.
In fact, this mean and variance should approximate the correct mean and variance of the population from which the dataset $\mathcal{D}$ was sampled.
This solution eases also the search for <em>sufficiently orthogonal</em> filters, improving feature extraction.</p>

</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div>
<div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<h3 id="References">References<a class="anchor-link" href="#References">&#182;</a></h3><p><strong>RNNs</strong>: this <a href="http://karpathy.github.io/2015/05/21/rnn-effectiveness/">blog post</a> by Andrej Karpathy illustrates many applications of RNNs; Denny Britz's <a href="http://www.wildml.com/2015/09/recurrent-neural-networks-tutorial-part-1-introduction-to-rnns/">blog post</a> guides you through another implementation of RNNs and LSTMs from scratch.</p>
<p><strong>LSTMs</strong>: I strongly suggest to read Christopher Olah's <a href="http://colah.github.io/posts/2015-08-Understanding-LSTMs/">blog post</a> about Long Short-Term Memories.</p>

</div>
</div>
</div>
    </div>
  </div>
</body>

 
</html>