diff --git a/index.html b/index.html
index 8ce8ca7..51f32c6 100644
--- a/index.html
+++ b/index.html
@@ -3,4 +3,4 @@
 <span class=navbar-toggler-icon></span></button><div class="collapse navbar-collapse text-center" id=navigation><ul class="navbar-nav ml-auto"><li class=nav-item><a class="nav-link text-dark" href=/DS250-Cannon>Home</a></li><li class=nav-item><a class="nav-link text-dark" href=/DS250-Cannon/projects>Projects</a></li><li class=nav-item><a class="nav-link text-dark" href=/DS250-Cannon/contact>Contact</a></li><li class=nav-item><a class="nav-link text-dark" href=/DS250-Cannon/course-materials>Materials</a></li><li class="nav-item dropdown"><a class="nav-link dropdown-toggle text-dark" href=# role=button data-toggle=dropdown aria-haspopup=true aria-expanded=false>Navigate</a><div class=dropdown-menu><a class=dropdown-item href=/DS250-Cannon/slides>Slides</a>
 <a class=dropdown-item href=/DS250-Cannon/course-materials/syllabus/>Syllabus</a>
 <a class=dropdown-item href=/DS250-Cannon/faq>FAQ</a></div></li></ul></div></div></nav><div class="container section"><div class=row><div class="col-lg-8 text-center mx-auto"><h1 class="text-white mb-3">CSE 250: Data Science Programming</h1><p class="text-white mb-4">Using pandas, Altiar, scikit-learn, and NumPy to program with data</p><div class=position-relative><input id=search class=form-control placeholder="Have a question? Just ask here or enter terms">
-<i class="ti-search search-icon"></i><script>$(function(){var projects=[{value:"Introduction",label:"<p>A competent student should be able to finish the exercises within 60 minutes. You should work through it on your own. This serves as an assessment of your understanding of the assigned readings.\nBefore you start Make sure you have installed VS-code, pandas, and altair on your computer. You can install these package by typing this line in the terminal.\npip install pandas altair\nOR if you have more than one version of python\npip3.9 install pandas altair\npip3.9 indicates the version of python you are installing the packages to.\nPart 1 Get familiar with your tools Programming involves a lot of research. Unlike subjects like Mathematics or History, we are not required to remember every single function and its usage. It is natural for experienced programmers to look for answers on the internet, books, even from other people\u0026rsquo;s code. Programming will be extremely frustrating if we are not allowed to do web searches, so please get familiar with the tools you have and use them often.\nOffical Documentation This should be your first resort for understanding any code\/function. Scanning the documentation of a function will allow you to get an overview of its usage.\nHere is a link to the documentation of the assign() function:\n(https:\/\/pandas.pydata.org\/docs\/reference\/api\/pandas.DataFrame.assign.html)\nExample of assign() (as shown in the documentation)\nimport pandas as pd df1 = pd.DataFrame({\u0026#39;temp_c\u0026#39;: [17.0, 25.0]}, index=[\u0026#39;Portland\u0026#39;, \u0026#39;Berkeley\u0026#39;]) df2 = df1.assign(temp_f=df1.temp_c * 9 \/ 5 \u002b 32) Exercise 1: After reading the documentation for assign(), write a short paragraph to explain assign() as if you were talking to someone with zero programming experience (use the example above to help you explain assign()).\n What is the difference between df1 and df2? How was df2 derived from df1?)  Online textbook It pains us to see students would rather be stuck at problems for hours yet they refuse to use the textbook. This is another very useful resource since this is designed for this class. link to the textbook: (https:\/\/byuidatascience.github.io\/python4ds\/)\nExercise 2: Locate the section where the textbook talks about query() and answer these questions.\n What function in R\u0026rsquo;s dplyr is equivalent or comparable to query() in pandas (You should include the section number in your answer)? What is the easiest mistake for python beginner to make that was shown in the text about query() (You should include the section number in your answer)?  The internet Google is a programmer\u0026rsquo;s friend. Get used to googling thing, in fact, you want to be an expert in googling\n Question that cannot be answered by the textbook and documentation? Google it. A function you have never seen before? Google it. An error in your code? Google it.  Exercise 3: Provide at least 2 extra resources you could find about the pandas function drop() on the internet.\nTutor, TA (Through slack, zoom, or in-person) We want to help you with your work; we want to answer your questions; but most importantly, we want to help you succeed in this class. That will require you to put in the necessary time in understanding the readings, coding and debugging. When you ask us a question, we expect that you have read the documentation, searched the textbook, and done your own research. Then we can be most helpful and can provide insights on top of your understanding.\nExamples of bad questions  How does drop() work? We will ask you to read the documentation for drop(). How do you make a table in a markdown file? We will refer you to the textbook. I don\u0026rsquo;t want these columns in my data, how can I drop them? We will ask you if you have found any things on the internet.  Examples of good questions  I am still confused about the syntax of drop(). After reading the documentation, this is my understanding of the function\u0026hellip; . What am I missing? I tried making a table in markdown (show code), it is still not giving me what I want, how can I fix this? I am trying to drop these columns in my dataframe, I think drop() is what I am looking for. Am I in the right direction? If not, what keywords should I be googling?  Exercise 4:\nUsing the code and tools mentioned above, finish question 4 and 5 under 3.2.4 in the textbook.(use the data in mpg for your plot):\n# library import import pandas as pd import altair as alt # data import url = \u0026quot;https:\/\/github.com\/byuidatascience\/data4python4ds\/raw\/master\/data-raw\/mpg\/mpg.csv\u0026quot; mpg = pd.read_csv(url)   Question 4: Make a scatterplot of hwy vs cyl.\n  Question 5: What happens if you make a scatterplot of class vs drv? Why is the plot not useful?\n  After you have completed this skill builder with your team (or on your own) then compare your work to our script    See the script.   </p><p></p>",url:"https:\/\/byuistats.github.io\/DS250-Cannon\/skill_builders\/introduction\/"},{value:"Day 1: Welcome",label:"<p>Welcome to DS 250!  Teacher: Paul Cannon TA: Bracken Sant (san20050@byui.edu)  Announcements  Devotional Computing Lab 4:30PM - 6:30PM all weekdays except Wednesday. Saturday from 10AM-12PM  Slack channel #tutoring_lab   Data Science Society - Wednesday\u0026rsquo;s at 6PM, STC 394 Math Department Opening Social - Thursday 11:30 RKS 229  What is a Data Scientist? A Data Scientist has a C\u002b Talent Stack Class Structure  Problem Solving Improved coding skills Effective written\/visual communication Collaboration Timeliness and communication with \u0026ldquo;the boss\u0026rdquo;  Syllabus\nGot Slack? Are we all on the Slack channel? Follow the Slack invitation that is waiting in your student email. If you don\u0026rsquo;t see an invite, you can join through this link and then ask \u0026ldquo;@Paul Cannon\u0026rdquo; to add you to the class channel.\nWho are you?  Introduce yourself and learn the names\/majors\/origin story of your group members. Make a plan to get help this semester. How will you contact each other? Some ideas: Slack, I-Learn, emails, group texts, etc. If you were independently wealthy, what would you be doing right now? Would you change majors? Highlights of 2022  Problem Solving This is not a \u0026ldquo;see and repeat\u0026rdquo; programming class!\nHow would you go about fixing my motorcycle? Learn how to ask for help (1 hr rule)  Getting started on Project 0 Setting up your Programming Snvironment  Download Visual Studio Code Download Python v (3.10.8)  Be sure to select the \u0026ldquo;Add to Path\u0026rdquo; option during the install process  If you\u0026rsquo;re on a Mac be sure to click on \u0026ldquo;Install Certificates\u0026rdquo; at the end of the install   Install the Python packages and VS Code extensions you need (see this page)  pip install pandas pip install numpy pip install jupyter pip install tabulate pip install altair   Install Quarto CLI Quatro Instructions Start looking at Project 0 Complete the \u0026ldquo;Methods Checkpoint\u0026rdquo;  Installing Packages and Extensions Learn how to install packages by reading the assigned material and by watching the video tutorial on this page.\nThe readings mention a lot of different packages. For Project 0, you need to install at least pandas, altair, numpy, and jupyter.\nThe readings will also mention two VS Code extensions you need to install.\nA note on Jupyter Notebooks vs. Interactive Python Window The textbook will show you how to use VS Code\u0026rsquo;s interactive python windows and Quatro. Feel free to use Jupyter Notebooks.\nWe will do write-ups in Quarto, though, which can be rendered as a PDF or HTML\nIntroduction to Brother Cannon    What do you want to know?    What is a data scientist?    Brother Hathaway\u0026rsquo;s definition:\n A blend of programmer, statistician, and communicator that burns with curiosity.\n My definiton for DS 250:\n Someone who can extract insights from data and then communicate those insights with clarity.\n Learn more about the BYU-Idaho data science program here.\n   What is data science programming?    Data scientists write code as a means to an end, whereas software developers write code to build things. Data science is inherently different from software development in that data science is an analytic activity, whereas software development has much more in common with traditional engineering.\nData scientists tackle problems such as identifying fraudulent transactions, or predicting which employees are likely to leave a company. Software developers can take the data scientists models and turn them into fully functioning systems with production-quality code. Software developers tackle problems like getting an algorithm to run more efficiently, or building user interfaces.\n   Course Outcomes    Upon completing this course, you will be able to use data-driven programming in Python to handle, format, and visualize data. We will introduce you to data wrangling techniques (panadas), analytical methods (scikit-learn), and the grammar of graphics (Altair). Specifically, as a successful learner, you will be able to:\n Use functions, data structures, and other programming constructs efficiently to process and find meaning in data. Programmatically load data from various types of data sources, including files, databases, and remote services. Use data manipulation libraries to perform straightforward analysis, produce charts, and prepare data for machine learning algorithms. Use machine learning libraries to discover insights, make predictions, and interpret the success of these algorithms. Collaborate and share your work with industry-leading tools.     BYU-Idaho Mission Statement     Brigham Young University-Idaho was founded and is supported and guided by The Church of Jesus Christ of Latter-day Saints. Its mission is to develop disciples of Jesus Christ who are leaders in their homes, the Church, and their communities.\n  How would you describe a leader? What makes a leader powerful? What does a leader do with insights?  An example of a good leader.\nWhat (or who) is truth?\n   ## Course Format and Grading How hard is this class going to be?    The reality of CSE 250:\n We have done all we can to ensure that this is a 2-credit course for the average student. That means that we expect 4-6 hours outside of class for the average student to achieve an A. You have to put in the time if you want to build skills. The course is necessarily creative in nature. That fact usually makes it feel more challenging. We will be asking you to learn to write creative data science python code. If you have any concerns, please talk with me!     What is the structure of CSE 250?    The class uses 7 projects to teach data science programming in Python using pandas, Altair, scikit-learn, and numpy.\n Projects Syllabus     How do I get the grade I want?     Specification Grading Grading structure Competency Elements  Introduction Project \u0026mdash;\u0026mdash;\u0026mdash;\u0026mdash;\u0026mdash;\u0026mdash;\u0026mdash;\u0026mdash;\u0026mdash;\u0026mdash;\u0026mdash;\u0026mdash;\u0026mdash;\u0026mdash;\u0026mdash;\u0026mdash;\u0026mdash;\u0026mdash;\u0026mdash;\u0026mdash;\u0026gt;\nWhat is the goal?    Completing the introduction project will set you up for success the rest of the semester. The workflow followed in the introduction project (loading packages, writing code, saving images, compiling a final report) will be the same for every other project . If you have questions about this project, you need to seek help.   What exactly do I need to submit?    Make sure you carefully read the project instructions.\nYou will submit a single .pdf file to I-Learn. This pdf file should contain an project summary, your answers to the grand questions (including the plot you saved with altair_saver), and an appendix where you copy and paste your commented Python code.\n   --------------------------------------------------------   ----------------------------------------------- </p><p></p>",url:"https:\/\/byuistats.github.io\/DS250-Cannon\/slides\/introduction\/day01\/"},{value:"pandas and Altair",label:"<p>For this skill builder, we are exploring some important functions in the package of pandas and Altair. DS programming requires a lot of data wrangling. Using the proper functions, we can create concise and comprehensive codes. You should be exposed to a few functions through the readings this week.\nYou may want to at least scan the readings before beginning this task since this serves as an assessment of your understanding of the assigned readings. A prepared student should be able to finish the exercises within 60 minutes. You should work through it on your own.\nBefore you start Make sure you have installed VS-code, pandas, and Altair on your computer. You can install these packages by typing this line in the terminal:\npip install pandas altair\nOR if you have more than one version of python:\npip3.9 install pandas altair\npip3.9 indicates the version of python you are installing the packages to.\nData import Run the following code to import the data we need for this skill builder:\n# package import import numpy as np import pandas as pd import altair as al # data import dat = pd.read_csv(\u0026#34;https:\/\/vincentarelbundock.github.io\/Rdatasets\/csv\/AER\/Guns.csv\u0026#34;) Make sure the variable dat is correctly assigned in your environment and finish the following exercises. You can read the documentation of the data on this page - https:\/\/vincentarelbundock.github.io\/Rdatasets\/doc\/AER\/Guns.html\nExercise 1 One of the first things we can do to a freshly imported data is to check its columns. This will help us understand the basic structure of the dataframe(table).\n Using one line of code, select all the columns in dat, assign it to a variable called col_list.\n  Hint Every dataframe has an attribute \u0022columns\u0022. Accessing this attribute will give you a list of all column names  We often want to know the dimension of a dataframe. How many columns are in the dataset? How many rows are in the dataset?\n Using one line of code, show the number of columns and rows in dat.\n  Hint Every dataframe has an attribute \u0022shape\u0022. Accessing this attribute will give you the dimension of a datafarme  Now run dat.head(). It will print out the first 5 rows of data in dat.\n Just from looking at the output, what column(s) seems to be redundant with the row number?\n  Hint There is one column that serves as nothing but a row counter, that columns is redundant.  Exercise 2 After a brief investigation of the data, we will clean up the data. By cleaning up, we are trying to filter down dat so this only holds data we need. We will first get rid of the extra column we found in the previous excercise.\n Using one line of code, drop the redundant column using the variable col_list (created in excercise 1)\n  Hint Use `drop()`. Understand what \u0026ldquo;axis\u0026rdquo; is as a parameter of drop().\nYour function should looks like this:\ndat.drop([col_list[_]], axis = _)\nfill the \u0026ldquo;_\u0026quot;\u0026rsquo;s with the correct values and assign the output to dat.\n Don\u0026rsquo;t forget to save the changes in dat. Run dat.head() to make sure the column is dropped in dat.\nExercise 3 We have filtered dat vertically by dropping a column. Now we will try to filter dat horizontally, meaning we will get rid of some the rows.\nWe can do that by applying a condition to dat. A condition is an expression that can be evaluated as True\/False. For example, 8 \u0026gt; 5 is an expression that evaluates to be True. This is trivial because 8 will always be greater than 5.\nRun the code below:\n what is the difference between exp1 and exp2?\n exp1 = 8 \u0026gt; 5 exp2 = dat.violent \u0026lt; 300  Hint Try type() on else variable OR calling else variable.  Run ths code below:\n By putting dat.violent \u0026lt; 300, and the violent column from dat into a dataframe, what is the relationship between the two columns?\n exp = pd.DataFrame({\u0026quot;dat.violent \u0026lt; 300\u0026quot; : exp2, \u0026quot;violent value from dat\u0026quot; : dat.violent}) exp  Hint Try computing `dat.violent[n]  Using query(), filter down the dat so that it only contains the data for idaho\n  Hint query() takes in expressions and filters down data.  Don\u0026rsquo;t forget to save the changes in dat. Run dat.shape() to make sure the there are 23 rows and 13 columns.\nExercise 4 Besides filtering, we can manipulate the data by adding new data to it. By adding a new column to the data, we assign a new value to each row.\n Using assign(), create a new column that show the ratio between murder rate and violent rate.\n  Hint Use assign() You see get the ratio by computing this code:\ndat.murder\/dat.violent\n Exercise 5  Create a scatter plot that shows the relationship between murder rate and violent rate for the state of Idaho. Your chart should show murder rate as the x-axis, violent as the y-axis.\n  Hint Can you mimic this plot? (https:\/\/altair-viz.github.io\/gallery\/scatter_tooltips.html)\n  For an extra push Exercise 6  Using a line of code, filter down the data set so that it only shows the data in years between 1993 and 1997.\n Exercise 7  Create a line chart that show prisoners numbers for the state of Idaho, Utah, and Oregon.\n Your chart should show year as the x-axis, prisoner as the y-axis, states as different colours, along with an appropriate title.\nExercise 8  Without using query(), finshed the data wrangling in question 2,5 and 6.\n After you have completed this skill builder with your team (or on your own) then compare your work to our script    See the script.   </p><p></p>",url:"https:\/\/byuistats.github.io\/DS250-Cannon\/skill_builders\/pandas_altair\/"},{value:"JSONs \u0026 missing",label:"<p>UFO Sightings Data Link to json file\nExercise 1 Read in the json file as a pandas dataframe. After reading in the data, you\u0026rsquo;ll want to explore it and gain some intuition. Exploring data is a very important step — the more you know about your data the better! Answer the following questions to gain some insight into this dataset.\n How many rows are there? How many columns? What does a row represent in this dataset? What are the different ways missing values are encoded? How many np.nan in each column?  Some useful code for exploring data\n# Object\/Categorical Columns data.column_name.value_counts(dropna=False) data.column_name.unique() # Numeric Columns data.column_name.describe() # Counting missing values data.isna().sum() # Creates boolean dataframe and sums each column  Exercise 2 After learning different ways our data encodes missing values, now we will neatly manage them. There are many techniques we can use to handle missing values; for example, we can drop all rows that contain a missing value, impute with mean or median, or replace missing values with a new missing category. We will use some of these techniques in this exercise.\n shape_reported - replace missing values with missing string. distance_reported - change -999 values to np.nan. (-999 is a typical way of encoding missing values.) distance_reported - fill in missing values with the mean (imputation) were_you_abducted - replace - string with missing string.  The first 10 rows of your data should look like this after completion of the above steps.\n    city shape_reported distance_reported were_you_abducted estimated_size     0 Ithaca TRIANGLE 8521.9 yes 5033.9   1 Willingboro OTHER 7438.64 no 5781.03   2 Holyoke OVAL 7438.64 no 697203   3 Abilene DISK 7438.64 no 5384.61   4 New York Worlds Fair LIGHT 6615.78 missing 3417.58   5 Valley City DISK 7438.64 no 4280.1   6 Crater Lake CIRCLE 7377.89 no 528289   7 Alma DISK 7438.64 missing 4772.75   8 Eklutna CIGAR 5214.95 no 4534.03   9 Hubbard CYLINDER 8220.34 missing 4653.72    Some useful code for filling in missing data\ndata.column_name.replace(..., ..., inplace=True) data.column_name.fillna(..., inplace=True)  Exercise 3 Create a table that contains the following summary statistics.\n median estimated size by shape mean distance reported by shape count of reports belonging to each shape  Your table should look like this:\n   shape_reported median_est_size mean_distance_reported group_count     CIGAR 5899.68 6520.21 3   CIRCLE 266002 7408.26 2   CYLINDER 4550.58 8039.49 2   DISK 4581.8 7516.39 16   FIREBALL 5407.22 7097.78 3   FLASH 6108.34 7438.64 1   FORMATION 5104.4 8708.32 2   LIGHT 3850.25 7636.09 2   OTHER 4699.4 7473.98 4   OVAL 4943.63 7787.24 4   RECTANGLE 3668.1 6054.62 2   SPHERE 5076.78 7206.55 6   TRIANGLE 5033.9 8521.9 1   missing 250153 7438.64 2    Some useful code for grouping and getting summary statistics\n(data.groupby(...) .agg(..., ..., ...))  Exercise 4 The cities listed below reported their estimated size in square inches, not square feet. Create a new column named estimated_size_sqft in the dataframe, that has all the estimated sizes reported as sqft. (Hint: divide by 144 to go from sqin -\u0026gt; sqft)\n Holyoke Crater Lake Los Angeles San Diego Dallas  The head of your data should look like this.\n    city shape_reported distance_reported were_you_abducted estimated_size estimated_size_sqft     0 Ithaca TRIANGLE 8521.9 yes 5033.9 5033.9   1 Willingboro OTHER 7438.64 no 5781.03 5781.03   2 Holyoke OVAL 7438.64 no 697203 4841.69   3 Abilene DISK 7438.64 no 5384.61 5384.61   4 New York Worlds Fair LIGHT 6615.78 missing 3417.58 3417.58   5 Valley City DISK 7438.64 no 4280.1 4280.1   6 Crater Lake CIRCLE 7377.89 no 528289 3668.68   7 Alma DISK 7438.64 missing 4772.75 4772.75   8 Eklutna CIGAR 5214.95 no 4534.03 4534.03   9 Hubbard CYLINDER 8220.34 missing 4653.72 4653.72    Some useful code to fix the rows reported in sqin\nnp.where(..., # Condition ..., # If condition is true ...) # If condition is false  After you have completed this skill builder with your team (or on your own) then compare your work to our script    See the script.   </p><p></p>",url:"https:\/\/byuistats.github.io\/DS250-Cannon\/skill_builders\/json_missing\/"},{value:"SQL \u0026 databases",label:"<p>Skill builder (relational database) For this skill builder, we are exploring some important topics in relational databases. This exercise will require you to create SQL queries through python. You may want to at least scan the readings before beginning this task since this serves as an assessment of your understanding of the assigned readings.\nA competent student should be able to finish the exercises within 75 minutes.\nBefore you start Make sure you have installed VS-code, pandas, and Altair on your computer.\nAlso make sure you have gone through the tutorial on under course materials called SQL for Data Science: we assume that you have a connection to your data.\nExercise 1 Readme file A database can consist of more than one table\/data set. A relational database consists of tables\/data sets that share columns. These shared columns then establish the relationship between the tables, thus the name relational database. The relations are sometimes not easily found and they require careful investigations.\nTo understand what is in a relational database, we can start with understanding the tables and the columns within.\nHere is a link to the readme file of the baseball database.\n What is the name of the table that records data about pitchers in the regular seasons?\n  What do the HR and HBP columns mean in that table respectively?\n Excercise 2 SELECT and FROM The simplest SQL query is a query with SELECT and FROM. These are the keywords you will see again and again in SQL. Usually, when constructing a more complex query, it is easier to identify what goes into these two clauses first.\n Create a query that shows all columns from the table you found in Exercise 1, save the dataframe in a variable \u0026ldquo;pitch\u0026rdquo;\n You script should look something like:\nresult = pd.read_sql_query( \u0027SELECT _______ FROM _______\u0027, con) results Excercise 2 WHERE The WHERE keyword allows us to filter down the table horizontally (fewer rows).\nIt goes after SELECT and FROM.\n Using a SQL query, select all rows in the same table where HR is lesser than 10 and gs is greater than 25.\n  Find out what the columns mean and explain your query in words\n Excercise 3 ORDER BY ORDER BY sort the table you select by one or more columns and goes after WHERE\n Using the same query in exercise 2, edit it so that the table is ordered by the year of the season(nearest to furthermost) and the player ID(alphabetically).\n Excercise 4 Joins Joins are used when you wish to create a new table through two different tables. Keep in mind that you have to identify the relationship between two tables before you can correctly join them.\nJOIN goes between FROM and WHERE.\n Identify the shared columns (keys) and join the table in exercise 2 with the salaries table, then filter the data so that it shows only pitchers in the year 1986.\n You should get a dataframe with 306 rows.\nExercise 5 Group by Group by is a keyword we use to lower the level of granularity of a table. Meaning we are combining rows into one by the given column(s).\nCreate a query that captures the number of pitchers the Washington Nationals used in each year, then sort the table by year\nYou should get a dataframe with 23 rows.\nFor the overachievers Excercise 6 Research the order of operations for SQL and put the following keywords in that order.\n SELECT FROM JOIN WHERE HAVING ORDER BY GROUP BY LIMIT  After you have completed this skill builder with your team (or on your own) then compare your work to our script    See the script.   </p><p></p>",url:"https:\/\/byuistats.github.io\/DS250-Cannon\/skill_builders\/relational_data\/"},{value:"Machine Learning",label:"<p>Intro to Titanic Machine Learning Skill Builder Link to data\nFor this skill builder, we\u0026rsquo;ll be putting our machine learning hats on. We\u0026rsquo;ll be creating a model that predicts whether a passenger survived. With machine learning, there is a lot of jargon! It can be quite overwhelming at times. This skill builder attempts to keep things basic and simple. With that being said, there are some terms that are important to understand. Let\u0026rsquo;s look at the first few rows of our dataset before proceeding with the definitions.\nThe titanic dataset will be used for examples of each definition.\n   survived pclass sex age siblings_spouses_aboard parents_children_aboard fare     0 3 1 22 1 0 7.25   1 1 0 38 1 0 71.2833   1 3 0 26 0 0 7.925   1 1 0 35 1 0 53.1   0 3 1 35 0 0 8.05    Important Terms:  features: measurable property of the object you\u0026rsquo;re trying to predict. We use this information to predict our target of interest.  Example: pclass, sex, age, siblings_spouses_aboard , parents_children_aboard, fare columns are all examples of different features. Synonyms: attributes, explanatory variables, independent variables, variables, X\u0026rsquo;s, covariates   target: the feature that you are wanting to gain more insight into. The thing you are trying to predict.  Example: in the titanic dataset our target is survived Synonyms: label, dependent variable, y   train set: Usually 70% of the rows from the original dataset are randomly sampled to create this training data. It\u0026rsquo;s used by the algorithm, to determine, or learn, the optimal combinations of variables that will generate a good predictive model  Example: Random sample of 70% of the original titanic dataset rows Synonyms: training data, train data, X_train, y_train   test set: Usually the remaining 30% of the rows in the original dataset are used to create this dataset. The testing data is a set of rows used only to assess the performance (i.e. generalization) of a model. To do this, the final model is used to predict classifications of examples in the test set. Those predictions are compared to the examples\u0026rsquo; true classifications to assess the model\u0026rsquo;s accuracy.  Example: Random sample of 30% of the original titanic dataset rows Synonyms: testing data, test data, X_test, y_test   evaluation metrics: A statistic that tells you how well your predictions align with the actual values. Other words, tells you how good your model is.  Example: Accuracy, Precision, Recall, MSE, MAE, Rsquared Synonyms: performance metric    Again, this is a very light and oversimplified treatment of machine learning. The purpose of this project is to help you understand the main concepts of ml and walk you through the process of building a machine learning model. A simplified work flow of a machine learning project is shown below. Spend some time getting familiar with this flow \u0026amp;mdash as you are about to code it\u0026hellip; Exciting!\nNote in order to do this skill builder you will need to have scikit-learn installed on your machine. Run the following command in your terminal if you haven\u0026rsquo;t already.\npip install scikit-learn\nData Link to csv file\nExercise 0 (Imports and Loading in data) # Loading in packages import pandas as pd import numpy as np import altair as alt from sklearn.model_selection import train_test_split from sklearn.ensemble import RandomForestClassifier from sklearn.metrics import accuracy_score # Loading in data data = pd.read_csv(___)  Exercise 1 Create a chart exploring the relationship between age and survived in the titanic dataset. A strip plot, density plot, or boxplot might be useful here. Below is an example of a density plot. Feel free to replicate this chart or create your own.\nThe purpose of making this chart is to explore the relationships between a feature and the target. We want to see if the feature contains predictive information about the target. This is a large part of machine learning called Exploratory Data Analysis that should never be skipped! Spend time getting to know your features and how they interact with other features and the target.\n Exercise 2 Build a random forest model that is able to predict whether a passenger survived. This exercise is the bulk of the skill builder and contains several steps.\nStep 0: Split the data into X and y variables The X variable will contain all your features\n# Removes the target and keeps all features X = data.drop(___, axis=1) The y variable will hold the target\n# Selects the target column y = data[\u0026#39;___\u0026#39;] Step 1: Split data into train and test sets The train_test_split function is useful for this task. Review the train_test_split function documentation\n# Splitting X and y variables into train and test sets using stratified sampling X_train, X_test, y_train, y_test = train_test_split(___, ___, test_size=0.3, random_state=24, stratify=y) Step 2: Train the model Explore the RandomForestClassifier documentation for the RandomForestClassifier. It\u0026rsquo;s not necessary to understand the inner workings of the Random Forest algorithm for this class - just learn the syntax of fitting the model.\n# Creating random forest object rf = RandomForestClassifier(random_state=24) # Fit with the training data rf.fit(___, ___) Step 3: Use test set to make predictions # Using the features in the test set to make predictions y_pred = rf.predict(___) Step 4: Compare test set predictions to actual values. Calculate the accuracy. # Comparing predictions to actual values accuracy_score(___, ___)  Exercise 3 What is the most important feature in making predictions? Why do you think this is?\nCreate a table that shows the feature importances in descending order. The random forest classifier has a feature importances attribute. It can be accessed by rf.feature_importances_. The table should look something like this.\n   feature names importances     fare 0.288051   sex 0.281853   age 0.266491   pclass 0.0814224   siblings_spouses_aboard 0.0475633   parents_children_aboard 0.034619    After you have completed this skill builder with your team (or on your own) then compare your work to our script    See the script.   </p><p></p>",url:"https:\/\/byuistats.github.io\/DS250-Cannon\/skill_builders\/ml_sklearn\/"},{value:"Munging data",label:"<p>Intro to cleaning movies data Link to the data\nThis skill builder focuses on munging (formatting) data into a machine learning ready dataset. We will be using an IMDB Ratings dataset. It contains columns that are categorical. Sklearn cannot handle columns that are strings, so we need to convert these into a numerical representation. We accomplish this by either one hot encoding, label encoding, or taking just one value of the range provided. There are many other ways to represent these columns as numbers, but they are beyond the scope of this course.\nOnce you\u0026rsquo;ve converted all columns to numeric, in an intelligent way, you will be asked to recreate a graph using altair. Here is the head of the data you will be working with. Enjoy!\n   star_rating content_rating genre duration box_office_rev major_hit     9.3 R Crime 142 €1924521976 - €1925521976 no   9.2 R Crime 175 €177034987 - €178034987 no   9.1 R Crime 200 €2617541398 - €2618541398 no   9 PG-13 Action 152 €996115723 - €997115723 no   8.9 R Crime 154 €1172054364 - €1173054364 no    Data Link to csv file: ...\n Exercise 0  Grab the high range value for each movie and put it into a new column called high_range_rev.  Make sure the data type of this new column is numeric!!   Remove the box_office_rev column from the dataset.  The .str.split() and .astype() methods might be of use! Also, to get the euro sign just copy it from here, €, and put it in your code.\nThe first 5 rows of the resulting dataframe should look like this\n   star_rating content_rating genre duration major_hit high_range_rev     9.3 R Crime 142 no 2345444803   9.2 R Crime 175 no 2182412593   9.1 R Crime 200 no 1604872807   9 PG-13 Action 152 no 284317976   8.9 R Crime 154 yes 1791932201     Exercise 1 Convert the major_hit column to 1\/0\u0026rsquo;s. yes -\u0026gt; 1 and no -\u0026gt; 0. Again, there are several ways to accomplish this. Using our old friend np.where is probably the easiest though.\nThe first 5 rows of the resulting dataframe should like this\n   star_rating content_rating genre duration major_hit high_range_rev     9.3 R Crime 142 0 1925521976   9.2 R Crime 175 0 178034987   9.1 R Crime 200 0 2618541398   9 PG-13 Action 152 0 997115723   8.9 R Crime 154 0 1173054364     Exercise 2 Convert the content_rating column using label encoding. We\u0026rsquo;re using label encoding in this case because the movie ratings already have a natural ordering to them. We will replace each rating with a number in it\u0026rsquo;s natural ascending order.\nTo be more specific, here is how we will do it.\n G: 0 PG: 1 PG-13: 2 R: 3  A dictionary and the .map() method could be useful for this exercise. There are other ways of tackling this problem though. Be creative!\nThe first 5 rows of the resulting dataframe should look like\n   star_rating content_rating genre duration major_hit high_range_rev     9.3 3 Crime 142 0 1925521976   9.2 3 Crime 175 0 178034987   9.1 3 Crime 200 0 2618541398   9 2 Action 152 0 997115723   8.9 3 Crime 154 0 1173054364     Exercise 3 The last column that we need to take care of is genre. We will use one hot encoding for this. Make sure to ONLY one hot encode the genre column!\nA useful function for one hot encoding is pd.get_dummies(). I recommend checking out the documentation.\nThe resulting dataframe should look like the following example; don\u0026rsquo;t worry if your high_range_rev column turned into scientific notation—Pandas does this sometimes.\n    star_rating content_rating duration major_hit high_range_rev genre_Action genre_Adventure genre_Animation genre_Biography genre_Comedy genre_Crime genre_Drama genre_Family genre_Fantasy genre_Horror genre_Mystery genre_Sci-Fi genre_Thriller genre_Western     0 9.3 3 142 0 1.92552e\u002b09 0 0 0 0 0 1 0 0 0 0 0 0 0 0   1 9.2 3 175 0 1.78035e\u002b08 0 0 0 0 0 1 0 0 0 0 0 0 0 0   2 9.1 3 200 0 2.61854e\u002b09 0 0 0 0 0 1 0 0 0 0 0 0 0 0   3 9 2 152 0 9.97116e\u002b08 1 0 0 0 0 0 0 0 0 0 0 0 0 0   4 8.9 3 154 0 1.17305e\u002b09 0 0 0 0 0 1 0 0 0 0 0 0 0 0     Exercise 4 Recreate this graph as best you can. You\u0026rsquo;ll need to use the original data that specifies the actual rating.\nAfter you have completed this skill builder with your team (or on your own) then compare your work to our script    See the script.   </p><p></p>",url:"https:\/\/byuistats.github.io\/DS250-Cannon\/skill_builders\/munging\/"},{value:"GitHub and git",label:"<p>Complete the Hello World GitHub Guide\n</p><p></p>",url:"https:\/\/byuistats.github.io\/DS250-Cannon\/skill_builders\/git_github\/"},{value:"Week 1: Introduction",label:"<p>  Introduction Project Syllabus   </p><p></p>",url:"https:\/\/byuistats.github.io\/DS250-Cannon\/slides\/introduction\/"},{value:"DS250",label:"<p></p>",url:"https:\/\/byuistats.github.io\/DS250-Cannon\/"},{value:"Frequently Asked Questions",label:"<p> What do you mean by data science programming?    Most likely, you have had 1-2 courses of programming before you have taken CSE 250. Unlike traditional computer science courses, CSE 250 uses Python in an interactive mode instead of building programs. The data provider usually has some big questions that need answering; However, there are hundreds of little issues and responses along the way. We use programming to facilitate this investigation.\nThere are similarities with User Experience Designers. In our case, we don\u0026rsquo;t get to ask users about their experience. We use programming to ask data about its background, and each data set has its own history. We want our analysis to mold to that experience. You can think of data science programming like a first date with your data. You can\u0026rsquo;t write one long program nieve of the issues and nuances each living data set provides.\n   How does CSE 250 compare to CSE 350 or Math 335?    The two courses have similarities. You could think of CSE 250 as an introduction to data wrangling and visualization. Both classes use real-world data and are built around data science projects. There are some critical differences between the two courses.\n In this course, we use Python, and CSE 350 uses R. We are introducing the principles of data science programming in CSE 250. The course is only 2-credits. CSE 250 is intended to introduce visualization, wrangling, and modeling.     How does CSE 250 prepare me for CSE 350, Math 335 and CSE 450?    You will be comfortable with interactive programming and have an introduction to the principles of data formats for data science applications. You will be introduced to principles related to machine learning, data wrangling, and data visualization.   What programming languages do we use in this course?    The course is done using Python. We focus on the pandas and Altair packages.   What are the prerequisites for this course?    Using the new courses at BYU-I, the prerequisite is CSE 110. However, if you have experience programming from other classes, you most likely are prepared for this course.   Why Python instead of R?    The computer science and software engineering programs at BYU-I use Python as their foundational courses. The standard student will have some experience with Python before CSE 250. Python is an essential programming language for data scientists, and we already have CSE 350\/Math 335, which is taught in R.   What is pandas?    pandas is the foundational data science package in Python. If you are using tabular data you will be in pandas.   Why are we using Altair instead of Seaborn or Matplotlib?    Matplotlib was the first visualization package to gain a following in Python. Seaborn is built on top of Matplotlib. Many data scientists use both in their work—neither leverage the grammar of graphics as developed by Leland Wilkinson. Altair is built on Vega-Lite, which uses the Vega visualization grammar. It is declarative and actively developed. We expect that it will become the predominant visualization package in Python (https:\/\/youtu.be\/FytuB8nFHPQ and https:\/\/youtu.be\/vTingdk_pVM).   </p>",url:"https:\/\/byuistats.github.io\/DS250-Cannon\/faq\/"},{value:"Skill Builders",label:"<p>These short activites are provided for you to gain some additional skills to help with the class projects.\n</p>",url:"https:\/\/byuistats.github.io\/DS250-Cannon\/skill_builders\/"},{value:"Slack",label:"<p>If you haven\u0026rsquo;t already, please join Slack. This will be a lifesaver.\nhttps:\/\/join.slack.com\/t\/byuidss\/signup\n</p>",url:"https:\/\/byuistats.github.io\/DS250-Cannon\/slack\/"},{value:"Slides",label:"<p>Use the navigation pane on the left to review the class slides.\n</p>",url:"https:\/\/byuistats.github.io\/DS250-Cannon\/slides\/"},{value:"",label:"<p>Details Your coding challenge will help you demonstrate the skills you have developed this semester. Here are a few essential items.\n Your goal is to demonstrate your data science coding abilities. Get through as many items with a rough implementation as possible. Get your code to match our outputs as close as possible, but don\u0026rsquo;t stress over minute details. Keep most of the code you type. If you end up not using specific parts, comment them out and include them at the bottom. Use the entire hour and may not finish. Submit a .md and a .pdf report with your output and code for each challenge.  Please use the challenge template to submit your work.\nimport pandas as pd import altair as alt import numpy as np from sklearn.model_selection import train_test_split from sklearn import tree from sklearn.ensemble import GradientBoostingClassifier from sklearn import metrics Challenge 1 Split Entry houses are a failed building experiment in the United States. Use the data from our Denver homes project, as shown below, to recreate the following graphic.\nurl = \u0026#39;https:\/\/github.com\/byuidatascience\/data4dwellings\/raw\/master\/data-raw\/dwellings_denver\/dwellings_denver.csv\u0026#39; dat_home = pd.read_csv(url).sample(n=4500, random_state=15) Challenge 2 Our computations can\u0026rsquo;t be done with missing values. Programmatically replace all the lost values with 125 and make a box-plot.\nmister = pd.Series([\u0026#34;lost\u0026#34;, 15, 22, 45, 31, \u0026#34;lost\u0026#34;, 85, 38, 129, 80, 21, 2]) Challenge 3 Our computations can\u0026rsquo;t be done with missing values. Programmatically replace all the lost values with 125 and report the mean rounded to two decimals.\nmister = pd.Series([\u0026#34;lost\u0026#34;, 15, 22, 45, 31, \u0026#34;lost\u0026#34;, 85, 38, 129, 80, 21, 2]) Challenge 4 Programmatically read in the following JSON file, keep only the cases column and return a markdown table that has country in the rows and cases for 1999 and 2000 in the columns. Your table will have six cells with values.\nurl = \u0026#39;https:\/\/github.com\/byuidatascience\/data4python4ds\/raw\/master\/data-raw\/table1\/table1.json\u0026#39; Challenge 5 Use our cleaned example of the star wars data from project 6 to predict the gender of the respondent to the survey. Report your precision and a feature importance plot.\n Use test_size = .20 and random_state = 2020 in train_test_split() Use the GradientBoostingClassifier() method.  url = \u0026#34;http:\/\/byuistats.github.io\/CSE250-Course\/data\/clean_starwars.csv\u0026#34; dat = pd.read_csv(url) </p>",url:"https:\/\/byuistats.github.io\/DS250-Cannon\/final_coding_challenge\/sp22\/"},{value:"Categories",label:"<p></p>",url:"https:\/\/byuistats.github.io\/DS250-Cannon\/categories\/"},{value:"Final_coding_challenges",label:"<p></p>",url:"https:\/\/byuistats.github.io\/DS250-Cannon\/final_coding_challenge\/"},{value:"Office Hours",label:"<p>Schedule a visit with Brother Cannon at an available time. https:\/\/calendly.com\/cannonp\n</p>",url:"https:\/\/byuistats.github.io\/DS250-Cannon\/contact\/"},{value:"Tags",label:"<p></p>",url:"https:\/\/byuistats.github.io\/DS250-Cannon\/tags\/"},];$("#search").autocomplete({source:projects}).data("ui-autocomplete")._renderItem=function(ul,item){return $("<li>").append("<a href="+item.url+" + \" &quot;\" +  >"+item.value+"</a>"+item.label).appendTo(ul);};});</script></div></div></div></div></header><section class=section><div class=container><div class="row justify-content-center"><div class="col-12 text-center"><h2 class=section-title></h2></div><div class="col-lg-4 col-sm-6 mb-4"><a href=https://byuistats.github.io/DS250-Cannon/skill_builders/ class="px-4 py-5 bg-white shadow text-center d-block match-height"><i class="ti-ruler-pencil icon text-primary d-block mb-4"></i><h3 class="mb-3 mt-0">Skill Builders</h3><p class=mb-0>Build skills for the projects.</p></a></div><div class="col-lg-4 col-sm-6 mb-4"><a href=https://byuistats.github.io/DS250-Cannon/slack/ class="px-4 py-5 bg-white shadow text-center d-block match-height"><i class="https://img.shields.io/badge/slack-@oresoftware/npp-yellow.svg?logo=slack icon text-primary d-block mb-4"></i><h3 class="mb-3 mt-0">Slack</h3><p class=mb-0>Link to Slack signup</p></a></div><div class="col-lg-4 col-sm-6 mb-4"><a href=https://byuistats.github.io/DS250-Cannon/slides/ class="px-4 py-5 bg-white shadow text-center d-block match-height"><i class="ti-layout-slider-alt icon text-primary d-block mb-4"></i><h3 class="mb-3 mt-0">Slides</h3><p class=mb-0>Class material for every day.</p></a></div></div></div></section><footer class="section pb-4"><div class=container><div class="row align-items-center"><div class="col-md-8 text-md-left text-center"><p class="mb-md-0 mb-4">J. Hathaway and BYU-I ©</p></div><div class="col-md-4 text-md-right text-center"><ul class=list-inline><li class=list-inline-item><a class="text-color d-inline-block p-2" href=https://github.com/byuidatascience><i class=ti-github></i></a></li><li class=list-inline-item><a class="text-color d-inline-block p-2" href=https://www.linkedin.com/groups/13537407/><i class=ti-linkedin></i></a></li></ul></div></div></div></footer><script src=https://byuistats.github.io/DS250-Cannon/js/script.min.js></script></body></html>
\ No newline at end of file
+<i class="ti-search search-icon"></i><script>$(function(){var projects=[{value:"Introduction",label:"<p>A competent student should be able to finish the exercises within 60 minutes. You should work through it on your own. This serves as an assessment of your understanding of the assigned readings.\nBefore you start Make sure you have installed VS-code, pandas, and altair on your computer. You can install these package by typing this line in the terminal.\npip install pandas altair\nOR if you have more than one version of python\npip3.9 install pandas altair\npip3.9 indicates the version of python you are installing the packages to.\nPart 1 Get familiar with your tools Programming involves a lot of research. Unlike subjects like Mathematics or History, we are not required to remember every single function and its usage. It is natural for experienced programmers to look for answers on the internet, books, even from other people\u0026rsquo;s code. Programming will be extremely frustrating if we are not allowed to do web searches, so please get familiar with the tools you have and use them often.\nOffical Documentation This should be your first resort for understanding any code\/function. Scanning the documentation of a function will allow you to get an overview of its usage.\nHere is a link to the documentation of the assign() function:\n(https:\/\/pandas.pydata.org\/docs\/reference\/api\/pandas.DataFrame.assign.html)\nExample of assign() (as shown in the documentation)\nimport pandas as pd df1 = pd.DataFrame({\u0026#39;temp_c\u0026#39;: [17.0, 25.0]}, index=[\u0026#39;Portland\u0026#39;, \u0026#39;Berkeley\u0026#39;]) df2 = df1.assign(temp_f=df1.temp_c * 9 \/ 5 \u002b 32) Exercise 1: After reading the documentation for assign(), write a short paragraph to explain assign() as if you were talking to someone with zero programming experience (use the example above to help you explain assign()).\n What is the difference between df1 and df2? How was df2 derived from df1?)  Online textbook It pains us to see students would rather be stuck at problems for hours yet they refuse to use the textbook. This is another very useful resource since this is designed for this class. link to the textbook: (https:\/\/byuidatascience.github.io\/python4ds\/)\nExercise 2: Locate the section where the textbook talks about query() and answer these questions.\n What function in R\u0026rsquo;s dplyr is equivalent or comparable to query() in pandas (You should include the section number in your answer)? What is the easiest mistake for python beginner to make that was shown in the text about query() (You should include the section number in your answer)?  The internet Google is a programmer\u0026rsquo;s friend. Get used to googling thing, in fact, you want to be an expert in googling\n Question that cannot be answered by the textbook and documentation? Google it. A function you have never seen before? Google it. An error in your code? Google it.  Exercise 3: Provide at least 2 extra resources you could find about the pandas function drop() on the internet.\nTutor, TA (Through slack, zoom, or in-person) We want to help you with your work; we want to answer your questions; but most importantly, we want to help you succeed in this class. That will require you to put in the necessary time in understanding the readings, coding and debugging. When you ask us a question, we expect that you have read the documentation, searched the textbook, and done your own research. Then we can be most helpful and can provide insights on top of your understanding.\nExamples of bad questions  How does drop() work? We will ask you to read the documentation for drop(). How do you make a table in a markdown file? We will refer you to the textbook. I don\u0026rsquo;t want these columns in my data, how can I drop them? We will ask you if you have found any things on the internet.  Examples of good questions  I am still confused about the syntax of drop(). After reading the documentation, this is my understanding of the function\u0026hellip; . What am I missing? I tried making a table in markdown (show code), it is still not giving me what I want, how can I fix this? I am trying to drop these columns in my dataframe, I think drop() is what I am looking for. Am I in the right direction? If not, what keywords should I be googling?  Exercise 4:\nUsing the code and tools mentioned above, finish question 4 and 5 under 3.2.4 in the textbook.(use the data in mpg for your plot):\n# library import import pandas as pd import altair as alt # data import url = \u0026quot;https:\/\/github.com\/byuidatascience\/data4python4ds\/raw\/master\/data-raw\/mpg\/mpg.csv\u0026quot; mpg = pd.read_csv(url)   Question 4: Make a scatterplot of hwy vs cyl.\n  Question 5: What happens if you make a scatterplot of class vs drv? Why is the plot not useful?\n  After you have completed this skill builder with your team (or on your own) then compare your work to our script    See the script.   </p><p></p>",url:"https:\/\/byuistats.github.io\/DS250-Cannon\/skill_builders\/introduction\/"},{value:"Day 1: Welcome",label:"<p>Welcome to DS 250!  Teacher: Paul Cannon TA: Bracken Sant (san20050@byui.edu)  Announcements  Devotional Computing Lab 4:30PM - 6:30PM all weekdays except Wednesday. Saturday from 10AM-12PM  Slack channel #tutoring_lab   Data Science Society - Wednesday\u0026rsquo;s at 6PM, STC 394 Math Department Opening Social - Thursday 11:30 RKS 229  What is a Data Scientist? A Data Scientist has a C\u002b Talent Stack Class Structure  Problem Solving Improved coding skills Effective written\/visual communication Collaboration Timeliness and communication with \u0026ldquo;the boss\u0026rdquo;  Syllabus\nGot Slack? Are we all on the Slack channel? Follow the Slack invitation that is waiting in your student email. If you don\u0026rsquo;t see an invite, you can join through this link and then ask \u0026ldquo;@Paul Cannon\u0026rdquo; to add you to the class channel.\nWho are you?  Introduce yourself and learn the names\/majors\/origin story of your group members. Make a plan to get help this semester. How will you contact each other? Some ideas: Slack, I-Learn, emails, group texts, etc. If you were independently wealthy, what would you be doing right now? Would you change majors? Highlights of 2022  Problem Solving This is not a \u0026ldquo;see and repeat\u0026rdquo; programming class!\nHow would you go about fixing my motorcycle? Learn how to ask for help (1 hr rule)  Getting started on Project 0 Setting up your Programming Snvironment  Download Visual Studio Code Download Python v (3.10.8)  Be sure to select the \u0026ldquo;Add to Path\u0026rdquo; option during the install process  Mac Users be sure to click on \u0026ldquo;Install Certificates\u0026rdquo; at the end of the install   Install the Python packages and VS Code extensions you need (see this page)  pip install pandas pip install numpy pip install jupyter pip install tabulate pip install altair   Install Quarto CLI Quatro Instructions Start looking at Project 0 Complete the \u0026ldquo;Methods Checkpoint\u0026rdquo;  Installing Packages and Extensions Learn how to install packages by reading the assigned material and by watching the video tutorial on this page.\nThe readings mention a lot of different packages. For Project 0, you need to install at least pandas, altair, numpy, and jupyter.\nThe readings will also mention two VS Code extensions you need to install.\nA note on Jupyter Notebooks vs. Interactive Python Window The textbook will show you how to use VS Code\u0026rsquo;s interactive python windows and Quatro. Feel free to use Jupyter Notebooks.\nWe will do write-ups in Quarto, though, which can be rendered as a PDF or HTML\nIntroduction to Brother Cannon    What do you want to know?    What is a data scientist?    Brother Hathaway\u0026rsquo;s definition:\n A blend of programmer, statistician, and communicator that burns with curiosity.\n My definiton for DS 250:\n Someone who can extract insights from data and then communicate those insights with clarity.\n Learn more about the BYU-Idaho data science program here.\n   What is data science programming?    Data scientists write code as a means to an end, whereas software developers write code to build things. Data science is inherently different from software development in that data science is an analytic activity, whereas software development has much more in common with traditional engineering.\nData scientists tackle problems such as identifying fraudulent transactions, or predicting which employees are likely to leave a company. Software developers can take the data scientists models and turn them into fully functioning systems with production-quality code. Software developers tackle problems like getting an algorithm to run more efficiently, or building user interfaces.\n   Course Outcomes    Upon completing this course, you will be able to use data-driven programming in Python to handle, format, and visualize data. We will introduce you to data wrangling techniques (panadas), analytical methods (scikit-learn), and the grammar of graphics (Altair). Specifically, as a successful learner, you will be able to:\n Use functions, data structures, and other programming constructs efficiently to process and find meaning in data. Programmatically load data from various types of data sources, including files, databases, and remote services. Use data manipulation libraries to perform straightforward analysis, produce charts, and prepare data for machine learning algorithms. Use machine learning libraries to discover insights, make predictions, and interpret the success of these algorithms. Collaborate and share your work with industry-leading tools.     BYU-Idaho Mission Statement     Brigham Young University-Idaho was founded and is supported and guided by The Church of Jesus Christ of Latter-day Saints. Its mission is to develop disciples of Jesus Christ who are leaders in their homes, the Church, and their communities.\n  How would you describe a leader? What makes a leader powerful? What does a leader do with insights?  An example of a good leader.\nWhat (or who) is truth?\n   ## Course Format and Grading How hard is this class going to be?    The reality of CSE 250:\n We have done all we can to ensure that this is a 2-credit course for the average student. That means that we expect 4-6 hours outside of class for the average student to achieve an A. You have to put in the time if you want to build skills. The course is necessarily creative in nature. That fact usually makes it feel more challenging. We will be asking you to learn to write creative data science python code. If you have any concerns, please talk with me!     What is the structure of CSE 250?    The class uses 7 projects to teach data science programming in Python using pandas, Altair, scikit-learn, and numpy.\n Projects Syllabus     How do I get the grade I want?     Specification Grading Grading structure Competency Elements  Introduction Project \u0026mdash;\u0026mdash;\u0026mdash;\u0026mdash;\u0026mdash;\u0026mdash;\u0026mdash;\u0026mdash;\u0026mdash;\u0026mdash;\u0026mdash;\u0026mdash;\u0026mdash;\u0026mdash;\u0026mdash;\u0026mdash;\u0026mdash;\u0026mdash;\u0026mdash;\u0026mdash;\u0026gt;\nWhat is the goal?    Completing the introduction project will set you up for success the rest of the semester. The workflow followed in the introduction project (loading packages, writing code, saving images, compiling a final report) will be the same for every other project . If you have questions about this project, you need to seek help.   What exactly do I need to submit?    Make sure you carefully read the project instructions.\nYou will submit a single .pdf file to I-Learn. This pdf file should contain an project summary, your answers to the grand questions (including the plot you saved with altair_saver), and an appendix where you copy and paste your commented Python code.\n   --------------------------------------------------------   ----------------------------------------------- </p><p></p>",url:"https:\/\/byuistats.github.io\/DS250-Cannon\/slides\/introduction\/day01\/"},{value:"pandas and Altair",label:"<p>For this skill builder, we are exploring some important functions in the package of pandas and Altair. DS programming requires a lot of data wrangling. Using the proper functions, we can create concise and comprehensive codes. You should be exposed to a few functions through the readings this week.\nYou may want to at least scan the readings before beginning this task since this serves as an assessment of your understanding of the assigned readings. A prepared student should be able to finish the exercises within 60 minutes. You should work through it on your own.\nBefore you start Make sure you have installed VS-code, pandas, and Altair on your computer. You can install these packages by typing this line in the terminal:\npip install pandas altair\nOR if you have more than one version of python:\npip3.9 install pandas altair\npip3.9 indicates the version of python you are installing the packages to.\nData import Run the following code to import the data we need for this skill builder:\n# package import import numpy as np import pandas as pd import altair as al # data import dat = pd.read_csv(\u0026#34;https:\/\/vincentarelbundock.github.io\/Rdatasets\/csv\/AER\/Guns.csv\u0026#34;) Make sure the variable dat is correctly assigned in your environment and finish the following exercises. You can read the documentation of the data on this page - https:\/\/vincentarelbundock.github.io\/Rdatasets\/doc\/AER\/Guns.html\nExercise 1 One of the first things we can do to a freshly imported data is to check its columns. This will help us understand the basic structure of the dataframe(table).\n Using one line of code, select all the columns in dat, assign it to a variable called col_list.\n  Hint Every dataframe has an attribute \u0022columns\u0022. Accessing this attribute will give you a list of all column names  We often want to know the dimension of a dataframe. How many columns are in the dataset? How many rows are in the dataset?\n Using one line of code, show the number of columns and rows in dat.\n  Hint Every dataframe has an attribute \u0022shape\u0022. Accessing this attribute will give you the dimension of a datafarme  Now run dat.head(). It will print out the first 5 rows of data in dat.\n Just from looking at the output, what column(s) seems to be redundant with the row number?\n  Hint There is one column that serves as nothing but a row counter, that columns is redundant.  Exercise 2 After a brief investigation of the data, we will clean up the data. By cleaning up, we are trying to filter down dat so this only holds data we need. We will first get rid of the extra column we found in the previous excercise.\n Using one line of code, drop the redundant column using the variable col_list (created in excercise 1)\n  Hint Use `drop()`. Understand what \u0026ldquo;axis\u0026rdquo; is as a parameter of drop().\nYour function should looks like this:\ndat.drop([col_list[_]], axis = _)\nfill the \u0026ldquo;_\u0026quot;\u0026rsquo;s with the correct values and assign the output to dat.\n Don\u0026rsquo;t forget to save the changes in dat. Run dat.head() to make sure the column is dropped in dat.\nExercise 3 We have filtered dat vertically by dropping a column. Now we will try to filter dat horizontally, meaning we will get rid of some the rows.\nWe can do that by applying a condition to dat. A condition is an expression that can be evaluated as True\/False. For example, 8 \u0026gt; 5 is an expression that evaluates to be True. This is trivial because 8 will always be greater than 5.\nRun the code below:\n what is the difference between exp1 and exp2?\n exp1 = 8 \u0026gt; 5 exp2 = dat.violent \u0026lt; 300  Hint Try type() on else variable OR calling else variable.  Run ths code below:\n By putting dat.violent \u0026lt; 300, and the violent column from dat into a dataframe, what is the relationship between the two columns?\n exp = pd.DataFrame({\u0026quot;dat.violent \u0026lt; 300\u0026quot; : exp2, \u0026quot;violent value from dat\u0026quot; : dat.violent}) exp  Hint Try computing `dat.violent[n]  Using query(), filter down the dat so that it only contains the data for idaho\n  Hint query() takes in expressions and filters down data.  Don\u0026rsquo;t forget to save the changes in dat. Run dat.shape() to make sure the there are 23 rows and 13 columns.\nExercise 4 Besides filtering, we can manipulate the data by adding new data to it. By adding a new column to the data, we assign a new value to each row.\n Using assign(), create a new column that show the ratio between murder rate and violent rate.\n  Hint Use assign() You see get the ratio by computing this code:\ndat.murder\/dat.violent\n Exercise 5  Create a scatter plot that shows the relationship between murder rate and violent rate for the state of Idaho. Your chart should show murder rate as the x-axis, violent as the y-axis.\n  Hint Can you mimic this plot? (https:\/\/altair-viz.github.io\/gallery\/scatter_tooltips.html)\n  For an extra push Exercise 6  Using a line of code, filter down the data set so that it only shows the data in years between 1993 and 1997.\n Exercise 7  Create a line chart that show prisoners numbers for the state of Idaho, Utah, and Oregon.\n Your chart should show year as the x-axis, prisoner as the y-axis, states as different colours, along with an appropriate title.\nExercise 8  Without using query(), finshed the data wrangling in question 2,5 and 6.\n After you have completed this skill builder with your team (or on your own) then compare your work to our script    See the script.   </p><p></p>",url:"https:\/\/byuistats.github.io\/DS250-Cannon\/skill_builders\/pandas_altair\/"},{value:"JSONs \u0026 missing",label:"<p>UFO Sightings Data Link to json file\nExercise 1 Read in the json file as a pandas dataframe. After reading in the data, you\u0026rsquo;ll want to explore it and gain some intuition. Exploring data is a very important step — the more you know about your data the better! Answer the following questions to gain some insight into this dataset.\n How many rows are there? How many columns? What does a row represent in this dataset? What are the different ways missing values are encoded? How many np.nan in each column?  Some useful code for exploring data\n# Object\/Categorical Columns data.column_name.value_counts(dropna=False) data.column_name.unique() # Numeric Columns data.column_name.describe() # Counting missing values data.isna().sum() # Creates boolean dataframe and sums each column  Exercise 2 After learning different ways our data encodes missing values, now we will neatly manage them. There are many techniques we can use to handle missing values; for example, we can drop all rows that contain a missing value, impute with mean or median, or replace missing values with a new missing category. We will use some of these techniques in this exercise.\n shape_reported - replace missing values with missing string. distance_reported - change -999 values to np.nan. (-999 is a typical way of encoding missing values.) distance_reported - fill in missing values with the mean (imputation) were_you_abducted - replace - string with missing string.  The first 10 rows of your data should look like this after completion of the above steps.\n    city shape_reported distance_reported were_you_abducted estimated_size     0 Ithaca TRIANGLE 8521.9 yes 5033.9   1 Willingboro OTHER 7438.64 no 5781.03   2 Holyoke OVAL 7438.64 no 697203   3 Abilene DISK 7438.64 no 5384.61   4 New York Worlds Fair LIGHT 6615.78 missing 3417.58   5 Valley City DISK 7438.64 no 4280.1   6 Crater Lake CIRCLE 7377.89 no 528289   7 Alma DISK 7438.64 missing 4772.75   8 Eklutna CIGAR 5214.95 no 4534.03   9 Hubbard CYLINDER 8220.34 missing 4653.72    Some useful code for filling in missing data\ndata.column_name.replace(..., ..., inplace=True) data.column_name.fillna(..., inplace=True)  Exercise 3 Create a table that contains the following summary statistics.\n median estimated size by shape mean distance reported by shape count of reports belonging to each shape  Your table should look like this:\n   shape_reported median_est_size mean_distance_reported group_count     CIGAR 5899.68 6520.21 3   CIRCLE 266002 7408.26 2   CYLINDER 4550.58 8039.49 2   DISK 4581.8 7516.39 16   FIREBALL 5407.22 7097.78 3   FLASH 6108.34 7438.64 1   FORMATION 5104.4 8708.32 2   LIGHT 3850.25 7636.09 2   OTHER 4699.4 7473.98 4   OVAL 4943.63 7787.24 4   RECTANGLE 3668.1 6054.62 2   SPHERE 5076.78 7206.55 6   TRIANGLE 5033.9 8521.9 1   missing 250153 7438.64 2    Some useful code for grouping and getting summary statistics\n(data.groupby(...) .agg(..., ..., ...))  Exercise 4 The cities listed below reported their estimated size in square inches, not square feet. Create a new column named estimated_size_sqft in the dataframe, that has all the estimated sizes reported as sqft. (Hint: divide by 144 to go from sqin -\u0026gt; sqft)\n Holyoke Crater Lake Los Angeles San Diego Dallas  The head of your data should look like this.\n    city shape_reported distance_reported were_you_abducted estimated_size estimated_size_sqft     0 Ithaca TRIANGLE 8521.9 yes 5033.9 5033.9   1 Willingboro OTHER 7438.64 no 5781.03 5781.03   2 Holyoke OVAL 7438.64 no 697203 4841.69   3 Abilene DISK 7438.64 no 5384.61 5384.61   4 New York Worlds Fair LIGHT 6615.78 missing 3417.58 3417.58   5 Valley City DISK 7438.64 no 4280.1 4280.1   6 Crater Lake CIRCLE 7377.89 no 528289 3668.68   7 Alma DISK 7438.64 missing 4772.75 4772.75   8 Eklutna CIGAR 5214.95 no 4534.03 4534.03   9 Hubbard CYLINDER 8220.34 missing 4653.72 4653.72    Some useful code to fix the rows reported in sqin\nnp.where(..., # Condition ..., # If condition is true ...) # If condition is false  After you have completed this skill builder with your team (or on your own) then compare your work to our script    See the script.   </p><p></p>",url:"https:\/\/byuistats.github.io\/DS250-Cannon\/skill_builders\/json_missing\/"},{value:"SQL \u0026 databases",label:"<p>Skill builder (relational database) For this skill builder, we are exploring some important topics in relational databases. This exercise will require you to create SQL queries through python. You may want to at least scan the readings before beginning this task since this serves as an assessment of your understanding of the assigned readings.\nA competent student should be able to finish the exercises within 75 minutes.\nBefore you start Make sure you have installed VS-code, pandas, and Altair on your computer.\nAlso make sure you have gone through the tutorial on under course materials called SQL for Data Science: we assume that you have a connection to your data.\nExercise 1 Readme file A database can consist of more than one table\/data set. A relational database consists of tables\/data sets that share columns. These shared columns then establish the relationship between the tables, thus the name relational database. The relations are sometimes not easily found and they require careful investigations.\nTo understand what is in a relational database, we can start with understanding the tables and the columns within.\nHere is a link to the readme file of the baseball database.\n What is the name of the table that records data about pitchers in the regular seasons?\n  What do the HR and HBP columns mean in that table respectively?\n Excercise 2 SELECT and FROM The simplest SQL query is a query with SELECT and FROM. These are the keywords you will see again and again in SQL. Usually, when constructing a more complex query, it is easier to identify what goes into these two clauses first.\n Create a query that shows all columns from the table you found in Exercise 1, save the dataframe in a variable \u0026ldquo;pitch\u0026rdquo;\n You script should look something like:\nresult = pd.read_sql_query( \u0027SELECT _______ FROM _______\u0027, con) results Excercise 2 WHERE The WHERE keyword allows us to filter down the table horizontally (fewer rows).\nIt goes after SELECT and FROM.\n Using a SQL query, select all rows in the same table where HR is lesser than 10 and gs is greater than 25.\n  Find out what the columns mean and explain your query in words\n Excercise 3 ORDER BY ORDER BY sort the table you select by one or more columns and goes after WHERE\n Using the same query in exercise 2, edit it so that the table is ordered by the year of the season(nearest to furthermost) and the player ID(alphabetically).\n Excercise 4 Joins Joins are used when you wish to create a new table through two different tables. Keep in mind that you have to identify the relationship between two tables before you can correctly join them.\nJOIN goes between FROM and WHERE.\n Identify the shared columns (keys) and join the table in exercise 2 with the salaries table, then filter the data so that it shows only pitchers in the year 1986.\n You should get a dataframe with 306 rows.\nExercise 5 Group by Group by is a keyword we use to lower the level of granularity of a table. Meaning we are combining rows into one by the given column(s).\nCreate a query that captures the number of pitchers the Washington Nationals used in each year, then sort the table by year\nYou should get a dataframe with 23 rows.\nFor the overachievers Excercise 6 Research the order of operations for SQL and put the following keywords in that order.\n SELECT FROM JOIN WHERE HAVING ORDER BY GROUP BY LIMIT  After you have completed this skill builder with your team (or on your own) then compare your work to our script    See the script.   </p><p></p>",url:"https:\/\/byuistats.github.io\/DS250-Cannon\/skill_builders\/relational_data\/"},{value:"Machine Learning",label:"<p>Intro to Titanic Machine Learning Skill Builder Link to data\nFor this skill builder, we\u0026rsquo;ll be putting our machine learning hats on. We\u0026rsquo;ll be creating a model that predicts whether a passenger survived. With machine learning, there is a lot of jargon! It can be quite overwhelming at times. This skill builder attempts to keep things basic and simple. With that being said, there are some terms that are important to understand. Let\u0026rsquo;s look at the first few rows of our dataset before proceeding with the definitions.\nThe titanic dataset will be used for examples of each definition.\n   survived pclass sex age siblings_spouses_aboard parents_children_aboard fare     0 3 1 22 1 0 7.25   1 1 0 38 1 0 71.2833   1 3 0 26 0 0 7.925   1 1 0 35 1 0 53.1   0 3 1 35 0 0 8.05    Important Terms:  features: measurable property of the object you\u0026rsquo;re trying to predict. We use this information to predict our target of interest.  Example: pclass, sex, age, siblings_spouses_aboard , parents_children_aboard, fare columns are all examples of different features. Synonyms: attributes, explanatory variables, independent variables, variables, X\u0026rsquo;s, covariates   target: the feature that you are wanting to gain more insight into. The thing you are trying to predict.  Example: in the titanic dataset our target is survived Synonyms: label, dependent variable, y   train set: Usually 70% of the rows from the original dataset are randomly sampled to create this training data. It\u0026rsquo;s used by the algorithm, to determine, or learn, the optimal combinations of variables that will generate a good predictive model  Example: Random sample of 70% of the original titanic dataset rows Synonyms: training data, train data, X_train, y_train   test set: Usually the remaining 30% of the rows in the original dataset are used to create this dataset. The testing data is a set of rows used only to assess the performance (i.e. generalization) of a model. To do this, the final model is used to predict classifications of examples in the test set. Those predictions are compared to the examples\u0026rsquo; true classifications to assess the model\u0026rsquo;s accuracy.  Example: Random sample of 30% of the original titanic dataset rows Synonyms: testing data, test data, X_test, y_test   evaluation metrics: A statistic that tells you how well your predictions align with the actual values. Other words, tells you how good your model is.  Example: Accuracy, Precision, Recall, MSE, MAE, Rsquared Synonyms: performance metric    Again, this is a very light and oversimplified treatment of machine learning. The purpose of this project is to help you understand the main concepts of ml and walk you through the process of building a machine learning model. A simplified work flow of a machine learning project is shown below. Spend some time getting familiar with this flow \u0026amp;mdash as you are about to code it\u0026hellip; Exciting!\nNote in order to do this skill builder you will need to have scikit-learn installed on your machine. Run the following command in your terminal if you haven\u0026rsquo;t already.\npip install scikit-learn\nData Link to csv file\nExercise 0 (Imports and Loading in data) # Loading in packages import pandas as pd import numpy as np import altair as alt from sklearn.model_selection import train_test_split from sklearn.ensemble import RandomForestClassifier from sklearn.metrics import accuracy_score # Loading in data data = pd.read_csv(___)  Exercise 1 Create a chart exploring the relationship between age and survived in the titanic dataset. A strip plot, density plot, or boxplot might be useful here. Below is an example of a density plot. Feel free to replicate this chart or create your own.\nThe purpose of making this chart is to explore the relationships between a feature and the target. We want to see if the feature contains predictive information about the target. This is a large part of machine learning called Exploratory Data Analysis that should never be skipped! Spend time getting to know your features and how they interact with other features and the target.\n Exercise 2 Build a random forest model that is able to predict whether a passenger survived. This exercise is the bulk of the skill builder and contains several steps.\nStep 0: Split the data into X and y variables The X variable will contain all your features\n# Removes the target and keeps all features X = data.drop(___, axis=1) The y variable will hold the target\n# Selects the target column y = data[\u0026#39;___\u0026#39;] Step 1: Split data into train and test sets The train_test_split function is useful for this task. Review the train_test_split function documentation\n# Splitting X and y variables into train and test sets using stratified sampling X_train, X_test, y_train, y_test = train_test_split(___, ___, test_size=0.3, random_state=24, stratify=y) Step 2: Train the model Explore the RandomForestClassifier documentation for the RandomForestClassifier. It\u0026rsquo;s not necessary to understand the inner workings of the Random Forest algorithm for this class - just learn the syntax of fitting the model.\n# Creating random forest object rf = RandomForestClassifier(random_state=24) # Fit with the training data rf.fit(___, ___) Step 3: Use test set to make predictions # Using the features in the test set to make predictions y_pred = rf.predict(___) Step 4: Compare test set predictions to actual values. Calculate the accuracy. # Comparing predictions to actual values accuracy_score(___, ___)  Exercise 3 What is the most important feature in making predictions? Why do you think this is?\nCreate a table that shows the feature importances in descending order. The random forest classifier has a feature importances attribute. It can be accessed by rf.feature_importances_. The table should look something like this.\n   feature names importances     fare 0.288051   sex 0.281853   age 0.266491   pclass 0.0814224   siblings_spouses_aboard 0.0475633   parents_children_aboard 0.034619    After you have completed this skill builder with your team (or on your own) then compare your work to our script    See the script.   </p><p></p>",url:"https:\/\/byuistats.github.io\/DS250-Cannon\/skill_builders\/ml_sklearn\/"},{value:"Munging data",label:"<p>Intro to cleaning movies data Link to the data\nThis skill builder focuses on munging (formatting) data into a machine learning ready dataset. We will be using an IMDB Ratings dataset. It contains columns that are categorical. Sklearn cannot handle columns that are strings, so we need to convert these into a numerical representation. We accomplish this by either one hot encoding, label encoding, or taking just one value of the range provided. There are many other ways to represent these columns as numbers, but they are beyond the scope of this course.\nOnce you\u0026rsquo;ve converted all columns to numeric, in an intelligent way, you will be asked to recreate a graph using altair. Here is the head of the data you will be working with. Enjoy!\n   star_rating content_rating genre duration box_office_rev major_hit     9.3 R Crime 142 €1924521976 - €1925521976 no   9.2 R Crime 175 €177034987 - €178034987 no   9.1 R Crime 200 €2617541398 - €2618541398 no   9 PG-13 Action 152 €996115723 - €997115723 no   8.9 R Crime 154 €1172054364 - €1173054364 no    Data Link to csv file: ...\n Exercise 0  Grab the high range value for each movie and put it into a new column called high_range_rev.  Make sure the data type of this new column is numeric!!   Remove the box_office_rev column from the dataset.  The .str.split() and .astype() methods might be of use! Also, to get the euro sign just copy it from here, €, and put it in your code.\nThe first 5 rows of the resulting dataframe should look like this\n   star_rating content_rating genre duration major_hit high_range_rev     9.3 R Crime 142 no 2345444803   9.2 R Crime 175 no 2182412593   9.1 R Crime 200 no 1604872807   9 PG-13 Action 152 no 284317976   8.9 R Crime 154 yes 1791932201     Exercise 1 Convert the major_hit column to 1\/0\u0026rsquo;s. yes -\u0026gt; 1 and no -\u0026gt; 0. Again, there are several ways to accomplish this. Using our old friend np.where is probably the easiest though.\nThe first 5 rows of the resulting dataframe should like this\n   star_rating content_rating genre duration major_hit high_range_rev     9.3 R Crime 142 0 1925521976   9.2 R Crime 175 0 178034987   9.1 R Crime 200 0 2618541398   9 PG-13 Action 152 0 997115723   8.9 R Crime 154 0 1173054364     Exercise 2 Convert the content_rating column using label encoding. We\u0026rsquo;re using label encoding in this case because the movie ratings already have a natural ordering to them. We will replace each rating with a number in it\u0026rsquo;s natural ascending order.\nTo be more specific, here is how we will do it.\n G: 0 PG: 1 PG-13: 2 R: 3  A dictionary and the .map() method could be useful for this exercise. There are other ways of tackling this problem though. Be creative!\nThe first 5 rows of the resulting dataframe should look like\n   star_rating content_rating genre duration major_hit high_range_rev     9.3 3 Crime 142 0 1925521976   9.2 3 Crime 175 0 178034987   9.1 3 Crime 200 0 2618541398   9 2 Action 152 0 997115723   8.9 3 Crime 154 0 1173054364     Exercise 3 The last column that we need to take care of is genre. We will use one hot encoding for this. Make sure to ONLY one hot encode the genre column!\nA useful function for one hot encoding is pd.get_dummies(). I recommend checking out the documentation.\nThe resulting dataframe should look like the following example; don\u0026rsquo;t worry if your high_range_rev column turned into scientific notation—Pandas does this sometimes.\n    star_rating content_rating duration major_hit high_range_rev genre_Action genre_Adventure genre_Animation genre_Biography genre_Comedy genre_Crime genre_Drama genre_Family genre_Fantasy genre_Horror genre_Mystery genre_Sci-Fi genre_Thriller genre_Western     0 9.3 3 142 0 1.92552e\u002b09 0 0 0 0 0 1 0 0 0 0 0 0 0 0   1 9.2 3 175 0 1.78035e\u002b08 0 0 0 0 0 1 0 0 0 0 0 0 0 0   2 9.1 3 200 0 2.61854e\u002b09 0 0 0 0 0 1 0 0 0 0 0 0 0 0   3 9 2 152 0 9.97116e\u002b08 1 0 0 0 0 0 0 0 0 0 0 0 0 0   4 8.9 3 154 0 1.17305e\u002b09 0 0 0 0 0 1 0 0 0 0 0 0 0 0     Exercise 4 Recreate this graph as best you can. You\u0026rsquo;ll need to use the original data that specifies the actual rating.\nAfter you have completed this skill builder with your team (or on your own) then compare your work to our script    See the script.   </p><p></p>",url:"https:\/\/byuistats.github.io\/DS250-Cannon\/skill_builders\/munging\/"},{value:"GitHub and git",label:"<p>Complete the Hello World GitHub Guide\n</p><p></p>",url:"https:\/\/byuistats.github.io\/DS250-Cannon\/skill_builders\/git_github\/"},{value:"Week 1: Introduction",label:"<p>  Introduction Project Syllabus   </p><p></p>",url:"https:\/\/byuistats.github.io\/DS250-Cannon\/slides\/introduction\/"},{value:"DS250",label:"<p></p>",url:"https:\/\/byuistats.github.io\/DS250-Cannon\/"},{value:"Frequently Asked Questions",label:"<p> What do you mean by data science programming?    Most likely, you have had 1-2 courses of programming before you have taken CSE 250. Unlike traditional computer science courses, CSE 250 uses Python in an interactive mode instead of building programs. The data provider usually has some big questions that need answering; However, there are hundreds of little issues and responses along the way. We use programming to facilitate this investigation.\nThere are similarities with User Experience Designers. In our case, we don\u0026rsquo;t get to ask users about their experience. We use programming to ask data about its background, and each data set has its own history. We want our analysis to mold to that experience. You can think of data science programming like a first date with your data. You can\u0026rsquo;t write one long program nieve of the issues and nuances each living data set provides.\n   How does CSE 250 compare to CSE 350 or Math 335?    The two courses have similarities. You could think of CSE 250 as an introduction to data wrangling and visualization. Both classes use real-world data and are built around data science projects. There are some critical differences between the two courses.\n In this course, we use Python, and CSE 350 uses R. We are introducing the principles of data science programming in CSE 250. The course is only 2-credits. CSE 250 is intended to introduce visualization, wrangling, and modeling.     How does CSE 250 prepare me for CSE 350, Math 335 and CSE 450?    You will be comfortable with interactive programming and have an introduction to the principles of data formats for data science applications. You will be introduced to principles related to machine learning, data wrangling, and data visualization.   What programming languages do we use in this course?    The course is done using Python. We focus on the pandas and Altair packages.   What are the prerequisites for this course?    Using the new courses at BYU-I, the prerequisite is CSE 110. However, if you have experience programming from other classes, you most likely are prepared for this course.   Why Python instead of R?    The computer science and software engineering programs at BYU-I use Python as their foundational courses. The standard student will have some experience with Python before CSE 250. Python is an essential programming language for data scientists, and we already have CSE 350\/Math 335, which is taught in R.   What is pandas?    pandas is the foundational data science package in Python. If you are using tabular data you will be in pandas.   Why are we using Altair instead of Seaborn or Matplotlib?    Matplotlib was the first visualization package to gain a following in Python. Seaborn is built on top of Matplotlib. Many data scientists use both in their work—neither leverage the grammar of graphics as developed by Leland Wilkinson. Altair is built on Vega-Lite, which uses the Vega visualization grammar. It is declarative and actively developed. We expect that it will become the predominant visualization package in Python (https:\/\/youtu.be\/FytuB8nFHPQ and https:\/\/youtu.be\/vTingdk_pVM).   </p>",url:"https:\/\/byuistats.github.io\/DS250-Cannon\/faq\/"},{value:"Skill Builders",label:"<p>These short activites are provided for you to gain some additional skills to help with the class projects.\n</p>",url:"https:\/\/byuistats.github.io\/DS250-Cannon\/skill_builders\/"},{value:"Slack",label:"<p>If you haven\u0026rsquo;t already, please join Slack. This will be a lifesaver.\nhttps:\/\/join.slack.com\/t\/byuidss\/signup\n</p>",url:"https:\/\/byuistats.github.io\/DS250-Cannon\/slack\/"},{value:"Slides",label:"<p>Use the navigation pane on the left to review the class slides.\n</p>",url:"https:\/\/byuistats.github.io\/DS250-Cannon\/slides\/"},{value:"",label:"<p>Details Your coding challenge will help you demonstrate the skills you have developed this semester. Here are a few essential items.\n Your goal is to demonstrate your data science coding abilities. Get through as many items with a rough implementation as possible. Get your code to match our outputs as close as possible, but don\u0026rsquo;t stress over minute details. Keep most of the code you type. If you end up not using specific parts, comment them out and include them at the bottom. Use the entire hour and may not finish. Submit a .md and a .pdf report with your output and code for each challenge.  Please use the challenge template to submit your work.\nimport pandas as pd import altair as alt import numpy as np from sklearn.model_selection import train_test_split from sklearn import tree from sklearn.ensemble import GradientBoostingClassifier from sklearn import metrics Challenge 1 Split Entry houses are a failed building experiment in the United States. Use the data from our Denver homes project, as shown below, to recreate the following graphic.\nurl = \u0026#39;https:\/\/github.com\/byuidatascience\/data4dwellings\/raw\/master\/data-raw\/dwellings_denver\/dwellings_denver.csv\u0026#39; dat_home = pd.read_csv(url).sample(n=4500, random_state=15) Challenge 2 Our computations can\u0026rsquo;t be done with missing values. Programmatically replace all the lost values with 125 and make a box-plot.\nmister = pd.Series([\u0026#34;lost\u0026#34;, 15, 22, 45, 31, \u0026#34;lost\u0026#34;, 85, 38, 129, 80, 21, 2]) Challenge 3 Our computations can\u0026rsquo;t be done with missing values. Programmatically replace all the lost values with 125 and report the mean rounded to two decimals.\nmister = pd.Series([\u0026#34;lost\u0026#34;, 15, 22, 45, 31, \u0026#34;lost\u0026#34;, 85, 38, 129, 80, 21, 2]) Challenge 4 Programmatically read in the following JSON file, keep only the cases column and return a markdown table that has country in the rows and cases for 1999 and 2000 in the columns. Your table will have six cells with values.\nurl = \u0026#39;https:\/\/github.com\/byuidatascience\/data4python4ds\/raw\/master\/data-raw\/table1\/table1.json\u0026#39; Challenge 5 Use our cleaned example of the star wars data from project 6 to predict the gender of the respondent to the survey. Report your precision and a feature importance plot.\n Use test_size = .20 and random_state = 2020 in train_test_split() Use the GradientBoostingClassifier() method.  url = \u0026#34;http:\/\/byuistats.github.io\/CSE250-Course\/data\/clean_starwars.csv\u0026#34; dat = pd.read_csv(url) </p>",url:"https:\/\/byuistats.github.io\/DS250-Cannon\/final_coding_challenge\/sp22\/"},{value:"Categories",label:"<p></p>",url:"https:\/\/byuistats.github.io\/DS250-Cannon\/categories\/"},{value:"Final_coding_challenges",label:"<p></p>",url:"https:\/\/byuistats.github.io\/DS250-Cannon\/final_coding_challenge\/"},{value:"Office Hours",label:"<p>Schedule a visit with Brother Cannon at an available time. https:\/\/calendly.com\/cannonp\n</p>",url:"https:\/\/byuistats.github.io\/DS250-Cannon\/contact\/"},{value:"Tags",label:"<p></p>",url:"https:\/\/byuistats.github.io\/DS250-Cannon\/tags\/"},];$("#search").autocomplete({source:projects}).data("ui-autocomplete")._renderItem=function(ul,item){return $("<li>").append("<a href="+item.url+" + \" &quot;\" +  >"+item.value+"</a>"+item.label).appendTo(ul);};});</script></div></div></div></div></header><section class=section><div class=container><div class="row justify-content-center"><div class="col-12 text-center"><h2 class=section-title></h2></div><div class="col-lg-4 col-sm-6 mb-4"><a href=https://byuistats.github.io/DS250-Cannon/skill_builders/ class="px-4 py-5 bg-white shadow text-center d-block match-height"><i class="ti-ruler-pencil icon text-primary d-block mb-4"></i><h3 class="mb-3 mt-0">Skill Builders</h3><p class=mb-0>Build skills for the projects.</p></a></div><div class="col-lg-4 col-sm-6 mb-4"><a href=https://byuistats.github.io/DS250-Cannon/slack/ class="px-4 py-5 bg-white shadow text-center d-block match-height"><i class="https://img.shields.io/badge/slack-@oresoftware/npp-yellow.svg?logo=slack icon text-primary d-block mb-4"></i><h3 class="mb-3 mt-0">Slack</h3><p class=mb-0>Link to Slack signup</p></a></div><div class="col-lg-4 col-sm-6 mb-4"><a href=https://byuistats.github.io/DS250-Cannon/slides/ class="px-4 py-5 bg-white shadow text-center d-block match-height"><i class="ti-layout-slider-alt icon text-primary d-block mb-4"></i><h3 class="mb-3 mt-0">Slides</h3><p class=mb-0>Class material for every day.</p></a></div></div></div></section><footer class="section pb-4"><div class=container><div class="row align-items-center"><div class="col-md-8 text-md-left text-center"><p class="mb-md-0 mb-4">J. Hathaway and BYU-I ©</p></div><div class="col-md-4 text-md-right text-center"><ul class=list-inline><li class=list-inline-item><a class="text-color d-inline-block p-2" href=https://github.com/byuidatascience><i class=ti-github></i></a></li><li class=list-inline-item><a class="text-color d-inline-block p-2" href=https://www.linkedin.com/groups/13537407/><i class=ti-linkedin></i></a></li></ul></div></div></div></footer><script src=https://byuistats.github.io/DS250-Cannon/js/script.min.js></script></body></html>
\ No newline at end of file
diff --git a/slides/introduction/day01/index.html b/slides/introduction/day01/index.html
index bf5afee..1ebc220 100644
--- a/slides/introduction/day01/index.html
+++ b/slides/introduction/day01/index.html
@@ -4,5 +4,5 @@
 <a class=dropdown-item href=/DS250-Cannon/course-materials/syllabus/>Syllabus</a>
 <a class=dropdown-item href=/DS250-Cannon/faq>FAQ</a></div></li></ul></div></div></nav></header><section class="single section-sm pb-0"><div class=container><div class=row><div class=col-lg-3><div class=sidebar><ul class=list-styled><a class=back-btn href=/DS250-Cannon></a><li data-nav-id=https://byuistats.github.io/DS250-Cannon/slides/ title=Slides class=sidelist><a href=https://byuistats.github.io/DS250-Cannon/slides/>Slides</a><ul><li data-nav-id=https://byuistats.github.io/DS250-Cannon/slides/introduction/ title="Week 1: Introduction" class=sidelist><a href=https://byuistats.github.io/DS250-Cannon/slides/introduction/>Week 1: Introduction</a><ul><li data-nav-id=https://byuistats.github.io/DS250-Cannon/slides/introduction/day01/ title="Day 1: Welcome" class="sidelist
 active"><a href=https://byuistats.github.io/DS250-Cannon/slides/introduction/day01/>Day 1: Welcome</a></li></ul></li></ul></li></ul></div></div><div class=col-lg-9><div class="p-lg-5 p-4 bg-white"><h2 class=mb-5>Day 1: Welcome</h2><div class=content><h2 id=welcome-to-ds-250>Welcome to DS 250!</h2><ul><li>Teacher: Paul Cannon</li><li>TA: Bracken Sant (<a href=mailto:san20050@byui.edu>san20050@byui.edu</a>)</li></ul><br><h2 id=announcements>Announcements</h2><ol><li>Devotional</li><li><a href=https://byuidatascience.github.io/lab/>Computing Lab</a> 4:30PM - 6:30PM all weekdays except Wednesday. Saturday from 10AM-12PM<ul><li>Slack channel #tutoring_lab</li></ul></li><li>Data Science Society - Wednesday&rsquo;s at 6PM, STC 394</li><li>Math Department Opening Social - Thursday 11:30 RKS 229</li></ol><h2 id=what-is-a-data-scientist>What is a Data Scientist?</h2><p><img src=KLR2.png alt>
-<img src=ds_4venn.png alt></p><h2 id=a-data-scientist-has-a-c-talent-stack>A Data Scientist has a C+ Talent Stack</h2><h2 id=class-structure>Class Structure</h2><ol><li>Problem Solving</li><li>Improved coding skills</li><li>Effective written/visual communication</li><li>Collaboration</li><li>Timeliness and communication with &ldquo;the boss&rdquo;</li></ol><p><a href=https://byuistats.github.io/DS250-Cannon/course-materials/syllabus/>Syllabus</a></p><h2 id=got-slack>Got Slack?</h2><h4 id=are-we-all-on-the-slack-channel>Are we all on the Slack channel?</h4><p>Follow the Slack invitation that is waiting in your student email. If you don&rsquo;t see an invite, you can join through <a href=https://join.slack.com/t/byuidss/signup>this link</a> and then ask &ldquo;@Paul Cannon&rdquo; to add you to the class channel.</p><br><h2 id=who-are-you>Who are you?</h2><ol><li>Introduce yourself and learn the names/majors/origin story of your group members.</li><li>Make a plan to get help this semester. How will you contact each other? Some ideas: Slack, I-Learn, emails, group texts, etc.</li><li>If you were independently wealthy, what would you be doing right now? Would you change majors?</li><li>Highlights of 2022</li></ol><h2 id=problem-solving>Problem Solving</h2><p>This is not a &ldquo;see and repeat&rdquo; programming class!</p><h3 id=how-would-you-go-about-fixing-my-motorcycle>How would you go about fixing my motorcycle?</h3><p><img src=PXL_20221015_211101230.jpg alt></p><h4 id=learn-how-to-ask-for-help-1-hr-rule>Learn how to ask for help (1 hr rule)</h4><br><hr><br><p><img src=googleit.jpg alt></p><h2 id=getting-started-on-project-0>Getting started on Project 0</h2><h4 id=setting-up-your-programming-snvironment>Setting up your Programming Snvironment</h4><ol><li>Download <a href=https://code.visualstudio.com/>Visual Studio Code</a></li><li>Download <a href=https://www.python.org/downloads/>Python</a> v <a href=https://www.python.org/downloads/release/python-3108/>(3.10.8)</a><ul><li>Be sure to select the <em>&ldquo;Add to Path&rdquo;</em> option during the install process</li><li><img src=image.png alt></li><li>If you&rsquo;re on a <strong>Mac</strong> be sure to click on <em>&ldquo;Install Certificates&rdquo;</em> at the end of the install</li></ul></li><li>Install the Python packages and VS Code extensions you need (see <a href=https://byuistats.github.io/DS250-Cannon/course-materials/python-for-data-science/>this page</a>)<ul><li>pip install pandas</li><li>pip install numpy</li><li>pip install jupyter</li><li>pip install tabulate</li><li>pip install altair</li></ul></li><li>Install Quarto CLI <a href=https://byuistats.github.io/DS250-Cannon/course-materials/quarto-for-data-science/>Quatro Instructions</a></li><li>Start looking at Project 0</li><li>Complete the &ldquo;Methods Checkpoint&rdquo;</li></ol><h4 id=installing-packages-and-extensions>Installing Packages and Extensions</h4><p>Learn how to install packages by reading the assigned material and by watching the video tutorial on <a href=https://byuistats.github.io/DS250-Cannon/course-materials/python-for-data-science/>this page</a>.</p><p>The readings mention a lot of different packages. For Project 0, you need to install at least <code>pandas</code>, <code>altair</code>, <code>numpy</code>, and <code>jupyter</code>.</p><p>The readings will also mention two VS Code extensions you need to install.</p><h4 id=a-note-on-jupyter-notebooks-vs-interactive-python-window>A note on Jupyter Notebooks vs. Interactive Python Window</h4><p>The textbook will show you how to use VS Code&rsquo;s interactive python windows and Quatro. <strong>Feel free to use Jupyter Notebooks.</strong><br>We will do write-ups in Quarto, though, which can be rendered as a PDF or HTML</p><br></div><p class="post-meta border-bottom pb-3 mb-0 mt-3">Updated on 15 Sep 2020</p><nav class="pagination mt-3"><a class="nav nav-prev" href=https://byuistats.github.io/DS250-Cannon/slides/introduction/><i class="ti-arrow-left mr-2"></i><span class="d-none d-md-block">Week 1: Introduction</span></a>
+<img src=ds_4venn.png alt></p><h2 id=a-data-scientist-has-a-c-talent-stack>A Data Scientist has a C+ Talent Stack</h2><h2 id=class-structure>Class Structure</h2><ol><li>Problem Solving</li><li>Improved coding skills</li><li>Effective written/visual communication</li><li>Collaboration</li><li>Timeliness and communication with &ldquo;the boss&rdquo;</li></ol><p><a href=https://byuistats.github.io/DS250-Cannon/course-materials/syllabus/>Syllabus</a></p><h2 id=got-slack>Got Slack?</h2><h4 id=are-we-all-on-the-slack-channel>Are we all on the Slack channel?</h4><p>Follow the Slack invitation that is waiting in your student email. If you don&rsquo;t see an invite, you can join through <a href=https://join.slack.com/t/byuidss/signup>this link</a> and then ask &ldquo;@Paul Cannon&rdquo; to add you to the class channel.</p><br><h2 id=who-are-you>Who are you?</h2><ol><li>Introduce yourself and learn the names/majors/origin story of your group members.</li><li>Make a plan to get help this semester. How will you contact each other? Some ideas: Slack, I-Learn, emails, group texts, etc.</li><li>If you were independently wealthy, what would you be doing right now? Would you change majors?</li><li>Highlights of 2022</li></ol><h2 id=problem-solving>Problem Solving</h2><p>This is not a &ldquo;see and repeat&rdquo; programming class!</p><h3 id=how-would-you-go-about-fixing-my-motorcycle>How would you go about fixing my motorcycle?</h3><p><img src=PXL_20221015_211101230.jpg alt></p><h4 id=learn-how-to-ask-for-help-1-hr-rule>Learn how to ask for help (1 hr rule)</h4><br><hr><br><p><img src=googleit.jpg alt></p><h2 id=getting-started-on-project-0>Getting started on Project 0</h2><h4 id=setting-up-your-programming-snvironment>Setting up your Programming Snvironment</h4><ol><li>Download <a href=https://code.visualstudio.com/>Visual Studio Code</a></li><li>Download <a href=https://www.python.org/downloads/>Python</a> v <a href=https://www.python.org/downloads/release/python-3108/>(3.10.8)</a><ul><li>Be sure to select the <em>&ldquo;Add to Path&rdquo;</em> option during the install process</li><li><img src=image.png alt></li><li><strong>Mac Users</strong> be sure to click on <em>&ldquo;Install Certificates&rdquo;</em> at the end of the install</li></ul></li><li>Install the Python packages and VS Code extensions you need (see <a href=https://byuistats.github.io/DS250-Course/Setup/python_setup.html>this page</a>)<ul><li>pip install pandas</li><li>pip install numpy</li><li>pip install jupyter</li><li>pip install tabulate</li><li>pip install altair</li></ul></li><li>Install Quarto CLI <a href=https://byuistats.github.io/DS250-Course/Setup/quarto_setup.html>Quatro Instructions</a></li><li>Start looking at Project 0</li><li>Complete the &ldquo;Methods Checkpoint&rdquo;</li></ol><h4 id=installing-packages-and-extensions>Installing Packages and Extensions</h4><p>Learn how to install packages by reading the assigned material and by watching the video tutorial on <a href=https://byuistats.github.io/DS250-Course/Setup/python_setup.html>this page</a>.</p><p>The readings mention a lot of different packages. For Project 0, you need to install at least <code>pandas</code>, <code>altair</code>, <code>numpy</code>, and <code>jupyter</code>.</p><p>The readings will also mention two VS Code extensions you need to install.</p><h4 id=a-note-on-jupyter-notebooks-vs-interactive-python-window>A note on Jupyter Notebooks vs. Interactive Python Window</h4><p>The textbook will show you how to use VS Code&rsquo;s interactive python windows and Quatro. <strong>Feel free to use Jupyter Notebooks.</strong><br>We will do write-ups in Quarto, though, which can be rendered as a PDF or HTML</p><br></div><p class="post-meta border-bottom pb-3 mb-0 mt-3">Updated on 15 Sep 2020</p><nav class="pagination mt-3"><a class="nav nav-prev" href=https://byuistats.github.io/DS250-Cannon/slides/introduction/><i class="ti-arrow-left mr-2"></i><span class="d-none d-md-block">Week 1: Introduction</span></a>
 <a class="nav nav-next" href=https://byuistats.github.io/DS250-Cannon/final_coding_challenge/><span class="d-none d-md-block">Final_coding_challenges</span><i class="ti-arrow-right ml-2"></i></a></nav></div></div></div></div></section><footer class="section pb-4"><div class=container><div class="row align-items-center"><div class="col-md-8 text-md-left text-center"><p class="mb-md-0 mb-4">J. Hathaway and BYU-I ©</p></div><div class="col-md-4 text-md-right text-center"><ul class=list-inline><li class=list-inline-item><a class="text-color d-inline-block p-2" href=https://github.com/byuidatascience><i class=ti-github></i></a></li><li class=list-inline-item><a class="text-color d-inline-block p-2" href=https://www.linkedin.com/groups/13537407/><i class=ti-linkedin></i></a></li></ul></div></div></div></footer><script src=https://byuistats.github.io/DS250-Cannon/js/script.min.js></script></body></html>
\ No newline at end of file