-
Notifications
You must be signed in to change notification settings - Fork 1
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
1 parent
7a4ec98
commit 0c188f6
Showing
78 changed files
with
3,792 additions
and
204 deletions.
There are no files selected for viewing
Binary file added
BIN
+250 KB
_images/06ac9d5a79ccb8fddbceee8abac82997c1eaa5ba2f1f02111769fb1db878af0d.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added
BIN
+22.5 KB
_images/0fa4db0830ff26984c1fe8d1c44a6367452a0e82d4950c0267d70b36b3ffadea.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added
BIN
+68.4 KB
_images/2be9b0df877a3152c37a0f62e2a8217ffae6dac94848bfb4c85700af3deaad08.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added
BIN
+16.5 KB
_images/30da1c3bf87862db9810d755b93eb0e249caf08d9f39042e013a0fcdec3afb3e.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added
BIN
+151 KB
_images/577bba368e0b273b6e8a0c62e6586ec72b4a8c84afff863b70415202769f4582.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added
BIN
+44.6 KB
_images/5f1a84ec55676e8362b3c9cfe6d203099c8ca065ba193dc99fc22f4c3fe15375.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added
BIN
+27.8 KB
_images/78ccae0b5b4cf8743bb7d1e0fae7f8d86ccefc8b97066bf2f4abf3d1ca7401cb.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added
BIN
+27.7 KB
_images/8919a66c65870f264d5390311b60d0736e28af55c6f717e0675cc22c0a74bddd.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file removed
BIN
-27.7 KB
_images/a318d2119336d94c638413cedc2cb1f37d2e690f22748b0f6672e555b1fd25bf.png
Binary file not shown.
Binary file added
BIN
+186 KB
_images/c1a996b89b35fe1a49ae51461e73f52489eaaae0d3e3c339e4e84fb5cf6e4480.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file removed
BIN
-22.3 KB
_images/cb4ba6dd979009882fa9449a570a15db32ac73a50d2d04cc56875d2c768d4afc.png
Binary file not shown.
Binary file removed
BIN
-27.8 KB
_images/d19e1393f89f5806de6d1ac50487cb41ad1279f4e86fcbd9b791cd7d77fd5e9b.png
Binary file not shown.
Binary file removed
BIN
-16.5 KB
_images/e46da72d78ef33239fd09a8e58186baec3c7286dfc0be10dd51966609d3d0251.png
Binary file not shown.
Binary file added
BIN
+67.5 KB
_images/f873423b5a1eae89748911b893ba6b7b9f19f75c052aa2fc00a4b598783728c3.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -5,7 +5,7 @@ __Due: 2023-10-03__ | |
|
||
Eligible skills: | ||
- prepare 1 | ||
- access 1 | ||
- access 2 | ||
- python 1,2 | ||
|
||
|
||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,108 @@ | ||
# Assignment 7 | ||
|
||
[accept the assigment](https://classroom.github.com/a/q-cpZN-M) | ||
|
||
__Due: 2023-10-28__ | ||
|
||
|
||
Eligible skills: | ||
- evaluate level 2 | ||
- classification level 1,2 | ||
- summarize, 1,2 | ||
- visuailze 1,2 | ||
|
||
## Related notes | ||
|
||
- [](../notes/2024-10-17) | ||
<!-- - [](../notes/2023-10-19) --> | ||
|
||
::::{important} | ||
There is a large extra section in the notes, that should be of use for this assignment. | ||
|
||
You can use Gassian Naive Bayes **or** a Decision tree for the assignment. | ||
:::: | ||
|
||
## Dataset and EDA | ||
|
||
|
||
Choose a dataset that is well suited for classification and that has *all numerical features*. | ||
If you want to use a dataset with nonnumerical features you will have to convert | ||
the categorical features to numerical with one hot encoding. | ||
|
||
```{hint} | ||
Use the [UCI ML repository](https://archive.ics.uci.edu/datasets), it will let you filter data by the attributes of it you need. | ||
``` | ||
|
||
1. Include a basic description of the data(what the features are) | ||
1. Describe the classification task in your own words | ||
1. Use EDA to determine if you expect the classification to get a high accuracy or not. What types of mistakes do you think will happen most (think about the confusion matrix)? | ||
1. Hypothesize which classifier from the notes will do better and why you think that. Does the data meet the assumptions of Naive Bayes? What is important about this classifier for this application? | ||
|
||
```{important} | ||
You will get to reuse the above, and this dataset, for the clustering assignment *and* optionally one or both of A10 and A11. | ||
``` | ||
|
||
## Basic Classification | ||
|
||
1. Fit your chosen classifier with the default parameters on 80% of the data | ||
1. Inspect the model to answer the questions appropriate to your model. | ||
|
||
- Does this model make sense? | ||
- (if DT) Are there any leaves that are very small? | ||
- (if DT) Is this an interpretable number of levels? | ||
- (if GNB) do the parameters fit the data well? or do the paramters generate similar synthetic data (you can answer statistically only or with synthetic data & a plot) | ||
1. Test it on 20% held out data and generate a classification report | ||
2. Interpret the model and its performance in terms of the application in order to give a recommendation, "would you deploy this model" . Example questions to consider in your response include | ||
|
||
- do you think this model is good enough to use for real? | ||
- is this a model you would trust? | ||
- do you think that a more complex model should be used? | ||
- do you think that maybe this task cannot be done with machine learning? | ||
|
||
:::{note} | ||
You need to give a thorough answer to the deployment question and these bulleted questions will help you create a thorough response. | ||
::: | ||
|
||
## Exploring Problem Setups | ||
|
||
```{important} | ||
Understanding the impact of test/train size is a part of classifcation and helps with evaluation. This exercise is *also* a chance at python level 2. | ||
``` | ||
|
||
````{margin} | ||
```{tip} | ||
The summary statistics and visualization we used before are useful for helping to | ||
investigate the performance of our model. We can try fitting a model with different settings | ||
to create a new "dataset" for our experiments. | ||
The same skills apply. | ||
``` | ||
```{hint} | ||
The most important thing about the max depth here is that it's the same across all of the models. If you get an error, try making it smaller. | ||
``` | ||
```` | ||
Do an experiment to compare test set size vs performance: | ||
1. Use a loop to train a model on 10%, 30%, ... , 90% of the data. Compute the {term}`training accuracy` and test accuracy for each size training data. Create a DataFrame with columns ['train_pct','n_train_samples','n_test_samples','train_acc','test_acc'] | ||
2. Use EDA on this data frame to interpret the results of your experiment. How does training vs test size impact the model's performance? Does it impact training and test accuracy the same way? | ||
|
||
|
||
|
||
|
||
```{admonition} Thinking Ahead | ||
_ideas for level 3 evaluate, not required for A7_ | ||
Repeat the problem setup experiment with multiple test/train splits at each size and plot with error bars. | ||
- What is the tradeoff to be made in choosing a test/train size? | ||
- What is the best test/train size for this dataset? | ||
or with variations: | ||
- allowing it to figure out the model depth for each training size, and recording the depth in the loop as well. | ||
- repeating each size 10 items, then using summary statistics on that data | ||
Use the extensions above to experiment further with other model parameters. | ||
**some of this we'll learn how to automate in a few weeks, but getting the | ||
ideas by doing it yourself can help build understanding and intution** | ||
``` |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.