To fall or not to fall

In the project we tried to apply all our knowledge in the area of statisitcal learning by focusing on:

• feature extraction through signal processing techniques
• training of various classification models (SVM, KNN, Logistic, NN etc.)
• hyperparamenter selection for each model and model selection
• compression of deep learning models

We invite you to read REPORT.pdf for a more detailed explanation of the algorithms used and the reasoning followed.

The project has the following goals:

• to simulate a collection of falls by using specific sensors and experimenting with different fall modes
• to try to mitigate the effect of bias due to the synthetic (simulation) approach by special feature extraction algorithms using a small amount of real falls data.
• to develop fall classification algorithms both based on classical machine learning models and neural network models
• to use the best model obtained to implement a practical solution to fall detection, based on the TinyML framework.

Project Structure

ToFall_orNot_toFall-main
│   .gitignore                
│   LICENSE                   
│   README.md                 # The file you are reading right now
│   REPORT.pdf                # Detailed project report
│   desktop.ini               # Windows folder settings file
│   main.ipynb                # Jupyter notebook with the project's main code, using only classic ML algorithms (both supervied and unsup. frameworks)
│
├───data collection           # Scripts and instructions for data collection
│       Accell&Gyrosc.ino     # Arduino sketch for accelerometer and gyroscope signals collection 
│       README.md             # Instructions for data collection
│       collect.py            # Python script to collect data from Arduino 
│
├───deep learning             # Deep learning models and scripts
│   │   deep_learning.ipynb   # Notebook with deep learning model training and evaluation
│   │
│   ├───models                # TensorFlow Lite model (compressed version of best TF model)
│   │   │   model.tflite
│   │   │   model_no_quant.tflite
│   │   │
│   │   └───model             # Best TensorFlow model weights directory
│   │       │   keras_metadata.pb
│   │       │   saved_model.pb
│   │       │
│   │       └───variables     # Model variables
│   │               variables.data-00000-of-00001
│   │               variables.index
│   │
│   └───tinyML                # TinyML model and scaling scripts for microcontrollers
│           StandardScaler.cc # C++ standard scaler implementation
│           StandardScaler.h 
│           main.ino          # Arduino main script for deploying the TinyML model
│           model.h           
│
├───feature extraction        # Scripts for extracting features from the dataset
│       feature_extraction.py # Python script for feature extraction
│       file_csv.zip          # Zipped CSV files with real falls to extract interesting features
│
├───images                    # Various images used in documentation and reports
│       ...
│
└───main functions            # Helper functions for the project
        nice_plots.py         # Functions for generating nice plots
        supervised_funcs.py   # Supervised learning helper functions used in the main notebook

Data collection

Refer to the README.md file in the data collection directory of this repository for specific info about the collection precess. We used accelerometer and gyroscope sensors to collect falls and normal activities signals simulating real-life context.

Reverse fall	Step	Fall

Features extraction

You can find the code and additional data for feature extraction in the feature extraction directory.

Fourier analysis of signals.

Finding the most informative frequencies and applying a lowpass filter (we simply thresholded at the first k frequencies since the lower frequencies contain higher energy).

Wavelet-based algorithm for similarity with real falls data that you can find in feature_extraction\file_csv.zip file.

The final data set will consist of the following features:

• 5 peaks of the magnitude of the power spectrum density for the accelerometer signal and 5 peaks for the gyroscope;
• 3 statistics (median, mad and skewness) for the two PSDs;
• 1 cwt coefficient for the similarity with the accelerometer peak mother wavelet. 8

Binary classification

For simplicity we started with a simply classification of signals with or without falls using several ML algorithms.

Here a brief summary display of our results:

Multi-class classification

After scaling, resampling with a K-Means SMOTE and selecting a coherent number of classes through an Unsupervised learning approach we got the following results:

1. The best number of classes according to a K-Means-based cluster analysis is 6 and, according to their coherence score we collapsed fall and light to the same label.

2. After the above analysis we fitted several ML algorithms:

KNN	SVM	Softmax

AdaBoost	Random forest	Perceptron

Multi-class classification with deep learning

We therefore decided to try a series of neural architectures that did not require the use of special feature extraction techniques. As we expected, we obtained results comparable or superior to the best classical machine learning models.

We noticed how, therefore, the deep learning approach avoids great efforts to extract fundamental features from the signals, at the expense of a lack of interpretability of the results.

Here we report the results for our best deep learning model: the CBAM-EDU (Convolutional-Bidirectional LSTM with Educational Enhancements).