Application of deep learning techniques to images collected with Charge Coupled Devices to search for Dark Matter
🎓 Publication: https://repositorio.unican.es/xmlui/handle/10902/20627
The Standard Model of particle physics, while being able to make accurate predictions, has been proved to fail to explain various phenomena, such as astronomical dark matter observations. In this work, a machine learning application has been implemented with the goal of studying dark matter candidates. Images from Charge Coupled Devices (CCDs) in different experiments DAMIC/DAMIC-M located underground are used to test different deep learning algorithms. A U-Net model is trained with Python. The model performs multi-class image segmentation in order to detect dark matter particle signals among background noise.
ℹ️ For more information regarding dark matter please read Theoretical Concepts: Dark Matter (DM)
|
|
|
These instructions explain how to get a copy of the project to run it on your local machine for development and testing purposes.
The project can be either cloned or downloaded to your own device. The source code of the application is implemented in Python, with the requirement of the following open-source libraries (version numbers are up to date: 21.06.2020):
- TensorFlow (2.1.0)
- Keras (2.3.1)
- Numpy (1.18.1)
- scikit-learn (0.22.1)
- scikit-image (0.9.3)
- OpenCV-Python (4.0.1)
- imgaug (0.4.0)
- Matplotlib (3.2.1)
- Pillow (7.1.2)
The libraries can manually be installed from the anaconda prompt with the command pip install 'library_name'. Make sure the correct working environment is activated.
If a module cannot be properly installed (installing tensorflow might sometimes be troublesome), doing it through the anaconda navigator is a good option.
In this section the core of the project is dissected. Every employed method is explained; from the origination of an image, to the training of the model, every necessary step in the creation of the deep learning application is analyzed.
📈 Function: sets details of the simulated images that are created in image_simulation.py. The pixel intensity value of each element in the image can be defined.
ℹ️ For more information regarding the image simulation please read Theoretical Concepts: Image Simulation
📈 Function: creates simulated images. Parameters such as number of images, elements, noise, glowing, etc. can be defined.
Images are saved to the saving path; arrays containing the predefined pixel intensity values are saved.
Testing and training images can be created. There is no need to create a validation dataset. The model automatically shuffles all images and creates a validation split in the training process.
Simulated 256X256 pixel CCD image containing glowing, hot pixels and pixel clusters. The pixel intensity values are given in ADCs.
👮 Requirements: working directory path must contain image_details.py and Cluster.pkl.
ℹ️ For more information regarding the image simulation please read Theoretical Concepts: Image Simulation
📈 Function: calculates the weights for the loss function and processes FITS files (DAMIC images). Originaly created to load the datasets as PNG files into arrays (unused in version 2.0).
-
load_images(commented) is no longer used in version 2.0. It was used in version 1.0 for loading the PNG images.
-
get_weights is used to avoid issues with imbalanced datasets, where images are too biased towards one class. In a case where >95% of the pixels are labeled as background, and <1% as clusters, the model can give a 95% accuracy prediction, but all pixels might be predicted as background, giving a meaningless output. Please read Theoretical Concepts: Network Implementation (Loss Function) for more information. The weight of each class is obtained as the inverse of its frequency in the training samples. The weights are then normalized to the number of classes. These weights are used by the model in the weighted_categorical_crossentropy loss function.
-
process_fits is used to process the DAMIC images, which are given in FITS (Flexible Image Transport System) file format. These files are read and saved as arrays that contain the collected charge by each CCD pixel. Since 256X256 pixel images are used for training the model, the DAMIC image is divided into sections of the same size, so they can be individually passed through the trained model, obtaining their respective predicted labels. The possibility to normalize, cut and resize the image is given.
-
images_small2big is used to reconstruct the predictions of all sections into a full segmentation map.
-
check_one_object is used to analyze the final output. It looks for a chosen category (i.e. 'Clusters') section by section. It returns a modified predicted label; it only shows background and the pixels classified as a chosen class.
📈 Function: creates the labels in an automated way, visualizes the output labels and can analyze and modify the images.
- get_monochrome converts the input image into a monochrome image. Input shape = (number of images, height, width, 3(RGB)) --> Output shape = (number of images, height, width, 1).
- get_class classifies each pixel as a certain class depending on its intensity value.
- get_mask creates labels from input images. Input shape = (number of images, height, width, 3(RGB)) --> Output shape = (number of images, height, width, number of classes), where number of classes = 4 in this project.
- get_max_in_mask takes the position of the maximum value, i.e., the class. Input shape = (number of images, height, width, number of classes) --> Output shape = (number of images, height, width, 1).
- mask_to_label creates the segmentatin map that can be visualized. It applies a color multiplier to each class. Input shape = (number of images, height, width, 1) --> Output shape = (number of images, height, width, 3(RGB)).
- mask_to_label_one_object creates the segmentatin map to visualize a chosen class. It is used by check_one_object. It applies a color multiplier to that one class. Input shape = (number of images, height, width, 1) --> Output shape = (number of images, height, width, 3(RGB)).
- statistics shows the number of classes and their frequency on the dataset.
- get_percentages simply returns the percentages of each class. This is used to calculate the weights for the loss function by get_weights.
- visualize_label is used to visualize the created label.
- create_masks takes the images as input and returns the labels. These labels are used by the model to train and evaluate the model while training.
- create_labels takes the images as input and returns the segmentation maps that can be visualized.
- create_labels_noStat_noPrint is the same as create_labels but it does not print the information in the console. Created in order to avoid the repeated information shown by the console.
- output_to_label takes the model outputs and converts them into labels that can be visualized.
- output_to_label_one_object takes the model outputs and converts them into labels where a chosen class is visualized.
- rgb2random randomly changes the RGB colors of the images (unused).
- rgb2gray converts the images to grayscale (unused).
- contrast augments the contrast of the image (unused).
- percentage_result gives the percentage of each class in an output label.
Image label example, where each pixel is classified as a class and its corresponding depth channel takes value 1. Adapted Digital Image. Jordan, J. "An overview of semantic image segmentation". (2018). Link.
Labels can perfectly be created using a labeling software. However, for the purpose of this project, automatic pixel-wise labeling is a practical solution. Remember that, in case of using your own labels, image and label names must match.
ℹ️ For more information regarding the labeling process please read Theoretical Concepts: Image Labeling
📈 Function: applies data augmentation techniques to both images and corresponding labels. Please read the imgaug documentation for more information on augmentation techniques. This is an optional step; it is applied when only a few training samples are available, or when the desired property is not present in the dataset.
- augmentation_sequence_Color and augmentation_Color apply color dropout, rotation, flipping.
- augmentation_sequence_Invert and augmentation_Invert apply color channel inversion, dropout, logContrast, hue, gammaContrast.
Image and label augmentation example. The applied transformations are translation and scaling.
ℹ️ For more information regarding data augmentation please read Theoretical Concepts: Data Augmentation
📈 Function: defines the model architecture and layer features. The model has U-Net architecture. The code is already prepared to add or remove layers in the model. Additionally, pretrained weights from an already trained model can be used.
- weighted_categorical_crossentropy loss function is used to calculate the categorical crossentropy loss of the model with the modification of taking into account the weight of each class.
Implemented U-Net architecture. Adapted from Ronneberger, O. et al. "Convolutional Networks for Biomedical Image Segmentation". (2015). Link.
ℹ️ For more information regarding ML (CNNs, layers, image segmentation) please read Theoretical Concepts: Machine Learning
ℹ️ For more information regarding the implementation of the network (employed layers, activation functions, loss function, optimizer) please read Theoretical Concepts: Network Implementation
📈 Function: trains a model with the defined parameters. Process:
- Loads the datasets.
- Creates the labels.
- (Optional) Applies augmentation on both images and labels.
- Trains the model with the defined hyperparameters and callbacks. For more information regarding callbacks please read the keras callbacks documentation.
- Plots and saves the accuracy and loss over the training process.
- Gives the predicted outputs of training and testing images.
- Evaluates the model on the test set.
- Gives a classification report that analyzes the performance of each class. For more information regarding the classification reports please read the scikit-learn classification report documentation.
The trained model and all the figures are saved in the defined paths.
Example of the console display while training
👮 Requirements: working directory path must contain models.py, mask.py and augmentation.py(if used).
ℹ️ For more information regarding the model training process please read Theoretical Concepts: Model Training
📈 Function: loads an already trained model and makes predictions on the FITS file (the DAMIC CCD image). Process:
- Loads the trained model.
- Loads and processes the FITS file.
- Creates small sections from the DAMIC image.
- Passes each section individually through the model.
- Reconstructs all sections into a prediction label.
- Looks for a chosen category (i.e. 'Clusters') section by section.
👮 Requirements: working directory path must contain load_dataset.py, models.py and mask.py.
The Python application consists on the 8 files previously explained. Only the last two (train.py and load_model.py) are executed.
Python implementation summary.
An imbalanced dataset entails further problems. A good solution to this issue is creating balanced images, with approximately the same percentage of presence of each class. The classes are not mixed in order to avoid confusion to the model when labeling the images. Additionally, only 60% of the images contain glowing, and it does not always start from the first pixel. This way, the model does not learn that all predictions should have a glowing column, nor where should it be. Here is an example of an image used for training the model:
Simulated 256×256 pixel image with a similar number of pixels belonging to each class.
For this project 180 training and 42 test images are created. The model is trained with the following hyperparameters:
- epochs = 100
- batch_size = 1
- split = 0.21
- dropout = 0.18
- activation = 'elu'
- loss = weighted_categorical_crossentropy(weights)
- optimizer = 'adadelta'
Please note that these parameters work well for this particular dataset, but do not assure reliable results for other datasets.
The loss and accuracy of the training and evaluation set verify that the model does not suffer from overfitting.
Training process of the model, that reaches its optimum state at epoch 40.
LEFT: Training and validation accuracy.
RIGHT: Training and validation loss.
The model performs correctly on the test dataset, segmenting the objects within the images.
LEFT: Test image to be passed through the trained model.
MIDDLE: Correct label of the test image.
RIGHT: Predicted label by the model.
The accuracy of the model on the test set is: 99.21%
The loss of the model on the test set is: 0.304
The classification report showes how efficiently each class performed:
| Class | Precision | Recall | F1-score | Support |
|---|---|---|---|---|
| Background | 1.00 | 0.99 | 1.00 | 2535931 |
| Glowing | 1.00 | 1.00 | 1.00 | 187442 |
| Hot pixel | 0.99 | 0.98 | 0.95 | 10594 |
| Cluster | 0.44 | 0.63 | 0.52 | 18545 |
Precision: percentage of correctly classified pixels among all pixels classified as the given class.
Recall: percentage of correctly classified pixels among all pixels that truly are of the given class.
F1-score: harmonic mean between precision and recall.
Best score is 1.00 and worst score is 0.00.
Support: number of pixels of the given class in the dataset.
The model also gives a seemingly correct prediction of a real image. Due to the small size of the objects, these cannot be seen when the whole image was displayed. If the 256×256 sections are individually observed instead, the segmented clusters can be analyzed.
LEFT: Real image 256X256 section to be passed through the trained model. The pixel intensity values are given in ADCs.
RIGHT: Predicted label with segmented clusters.
The application’s shortcomings are revealed when a more complex image is passed through the model, obtaining a prediction that is not entirely correct.
LEFT: DAMIC image (T=240K). The pixel intensity values are given in ADCs.
RIGHT: Predicted label with unusual hot pixel and cluster segmentation on the bottom right side..
- In order to deal with this issue, defining geometrical restrictions is a possible solution. For instance, imposing a hot pixel prediction to have its expected shape, which is avertical or horizontal line. Likewise a maximum cluster size can be defined.
- Additionally, glowing predictions are likely to improve if different, more realistic shapes are generated in the simulated dataset.
- The results might suggest that a model is limited to the range of intensities to which it is trained. However, applying different normalization techniques, such as linear scaling or clipping, to the DAMIC images is a seemingly good approach to solve this limitation.
- Moreover, in future work more classification categories can be added, allowing particle identification by measuring their energy, and possible DM signals could be discerned.
Feel free to submit pull requests.
Please read CONTRIBUTING.md for details on the code of conduct, and the process for submitting pull requests.
If you use this code in a publicly available project, please post an issue or create a pull request and your project link will be added here.
-
First release: v1.0 CNN on simulated images. Only for simulated images. Not for real images.
-
Second release: v2.0 CNN Image Segmentation. For both simulated and real images.
I express my sincere gratitude to my director, Rocío Vilar Cortabitarte, and co-director, Alicia Calderón Tazón, for providing their expertise and guidance throughout the course of this project. I also thank the rest of my advisors, Agustín Lantero Barreda and Núria Castelló-Mor, who contributed so thoroughly through their assistance and dedicated involvement.
Copyright 2020, Aritz Lizoain. License.