From b1207b71c099804d525b8d8024ee8b6c7f92ee32 Mon Sep 17 00:00:00 2001 From: prachi-0319 Date: Sat, 26 Oct 2024 21:20:30 +0200 Subject: [PATCH 1/2] Updated the tutorial.md and project-structure.md files --- docs/project-structure.md | 56 ++++++++++++++++++++++++++++++++++----- docs/tutorial.md | 6 ++++- 2 files changed, 54 insertions(+), 8 deletions(-) diff --git a/docs/project-structure.md b/docs/project-structure.md index aa1242ec..341449a3 100644 --- a/docs/project-structure.md +++ b/docs/project-structure.md @@ -4,14 +4,23 @@ The directory layout of **Predictive Calc** is organized in a way that separates ``` predictive-calc/ -├── app.py # Main entry point for the Streamlit web app -├── docs/ # Documentation files -├── form_configs/ # Configuration files for the forms +├── app.py # Main entry point for the Streamlit web app +├── docs/ # Documentation files +│ ├── project-structure.md # Directory layout of the repository +│ ├── tutorial.md # Steps to integrate a new machine learning model into the repository +├── form_configs/ # Configuration files for the forms +│ ├── customer_income.json # JSON configuration for customer income model form input fields +│ ├── gold_price_prediction.json # JSON configuration for gold price prediction model form input fields │ ├── house_price.json # JSON configuration for house price model form input fields │ ├── loan_eligibility.json # JSON configuration for loan eligibility model form input fields -├── models/ # Folder containing machine learning models -│ ├── house_price/ # Example model directory (for house price prediction) -│ │ ├── data/ # Datasets used by the models +│ ├── parkinson_detection.json # JSON configuration for parkinson disease detection model form input fields +│ ├── stress_detection.json # JSON configuration for stress detection model form input fields +├── models/ # Folder containing machine learning models +│ ├── PDF_malware_detection +│ ├── customer_income +│ ├── gold_price_prediction +│ ├── house_price/ # Example model directory (for house price prediction) +│ │ ├── data/ # Datasets used by the models │ │ ├── notebooks/ # Jupyter notebooks for training and experimentation │ │ │ ├── house_price.ipynb │ │ ├── saved_models/ # Serialized (pickled) model files and scalers @@ -20,13 +29,33 @@ predictive-calc/ │ │ ├── model.py # Code to define and train the model │ │ ├── predict.py # Code to make predictions using the trained model │ ├── modelEvaluation.py # Model evaluation class +│ ├── parkinson_disease_detector +│ ├── stress_level_detect +│ ├── text_sumarization +│ ├── translator_app ├── pages/ # Streamlit pages representing different calculators +│ ├── Customer_Income_Estimator.py +│ ├── Gold_Price_Predictor.py │ ├── House_Price_Estimator.py │ ├── Loan_Eligibility_Estimator.py +│ ├── PDF_Malware_Detection.py +│ ├── Parkinson_Disease_Detector.py +│ ├── Stress_Level_Detector.py +│ ├── Text Summarizer.py +│ ├── Translator.py │ ├── pages.json # Configuration file for managing page details and settings +├── assets/images/ # All assets using in the project +│ ├── machine_learning.png +│ ├── machine-learning.gif +├── Dockerfile # Instructions for building a Docker image +├── dev-requirements.txt # List of development dependencies for the project +├── docker-compose.debug.yml # To debug inside the docker container using Debugpy +├── docker-compose.yml # To set up a docker container for streamlit ├── form_handler.py # Handles dynamic form generation based on JSON configs ├── page_handler.py # Manages the page rendering logic and handles model predictions ├── requirements.txt # List of Python dependencies +├── packages.txt # List of Python packages +├── readme.md # Overview of the project and setup instructions ``` ### Key Components @@ -59,4 +88,17 @@ The `form_configs/` folder contains JSON configuration files that define the inp This script dynamically generates the input forms based on the JSON configuration files. It maps user inputs to the model’s expected parameters and passes the data to the prediction logic. #### 8. `docs/` -Contains the project's documentation. This is where you can find information about how the system is structured and how to contribute to the project. +Contains the project's documentation. This is where you can find information about how the system is structured and how to contribute to the project, more specifically the `tutorial.md` and `project-stuctures.md` files. + +#### 9. `docker-compose.yml` +This file helps you set up and run a Docker container for the Streamlit web application. It also monitors the app health to restart or alert if the app becomes unresponsive. +- `docker-compose.debug.yml` file is configured to run a service for remote debugging using debugpy, a popular debugging tool.Debugpy allows external tools to connect and debug the Python app inside the container. + +#### 10. `dev-requirements.txt` +This file lists packages that are used during the development of the project but are not necessary for running the Streamlit application in a production environment. These are typically used in developing, testing, and prototyping. + +#### 11. `Dockerfile` +Used to create a Docker image that contains a Streamlit application. Thsi file sets up the environment, installs necessary dependencies, copies the application code, and specifies how to run the application. + +#### 12. `assets/images/` +This folder contains all the assets used in the main project. \ No newline at end of file diff --git a/docs/tutorial.md b/docs/tutorial.md index 33d00992..2377c366 100644 --- a/docs/tutorial.md +++ b/docs/tutorial.md @@ -34,4 +34,8 @@ In the `pages/pages.json` file, add an entry for your new model. This configurat - The page name on the sidebar will be same as the file name. ### 7. Update the Main App -In `app.py`, update the list of available pages \ No newline at end of file +In `app.py`, update the list of available pages + +### 8. Update and Test the Dependencies +- If you've added or updated any dependencies (packages/modules/libraries) for your model, update the `requirements.txt` file accordingly. +- After updating, test the entire project to ensure there are no version conflicts between packages. This helps maintain a stable and reproducible environment for all the contributors. \ No newline at end of file From 6b9fefdda952cbf15c9bfdc7a09b22603cb947b7 Mon Sep 17 00:00:00 2001 From: Yashasvini Sharma <100478608+yashasvini121@users.noreply.github.com> Date: Sun, 27 Oct 2024 10:00:34 +0530 Subject: [PATCH 2/2] Update project-structure.md --- docs/project-structure.md | 111 +++++++++++++++++--------------------- 1 file changed, 49 insertions(+), 62 deletions(-) diff --git a/docs/project-structure.md b/docs/project-structure.md index 341449a3..ab85d07b 100644 --- a/docs/project-structure.md +++ b/docs/project-structure.md @@ -6,55 +6,42 @@ The directory layout of **Predictive Calc** is organized in a way that separates predictive-calc/ ├── app.py # Main entry point for the Streamlit web app ├── docs/ # Documentation files -│ ├── project-structure.md # Directory layout of the repository -│ ├── tutorial.md # Steps to integrate a new machine learning model into the repository +│ ├── project-structure.md # Directory layout of the repository +│ ├── tutorial.md # Steps to integrate a new machine learning model into the repository ├── form_configs/ # Configuration files for the forms -│ ├── customer_income.json # JSON configuration for customer income model form input fields -│ ├── gold_price_prediction.json # JSON configuration for gold price prediction model form input fields -│ ├── house_price.json # JSON configuration for house price model form input fields -│ ├── loan_eligibility.json # JSON configuration for loan eligibility model form input fields -│ ├── parkinson_detection.json # JSON configuration for parkinson disease detection model form input fields -│ ├── stress_detection.json # JSON configuration for stress detection model form input fields +│ ├── house_price.json # JSON configuration for house price model form input fields +│ ├── loan_eligibility.json # JSON configuration for loan eligibility model form input fields +│ ├── ... ├── models/ # Folder containing machine learning models -│ ├── PDF_malware_detection -│ ├── customer_income -│ ├── gold_price_prediction -│ ├── house_price/ # Example model directory (for house price prediction) -│ │ ├── data/ # Datasets used by the models -│ │ ├── notebooks/ # Jupyter notebooks for training and experimentation +│ ├── house_price/ # Example model directory (for house price prediction) +│ │ ├── data/ # Datasets used by the models +│ │ ├── notebooks/ # Jupyter notebooks for dataset exploration and model training │ │ │ ├── house_price.ipynb -│ │ ├── saved_models/ # Serialized (pickled) model files and scalers -│ │ │ ├── model.pkl # Trained model for predictions -│ │ │ ├── scaler.pkl # Scaler for data normalization -│ │ ├── model.py # Code to define and train the model -│ │ ├── predict.py # Code to make predictions using the trained model -│ ├── modelEvaluation.py # Model evaluation class -│ ├── parkinson_disease_detector -│ ├── stress_level_detect -│ ├── text_sumarization -│ ├── translator_app +│ │ ├── saved_models/ # Serialized (pickled) model files and scalers +│ │ │ ├── model.pkl # Trained model for predictions +│ │ │ ├── scaler.pkl # Scaler for data normalization +│ │ ├── model.py # Code to define and train the model +│ │ ├── predict.py # Code to make predictions using the trained model. Contain get_prediction() function +│ ├── modelEvaluation.py # Model evaluation class generates plots and metrics +│ ├── ... ├── pages/ # Streamlit pages representing different calculators -│ ├── Customer_Income_Estimator.py -│ ├── Gold_Price_Predictor.py +│ ├── pages.json # Configuration file for managing page details and settings │ ├── House_Price_Estimator.py │ ├── Loan_Eligibility_Estimator.py -│ ├── PDF_Malware_Detection.py -│ ├── Parkinson_Disease_Detector.py -│ ├── Stress_Level_Detector.py -│ ├── Text Summarizer.py -│ ├── Translator.py -│ ├── pages.json # Configuration file for managing page details and settings -├── assets/images/ # All assets using in the project -│ ├── machine_learning.png -│ ├── machine-learning.gif -├── Dockerfile # Instructions for building a Docker image -├── dev-requirements.txt # List of development dependencies for the project -├── docker-compose.debug.yml # To debug inside the docker container using Debugpy +│ ├── ... +├── assets/ +│ ├── images/ # Image assets used in the Streamlit app +│ │ ├── machine-learning.png +│ │ ├── machine-learning.gif +│ │ ├── ... +├── form_handler.py # Class to handle dynamic form generation based on JSON configs +├── page_handler.py # Class to manage the page rendering logic and handles model predictions +├── requirements.txt # List of Python dependencies required for the Streamlit App +├── dev-requirements.txt # Development dependencies, including Jupyter and tools for local use, excluding those needed for production Streamlit +├── packages.txt # List of Ubuntu packages required for Streamlit App +├── Dockerfile # Instructions for building a Docker Image ├── docker-compose.yml # To set up a docker container for streamlit -├── form_handler.py # Handles dynamic form generation based on JSON configs -├── page_handler.py # Manages the page rendering logic and handles model predictions -├── requirements.txt # List of Python dependencies -├── packages.txt # List of Python packages +├── docker-compose.debug.yml # To debug inside the docker container using Debugpy ├── readme.md # Overview of the project and setup instructions ``` @@ -63,7 +50,13 @@ predictive-calc/ #### 1. `app.py` This is the entry point for the **Streamlit** application. It initializes the app and renders the home page, and loads the model calculators having their pages in the `pages/` directory. -#### 2. `models/` +#### 2. `page_handler.py` +The page_handler.py file manages the rendering of pages in the Predictive Calc application by reading configurations from the pages.json file. It dynamically loads page titles, icons, and model paths, ensuring a smooth user experience. The class integrates model prediction logic and utilizes the `FormHandler` to generate dynamic forms based on the specified configurations. It also manages multiple tabs, enhancing functionality and allowing for easy updates or additions of new models while maintaining a cohesive interface. + +#### 3. `form_handler.py` +This script dynamically generates the input forms based on the JSON configuration files. It maps user inputs to the model’s expected parameters and passes the data to the prediction logic. + +#### 4. `models/` Each model gets its own folder within the `models/` directory, which contains all necessary files for that particular model. This includes: - **notebooks/**: Jupyter notebooks for model training and experimentation. - **model.py**: Code defining and training the finally chosen machine learning model. @@ -72,33 +65,27 @@ Each model gets its own folder within the `models/` directory, which contains al - **data/**: Raw datasets used for training the model. - **modelEvaluation.py**: Scripts for model evaluation and reporting. -#### 3. `pages/` +#### 5. `pages/` Each model has a corresponding Streamlit page in the `pages/` folder. This page handles the frontend logic, rendering the forms for user input and displaying the prediction results. For example, the `house_price_calculator.py` page contains the interface for the house price prediction model. -#### 4. `pages/pages.json` +#### 6. `pages/pages.json` This file manages the configuration for all pages in the application, defining attributes such as titles, icons, model paths, and tab configurations for each calculator. This centralizes the configuration for easier updates and management of multiple pages. -#### 5. `page_handler.py` -The page_handler.py file manages the rendering of pages in the Predictive Calc application by reading configurations from the pages.json file. It dynamically loads page titles, icons, and model paths, ensuring a smooth user experience. The class integrates model prediction logic and utilizes the `FormHandler` to generate dynamic forms based on the specified configurations. It also manages multiple tabs, enhancing functionality and allowing for easy updates or additions of new models while maintaining a cohesive interface. - -#### 6. `form_configs/` +#### 7. `form_configs/` The `form_configs/` folder contains JSON configuration files that define the input fields required by each model. These JSON files dictate how the forms are dynamically generated by `form_handler.py`. For example, the `house_price.json` file specifies the input fields (e.g., square footage, number of bedrooms, etc.) needed for the house price prediction model. -#### 7. `form_handler.py` -This script dynamically generates the input forms based on the JSON configuration files. It maps user inputs to the model’s expected parameters and passes the data to the prediction logic. - #### 8. `docs/` Contains the project's documentation. This is where you can find information about how the system is structured and how to contribute to the project, more specifically the `tutorial.md` and `project-stuctures.md` files. -#### 9. `docker-compose.yml` -This file helps you set up and run a Docker container for the Streamlit web application. It also monitors the app health to restart or alert if the app becomes unresponsive. -- `docker-compose.debug.yml` file is configured to run a service for remote debugging using debugpy, a popular debugging tool.Debugpy allows external tools to connect and debug the Python app inside the container. - -#### 10. `dev-requirements.txt` -This file lists packages that are used during the development of the project but are not necessary for running the Streamlit application in a production environment. These are typically used in developing, testing, and prototyping. +#### 9. `requirements.txt`, `dev-requirements.txt`, `packages.txt` +- `requirements.txt` contains the Python dependencies required to run the Streamlit application. +- `dev-requirements.txt` includes additional dependencies for development purposes, such as Jupyter notebooks and other tools. +- `packages.txt` lists the Ubuntu packages required for the Streamlit application to run correctly. -#### 11. `Dockerfile` -Used to create a Docker image that contains a Streamlit application. Thsi file sets up the environment, installs necessary dependencies, copies the application code, and specifies how to run the application. +#### 10. `Dockerfile`, `docker-compose.yml`, `docker-compose.debug.yml` +- `Dockerfile` contains the instructions for building a Docker image that can run the Streamlit application. +- `docker-compose.yml` sets up a Docker container for the Streamlit application. +- `docker-compose.debug.yml` allows debugging inside the Docker container using Debugpy. -#### 12. `assets/images/` -This folder contains all the assets used in the main project. \ No newline at end of file +#### 11. `assets/` +This folder contains all the assets used in the main project.