Add colabs for ultralytics integration (#438)

* add: inference notebooks + assets

* update: inference notebook with download links for images

* add: train val notebook

* update: inference notebook

* update: train val notebook

* update: inference notebook

* updated: colabs

* update: readme

colabs/README.md

@@ -15,6 +15,8 @@
 | HuggingFace | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](http://wandb.me/huggingface-colab) |
 | spaCy v3 | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](http://wandb.me/spacy-colab) |
 | YOLOv5 | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](http://wandb.me/yolo-colab) |
+| Ultralytics Train | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](http://wandb.me/ultralytics-train) |
+| Ultralytics Inference | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](http://wandb.me/ultralytics-inference) |
 | Ray/Tune | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](http://wandb.me/raytune-colab) |
 | 🤗 Diffusers | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](http://wandb.me/diffusers-uncond-colab) |
 | Kaolin-Wisp | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](http://wandb.me/vqad-colab) |

colabs/ultralytics/00_inference.ipynb

@@ -0,0 +1,161 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# 🔥🔥 Explore Predictions from Ultralytics models using Weights & Biases 🪄🐝\n",
+    "\n",
+    "<!--- @wandbcode{ultralytics-inference} -->\n",
+    "\n",
+    "This notebook demonstrates a typical workflow of using an [Ultralytics](https://docs.ultralytics.com/modes/predict/) model for inference and visualizing the results using [Weights & Biases](https://wandb.ai/site).\n",
+    "\n",
+    "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](http://wandb.me/ultralytics-inference)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "![](./assets/interactive_bbox.gif)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# Install Dependencies\n",
+    "\n",
+    "- Install Ultralytics using `pip install ultralytics`. In order to learn about more ways to install Ultralytics, you can check out the [official docs](https://docs.ultralytics.com/quickstart/#install-ultralytics).\n",
+    "\n",
+    "- Then, you need to install the [`feat/ultralytics`](https://github.com/wandb/wandb/tree/feat/ultralytics) branch from W&B, which currently houses the out-of-the-box integration for Ultralytics."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Install Ultralytics\n",
+    "!pip install -q ultralytics\n",
+    "\n",
+    "# Install the `feat/ultralytics`` branch from W&B,\n",
+    "# which currently houses the out-of-the-box integration for Ultralytics.\n",
+    "!pip install -q git+https://github.com/wandb/wandb@feat/ultralytics"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "**Note:** The Ultralytcs integration will be soon available as a fully supported feature on Weights & Biases once [this pull request](https://github.com/wandb/wandb/pull/5867) is merged."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# Using Ultralytics with Weights & Biases\n",
+    "\n",
+    "In order to use the W&B integration with Ultralytics, we need to import the `wandb.yolov8.add_wandb_callback` function."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import wandb\n",
+    "from wandb.yolov8 import add_wandb_callback\n",
+    "\n",
+    "from ultralytics.engine.model import YOLO"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Now, let us download a few images to test the integration on. You can use your own images, videos or camera sources. For more information on inference sources, you can check out the [official docs](https://docs.ultralytics.com/modes/predict/)."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "!wget https://raw.githubusercontent.com/wandb/examples/ultralytics/colabs/ultralytics/assets/img1.png\n",
+    "!wget https://raw.githubusercontent.com/wandb/examples/ultralytics/colabs/ultralytics/assets/img2.png\n",
+    "!wget https://raw.githubusercontent.com/wandb/examples/ultralytics/colabs/ultralytics/assets/img4.png\n",
+    "!wget https://raw.githubusercontent.com/wandb/examples/ultralytics/colabs/ultralytics/assets/img5.png"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Next, we initialize a W&B [run](https://docs.wandb.ai/guides/runs) using `wandb.init`."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Initialize Weights & Biases run\n",
+    "wandb.init(project=\"ultralytics\", job_type=\"inference\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Next, we initialize the `YOLO` model of our choice, and invoke the `add_wandb_callback` function on it before performing inference with the model. This would ensure that when we perform inference, it would automatically log the images overlayed with our [interactive overlays for computer vision tasks](https://docs.wandb.ai/guides/track/log/media#image-overlays-in-tables) along with additional insights in a [`wandb.Table`](https://docs.wandb.ai/guides/data-vis)."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "model_name = 'yolov8n' #@param {type:\"string\"}\n",
+    "\n",
+    "# Initialize YOLO Model\n",
+    "model = YOLO(f\"{model_name}.pt\")\n",
+    "\n",
+    "# Add Weights & Biases callback for Ultralytics\n",
+    "add_wandb_callback(model, enable_model_checkpointing=True)\n",
+    "\n",
+    "# Perform prediction which automatically logs to a W&B Table\n",
+    "# with interactive overlays for bounding boxes, segmentation masks\n",
+    "model([\"./assets/img1.jpeg\", \"./assets/img3.png\", \"./assets/img4.jpeg\", \"./assets/img5.jpeg\"])\n",
+    "\n",
+    "# Finish the W&B run\n",
+    "wandb.finish()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Next, you can check out the following notebook to learn how to perform experiment tracking and visualize validation predictions during training using Weights & Biases in the following notebook:\n",
+    "\n",
+    "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](http://wandb.me/ultralytics-train)\n",
+    "\n",
+    "In order to learn more about using Weights & Biases with Ultralytics, you can also read the report: [**Supercharging Ultralytics with Weights & Biases**](https://wandb.ai/geekyrakshit/ultralytics/reports/Supercharging-Ultralytics-with-Weights-Biases--Vmlldzo0OTMyMDI4)"
+   ]
+  }
+ ],
+ "metadata": {
+  "language_info": {
+   "name": "python"
+  },
+  "orig_nbformat": 4
+ },
+ "nbformat": 4,
+ "nbformat_minor": 2
+}

colabs/ultralytics/01_train_val.ipynb

@@ -0,0 +1,150 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# 🔥🔥 Explore Predictions from Ultralytics models using Weights & Biases 🪄🐝\n",
+    "\n",
+    "<!--- @wandbcode{ultralytics-train} -->\n",
+    "\n",
+    "This notebook demonstrates a typical workflow of using an [Ultralytics](https://docs.ultralytics.com/modes/predict/) model for training, fine-tuning, and validation and performing experiment tracking, model-checkpointing, and visualization of the model's performance using [Weights & Biases](https://wandb.ai/site).\n",
+    "\n",
+    "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](http://wandb.me/ultralytics-train)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# Install Dependencies\n",
+    "\n",
+    "- Install Ultralytics using `pip install ultralytics`. In order to learn about more ways to install Ultralytics, you can check out the [official docs](https://docs.ultralytics.com/quickstart/#install-ultralytics).\n",
+    "\n",
+    "- Then, you need to install the [`feat/ultralytics`](https://github.com/wandb/wandb/tree/feat/ultralytics) branch from W&B, which currently houses the out-of-the-box integration for Ultralytics."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Install Ultralytics\n",
+    "!pip install -q ultralytics\n",
+    "\n",
+    "# Install the `feat/ultralytics`` branch from W&B,\n",
+    "# which currently houses the out-of-the-box integration for Ultralytics.\n",
+    "!pip install -q git+https://github.com/wandb/wandb@feat/ultralytics"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "**Note:** The Ultralytcs integration will be soon available as a fully supported feature on Weights & Biases once [this pull request](https://github.com/wandb/wandb/pull/5867) is merged."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# Using Ultralytics with Weights & Biases\n",
+    "\n",
+    "In order to use the W&B integration with Ultralytics, we need to import the `wandb.yolov8.add_wandb_callback` function."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import wandb\n",
+    "from wandb.yolov8 import add_wandb_callback\n",
+    "\n",
+    "from ultralytics.engine.model import YOLO"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Next, we initialize the `YOLO` model of our choice, and invoke the `add_wandb_callback` function on it before performing inference with the model. This would ensure that when we perform training, fine-tuning, validation, or inference, it would automatically log the experiment logs and the images overlayed with both ground-truth and the respective prediction results using the [interactive overlays for computer vision tasks](https://docs.wandb.ai/guides/track/log/media#image-overlays-in-tables) on W&B along with additional insights in a [`wandb.Table`](https://docs.wandb.ai/guides/data-vis)."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "model_name = \"yolov8n\" #@param {type:\"string\"}\n",
+    "dataset_name = \"coco128.yaml\" #@param {type:\"string\"}\n",
+    "\n",
+    "# Initialize YOLO Model\n",
+    "model = YOLO(f\"{model_name}.pt\")\n",
+    "\n",
+    "# Add Weights & Biases callback for Ultralytics\n",
+    "add_wandb_callback(model, enable_model_checkpointing=True)\n",
+    "\n",
+    "# Train/fine-tune your model\n",
+    "# At the end of each epoch, predictions on validation batches are logged\n",
+    "# to a W&B table with insightful and interactive overlays for\n",
+    "# computer vision tasks\n",
+    "model.train(project=\"ultralytics\", data=dataset_name, epochs=5, imgsz=640)\n",
+    "model.val()\n",
+    "\n",
+    "# Finish the W&B run\n",
+    "wandb.finish()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# Sample Experiment Tracking"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "![](./assets/experiment.gif)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# Epoch-wise results visualized"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "![](./assets/table.gif)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Next, you can check out the following notebook to learn how to perform inference and visualize predictions during training using Weights & Biases in the following notebook:\n",
+    "\n",
+    "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](http://wandb.me/ultralytics-inference)\n",
+    "\n",
+    "In order to learn more about using Weights & Biases with Ultralytics, you can also read the report: [**Supercharging Ultralytics with Weights & Biases**](https://wandb.ai/geekyrakshit/ultralytics/reports/Supercharging-Ultralytics-with-Weights-Biases--Vmlldzo0OTMyMDI4)"
+   ]
+  }
+ ],
+ "metadata": {
+  "language_info": {
+   "name": "python"
+  },
+  "orig_nbformat": 4
+ },
+ "nbformat": 4,
+ "nbformat_minor": 2
+}