roboflow modular service

This module implements the rdk vision API in a viam-labs:vision:roboflow model.

This model leverages the Roboflow python library to allow for object detection and classification from Roboflow Universe models.

Roboflow models can be used either remotely (via Roboflow APIs) or locally (automatically downloaded from Roboflow).

Prerequisites

In order to use Roboflow models, you must sign up for a Roboflow account and retrieve a private API key.

If you wish to run Roboflow models locally (on your robot or smart machine), you must also install Docker. This step is not required if you plan on only using hosted Roboflow models (Roboflow Inference API).

 curl -fsSL https://get.docker.com -o get-docker.sh
 sudo sh get-docker.sh

Build and Run

To use this module, follow these instructions to add a module from the Viam Registry and select the viam-labs:vision:roboflow model from the viam-labs roboflow module.

Configure your vision

Note

Before configuring your vision, you must create a machine.

Navigate to the Config tab of your robot’s page in the Viam app. Click on the Components subtab and click Create component. Select the vision type, then select the viam-labs:vision:roboflow model. Enter a name for your vision and click Create.

On the new component panel, copy and paste the following attribute template into your vision service's Attributes box:

{
  "project": "<roboflow_project_identifier>",
  "version": "<roboflow_model_version>",
  "workspace": "<roboflow_workspace_identifier>",
  "api_key": "<roboflow_private_api_key>",
  "local": "<run_locally_true_or_false>"
}

Note

For more information, see Configure a Robot.

Attributes

The following attributes are available for viam-labs:vision:roboflow model:

Name	Type	Inclusion	Description
project	string	Required	Roboflow project identifier
version	int	Required	Roboflow model version
workspace	string		Roboflow workspace identifier, not required for public projects
api_key	string	Required	Roboflow private API key
local	bool		Whether or not to run this model locally (default: false)
jetpack	string		If running on Nvidia, you must specify Jetpack version (4.5, 4.6, 5.1 are supported)

Example Configurations

Bone Fracture Detection:

{
  "project": "bone-fracture-detection-khatl",
  "version": 8,
  "api_key": "YOURAPIKEYGOESHERE"
}

API

The roboflow resource provides the following methods from Viam's built-in rdk:service:vision API

get_detections(image=binary)

get_detections_from_camera(camera_name=string)

Note: if using this method, any cameras you are using must be set in the depends_on array for the service configuration, for example:

      "depends_on": [
        "cam"
      ]

get_classifications(image=binary)

get_classifications_from_camera(camera_name=string)

Note: if using this method, any cameras you are using must be set in the depends_on array for the service configuration, for example:

      "depends_on": [
        "cam"
      ]

Troubleshooting

Service will not initialize properly, with "port is already allocated" error in logs

If you are running in local mode, a Docker container named 'viam-roboflow-' is created. If you rename the service, the old container may still be running and will need to be stopped manually.

docker stop <container_name>