Cannot submit label on new ohif update

[JackTheSturgeon]

Hello Everyone,

I have been using MonaiLabel and OHIF to perform active training on my segmentation model on a remote server. The workflow was to infer a new segmentation mask with my model, then correct it, submit the label and train the model on the submited label before moving to the next sample.
However, I recently updated my conda environment and saw OHIF has been updated. Problem is, there is no more submit label button, and when I infer a new segmentation mask it is not forwarded to my orthanc server and I cannot modify and correct it.
I don't see anything talking about this and I don't see any documentation that hints to a solution to add my infered label to the study. Thus I cannot perform active training anymore which is pretty annoying.

Does anybody have a solution or somewhere I could look for a solution?
Thank you

Ps: If needed, I can provide more precisions

[SachidanandAlle]

OHIF v3 is lot under development. And OHIF v2 version is kind reaching EOL.
You can save your seg via direct OHIF v3 feature.

Instead of feeling annoyed, please feel free to jump in if you can. At the end it is an open source to make it useful and available for everyone to use and contribute and make it better. We are looking for more contributions on this front.

[SachidanandAlle]

Related code which needs some contribution/fix to work with latest OHIF v3.

https://github.com/Project-MONAI/MONAILabel/blob/main/plugins/ohifv3/extensions/monai-label/src/components/actions/ActiveLearning.tsx#L250-L260

[JackTheSturgeon]

Hello, Thanks for your answer! I don't have the segmentation pannel appearing when my model has finished running. I can see the segmentation mask but no menu appearing. Maybe it is a problem on my side ? I need the active learning to be working so if I have to I will be trying to find a way to make it work. How was the label submission implemented in the previous version, do you have the code? Best Regards De: "SACHIDANAND ALLE" ***@***.***> À: "Project-MONAI/MONAILabel" ***@***.***> Cc: "JackTheSturgeon" ***@***.***>, "Author" ***@***.***> Envoyé: Lundi 20 Novembre 2023 16:49:25 Objet: Re: [Project-MONAI/MONAILabel] Cannot submit label on new ohif update (Discussion #1596) OHIF v3 is lot under development. And OHIF v2 version is kind reaching EOL. You can save your seg via direct OHIF v3 feature. [ https://user-images.githubusercontent.com/7339051/284400514-80a923a6-6c43-4552-8dc9-1cee4596d497.png ] Instead of feeling annoyed, please feel free to jump in if you can. At the end it is an open source to make it useful and available for everyone to use and contribute and make it better. We are looking for more contributions on this front. — Reply to this email directly, [ #1596 (comment) | view it on GitHub ] , or [ https://github.com/notifications/unsubscribe-auth/ANMQX767FGZRAPO65XU5TKDYFPF6LAVCNFSM6AAAAAA7TR2WD6VHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM3TMMRTHE4DA | unsubscribe ] . You are receiving this because you authored the thread. Message ID: <Project-MONAI/MONAILabel/repo-discussions/1596/comments/7623980 @ github . com>

[JackTheSturgeon]

Hi @diazandr3s,

Thanks again for your answer, I reinstalled a fresh python env and did pip install monailabel. So I should have the latest version of monai and ohif.

I have 2 problems:
1. Image ID undefined
When I run my autosegmentation I have the following error message:
[2023-11-28 15:39:43,856] [158160] [MainThread] [INFO] (monailabel.endpoints.infer:171) - Infer Request: {'model': 'segmentation_unetr', 'image': 'undefined', 'device': ['NVIDIA RTX A6000:0', 'NVIDIA RTX A6000:1'], 'result_extension': '.nrrd', 'result_dtype': 'uint16', 'result_compress': False, 'restore_label_idx': False} [2023-11-28 15:39:43,856] [158160] [MainThread] [INFO] (monailabel.datastore.dicom:78) - Image ID: undefined [2023-11-28 15:39:43,856] [158160] [MainThread] [INFO] (monailabel.datastore.dicom:80) - Image Dir (cache): /home/fred/.cache/monailabel/dicom/488fca1f7f3f98331f6f7756beca0d35/undefined [2023-11-28 15:39:43,856] [158160] [MainThread] [INFO] (dicomweb_client.web:2128) - search for series [2023-11-28 15:39:43,860] [158160] [MainThread] [ERROR] (uvicorn.error:434) - Exception in ASGI application Traceback (most recent call last): File "/home/fred/.local/lib/python3.9/site-packages/uvicorn/protocols/http/h11_impl.py", line 429, in run_asgi result = await app( # type: ignore[func-returns-value] File "/home/fred/.local/lib/python3.9/site-packages/uvicorn/middleware/proxy_headers.py", line 78, in __call__ return await self.app(scope, receive, send) File "/home/fred/.local/lib/python3.9/site-packages/fastapi/applications.py", line 276, in __call__ await super().__call__(scope, receive, send) File "/home/fred/.local/lib/python3.9/site-packages/starlette/applications.py", line 122, in __call__ await self.middleware_stack(scope, receive, send) File "/home/fred/.local/lib/python3.9/site-packages/starlette/middleware/errors.py", line 184, in __call__ raise exc File "/home/fred/.local/lib/python3.9/site-packages/starlette/middleware/errors.py", line 162, in __call__ await self.app(scope, receive, _send) File "/home/fred/.local/lib/python3.9/site-packages/starlette/middleware/cors.py", line 92, in __call__ await self.simple_response(scope, receive, send, request_headers=headers) File "/home/fred/.local/lib/python3.9/site-packages/starlette/middleware/cors.py", line 147, in simple_response await self.app(scope, receive, send) File "/home/fred/.local/lib/python3.9/site-packages/starlette/middleware/exceptions.py", line 79, in __call__ raise exc File "/home/fred/.local/lib/python3.9/site-packages/starlette/middleware/exceptions.py", line 68, in __call__ await self.app(scope, receive, sender) File "/home/fred/.local/lib/python3.9/site-packages/fastapi/middleware/asyncexitstack.py", line 21, in __call__ raise e File "/home/fred/.local/lib/python3.9/site-packages/fastapi/middleware/asyncexitstack.py", line 18, in __call__ await self.app(scope, receive, send) File "/home/fred/.local/lib/python3.9/site-packages/starlette/routing.py", line 718, in __call__ await route.handle(scope, receive, send) File "/home/fred/.local/lib/python3.9/site-packages/starlette/routing.py", line 276, in handle await self.app(scope, receive, send) File "/home/fred/.local/lib/python3.9/site-packages/starlette/routing.py", line 66, in app response = await func(request) File "/home/fred/.local/lib/python3.9/site-packages/fastapi/routing.py", line 237, in app raw_response = await run_endpoint_function( File "/home/fred/.local/lib/python3.9/site-packages/fastapi/routing.py", line 163, in run_endpoint_function return await dependant.call(**values) File "/home/fred/.local/lib/python3.9/site-packages/monailabel/endpoints/infer.py", line 206, in api_run_inference return run_inference(background_tasks, model, image, session_id, params, file, label, output) File "/home/fred/.local/lib/python3.9/site-packages/monailabel/endpoints/infer.py", line 172, in run_inference result = instance.infer(request) File "/home/fred/.local/lib/python3.9/site-packages/monailabel/interfaces/app.py", line 289, in infer request["image"] = datastore.get_image_uri(request["image"]) File "/home/fred/.local/lib/python3.9/site-packages/monailabel/datastore/dicom.py", line 83, in get_image_uri dicom_web_download_series(None, image_id, image_dir, self._client, self._fetch_by_frame) File "/home/fred/.local/lib/python3.9/site-packages/monailabel/datastore/utils/dicom.py", line 70, in dicom_web_download_series [ IndexError: list index out of range

The image ID is undefined and it seems to be because for some reason monailabel couldn't access the study ID linked to the image. Moreover, sometimes the auto segmentation works and if I try again on the same patient I get this same error. I am using orthanc dycom server.
What do you think might be happening and how can I fix this ?

2. Cannot edit my segmentation
When the auto segmentation does work I have nothing appearing to be able to save or edit my segmentation.
Here are the logs for when the segmentation works:
[2023-11-28 15:40:01,272] [158160] [MainThread] [INFO] (monailabel.endpoints.infer:171) - Infer Request: {'model': 'segmentation_unetr', 'image': '1.2.826.0.1.3680043.8.274.1.1.8323329.758566.1700684574.80936', 'device': ['NVIDIA RTX A6000:0', 'NVIDIA RTX A6000:1'], 'result_extension': '.nrrd', 'result_dtype': 'uint16', 'result_compress': False, 'restore_label_idx': False} [2023-11-28 15:40:01,272] [158160] [MainThread] [INFO] (monailabel.datastore.dicom:78) - Image ID: 1.2.826.0.1.3680043.8.274.1.1.8323329.758566.1700684574.80936 [2023-11-28 15:40:01,272] [158160] [MainThread] [INFO] (monailabel.datastore.dicom:80) - Image Dir (cache): /home/fred/.cache/monailabel/dicom/488fca1f7f3f98331f6f7756beca0d35/1.2.826.0.1.3680043.8.274.1.1.8323329.758566.1700684574.80936 [2023-11-28 15:40:01,273] [158160] [MainThread] [INFO] (monailabel.tasks.infer.basic_infer:280) - Infer Request (final): {'device': 'cuda:0', 'model': 'segmentation_unetr', 'image': '/home/fred/.cache/monailabel/dicom/488fca1f7f3f98331f6f7756beca0d35/1.2.826.0.1.3680043.8.274.1.1.8323329.758566.1700684574.80936.nii.gz', 'result_extension': '.nrrd', 'result_dtype': 'uint16', 'result_compress': False, 'restore_label_idx': False, 'description': 'Cortical Vessel Segmentation using the UnetR model'} RUNNING PRE TRANSFORMS in INFER [2023-11-28 15:40:01,274] [158160] [MainThread] [INFO] (monailabel.interfaces.utils.transform:76) - PRE - Run Transform(s) [2023-11-28 15:40:01,274] [158160] [MainThread] [INFO] (monailabel.interfaces.utils.transform:77) - PRE - Input Keys: ['device', 'model', 'image', 'result_extension', 'result_dtype', 'result_compress', 'restore_label_idx', 'description', 'image_path'] [2023-11-28 15:40:01,564] [158160] [MainThread] [INFO] (monailabel.interfaces.utils.transform:122) - PRE - Transform (LoadImaged): Time: 0.2901; image: torch.Size([448, 448, 128])(torch.float32) [2023-11-28 15:40:01,565] [158160] [MainThread] [INFO] (monailabel.interfaces.utils.transform:122) - PRE - Transform (EnsureChannelFirstd): Time: 0.0003; image: torch.Size([1, 448, 448, 128])(torch.float32) [2023-11-28 15:40:01,565] [158160] [MainThread] [INFO] (monailabel.interfaces.utils.transform:122) - PRE - Transform (EnsureTyped): Time: 0.0001; image: torch.Size([1, 448, 448, 128])(torch.float32) [2023-11-28 15:40:01,746] [158160] [MainThread] [INFO] (monailabel.interfaces.utils.transform:122) - PRE - Transform (NormalizeIntensityd): Time: 0.1808; image: torch.Size([1, 448, 448, 128])(torch.float32) RUNNING INFERER in INFER [2023-11-28 15:40:01,746] [158160] [MainThread] [INFO] (monailabel.tasks.infer.basic_infer:504) - Inferer:: cuda:0 => unetrInferer => {'threshold': 0.5, 'roi_size': (224, 224, 128), 'spatial_size': (64, 64, 64), 'device': device(type='cuda')} [2023-11-28 15:40:01,746] [158160] [MainThread] [INFO] (monailabel.tasks.infer.basic_infer:432) - Infer model path: /home/fred/monailabel/apps/radiology/model/segmentation_unetr.pt CALL TO UNETR INFERER Using device: cuda 98it [00:00, 104.71it/s] output shape: torch.Size([1, 448, 448, 128]) output neg shape: torch.Size([1, 448, 448, 128]) output concat shape: torch.Size([2, 448, 448, 128]) output concat unsqueeze shape: torch.Size([1, 2, 448, 448, 128]) [2023-11-28 15:40:03,036] [158160] [MainThread] [INFO] (monailabel.interfaces.utils.transform:76) - POST - Run Transform(s) [2023-11-28 15:40:03,037] [158160] [MainThread] [INFO] (monailabel.interfaces.utils.transform:77) - POST - Input Keys: ['device', 'model', 'image', 'result_extension', 'result_dtype', 'result_compress', 'restore_label_idx', 'description', 'image_path', 'latencies', 'pred'] [2023-11-28 15:40:03,037] [158160] [MainThread] [INFO] (monailabel.interfaces.utils.transform:122) - POST - Transform (EnsureTyped): Time: 0.0001; image: torch.Size([1, 448, 448, 128])(torch.float32); pred: torch.Size([2, 448, 448, 128])(torch.float32) [2023-11-28 15:40:03,037] [158160] [MainThread] [INFO] (monailabel.interfaces.utils.transform:122) - POST - Transform (Activationsd): Time: 0.0004; image: torch.Size([1, 448, 448, 128])(torch.float32); pred: torch.Size([2, 448, 448, 128])(torch.float32) [2023-11-28 15:40:03,039] [158160] [MainThread] [INFO] (monailabel.interfaces.utils.transform:122) - POST - Transform (AsDiscreted): Time: 0.0013; image: torch.Size([1, 448, 448, 128])(torch.float32); pred: torch.Size([1, 448, 448, 128])(torch.float32) [2023-11-28 15:40:03,039] [158160] [MainThread] [INFO] (monailabel.interfaces.utils.transform:122) - POST - Transform (Restored): Time: 0.0001; image: torch.Size([1, 448, 448, 128])(torch.float32); pred: torch.Size([448, 448, 128])(torch.float32) [2023-11-28 15:40:03,039] [158160] [MainThread] [INFO] (monailabel.tasks.infer.basic_infer:604) - Writing Result... [2023-11-28 15:40:03,039] [158160] [MainThread] [INFO] (monailabel.transform.writer:196) - Result ext: .nrrd; write_to_file: True; dtype: uint16 [2023-11-28 15:40:05,315] [158160] [MainThread] [INFO] (monailabel.tasks.infer.basic_infer:335) - ++ Latencies => Total: 4.0428; Pre: 0.4730; Inferer: 1.2899; Invert: 0.0000; Post: 0.0033; Write: 2.2762 [2023-11-28 15:40:05,315] [158160] [MainThread] [INFO] (monailabel.tasks.infer.basic_infer:370) - Result File: /tmp/tmp2kik_yf6.nrrd [2023-11-28 15:40:05,315] [158160] [MainThread] [INFO] (monailabel.tasks.infer.basic_infer:371) - Result Json Keys: ['label_names', 'latencies', 'centroids']
And here is an image of what I see on OHIF:

We can see the output of my model (which is the red segmentation) but there is nothing to edit or save it.

[diazandr3s]

Hi @JackTheSturgeon,

Thanks for the update.
For this:

1. Image ID undefined

Can you please remove the folder /home/fred/.cache/monailabel/ and try again? For some reason, the cache creates issues with the ID.

For this:

2. Cannot edit my segmentation

It is strange. I still don't see the Segmentation mode in the print screen you've shared. How are you accessing OHIF? Via the MONAI Label IP address?

Let us know,

[JackTheSturgeon]

Hello @diazandr3s,

1. I removed the .cache/monailabel folder but it didn't resolve the problem. Is it the existence of a cache before running the monailabel server that creates issues or the sole existence of a cache ? I am asking because the cache folder gets recreated when I launch the monai server.

Also, something odd is when I connect to MONAI Label in OHIF, the dicomweb client retrieves metadata and shows me the Series ID and Study ID. If OHIF is able to see the Study ID why Monai isn't able to do so ?

2. I am accessing OHIF via the MONAI Label IP adress: localhost/8000/ohif

Update: When I use the segmentation model, the segmentation tools and segmentation pannels appear when connection with monailabel is established. This is not the case when I am using my custom model. However, the Study ID being undefined is a persistent issue for any model that I am trying to use.

[diazandr3s]

Hi @JackTheSturgeon,

Thanks for the update.

I removed the .cache/monailabel folder but it didn't resolve the problem. Is it the existence of a cache before running the monailabel server that creates issues or the sole existence of a cache ? I am asking because the cache folder gets recreated when I launch the monai server.

It is the existence of a cache before starting/running MONAI Label. MONAI Label creates a cache on every run.

Update: When I use the segmentation model, the segmentation tools and segmentation pannels appear when connection with monailabel is established. This is not the case when I am using my custom model. However, the Study ID being undefined is a persistent issue for any model that I am trying to use.

What did you change on the custom model? Was it only the folder name or also the class names?

[JackTheSturgeon]

Hi @diazandr3s,

1. I resolved the study ID issue. The problem was linked to the way I uploaded my dicom images to orthanc.
   The way I used to upload to orthanc was this: python -m pynetdicom storescu 127.0.0.1 4242 images_folder_name -aet MONAILABEL -r
   It allowed me to upload from the remote server I am working on, and upload every images at once.
   Solution: Manually upload to orthanc using orthanc's GUI
   Is there a better way to upload to orthanc than this ?

2. For my custom unetr model, I added a segmentation_unetr.py in radiology/lib/configs, radiology/lib/infers, radiology/lib/trainers. I adapted these files from the default MONAI segmentation_spleen model.
   My Class name is also changed to "SegmentationUnetrCorticalVessels" for the config, trainer and inferer.
   To accomodate this change I customized the radiology/main.py file by adding this function:
   `
   Adding a custom entry to the configs if it is specified in models

    ex: monailabel start_server --conf models custom:model_name.model_className
    We store this in the config as configs[custom]=path_to_model
    
    @arg model_string str: string containing the name of a pretrained model or has the following structure -> customXXX:path_to_model
    @arg config_dic dict: configuration dictionnary containing the paths to the models
   
    def is_custom(model_string, config_dic):
        custom_model = model_string.split(":")
        # If it is not a custom model, then we just return the string as is
        if custom_model[0] != "custom":
            return model_string.strip()
        # If the user wants to use a custom model, then we add it to the configuration dictionnary
        else:
            custom_model[0] = custom_model[0].strip()
            model_name, model_className = custom_model[1].split(".")[0], custom_model[1].split(".")[1]
            
            config_dic[model_name]="lib.configs." + model_name + "." + model_className
            # custom_model[0] is the key to the path for finding the model
            return model_name`
   

And replacing the line:
models = [m.strip() for m in models]
By the following line:
models = [is_custom(m, configs) for m in models]

By using this strategy, I am giving to the main function the path to the pretrained model weights and the class name of my custom model.

Thus, I can start my monailabel server this way:
monailabel start_server --app monailabel/apps/radiology --studies http://127.0.0.1:8042/dicom-web --conf models custom:segmentation_unetr.SegmentationUnetrCorticalVessels

[JackTheSturgeon]

1. I resolved the study ID issue by manually uploading my Images to Orthanc using the GUI (through the upload button when accessing orthanc on localhost:8042)

The way I originally did it was by using pynetdicom
python -m pynetdicom storescu 127.0.0.1 4242 images_folder_name -aet MONAILABEL -r
which is way more convenient (because I can just upload all images in one line) but not monai friendly.

2. I figured why the segmentation pannel was not showing
You need to use 2 or more labels in your model config file (Im talking about self.labels in radiology/lib/config/segmentation_unetr.py).
Example:
self.labels = {"background":0, "Blood Vessel": 1}

[diazandr3s]

Many thanks for sharing your experience, @JackTheSturgeon. Glad to hear you got it working