Minimal example to get depth for a point in 2D image.

[micheldom]

I am using the O3R225 and the M04239 VPU (functionally the same as the OVP800 I think)

I am curious if there is a known, accurate way that, given a point/coords in the 2D image, to get the depth from the corresponding distance image.

I have tried to scale, crop and align both images (due to difference in image size and FOV) and then slice the distance image accordingly to get the depth but it involves a lot of manual adjustment so I want to see if there is way to do it with any inbuilt function/known examples

I have seen examples like the registration_2d_3d.py using RGBInfoV1 and TOFInfoV4 but I am wholly unfamiliar with how to use it even after looking at this helpful image but if thats the proper solution then I will go with it. Any help or nudge in the right direction is greatly appreciated

[lola-masson]

Hi @micheldom,
You are correct, the appropriate example for this is the 2d_3D_registration.py.
The method that does all the work is the colorize_point_cloud, which has comments throughout to guide you through the process.
Maybe also the explanation in the readme would be helpful.

Unfortunately, this is the most accurate way to get an overlay of the RGB and depth. Because of the different fields of view, resolution and optical parameters, you cannot simply "put one on top of the other" and expect them to match.

Let us know if you have any questions about the process and we'll be happy to go into details.

[micheldom]

Hi, thanks for the response and I understand whats happening in 2D_3D_registration.py now, but I have to ask if there's any way to, instead of going from 3D point cloud to 2D image, to go the other way around from 2D image to 3D point cloud. I have ROI's in the 2D image which I want to get the depth for but I don't want to process the whole point cloud every frame.

[lola-masson]

Yes, to do this you would need to follow the operations in the reverse order, finding the unit vectors for your 2D pixels, and projecting those to the 3D. The same ideas apply, you just need to substitute the intrinsic calibration for inverse intrinsic calibration, and vice versa.