Official Support for Matterport Partner 360 Cameras to Generate Area Targets

[thechargedneutron]

Hi, I want to know what cameras are compatible for capturing the 3D scans. I understand that we need matterport scans for larger areas. The vuforia docs here mention Matterport and Leica scanners. These are pretty expensive for hobbyists. When I look for matterport compatible cameras here: https://matterport.com/cameras/360-cameras, I see a lot of affordable cameras like the Insta360 X3, Ricoh Theta X. Are these cameras not compatible for vuforia scans?

If the matterport portal allows scanning using these affordable cameras and creating 3D scans, I do not see why we cannot follow the steps given here: https://developer.vuforia.com/library/scanning-environment/area-targets-matterporttm-scans to import the 3D scans in vuforia.

Am I missing some steps that is making these affordable cameras incompatible or should it still work in Matterport supports it?

Thanks

[ptc-pscheper]

Hi there, though we do not recommend these third-party 360 cameras from Matterport you are indeed theoretically able to use them to generate Area Targets! The reason why we do not officially support them is because the accuracy of the 3D reconstruction of these cameras can be potentially off which could cause bad localization and tracking performance.

Matterport also elaborates on this in their documentation:

For MatterPaks generated by Synthesized or Hybrid Depth sources that use 360° imagery and our Cortex AI with deep learning models to estimate depth, Matterport does not guarantee the accuracy of the generated point cloud or mesh files.

I've converted this ticket into a device request to officially support third-party 360 cameras from Matterport to generate Area Targets.

[thechargedneutron]

Thanks, this support would be greatly helpful. Just to clarify, should I go ahead with purchasing and recording with a third-party camera while understanding the risk with accuracy or will that not work and I'll get an incompatibility error?

[ptc-pscheper]

Hi there, since this is currently not officially supported by PTC I cannot guarantee that you will be met without issues. However based on the documentation of Matterport, as long as you can generate a Matterpak it could work within the Area Target Generator. Please share your findings if you decide to test out this workflow!

[thechargedneutron]

Sure, I will share my observations. It would be great to have a low-end camera working with AR for prototyping.

[ptc-emaggio]

@thechargedneutron you can use the Vuforia Creator App to capture an area target with any iOS device with Lidar sensor (e.g., iPhone Pros) https://developer.vuforia.com/library/tools/creator-app . The app calls the Area Target Capture API https://developer.vuforia.com/library/develop-area-targets/capture-api

[thechargedneutron]

@ptc-emaggio Yes, I am aware of that and have already tested that. I want to use a 360 camera to test the matterport and matterpak workflow for a slightly larger area (my home). iOS capture was good for my room. The ability to use 360 cameras would enable a step between jumping from iOS to (expensive) matterport.

[thechargedneutron]

@ptc-emaggio @ptc-pscheper I tried Insta360 X3 with matterport and it works just fine. I did not encounter any error in the whole pipeline mentioned here: https://developer.vuforia.com/library/scanning-environment/area-targets-matterporttm-scans

I was expecting the quality to not be top-notch but the 3D model were below my expectations. I would say the quality is almost the same as iPhone (with Lidar), maybe slightly better.

Two related questions:

1. I was trying to get an apples to apples comparison by scanning my whole house with iPhone but I run into maximum size reached with the Vuforia capture app. Is the limit only for free plans? If not, why is there a limit on iPhone capture?

2. Do I need to purchase Matterpak per scan? That seems a bit costly for small scans.

3. Why does the doc say we need a minimum of professional matterport plan? I got the professional plan but the matterpak package said we can use it for any subscription above free and we have a starter plan between free and professional.

[ptc-emaggio]

@thechargedneutron regarding (1) the maximum size (CAPACITY_WARNING) is a soft-bound due to device memory limitations. Continuing scanning after the warning will have limited impact at the beginning but if you can keep on scanning the target information will become sparser and sparse (per unit area scanned), and eventually this will impact on localization performance and authoring mesh accuracy. Still you can experiment and check what loss is acceptable to you.

There is also the alternative to create multiple targets within the same session (e.g., one per room). The targets should be reasonably aligned with each other, as long as ARKit tracking does not break. Please check this information on how to deal with multiple targets https://developer.vuforia.com/library/develop-area-targets/multiple-area-targets

[ptc-emaggio]

@thechargedneutron regarding (2) any plan where you can use the Matterpak and the rest API should be enough. Note that for the officially supported Pro scanners a Professional plan is indeed required.

[thechargedneutron]

@ptc-emaggio I will check out the multiple target and see if that can useful in my case. Thanks for the help!

Feel free to close this issue or keep it open if you want to officially support 360 cameras.

[ptc-pscheper]

We can keep this open as your observations will be useful for anyone else trying these unofficially supported cameras! Thank you!