Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Hololens 2 support #109

Closed
rahulswa08 opened this issue May 20, 2023 · 9 comments
Closed

Hololens 2 support #109

rahulswa08 opened this issue May 20, 2023 · 9 comments

Comments

@rahulswa08
Copy link

Does the repo support build for hololens2?

@hoffmann-stefan
Copy link
Member

Hi @rahulswa08,

I'm not sure about the current state of ros2_dotnet for Hololens 2 and therefore UWP.

As UWP is an sandboxed environment that the software needs to support, for example not using normal WIN32 API that are not available in UWP.
As far as I know there was support for at least some of the core ROS 2 packages for UWP, but with the time those packages did loose support for UWP again, as UWP isn't actively testet in CI on those packages.
In that sense this repo should support UWP as I wouldn't know about something that broke it, but the depended ROS 2 packages would need to support it again first so we can test this.

ooeygui did try to fix this and upstream the changes to the needed ROS 2 packages and dependencies, see #92, but there isn't much recent information on the issue.

@ooeygui: Do you have any updates on this? Is this all done and we can try enabling it again or is there work left to do? Is there some more recent TODO list somewhere?
In some recent news article I saw that Hololens 2 gets Windows 11 support. Maybe this means we don't have to use UWP for this but rather can depend on normal WIN32 API and modern .NET versions?

@rahulswa08: If you mind to share, what would be your use-case for ROS 2 on the Hololens? Is this some student project, a hobby or some professional use? Maybe you could assist in getting this to work again?

@ooeygui
Copy link
Member

ooeygui commented May 22, 2023

Hi @hoffmann-stefan,
In order to run ROS2 on current generation of Hololens, all of the native code needs to be compiled to be UWP container friendly. (This was not the case for Hololens 1). I had made significant changes to the ROS2 runtime itself, FastDDS, and numerous dependencies. The current codebase, which includes the repos file - lives at http://aka.ms/ros/mrtk_native.

I'm not actively working on it.

@rahulswa08
Copy link
Author

Hi @rahulswa08,

I'm not sure about the current state of ros2_dotnet for Hololens 2 and therefore UWP.

As UWP is an sandboxed environment that the software needs to support, for example not using normal WIN32 API that are not available in UWP. As far as I know there was support for at least some of the core ROS 2 packages for UWP, but with the time those packages did loose support for UWP again, as UWP isn't actively testet in CI on those packages. In that sense this repo should support UWP as I wouldn't know about something that broke it, but the depended ROS 2 packages would need to support it again first so we can test this.

ooeygui did try to fix this and upstream the changes to the needed ROS 2 packages and dependencies, see #92, but there isn't much recent information on the issue.

@ooeygui: Do you have any updates on this? Is this all done and we can try enabling it again or is there work left to do? Is there some more recent TODO list somewhere? In some recent news article I saw that Hololens 2 gets Windows 11 support. Maybe this means we don't have to use UWP for this but rather can depend on normal WIN32 API and modern .NET versions?

@rahulswa08: If you mind to share, what would be your use-case for ROS 2 on the Hololens? Is this some student project, a hobby or some professional use? Maybe you could assist in getting this to work again?

Hi, @hoffmann-stefan

Thanks for your reply. we are building a human robot interaction platform for a research study using hololens 2. we need to stream a video into hololens 2 and the other way communication too.For this purpose I am looking for some resources . I don't need ros2 in particular and not sure if needed ros at all. Any suggestions would be appriciated.

Thank you

@ooeygui
Copy link
Member

ooeygui commented May 24, 2023

Take a peek at https://github.com/microsoft/MixedReality-WebRTC. (although, It is deprecated). You can use WebRTC to display remote cameras. I've been contributing to https://github.com/RobotWebTools/webrtc_ros, including supporting ROS2 - which allows you to subscribe to a ROS camera, and publish it as RTC. (I've done a video on how to use this ROS node to output to a steamdeck, but webrtc to webrtc should work as well - https://www.youtube.com/watch?v=osiSNGhKwQo&list=PL2dJBq8ig-vh42rfm8QEZFoCpYaBG7x2u&index=3.)

@rahulswa08
Copy link
Author

Thanks @ooeygui for your input. I have actually tried Mixedreality-WebRTC. But the repo is only supported on windows where as we are running a linux system on robot end. Can i use webrtc_ros repo to stream video to MR-webrtc app?

@ooeygui
Copy link
Member

ooeygui commented May 25, 2023

Sorry if I wasn't clear - the WebRTC_Ros node only works on linux - you would run that end on your ROS robot. The MRWebRTC component runs on the hololens. With some custom signaling, you should be able to connect them.

@rahulswa08
Copy link
Author

Ohh great!!. I will definitely try that.

@hoffmann-stefan
Copy link
Member

I had made significant changes to the ROS2 runtime itself, FastDDS, and numerous dependencies. The current codebase, which includes the repos file - lives at http://aka.ms/ros/mrtk_native.

@ooeygui Thanks for the information, I crated an PR that points to http://aka.ms/ros/mrtk for the current Hololens support and removes the duplicate install instructions from this repository's README.md

@hoffmann-stefan
Copy link
Member

closing this ass duplicate of #82

@rahulswa08 feel free to ask more questions or report bugs in new issues :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants