Home

Purpose

This project use SRworks SDK to demonstrate 3 AR games, developer can save time by learning from this demo to make AR illusion and interaction.

Category

There are 3 categories for coding structure.

1. GameManager: handle the game state & logic. .
   
2. ARRender: handle the render works.
3. ReconstructManager: handle the SRWorks reconstruct operation.

Subject code

Interaction and vision quality is important for AR experience, we use SRWorks SDK to scan real world and mix virtual/real object to perform AR illusions.

• Open door shader : create an open door illusion from real wall.
• Reconstruct : Static: scanning and reconstruct static real world. Dynamic: reconstruct dynamic object realtime(e.g. user's hand).
• Culling : make reasonable culling between the real/virtual object.
• Shadow : the virtual model can cast shadow on real world object
• Lighting : the virtual (spot/point)light can illuminate real world object

AR open door

Open a door from real wall is shader trick, that we record the virtual door's world matrix before open(rotate/move), and use the recorded constant world matrix to sample real camera's image on the door, thus, door can rotate with wall's texture, and there is a board open on the wall.

Please refer the relative code.

Advance render

SRWorks is good for camera and real world reconstruction, but there are not enough functions for user to render AR effects (lighting, culling shadow), so, I made an additional render system to improve AR visualization quality. The AR effects only need a depth map, which is render scanned real object once time per frame, and calculate in post processing, I think it’s only cost slightly resource on GPU.

1. VR/Real soft edge culling: When scanning real environment to 3D mesh, there are always produce unsmooth edge not so fit real objects, causes real object over culling the VR object, so I create an adjustable soft edge map to fading culling the VR/Real.>
   

2. Shadows on real object: There already a simple shadow method provides in SRWorks, It calling the unity’s shader function SHADOW_ATTENUATION to get the shadow intensity, but the shadow perform is not good, and not easy fine tune individually, so, I made a shadow system to render virtual object’s shadow and control how quality to render on scanned real objects.
   .
   
   We can fine tune parameter to make approximately ambient occlusion on real object
   .
   .
   
   Add intensity while closing floor, less intensity while leaving floor, shadow can improve the feeling of object is really on the floor.

3. Point & Spot lights on real object: There doesn’t a function is easy to call for lighting in unity shader (SRWorks only provides simple shadow calculation), so I made a simple deferred lighting system to light up the real objects, It will break the unity’s lighting limitation (unlimited light amount) and can adjust lighting parameter only for MR visualization, thus unity lights can interact with VR & Real object.
   

4. Render flow #