AR demo in livingroom

AR Semantic Scene Generation

Developers of AR apps often place virtual objects arbitrarily in front of the user with no consideration of their surroundings. However, resulting illogical placements make the AR feel less realistic. We seek a more nuanced method of virtual object placement that takes objects in the user’s environment into account.

We developed an interface to generate heatmaps indicating likely placements for new objects of different types (eg. bed, chair, table, etc.). This interface can be run from Unity or as an AR application on IOS.

Xiyu Zhai
Software Engineer

My research interests include robotics, multi-agent control and graphics.