Combining Multiple Depth Cameras and Projectors for Interactions On, Above, and Between Surfaces
Authors - Andrew D. Wilson and Hrvoje Benko
Authors Bios - Andrew D. Wilson is a researcher at Microsoft Research and has a PhD from MIT Media Laboratory.
Hrvoje Benko is a researcher at Microsoft Research and has a PhD from Columbia University.
Venue - This paper was presented at the UIST '10 Proceedings of the 23rd annual ACM symposium on User interface software and technology.
Summary
Hypothesis - In this paper, the researchers will develop a system capable of making non-instrumented surfaces interactive using a series of projectors and cameras. The hypothesis is that this system, called LightSpace, will explore more capabilities of depth cameras and help lead us to a future in which even the smallest corner of a room will be interactive.
Content - 4 interactions that will be implemented in this system include simulated interactive surfaces (support hand gestures and touch input), through-body transitions between surfaces (ability to touch 2 surfaces to transfer objects), picking up objects (ability to carry an object in a user's hand), and spatial menus (use human as interactive surface).
Methods - The researchers built their system using 3 InFocus projectors and 3 PrimeSense depth cameras mounted on a single aluminum truss and aimed so as to maximize effectiveness. The depth cameras use 2 types of cameras and a light source to accurately form a 3D model of what is being viewed. All cameras contribute to a single system-stored 3D representation of the test area to eliminating the need to know which camera is located where. The cameras are calibrated individually using 3 points of reference and them the projectors are inserted into the 3D grid created by the camera calibration. The interactive surfaces must be designated by identifying 3 corners of the surface in question and the surface must be rectangular, flat, and immobile. A virtual camera's view is created by using the data collected by all 3 cameras in order to form an overall view of the interactive area to better analyze what is being performed. Connectivity of two objects by a person is detected by checking for intersecting images with the interactive surfaces. Picking up and dropping objects is handled by simply checking what a user is touching. The spatial menu implementation can easily be applied to other spatial objects in the future. The system was demoed to an audience of 800 at a special event.
Results - The demo showed that a maximum of 6 users could use the system at any one time to any real effect although, any more than 3 users slowed the system down considerably. Whenever actions were not detected by the cameras, it was usually because some object or person was blocking the view which brought the question of camera placement back to the creators' minds. Some unique interactions were also discovered during the demo such as objects transferring between surfaces because users with those objects shook hands. Future plans for the system mostly center around making the system more robust and removing limiting requirements.
Conclusion - The researchers conclude by saying they have presented a system that is capable of transitioning the user space from a single computer to the entire room around them.
Discussion
I think the researchers prove their hypothesis that depth cameras provide a viable option in creating more interactivity in currently non-interactive environments by simply bringing in a somewhat portable system that can then calibrate to that space. I think the future in this study could produce great results that lead some day to an interactive classroom followed by the elusive "smart" house. The only faults I can find with this work is that the use of an external system, as opposed to an internal one, can be very limited by the location of objects and users if a standard for depth camera positioning is not established by an extensive study on the topic.
No comments:
Post a Comment