Microsoft is moving beyond one surface onto multiple surfaces now. With their LightSpace research project, they are tracking virtual objects as they move off a surface and onto users’ hands to be carried around the room. Projectors keep the virtual objects in sync with the real-world objects. So you can write a virtual note, carry it around, and “drop” it onto a wall. This is apparently made possible through the use of something called “depth cameras,” important for Microsoft’s Kinect gaming platform:
“Depth cameras (such as those from PrimeSense1, 3DV, and Canesta) are able to directly sense range to the nearest physical surface at each pixel location. They are unique in that they enable inexpensive real time 3D modeling of surface geometry, making some traditionally difficult computer vision problems easier. For example, with a depth camera it is trivial to composite a false background in a video conferencing application.