Capybara

A motion sensing camera

What it does:

Capybara is a camera that uses gyros or optical tracking to detect the motion of the camera. It can sweep in a panoramic scene as you move it around by grabbing multiple images and blending them together. Then, you can play back the captured panoramic scene in a virtual reality mode, where it appears to float around you. When you play it back, the experience is almost exactly like using a regular camcorder, except what you see in the viewfinder is from some other place.

Additionally, Capybara has a user interface which is based on motion tracking. To use it, you look in the viewfinder and move the camera around. You see a virtual workspace floating before you, and you have a cursor that is fixed to the camera. You just point the camera at what you want to select, and click the shoot button. For example, to select an image for review, you can pull up a sheet of thumbnail images. The images are stabilized, so they appear as if they are attached to a wall. If you move the camera body around, the images stay fixed in space. You can just point the camera at the one you want to select, and click.

Technologies

(Cubicle sweet cubicle)

One of the main technologies is motion tracking. Here are 100 photos captured and pasted in position based on readings from a gyro module. I developed the firmware for the module (Atmel Mega8 and TI MSP430). It converts raw angular rate sensor readings into an accurate position reading. It automatically and dynamically calibrates itself.

As you can see, the raw positions are not sufficient to make a smooth panorama, so my colleague and I developed blending techniques to make the panorama seamless.

History

In 1998, I was working with the HP Labs team that was developing the optical mouse chip, and I was working with the team that was developing software for panoramic capture (a feature that is in the HP 817 camera). I had an idea for a camera that could use motion tracking to do a bunch of cool new things. I built a prototype and I filed for a patent. The prototype attracted a lot of attention, and it was easy to find resources and then put together a team to build a more advanced version. A big group of people helped me realize the original ideas and to go beyond these ideas into more advanced areas.

Patents

20050226531 System and method for blending images into a single image

20040085455 Pointing device for digital camera display

20050146622 Pointing device for digital camera display

20040179121 System and method for displaying captured images according to imaging device position

And others

This technology is available for licensing.