Skinput is a technology that appropriates the human body for acoustic transmission, allowing the skin to be used as a finger input surface. In particular, we resolve the location of finger taps on the arm and hand by analyzing mechanical vibrations that propagate through the body. We collect these signals using a novel array of sensors worn as an armband. This approach provides an always-available, naturally-portable, and on-body interactive surface. To illustrate the potential of our approach, we developed several proof-of-concept applications on top of our sensing and classification system

Selected Press: CNN, Popular Science, CNET, Engadget, Gizmodo, New Scientist

A cord, although simple in form, has many interesting physical affordances that make it powerful as an input device. Not only can a length of cord be grasped in different locations, but also pulled, twisted and bent — four distinct and expressive dimensions that could potentially act in concert. Such an input mechanism could be readily integrated into headphones, backpacks, and clothing. Once grasped in the hand, a cord can be used in an eyes-free manner to control mobile devices, which often feature small screens and cramped buttons. We built a proof-of-concept cord-based sensor, which senses three of the four input dimensions we propose.

Minput is a sensing and input method that enables intuitive and accurate interaction on very small devices – ones too small for practical touch screen use and with limited space to accommodate physical buttons. We achieve this by adding two, inexpensive and high-precision optical sensors (like those found in optical mice) to the underside of the device. This allows the entire device to be used as an input mechanism, instead of the screen, avoiding occlusion by fingers. In addition to x/y translation, our system also captures twisting motion, enabling many interesting interaction opportunities typically found in larger and far more complex systems.

Selected Press: Popular Science, CNET GadgetVenue, ACM TechNews

Whack Gestures seeks to provide a simple means to interact with devices with minimal attention from the user – in particular, without the use of fine motor skills or detailed visual attention (requirements found in nearly all conventional interaction techniques). For mobile devices, this could enable interaction without “getting it out,” grasping, or even glancing at the device. Users can simply interact with a device by striking it with open palm or heel of the hand.

Selected Press: Gizmodo, New Scientist, MaximumPC

We developed a magnetically-driven input approach that makes use of the (larger) space around a (very small) device. Our technique provides robust, inexpensive, and wireless input from fingers, without requiring powered external components. By extending the input area to many times the size of the device’s screen, our approach is able to offer a high C-D gain, enabling fine motor control. Additionally, screen occlusion can be reduced by moving interaction off of the display and into unused space around the device.

We invented a method for identifying the order of stacked items on interactive surfaces. This is achieved using conventional, passive fiducial markers, which in addition to reflective regions, also incorporate structured areas of transparency. This allows particular orderings to appear as unique marker patterns. We discuss how such markers are encoded and fabricated, and include relevant mathematics. To motivate our approach, we comment on various scenarios where stacking could be especially useful. We conclude with details from our proof-of-concept implementation, built on Microsoft Surface.

Physical buttons have the unique ability to provide low-attention and vision-free interactions through their intuitive tactile clues. Unfortunately, the physicality of these interfaces makes them static, limiting the number and types of user interfaces they can support. On the other hand, touch screen technologies provide the ultimate interface flexibility, but offer no inherent tactile qualities. In this paper, we describe a technique that seeks to occupy the space between these two extremes – offering some of the flexibility of touch screens, while retaining the beneficial tactile properties of physical interfaces.

Selected Press: Technology Review, Wired, Gizmodo, Engagdet, Crunchgear, Slashdot, Register,Popular Mechanics

Despite touch being a rich sensory channel, tactile output is almost exclusively vibrotactile in nature. We explored a different style of tactile display - one that can assume several different textural states (e.g., sticky, bumpy, smooth, course, gritty). In contrast to conventional vibrotactile approaches, these displays provide information passively. Only when they are explicitly handled by the user, either with intent to inquire about the information, or in the course of some other action, can state be sensed. This inherently reduces their attention demand and intrusiveness. We call this class of devices texture displays.

One way to draw a wearer’s attention is through vibrotactile stimulation, commonly employed in mobile devices to alert users (e.g., to an incoming call). Also popular, although not tied to any particular body location, are audio alerts. Visual displays offer an alternative notification method. However, there has been little research into their optimal body placement, despite being an unobtrusive, lightweight, and low-powered information delivery mechanism. Furthermore, visual stimuli have the added benefit of being able to work alone or in concert with conventional methods, including auditory and vibrotactile alerts.

Using the infrared camera in the Wii remote and a head mounted sensor bar (two IR LEDs), you can accurately track the location of your head and render view dependent images on the screen. This effectively transforms your display into a portal to a virtual environment. The display properly reacts to head and body movement as if it were a real window creating a realistic illusion of depth and space.

Selected Press: Slashdot, Joystiq, Gizmodo, HackADay, Engadget, New York Times

Scratch Input is an acoustic-based input technique that relies on the unique sound produced when a fingernail is dragged over the surface of a textured material, such as wood, fabric, or wall paint. We employ a simple sensor that can be easily coupled with existing surfaces - such as walls and tables - turning them into large, unpowered and ad hoc finger input surfaces. Our sensor is sufficiently small that it could be incorporated into a mobile device, allowing any suitable surface on which it rests to be appropriated as a gestural input surface.

Selected Press: Slashdot, Slashdot, HackADay, Technology Review, Make, Gizmodo, Wired, ReadWriteWeb, CrunchGear

Numerous methods have been proposed that allow mobile devices to determine where they are located (e.g., home or office) and in some cases, predict what activity the user is currently engaged in (e.g., walking, sitting, or driving). While useful, this sensing currently only tells part of a much richer story. To allow devices to act most appropriately to the situation they are in, it would also be very helpful to know about their placement – for example whether they are sitting on a desk, hidden in a drawer, placed in a pocket, or held in one’s hand – as different device behaviors may be called for in each of these situations.

Since the Wiimote can track sources of infrared (IR) light, you can track pens that have an IR led in the tip. By pointing a wiimote at a projection screen or LCD display, you can create very low-cost interactive whiteboards or tablet displays. Since the Wiimote can track upto 4 points, up to 4 pens can be used. It also works great with rear-projected displays.

Selected Press: Gizmodo, ArsTechnica, HackADay, Engadget

Using an LED array and some reflective tape, you can use the infrared camera in the Wii remote to track objects, like your fingers, in 2D space. This lets you interact with your computer simply by waving your hands in the air similar to the interaction seen in the movie "Minority Report". The Wiimote can track upto 4 points simultaneously. The multipoint grid software is a custom C# DirectX program.

Selected Press: Slashdot, Joystiq, Gizmodo, HackADay, Engadget

When conversing with someone via video conference, you are provided with a virtual window into their space. However, this currently remains both flat and fixed, limiting its immersiveness. We developed a method for producing a pseudo-3D experience using only a single generic webcam at each end. This means nearly any computer currently able to video conference can use our technique, making it readily adoptable. Although using comparatively simple techniques, the 3D result is compelling.

Selected Press: Slashdot, Make, DailyDIY, HackADay, Wired

Pediluma is a shoe accessory that tracks and visualizes the wearer’s physical activity by varying the intensity of a lighted housing. In particular, the more physically active the wearer is, the more the device glows. We hoped the desire to maintain a positive, glowing state would drive users to engage in more physical activity. We ran a two-week, four-condition user study to investigate the device's effects. Results indicate participants wearing our device were significantly more physically active than participants in control groups.

Modern computer displays tend to be in fixed size, rigid, and rectilinear rendering them insensitive to the visual area demands of an application or the desires of the user. Foldable displays offer the ability to reshape and resize the interactive surface at our convenience and even permit us to carry a very large display surface in a small volume. We implemented four interactive foldable display designs using image projection with low-cost tracking and explore display behaviors using orientation sensitivity.

The size and resolution of computer displays has increased dramatically, allowing more information than ever to be rendered on-screen. However, items can now be so small or screens so cluttered that users need to lean forward to properly examine them. This behavior may be detrimental to a user’s posture and eyesight. Our Lean and Zoom system detects a user’s proximity to the display using a camera and magnifies the on-screen content proportionally. This alleviates dramatic leaning and makes items more readable.

Selected Press: Fox Business, CMU

Projection technology typically places several constraints on the geometric relationship between the projector and the projection surface to obtain an undistorted, properly sized image. We developed a simple, robust, fast, and low-cost method for automatic projector calibration that eliminates many of these constraints. This technique can be expanded to automatically stitch multiple projectors, calibrate onto non-planar surfaces for object decoration, and provide a method for simple geometry acquisition.

Selected Press: Engadget, HackADay

By projecting smaller patterns over the discovered locations of the sensors, we can obtain location updates much faster sufficient to do interactive tracking of hand-held surfaces and objects. Additionally, small patterns free up pixels that can be used for application content. This work also describes a technique for projecting the patterns in a frequency modulated (FM) manner such that they are imperceptible to the human eye. The result is that the patterns appear as solid gray squares eliminating the high-contrast black and white patterns from before.

We developed a first design and implementation of a high-resolution, scalable, general purpose invisible near-infrared projector that can be manufactured in a practical manner. This approach is compatible with simultaneous visible light projection and integrates well with future Digital Light Processing (DLP) projector designs. By unifying both the visible and non-visible pattern projection into a single device, we can greatly simply the implementation and execution of interactive projection systems. Additionally, we can inherently provide location discovery and tracking capabilities that are unattainable using other approaches.

The home deployment of sensor-based systems offers many opportunities, particularly in the area of using sensor-based systems to support aging in place by monitoring an elder's activities of daily living. But existing approaches to home activity recognition are typically expensive, difficult to install, or intrude into the living space. This research considered the feasibility of a new approach that "reaches into the home" via the existing infrastructure. Specifically, we deploy a small number of low-cost sensors at critical locations in a home's water distribution infrastructure to infer activities.

The Haptic Pen is a simple low-cost device that provides individualized tactile feedback for multiple simultaneous users and can operate on large touch screens as well as ordinary surfaces. A pressure-sensitive stylus is combined with a small solenoid to generate a wide range of tactile sensations. The physical sensations generated by the Haptic pen can be used to enhance our existing interaction with graphical user interfaces as well as to help make modern computing systems more accessible to those with visual or motor impairments.

Light Emitting Diodes (LEDs) offer long life, low cost, efficiency, brightness, and a full range of colors. Because of these properties, they are widely used for simple displays in electronic devices. A previously characterized, but little known property of LEDs allows them to be used as photo sensors. We show how this capability can be used to turn unmodified, off the shelf, LED arrays into touch sensitive input devices, (while still remaining capable of producing output). The technique is simple and requires little or no extra hardware - in some cases operating with the same micro-controller based circuitry normally used to produce output, requiring only software changes.

Toolkits and other tools have dramatically reduced the time and technical expertise needed to design and implement graphical user interfaces. Unfortunately the generation of functioning prototypes for physical interactive devices as not had similar support - it still requires substantial time and effort by individuals with highly specialized skills and tools. Calder is a development environment for rapidly exploring and prototyping functional physical interactive devices. The system provides a set of reusable small input and output components, and integration into existing interface prototyping environments. These components communicate with a computer using wired and wireless connections. Calder is a tool targeted toward product and interaction designers to aid them in their early design process.

Rapid, early, but rough system prototypes are becoming a standard and valued part of the user interface design process. In the case of physical forms, however, with embedded technology, there is a lack of tools for developing rapid, early prototypes. To help alleviate this problem, we developed a simple tool for very rapidly creating functioning, rough physical prototypes early in the design process - supporting what amounts to interactive physical sketching. Our tool allows a designer to combine exploration of form and interactive function, using objects constructed from materials such as thumbtacks, foil, cardboard and masking tape, enhanced with a small electronic sensor board.

Ambient information displays are designed to convey background or context information that the user may or may not wish to attend to at any given time. They are designed to work primarily in the periphery of a user's awareness, moving to the center of attention only when appropriate and desirable. This research describes a new ambient information display that is designed to give a rich medium of expression placed within an aesthetically pleasing decorative object. This display - The Information Percolator - is formed by bubbles of air rising up columns of water.