Daydream allows people to use their smartphones to have a rich VR experience with a comfortable headset and intuitive motion controller.
As interaction design lead on hardware for the Google VR team, Hayes helped originate the Daydream platform and Daydream View and Controller products, and carried it from early concept and prototypes to final production. He was intimately involved in the industrial and interaction design of the Daydream Controller, and helped manage the UX and human factors requirements for the headset.
Hayes led the interaction research team for Google Glass and was part of the core team from the time team members started making custom hardware through shipping our v1 and v2 hardware. He focused on developing new experiences which are uniquely enabled by the form factor. Wink to Photo and notification glance emphasized hands-free control. Entertainment (music and earbuds), novel input methods (eye gestures, head movement, voice and more), and Mini Games illustrate the Interaction Research, design strategy, technology development, and system prototyping Hayes’ team conducted.
Pop Goes The Cell Phone confronts the challenge that very young children experience the world largely in the “here and now,” and asynchronous messaging is, by definition, neither here nor now.
A jack-in-the box toy, Toaster, uses a mobile phone for children to playfully take and share electronic media. Orange Toaster allows children to create and share self-portraits; Family Toast allows children to browse family photos with physical tokens, and shares their self-portrait reactions with remote family members; Play with Elmo allows children and distant adults to asynchronously share playful video messages. Download the research paper.
Story Visit is a website where children and long-distance loved ones can read story books together. Story Visit combines video conferencing and connected story books: when a grown-up turns the page, the child’s page turns along with it. If the child points to something on his screen, the grown-up can see that on her screen too. Now families can read stories together even when they can’t be in the same place.
Our research showed that conversations with 3 year olds increased from several minutes (in a typical video call) to 15-20 minutes when they used Story Visit. Reading books together really is more fun!
Family Story Play supports grandparents to read books together with their grandchildren over the Internet. Story Play improves communication across generations and over a distance, and to supports parents and grandparents in fostering the literacy development of young children.
The interface encourages active child participation in the book reading experience by combining a paper book, a sensor-enhanced frame, video conferencing technology, and video content of a Sesame Street Muppet (Elmo). These features improve child engagement in long-distance communication. Story Play also encourages dialogic reading styles that are linked with literacy development.
Family Story Play leverages a synergy between young children’s education—in this case, a rich shared reading experience—and communication with long-distance family. Our goal is to create new kinds of toys and tools that support creativity, learning, and family togetherness.
Sound of Touch is a semi-acoustic instrument for real-time capture and sensitive physical stimulation of sound samples using digital convolution. A hand-held wand can be used to record sound and then playback the recording by brushing, scraping, striking or otherwise physically manipulating the wand against physical objects. During playback, the recorded sound is continuously filtered by the acoustic interaction of the wand and the material being touched.
Freestanding Texture kits invite visitors to sculpt sound by exploring the visual, tactile and acoustic ranges of familiar materials. Their designs are inspired by Mondrian’s mid-career abstract paintings of blocks and lines, and in the texture kits, familiar materials are decontextualized as repurposeable media content.
While an acoustic instrument’s resonance is typically determined by the materials from which it is built, digital audio tools are usually divorced from the world of physical acoustics. With the Sound of Touch, resonant materials can be chosen during the performance itself, allowing performers to shape the acoustics of digital sounds by leveraging their intuitions for the acoustics of physical objects. The Sound of Touch permits real-time exploitation of the sonic properties of a physical environment, to achieve a rich and expressive control of digital sound that is not typically possible in electronic sound synthesis and control systems.
As a researcher At the MIT media lab I have been pursuing my vision to create interactive toys that people bring to life through play and learning. My expertise is in designing products that navigate the boundary between the physical and digital worlds, building on over ten years inventing and bringing innovative designs to market. My work involves conceiving new ideas for interactive media, and helping manage prototyping, testing and marketing of those ideas. My goal is to reinvent children’s interactive technologies to reflect children’s passions, creativity and encourage collaboration.
Topobo is a 3D constructive assembly system with kinetic memory, the ability to record and playback physical motion. You can build a dog with Topobo, wiggle its body around with your hands and teach it to walk. The dog will then repeat your motions repeatedly. The same way kids can learn about static structures playing with blocks, they can learn about dynamic structures playing with Topobo. In this video interview with the Science Channel I explain the system in more detail.
Topobo integrates all of the things I love about design – a clean and emotionally engaging concept, strong aesthetics, clear interaction and enough flexibility for the user to put their own personality into the product. Furthermore, designing Topobo has been an amazing collaboration, and I have been able to learn skills from product designers, graphic designers, electrical and computer engineers, educators and kids. We have tested Topobo in classrooms with kids ages 5-14 throughout the design cycle. These interactions were enlightening for us, giving us insight how others made meaning of the system, seeing people build and animate creations we’d never imagined, and getting feedback about how to improve Topobo in future iterations. To our delight, both 2nd and 8th graders told us they thought Topobo was designed for them!
Our work led to four peer-reviewed conference papers, an ID Magazine Design Distinction, a Prix Ars Electronica Honorable Mention for interactive art, and numerous international shows in art, design and education. The system has also had a mature design cycle. In 2004 I received a Microsoft iCampus grant to conduct longitudinal research studies, and pursued a complete two-year redesign of the system including production manufacturing in Hong Kong. My experience conducting design work at the Chinese factory has given me a deeper understanding of the full design and manufacturing cycle of interactive media products.
Jabberstamp is the the first tool that allows children to synthesize their drawings and voices. To use Jabberstamp, children create drawings, collages or paintings on normal paper. They press a special rubber stamp onto the page to record sounds into their drawings. When children touch the marks of the stamp with a small trumpet, they can hear the sounds playback, retelling the stories they have created.
Children ages 4+ use Jabberstamp to embed names, narratives, characters’ voices and environmental sound effects in their original drawings. Children’s compositions help them communicate their stories with peers and adults, and allow them to record and situate stories in personally meaningful contexts before they have mastered writing.
I originally stumbled upon toy invention through an artist internship that helped shape my undergraduate sculpture studies at Yale. One summer I helped artist Michael Joaquin Grey conceive and design a new building toy called ZOOB, a haptic modeling system that allows people to reflexively learn about biological structures like bones and molecules. I later helped Michael start up a design and manufacturing company to bring ZOOB to market. ZOOB was a great success—we did over $3M in our first year, and won a number of awards including Dr. Toy Best Toy, Toy of the Year and ID Magazine Design Distinction for consumer products. Over the next several years I directed the company’s internal design department, overseeing all of the company’s print and product design.
After ZOOB, my independent art and design work led me to the MIT Media Lab, where I joined the Tangible Media Group to develop my technical and conceptual skills developing electronic media. My early work at MIT was driven by an ambition to create new product designs that engage people’s kinesthetic sense and love of play to support communication, entertainment and learning.
Super Cilia Skin is a literal membrane separating a computer from its environment. Like our skin, it is haptic I/O membrane that can sense and simulate movement and wind flow. Our intention is to have it be universally applied to sheath any surface. As a display, it can mimic another person’s gesture over a distance via a form of tangible telepresence. A hand-sized interface covered with Super Cilia Skin would produce subtle changes in surface texture that feel much like a telepresent "butterfly kiss."
Our extensive design studies spanned dozens of physical and electronic prototypes and led to a technical paper in CHI and a full journal paper in "Textile: The Journal of Cloth and Culture" (Berg 2004). The investigation focused on the haptics and scale: as a product, Super Cilia Skin is a "touch telephone." As wallpaper or carpet, we considered the potential for actuated, telepresent interior design. At the tectonic scale, sheathing tall buildings, the interface assumes dual roles as billboard size display or as wind-driven electromagnetic power generators.
The You’re In Control system uses computation to enhance the act of urination. Sensors in the back of a urinal detect the position of impact of a stream of urine, enabling the user to play interactive games on a screen mounted above the urinal.
In order to allow men and women to publicly interact with YIC, we built a customized game controller that can be strapped around the waist and pressurized to create a stream of water. Players can toggle between the "Flying Hamsters" and "You’re A Nation" games by flushing the urinal. It really is a pissing contest!
About fifty years ago my grandfather invented a polymer clay called Sculpey and reinvented how children can create and express themselves. His life as an inventor and entrepreneur inspires me to imagine how electronic toys can be more than flashy gadgets or media conduits, and actually amplify children’s creativity, expression and playful learning.
Natura maxime miranda in minimis
(Nature is greatest in little things)
Super Cilia Skin is a literal membrane separating a computer from its environment. Like our skin, it is haptic I/O membrane that can sense and simulate movement and wind flow. Our intention is to have it be universally applied to sheath any surface. As a display, it can mimic another person’s gesture over a distance via a form of tangible telepresence. A hand-sized interface covered with Super Cilia Skin would produce subtle changes in surface texture that feel much like a telepresent “butterfly kiss.”
A small object surrounded with Super Cilia Skin could propel itself across the floor, or stop to create
visually expressive changes in surface. Conversely, a Super Cilia Skin surface could propel objects across it using mechanical gestures like the movements of a centipede’s feet.