Story Visit

Story Visit is a website where children and long-distance loved ones can read story books together. Story Visit combines video conferencing and connected story books: when a grown-up turns the page, the child’s page turns along with it. If the child points to something on his screen, the grown-up can see that on her screen too. Now families can read stories together even when they can’t be in the same place.

Our research showed that conversations with 3 year olds increased from several minutes (in a typical video call) to 15-20 minutes when they used Story Visit. Reading books together really is more fun!

Story Time for the 21st Century

Hayes Raffle, Rafael “Tico” Ballagas, Glenda Revelle, et al., in Collaboration with Sesame Workshop

Family Story Play

Family Story Play supports grandparents to read books together with their grandchildren over the Internet. Story Play improves communication across generations and over a distance, and to supports parents and grandparents in fostering the literacy development of young children.

The interface encourages active child participation in the book reading experience by combining a paper book, a sensor-enhanced frame, video conferencing technology, and video content of a Sesame Street Muppet (Elmo). These features improve child engagement in long-distance communication. Story Play also encourages dialogic reading styles that are linked with literacy development.

Family Story Play leverages a synergy between young children’s education—in this case, a rich shared reading experience—and communication with long-distance family. Our goal is to create new kinds of toys and tools that support creativity, learning, and family togetherness.

Katie and her dad read, and then mimic Grover in the story while the grandparent reads to them over video chat.

Sound of Touch


Hayes Raffle
& David Merrill, with Roberto Aimi
Steel, wood, custom electronics, audio processing software running on a PC, and various materials

The Sound of Touch
brushing-beige-wide2.jpg
Sound Of Touch

Sound of Touch is a semi-acoustic instrument for real-time capture and sensitive physical stimulation of sound samples using digital convolution. A hand-held wand can be used to record sound and then playback the recording by brushing, scraping, striking or otherwise physically manipulating the wand against physical objects. During playback, the recorded sound is continuously filtered by the acoustic interaction of the wand and the material being touched.

{website | papers | press | exhibitions}

Freestanding Texture kits invite visitors to sculpt sound by exploring the visual, tactile and acoustic ranges of familiar materials. Their designs are inspired by Mondrian’s mid-career abstract paintings of blocks and lines, and in the texture kits, familiar materials are decontextualized as repurposeable media content.

While an acoustic instrument’s resonance is typically determined by the materials from which it is built, digital audio tools are usually divorced from the world of physical acoustics. With the Sound of Touch, resonant materials can be chosen during the performance itself, allowing performers to shape the acoustics of digital sounds by leveraging their intuitions for the acoustics of physical objects. The Sound of Touch permits real-time exploitation of the sonic properties of a physical environment, to achieve a rich and expressive control of digital sound that is not typically possible in electronic sound synthesis and control systems.

Topobo

Topobo Griffon
Topobo system
Topobo mixed parts
Topobo moose+hands

While at the MIT Media Lab Hayes invented Topobo, a construction toy with kinetic memory, the ability to record and playback physical motion. Since 2008 he has managed Topobo Co. as founder and principal.

With Topobo, you can snap together Passive (static) and Active (motorized) components to invent your own Topobo creature and animate it by pushing, pulling, and twisting its body. For example, you can make a dog and then teach it to dance and walk by twisting its body and legs. With the push of a button, the dog will dance and walk by itself. The same way children learn how buildings stand by stacking blocks, they can learn how animals walk by playing with Topobo.

Watch the Topobo video.

{website | videos | brochure | press | awards | papers | images}

Jabberstamp

Hayes Raffle, Cati Vaucelle and Ruibing Wang
Paper, pens, paint and other physical media, microphone, speakers,
wireless transmitters and position sensing tablet, PC running custom software

Jabberstamp - girls drawing+recording
Jabberstamp drawing - annotated
Jabberstamp drawing

Jabberstamp is the the first tool that allows children to synthesize their drawings and voices. To use Jabberstamp, children create drawings, collages or paintings on normal paper. They press a special rubber stamp onto the page to record sounds into their drawings. When children touch the marks of the stamp with a small trumpet, they can hear the sounds playback, retelling the stories they have created.

Children ages 4+ can use Jabberstamp to embed names, narratives, characters’ voices and environmental sound effects in their original drawings. Children’s compositions help them communicate their stories with peers and adults, and allow them to record and situate stories in personally meaningful contexts to share with others, before they have mastered writing.

Winner, Honorable Mention from I.D. Magazine Student Design Review.

{ website | brochure | video | papers | press }

Future of Play

As a researcher At the MIT media lab I have been pursuing my vision to create interactive toys that people bring to life through play and learning. My expertise is in designing products that navigate the boundary between the physical and digital worlds, building on over ten years inventing and bringing innovative designs to market. My work involves conceiving new ideas for interactive media, and helping manage prototyping, testing and marketing of those ideas. My goal is to reinvent children’s interactive technologies to reflect children’s passions, creativity and encourage collaboration.

Topobo moose

Topobo is a 3D constructive assembly system with kinetic memory, the ability to record and playback physical motion. You can build a dog with Topobo, wiggle its body around with your hands and teach it to walk. The dog will then repeat your motions repeatedly. The same way kids can learn about static structures playing with blocks, they can learn about dynamic structures playing with Topobo. In this video interview with the Science Channel I explain the system in more detail.

Topobo integrates all of the things I love about design – a clean and emotionally engaging concept, strong aesthetics, clear interaction and enough flexibility for the user to put their own personality into the product. Furthermore, designing Topobo has been an amazing collaboration, and I have been able to learn skills from product designers, graphic designers, electrical and computer engineers, educators and kids. We have tested Topobo in classrooms with kids ages 5-14 throughout the design cycle. These interactions were enlightening for us, giving us insight how others made meaning of the system, seeing people build and animate creations we’d never imagined, and getting feedback about how to improve Topobo in future iterations. To our delight, both 2nd and 8th graders told us they thought Topobo was designed for them!

Our work led to four peer-reviewed conference papers, an ID Magazine Design Distinction, a Prix Ars Electronica Honorable Mention for interactive art, and numerous international shows in art, design and education. The system has also had a mature design cycle. In 2004 I received a Microsoft iCampus grant to conduct longitudinal research studies, and pursued a complete two-year redesign of the system including production manufacturing in Hong Kong. My experience conducting design work at the Chinese factory has given me a deeper understanding of the full design and manufacturing cycle of interactive media products.
Kids playing with Topobo

Jabberstamp - girls drawing+recording

Jabberstamp is the the first tool that allows children to synthesize their drawings and voices. To use Jabberstamp, children create drawings, collages or paintings on normal paper. They press a special rubber stamp onto the page to record sounds into their drawings. When children touch the marks of the stamp with a small trumpet, they can hear the sounds playback, retelling the stories they have created.

Children ages 4+ use Jabberstamp to embed names, narratives, characters’ voices and environmental sound effects in their original drawings. Children’s compositions help them communicate their stories with peers and adults, and allow them to record and situate stories in personally meaningful contexts before they have mastered writing.

I originally stumbled upon toy invention through an artist internship that helped shape my undergraduate sculpture studies at Yale. One summer I helped artist Michael Joaquin Grey conceive and design a new building toy called ZOOB, a haptic modeling system that allows people to reflexively learn about biological structures like bones and molecules. I later helped Michael start up a design and manufacturing company to bring ZOOB to market.  ZOOB was a great success—we did over $3M in our first year, and won a number of awards including Dr. Toy Best Toy, Toy of the Year and ID Magazine Design Distinction for consumer products. Over the next several years I directed the company’s internal design department, overseeing all of the company’s print and product design.
zoob-lineup.jpg

After ZOOB, my independent art and design work led me to the MIT Media Lab, where I joined the Tangible Media Group to develop my technical and conceptual skills developing electronic media. My early work at MIT was driven by an ambition to create new product designs that engage people’s kinesthetic sense and love of play to support communication, entertainment and learning.

Super Cilia Skin
Super Cilia Skin is a literal membrane separating a computer from its environment. Like our skin, it is haptic I/O membrane that can sense and simulate movement and wind flow. Our intention is to have it be universally applied to sheath any surface.  As a display, it can mimic another person’s gesture over a distance via a form of tangible telepresence. A hand-sized interface covered with Super Cilia Skin would produce subtle changes in surface texture that feel much like a telepresent "butterfly kiss."  

Our extensive design studies spanned dozens of physical and electronic prototypes and led to a technical paper in CHI and a full journal paper in "Textile: The Journal of Cloth and Culture" (Berg 2004). The investigation focused on the haptics and scale: as a product, Super Cilia Skin is a "touch telephone." As wallpaper or carpet, we considered the potential for actuated, telepresent interior design. At the tectonic scale, sheathing tall buildings, the interface assumes dual roles as billboard size display or as wind-driven electromagnetic power generators.

The You’re In Control system uses computation to enhance the act of urination. Sensors in the back of a urinal detect the position of impact of a stream of urine, enabling the user to play interactive games on a screen mounted above the urinal.

In order to allow men and women to publicly interact with YIC, we built a customized game controller that can be strapped around the waist and pressurized to create a stream of water. Players can toggle between the "Flying Hamsters" and "You’re A Nation" games by flushing the urinal. It really is a pissing contest!

You’re In Control (Urine Control) interactive gaming system

About fifty years ago my grandfather invented a polymer clay called Sculpey and reinvented how children can create and express themselves. His life as an inventor and entrepreneur inspires me to imagine how electronic toys can be more than flashy gadgets or media conduits, and actually amplify children’s creativity, expression and playful learning.

+Hayes Raffle

{ contact | cv | portfolio }

 

Fuzzmail

Fuzzmail logo
Fuzzmail screenshot

With email, why does a love letter have to look the same as a business letter? We invented Fuzzmail to give people’s typed communications more emotional breadth. Fuzzmail records the act of writing and lets the author send it as an email. Dynamic changes, typoes, pauses and writeovers are captured and communicated. Fuzzmail was created to provide a more emotionally expressive alternative to email, so that an emailed love letter does not have to look the same as a business letter.

You can check out one of my favorites about family and caffeine here.

Fuzzmail allows you to create more expressive text communications by incorporating dramatic timing and rhythm into the content of the written message. The act of writing becomes more like singing or acting. A fuzzmail may be a carefully scripted performance or a spontaneous creation.

Give Fuzzmail a try at http://www.fuzzmail.org.

* Winner of the 2007 Web 2.0 Awards Honorable Mention, “Fun Stuff.”

Youre In Control (Urine Control) Interactive Gaming System

Dan Maynes-Aminzade and Hayes Raffle
Urinal, Electronics, PIC Microcontroller, PC gaming equipment running custom video game

You
Playing You’re In Control
proud-owner.jpg

The You’re In Control system uses computation to enhance the act of urination. Sensors in the back of a urinal detect the position of impact of a stream of urine, enabling the user to play interactive games on a screen mounted above the urinal.

While urination fulfills a basic bodily function, it is also an activity rich with social significance. Along with the refreshing release it provides, the act of micturition satisfies a primal urge to mark our territory. For women who visit the bathroom in groups and chat in neighboring stalls, urination can be a bonding ritual. For men who write their names in the snow, extinguish cigarettes, or congregate around lampposts to urinate, urination can be a test of skill and a way of asserting their masculinity.

Flush the urinal to play You’re a Nation, and drown political opposition as they campaign in key swing states.

Patented.

{website | video | paper}

Super Cilia Skin

Super Cilia Skin
Super Cilia Skin hand front
Super Cilia Skin hand

Natura maxime miranda in minimis
(Nature is greatest in little things)

Super Cilia Skin is a literal membrane separating a computer from its environment. Like our skin, it is haptic I/O membrane that can sense and simulate movement and wind flow. Our intention is to have it be universally applied to sheath any surface.  As a display, it can mimic another person’s gesture over a distance via a form of tangible telepresence.  A hand-sized interface covered with Super Cilia Skin would produce subtle changes in surface texture that feel much like a telepresent “butterfly kiss.”

A small object surrounded with Super Cilia Skin could propel itself across the floor, or stop to create
visually expressive changes in surface. Conversely, a Super Cilia Skin surface could propel objects across it using mechanical gestures like the movements of a centipede’s feet.

{website | video}

Publications

Included in Inform Material, Thema. 2008.

Included in Brownell, Blaine (Ed). Transmaterial 2: A Catalog of Materials That Redefine Our Physical Environment. Princeton, NJ: Princeton University Press (2008).

Raffle, H., and Tichenor, J. Super Cilia Skin: Interfaz de Texturas, Pasajes Construcción: Materiales. Number 31, September 2007, pp. 8-9 and front cover. Madrid: América Ibérica.

Raffle, H., Tichenor, J., and Ishii, H. Super Cilia Skin: A Textural Interface, Textile: The Journal of Cloth and Culture. London: Berg Publishers. 2004.

Raffle, H., Joachim, M., and Tichenor, J. Super Cilia Skin: An Interactive Membrane, Proceedings of CHI 2003.

Jiggi

Jiggi a
jiggi b

Jiggi a

Jiggi is a 3D modeling and visualization interface.

Jiggi extrudes the Sensetable graspable interface into a 3d projection plane system. A Jiggi device, is a system which tracks the positions of intelligent objects on to multiple tabletop surfaces, and projects information onto these objects. This allows the combination and the advantages of physical interaction with the dynamic qualities of graphical displays in 3d. The use of a small electronically tagged magnetic Quill serves as a physical icon for the containment, transport, and manipulation of
the 3d modeling data. In this case the data is Landscape Ecology information.

The Jiggi unit is used to alter anything from spheres of watershed engagement to plotting wind course corrections. An ecological model is projected in multiple views at once thus providing sectional layouts and vector based information. Here the projections can be manipulated to alter the object or reveal hidden relationships.

{website}

Artworks 1992-2001



Check it out in a separate window.


The Rafelandia website is an archive of my work from 1992-2002, presented as a full flash website. My sculpture, titled “love” is buried there among associations and metaphors, and some of my older professional work – “money” – is presented with detailed attention to aesthetics.

The site design is intended to reveal related work to web surfers who dive deeper into content that interests them, but it is a bit cumbersome for my speedier approach to viewing most websites today. Check it out in the frame above, or in a separate window.

This site was designed and built in collaboration with mogeworks design.

Rememberpen

Rememberpen is a pen computing concept I developed while considering the ways computers could fit into my life more seamlessly.

[kml_flashembed movie=”/wp-content/uploads/rememberpen.swf” height=”300″ width=”433″ /]


When I first started thinking about how computers might be designed a more fluid part of my life, I threw together a concept animation for Rememberpen, a simple pen computing concept that is now nearly productized with technologies like Anoto. However, there is still no seamlessness between physical and digital media, and creative work in the area of interaction design is much needed to make our increasingly complex machines simpler to use.