Hayes Raffle Ph.D.

About

I specialize in early-stage product development, taking concepts from “wouldn’t it be cool if…” through v1 launch. Currently at Google, I’m developing applied consumer AI experiences that make Search, YouTube, and Gemini more helpful for learning.

I recently returned to hands-on IC work after building and leading a cross-functional team of 23 (engineering, design, UXR) at Google LearnX. Our team launched Learn About, which won 6 international design awards and continues to inspire learning roadmaps across Google’s products. I made the shift back to IC to stay close to the emerging possibilities in AI—there’s still so much to build.

My focus on democratizing learning was shaped by a year spent traveling globally and researching technology use throughout the world. Seeing how many people had great phones and internet but lacked access to quality teachers reinforced my commitment to building educational experiences that enable anyone to improve their life through learning.

Previously, I helped shape the future of AR and VR as a core team member of Google Glass and lead for Daydream VR, where I invented the input methods and UI frameworks that became industry standard for mass-market 3DoF VR systems. I developed hands-free input methods for wearable computing including eye gaze tracking, eye gestures, and biometric input.

My work for children includes Family Communication tools for preschoolers to connect with long-distance loved ones. My toys and learning tools include the award-winning Topobo and ZOOB® building systems, which have served as foundational learning tools for over 25 years.

I hold a Ph.D. and M.S. in Media Arts and Sciences from the MIT Media Lab, and a B.A. in fine art from Yale. I’m inventor on over 75 US utility patents and have authored over 40 peer-reviewed academic publications in the areas of tangible media, technologies for learning, and virtual and augmented reality.

I gravitate toward products that ship efficiently and meaningfully improve how people learn, create, and connect. Particularly interested in early-stage products that combine technical innovation with social impact.

Google’s Learning Focus Area

During the pandemic I saw how people like my own high-school aged daughters were struggling, and I wanted to help. I joined the Learning FA, a new central team being formed to help make Google’s products better for learning. I focused on broad strategic goals (like defining new business areas) and spent a lot of time making our core products – like YouTube, Search and Gemini – more educational. I’ve also built a team of 20+ product designers, engineers and user researchers.

We’ve focused a lot of energy on fulfilling a long-held dream in educational technology: to create a personalized tutor for every learner that’s always available, and free. At Google I/O 2024, we announced LearnLM, our cross-company effort to make Google’s foundational AI models great for teaching and learning, and showed how we’re bringing it to learners with features on Search, YouTube, and Gemini. We’re also building some brand new products designed to help anyone learn anything.

Learn About is a Learning Labs experience that explores how information can turn into understanding by bringing together high-quality content, learning science and chat experiences. Ask a question and it helps guide you through any topic at your own pace — through pictures, videos, webpages and activities — and you can upload files or notes and ask clarifying questions along the way. The project won 6 design awards including iF Design Award, Red Dot Award, Webby Awards (Best User interface, Responsible AI, Best Navigation/Structure), and UX Design Awards. I spoke a few times about how curiosity motivates deeper and more meaningful learning, including Learn About: AI for Curious People at the ASU+GSV AI Show.

I’m proud that Learn About helped demonstrate how genAI products can go beyond text. We’re bringing a lot of that magic to our scaled products that reached hundreds of millions of learners every day.

Google’s AI Overviews let learners get an overview for almost any question, and dive deeper in AI Mode.

Periodic table lets people explore an interactive version of this classic chemistry mainstay. Clicking on any element opens an interactive 3d Bohr model with key facts about the element. People can even view an AR version of this model right in their living rooms or classrooms with a mobile phone.

YouTube is known for its endless library of how-to videos, but did you know that millions of people also use it to help them with their schoolwork every day? But it’s not always easy, and people told us it could be distracting. So, we removed distractions that used appear alongside academic videos, and now suggest only relevant academic videos when you’re studying. We also added new features like searchable transcripts that help students quickly hone in to the parts of videos that answer their questions.

On Gemini, we’ve introduced interactive quizzes and a learning mode where the AI asks you questions to make sure you really understand what you’re studying.

This work is both rewarding, and fun. Over 20 years ago, I made exhibits at the San Francisco Exploratorium and loved how kids would playfully explore, experience and learn new ideas driven by their curiosity. At Google, I love to ask how we can bring playful exploration to learners’ experiences with our products. Because who said learning can’t be fun?

Google VR

Daydream allows people to use their smartphones to have a rich VR experience with a comfortable headset and intuitive motion controller.

As interaction design lead on hardware for the Google VR team, Hayes helped originate the Daydream platform and Daydream View and Controller products, and carried it from early concept and prototypes to final production. He was intimately involved in the industrial and interaction design of the Daydream Controller, and helped manage the UX and human factors requirements for the headset.

Daydream View was announced in November, 2017 and was received as comfortable and easy to use. Our favorites called it an “eye pillow” with a “comfy sweatpants vibe”, pointed out how intuitive the UI is, and highlighted some of the clever things we did. We even got street cred from the fashion world for bringing some style to VR.

Google Glass

Hayes led the interaction research team for Google Glass and was part of the core team from the time team members started making custom hardware through shipping our v1 and v2 hardware. He focused on developing new experiences which are uniquely enabled by the form factor. Wink to Photo and notification glance emphasized hands-free control. Entertainment (music and earbuds), novel input methods (eye gestures, head movement, voice and more), and Mini Games illustrate the Interaction Research, design strategy, technology development, and system prototyping Hayes’ team conducted.

Story Time for the 21st Century

Hayes Raffle, Rafael “Tico” Ballagas, Glenda Revelle, et al., in Collaboration with Sesame Workshop

Family Story Play

Family Story Play supports grandparents to read books together with their grandchildren over the Internet. Story Play improves communication across generations and over a distance, and to supports parents and grandparents in fostering the literacy development of young children.

The interface encourages active child participation in the book reading experience by combining a paper book, a sensor-enhanced frame, video conferencing technology, and video content of a Sesame Street Muppet (Elmo). These features improve child engagement in long-distance communication. Story Play also encourages dialogic reading styles that are linked with literacy development.

Family Story Play leverages a synergy between young children’s education—in this case, a rich shared reading experience—and communication with long-distance family. Our goal is to create new kinds of toys and tools that support creativity, learning, and family togetherness.

Katie and her dad read, and then mimic Grover in the story while the grandparent reads to them over video chat.

Topobo

While at the MIT Media Lab Hayes invented Topobo, a construction toy with kinetic memory, the ability to record and playback physical motion. Since 2008 he has managed Topobo Co. as founder and principal.

With Topobo, you can snap together Passive (static) and Active (motorized) components to invent your own Topobo creature and animate it by pushing, pulling, and twisting its body. For example, you can make a dog and then teach it to dance and walk by twisting its body and legs. With the push of a button, the dog will dance and walk by itself. The same way children learn how buildings stand by stacking blocks, they can learn how animals walk by playing with Topobo.

Watch the Topobo video.

{website | videos | brochure | press | awards | papers | images}

Youre In Control (Urine Control) Interactive Gaming System

Dan Maynes-Aminzade and Hayes Raffle
Urinal, Electronics, PIC Microcontroller, PC gaming equipment running custom video game

The You’re In Control system uses computation to enhance the act of urination. Sensors in the back of a urinal detect the position of impact of a stream of urine, enabling the user to play interactive games on a screen mounted above the urinal.

While urination fulfills a basic bodily function, it is also an activity rich with social significance. Along with the refreshing release it provides, the act of micturition satisfies a primal urge to mark our territory. For women who visit the bathroom in groups and chat in neighboring stalls, urination can be a bonding ritual. For men who write their names in the snow, extinguish cigarettes, or congregate around lampposts to urinate, urination can be a test of skill and a way of asserting their masculinity.

Flush the urinal to play You’re a Nation, and drown political opposition as they campaign in key swing states.

Patented.

{website | video | paper}

ZOOB

ZOOB® is the first construction toy based on things that grow, from DNA to dinosaurs. Hayes worked with inventor Michael Grey to conceive and design the initial system. As employee #6 at Primordial, LLC, Hayes helped oversee all aspects of production design, including helping build and manage a design team to create product designs, packaging designs, and videos. The team won an ID Magazine award in the 45th Annual Design Review in Consumer Products and the toy has been popular for over twenty years.