Entertainment & Music

Musical Robot Companion Enhances Listener Experience

Atlanta, GA (June 26, 2012) — Wedding DJs everywhere should be worried about job security now that a new robot is on the scene.  

Shimi, a musical companion developed by Georgia Tech's Center for Music Technology, recommends songs, dances to the beat and keeps the music pumping based on listener feedback. The smartphone-enabled, one-foot-tall robot is billed as an interactive "musical buddy."

"Shimi is designed to change the way that people enjoy and think about their music,"said Professor Gil Weinberg, director of Georgia Tech’s Center for Music Technology and the robot’s creator. He will unveil the robot at Wednesday’s Google I/O conference in San Francisco. A band of three Shimi robots will strut its stuff for guests, dancing in sync to music created in the lab and composed according to its movements.

Shimi is essentially a docking station with a "brain" powered by an Android phone. Once docked, the robot gains the sensing and musical generation capabilities of the user's mobile device. In other words, if there’s an "app for that," Shimi is ready.

For instance, by using the phone’s camera and face-detecting software, the bot can follow a listener around the room and position its "ears," or speakers, for optimal sound. Another recognition feature is based on rhythm and tempo. If the user taps or claps a beat, Shimi analyzes it, scans the phone’s musical library and immediately plays the song that best matches the suggestion. Once the music starts, Shimi dances to the rhythm.

"Many people think that robots are limited by their programming instructions," said Music Technology Ph.D. candidate Mason Bretan. "Shimi shows us that robots can be creative and interactive."

Future apps in the works will allow the user to shake their head in disagreement or wave a hand in the air to alert Shimi to skip to the next song or increase/decrease the volume. The robot will also have the capability to recommend new music based on the user’s song choices and provide feedback on the music play list.

Weinberg hopes other developers will be inspired to create more apps to expand Shimi’s creative and interactive capabilities, allowing the machine to leave the lab and head into the real world.

"I believe that our center is ahead of a revolution that will see more robots in homes, bypassing some of the fears some people have about machines doing everyday functions in their lives," Weinberg said.

Weinberg is in the process of commercializing Shimi through an exclusive licensing agreement with Georgia Tech. A new start-up company, Tovbot, has been formed and Weinberg hopes to make the robot available to consumers by the 2013 holiday season. Shimi was developed in collaboration with the Media Innovation Lab at IDC Herzliya and led by Professor Guy Hoffmann. Entrepreneur Ian Campbell and robot designer Roberto Aimi were also part of the Shimi team.

This is the third robotic musician created by the Center for Music Technology. Haile is a percussionist that can listen to live players, analyze their music in real-time and improvise with music of its own. Shimon is an interactive marimba player.

"If robots are going to arrive in homes, we think that they will be these kind of machines - small, entertaining and fun," Weinberg said. "They will enhance your life and pave the way for more sophisticated service robots in our lives."

This project is supported by the National Science Foundation (NSF) (Award Number HCC-1017169). The content is solely the responsibility of the principal investigators and does not necessarily represent the official views of the NSF.

Story written by Jason Maderer and Liz Klipp. 

Related Links

For more information contact:

Liz Klipp
Media Relations
klipp@gatech.edu
404-894-6016

 

Photos

Click on an image below to see the full photo

  • Shimi
  • Gil Weinberg 2011

Media

Click on an image to launch a media file.

  • Shimi BandVideo Clip
  • Shimi ExplainedVideo Clip
  • Shimi: How it WorksVideo Clip

Faculty

  • Amy Bruckman

    Amy Bruckman

    Associate Professor
    School of Interactive Computing, College of Computing

    Areas of Expertise:
    Educational Technology, Social Networking/Online Communities, Wikipedia, Twitter, Facebook, Internet Research Ethics, Human Computer Interaction, Human Computer Interaction for Kids

  • Carl DiSalvo

    Carl DiSalvo

    Assistant Professor
    School of Literature, Communication and Culture, Ivan Allen College of Liberal Arts

    Areas of Expertise:
    Participatory Design, Critical Design, Design Studies, Robotics and Sensing in Art and Community Settings

  • Keith Edwards

    Keith Edwards

    Associate Professor
    School of Interactive Computing, College of Computing

    Areas of Expertise:
    Social Impacts of Technology, Home Network Security, Home Networking, Human-Computer Interaction

  • Irfan Essa

    Irfan Essa

    Professor
    School of Interactive Computing, College of Computing
    School of Electrical and Computer Engineering, College of Engineering

    Areas of Expertise:
    Computational Video, Computational Photography, Computational Journalism, Computational Media, Computational Perception

  • Beki Grinter

    Beki Grinter

    Associate Professor
    School of Interactive Computing, College of Computing

    Areas of Expertise:
    Societal Impacts of Technology, Human-Computer Interaction, Computer Supported Cooperative Work

  • Renu Kulkarni

    Renu Kulkarni

    Executive Director, FutureMedia

    Areas of Expertise:
    Convergence of digital, social, mobile and multimedia industries, Strategic Alliances, Industry Partnerships, Open Innovation Practices

  • Blair MacIntyre

    Blair MacIntyre

    Associate Professor
    School of Interactive Computing, College of Computing
    School of Literature Communication and Culture, Ivan Allen College of Liberal Arts

    Areas of Expertise:
    Augmented Reality, Virtual Reality, Mobile Games, Social Games, Augmented Reality Games, Video Game Design, Video Game Architecture

  • Ali Mazalek

    Ali Mazalek

    Assistant Professor
    School of Literature, Communication and Culture, Ivan Allen College of Liberal Arts

    Areas of Expertise:
    Tangible Interfaces, Experimental Media, Media Arts, Interaction Design, Emerging Technologies

  • Janet Murray

    Janet H. Murray

    Ivan Allen College Dean's Professor
    School of Literature, Communication and Culture, Ivan Allen College of Liberal Arts

    Areas of Expertise:
    Game Design, Interactive Narrative, Interactive Television, Media Convergence, Information Design, Digital Media and Education

  • Elizabeth Mynatt

    Elizabeth Mynatt

    Director, GVU Center
    Professor, School of Interactive Computing
    Associate Dean for Strategic Planning and Initiatives
    College of Computing

    Areas of Expertise:
    Human-Computer Interaction, Human-Centered Computing, Health Informatics, Ubiquitous Computing, Assistive Technologies

  • Ashwin Ram

    Ashwin Ram

    Associate Professor
    School of Interactive Computing, College of Computing

    Areas of Expertise:
    Artificial Intelligence (AI) (Case-Based Reasoning, Natural Language, & Game/Entertainment AI), Human-Centered Computing - Cognitive Science, Healthcare Informatics

  • Bruce Walker

    Bruce Walker

    Associate Professor
    School of Psychology, College of Sciences School of Interactive Computing, College of Computing

    Areas of Expertise:
    Interactive Music, Mobile Music, Human-Computer Interaction, Auditory Perception, Psychology