Interactive Music Challenge: The Reactable & the Reactable for Children with Autism Spectum Disorders

What I'd like to share today might be of interest to educators, techies, musicians, parents, and students. If you create a composition using the Reactable Mobile app and submit it, you might have a chance of winning your very own Reactable Live!  


Information about the challenge can be found on the Reactable website.

You can find information about creating music with Reactable Mobile for your phone or tablet (Android or Apple) on the Reactable Mobile site. I especially like that the company provides a load of support through the Reactable Community.



Acquisition of Joint Attention and Social Abilities of Children with Autism Spectrum Disorders

The Reactable provided the centerpiece of Lilia Villafuerte Bazan's work for her Master thesis.  Her project focused on the acquisition  of joint attention and social abilities of children with autism spectrum disorders through music. Lilia's work was through the Music Technology Group at Universitat Pompeu Fabra, under the direction of Dr. Sergi Jorda.   

I was delighted to see Lilia's presentation of her work, along with the video, during the recent CHI 2012 conference. 


Project website: 
http://www.villafuerte.info/ascchildrenandtuis
Master Thesis:
http://www.villafuerte.info/wp-content/uploads/2012/05/LiliaVillafuerte_MasterThesis.pdf


-Lilia Villafuerte Bazan

RELATED
REACTABLE LIVE

Comment:
If you follow this blog, you probably know that I am a school psychologist who has a background in technology, music, and related arts.  I spend much of my work week with young people who have severe autism and other disabilities. I also work at a magnet high school for technology and the arts.  I'd like a Reactable for students at both schools!

CHI 2012 SlideShow (quick)

Here are some pictures from CHI 2012:


 I'll be sharing about innovative, interesting interactive technologies in some of my future posts.

URBAN MUSICAL GAME: Play with musical-sensor sports balls and friends (Real-Time Musical Interactions team at IRCAM)

One of my favorite things at CHI 2012 was the Interactivity sessions.  I especially liked the  Urban Musical Game, played with musical sports balls.  I didn't want to stop playing!


This just might transform P.E. and recess.


Below is a version of the game from 2011 created for the Future en Seine festival by the Real-Time Musical Interaction team at the IRCAM, in collaboration with NoDesign, Phonotonic, and composer Andrea Cera:

The balls in the game contain wireless sensors, which trigger sounds and interactive music.  It is supported by MO  (Modular Musical Object) technology, and Max/MSP. Below is a video of the balls in action (in French).

Nicolas Rasamimanana - Phonotonic, Paris, France 
Frederic Bevilacqua - IRCAM, France 
Julien Bloit - IRCAM, France Norbert Schnell - IRCAM, France 
Emmanuel Flety - IRCAM, France Andrea Cera - IRCAM, France 
Uros Petrevski - NoDesign, France 
Jean-Louis Frechin - NoDesign, France


Comment:
This would be a great activity for the large therapy room at Wolfe, a program for students with significant disabilities (including autism), where I spend most of my work days as a school psychologist.  I'd love to have it demonstrated in-person for students at the CATA (Central Academy of Technology & Arts), the high school I also serve, to inspire students about the creative aspects of technology.


I would also like to create my experiments with this sort of technology, based on a number of interesting ideas that have popped up in my mind over the past decade or so.  I'm especially curious to know how this could work in environments that also have multiple large interactive visual displays....


How can I make it so?
(Update:  It is a small world.  I looked up one of the people involved in the Urban Musical Game on Facebook and found that we share three mutual FB "friends".)

Keepin' It Real: Organic, Natural, or Reality-Based Interactions? Video of TEI'12 panel discussion: Bill Buxton, Rob Jacob, Mili John Tharakan, and Roel Vertegaal (and more from TEI '12)

TEI '12 is finishing up in Kingston, Ontario.  If you missed it, like I did, don't worry.  Many of the presentations were streamed live, and are available to watch at your convenience.  
(The panel gets started at about 1: 48 on the timeline)


Keepin' It Real: Organic, Natural or Reality-Based Interactions?
Video streaming by UstreamModerator: Amanda Parkes, Columbia University 
Panelists
William Buxton, Microsoft Research
Rob Jacob, Tufts University 
Mili John Tharakan, The Swedish School of Textiles 
Roel Vertegaal, Queen's University


RELATED
Here are some TEI '12 presentations interest me!
Process Pad: A Low-Cost Multi-Touch Platform to Facilitate Multimodal Documentation of Complex Learning
Shima Salehi, Stanford University
Jain Kim, Stanford University
Colin Meltzer, Stanford University
Paulo Blikstein, Stanford University
Process Pad project website


Exploring peripheral interaction design for primary school teachers
Saskia Bakker Eindhoven University of Technology, the Netherlands 
Elise van den Hoven Eindhoven University of Technology, the Netherlands 
Berry Eggen Eindhoven University of Technology, the Netherlands 
Kees Overbeeke Eindhoven University of Technology, the Netherlands 


Others:
Exploring Tabletops as an Effective Tool to Foster Creativity Traits
Mobile Lorm Glove - Introducing a communication device for deaf-blind people
Mobile Lorm Glove project website
The HapticTouch Toolkit: Enabling Exploration of Haptic Interactions

HapticTouch Toolkit from Haptic Touch on Vimeo.

SIFTEO, the next-gen Siftables! (Tangible User Interfaces for All)

Despite my enthusiasm for TUI's , I somehow missed the news about the transformation of Siftables to a commercial version, Sifteo:

Sifteo Inc. Debuts Sifteo™ Cubes - A New Way To Play (PDF) 

"Sifteo cubes are 1.5 inch computers with full-color displays that sense their motion, sense each other, and wirelessly connect to your computer. You, your friends, and your family can play an ever-growing array of interactive games that get your brain and body engaged.

Sifteo’s initial collection of titles includes challenging games for adults, fun learning puzzles for kids, and games people can play together." -Sifteo website

For more information, see the Sifteo website,  blog, and YouTube  channel.  If you can't wait to get your own set,  take a look at Josh Blake's Sifteo Cube Unboxing Video!

RELATED

About two years ago, I was interviewed about my thoughts about the interactive, hands-on, programmable cubes, then called Siftables,  for an article published in IEEE's Computing Now magazine:  Siftables Offer New Interaction Mode  (James Figeuroa, Computing Now, 3/2009). 

For those of you who'd like more information about tangible user interfaces (TUIs) and  the development of Siftables, I've copied my 2009 post,   Tangible User Interfaces, Part I:  Siftables,  below:

TANGIBLE USER INTERFACES, PART I: SIFTABLES (2009)

In 1997, the vision of tangible user interfaces, also known as TUI's, was outlined by Hiroshi Ishii and Brygg Ullmer of the Tangible Media Group at MIT, in their paper, "Tangible Bits: Towards Seamless Interfaces between People, Bits, and Atoms" (pdf).   According to this vision, "the goal of Tangible Bits is to bridge the gaps between both cyberspace and the physical environment, as well as the foreground and background of human activities." This article is is a must-read for anyone interested in "new" interactive technologies.

The pictures in the article of the metaDesk, transBoard, activeLENS, and ambientRoom, along with the references, are worth a look, for those interested in this seminal work.

Another must-read is Hiroshi Ishii's 2008 article, Tangible Bits: Beyond Pixels (pdf). In this article, Ishii provides a good overview of TUI concepts as well as the contributions of his lab to the field since the first paper was written.

Related to Tangible User Interface research is the work of the Fluid Interfaces Group at MIT. The Fluid Interfaces Group was formerly known as the Ambient Intelligence Group, and many of the group's projects incorporate concepts related to TUI and ambient intelligence. 


According to the Fluid Interfaces website, the goal of this research group is to "radically rethink the human-machine interactive experience. By designing interfaces that are more immersive, more intelligent, and more interactive we are changing the human-machine relationship and creating systems that are more responsive to people's needs and actions, and that become true "accessories" for expanding our minds."

The Siftables project is an example of how TUI and fluid interface (FI) interaction can be combined. Siftables is the work of David Merrill and Pattie Maes, in collaboration with Jeevan Kalanithi, and was brought to popular attention through David Merrill's recent TED talk:

David Merrill's TED Talk: Siftables - Making the digital physical -Grasp Information Physically

"Siftables aims to enable people to interact with information and media in physical, natural ways that approach interactions with physical objects in our everyday lives. As an interaction platform, Siftables applies technology and methodology from wireless sensor networks to tangible user interfaces. Siftables are independent, compact devices with sensing, graphical display, and wireless communication capabilities. They can be physically manipulated as a group to interact with digital information and media. Siftables can be used to implement any number of gestural interaction languages and HCI applications....Siftables can sense their neighbors, allowing applications to utilize topological arrangement..No special sensing surface or cameras are needed."





Siftables Music Sequencer from Jeevan Kalanithi on Vimeo.

More about Siftables:
Rethinking display technology (Scott Kirsner, Boston Globe, 7/27/08)
TED: Siftable Computing Makes Data Physical
Siftables: Toward Sensor Network User Interfaces (pdf)

It seems that people really like the Siftable concept, or they don't see the point. I found the following humerous critique of Siftables on YouTube:

"Imagine if all the little programs you had on your iphone were little separate chicklets in your pocket.
You'd lose em.
Your cat would eat em.
You'd vacuum them up.
They'd fall down in the sofa.
They'd be all over the car floor.
You'd throw them away by mistake..."

In my opinion, it is exciting to learn that perhaps some of this technology has the potential of becoming main-stream.

Short documentary of the story behind the Reactable, a tangible user interface for creating music. (Includes an interview of Joel Bonasera, of Charlotte's Discovery Place museum.)

The following video provides a look into the history of the Reactable,  from the initial paper prototypes to the present, including the Reactable Mobile application designed for the iPad, iPhone, and iPod touch.   The video includes interviews of Sergi Jorda and Gunter Geiger, members of the original team at Pompeu Fabra University (Barcelona) that created the Reactable.  The other team members are Martin Kaltenbrunner and Marcos Alonso.




FYI:  At about 2:34 in the video, Joel Bonasera briefly discusses the Reactable installation at Charlotte's Discovery Place museum. Joel is a project manager at Discovery Place.


RELATED
How the Reactable Works
John Fuller, howstuffworks
Music Technology Group, Pompeu Fabra University
Reactable Website
Reactable Concepts
Reactable History
Discovery Place
Interactive Technology in the Carolinas: Discovery Place Science Center
(Includes a short video clip I took of the Reactable at Discovery Place)

Human-Machine-Music Interaction: KarmetiK Machine Orchestra (Video, links)

Here is an example of innovative interaction between humans, machines, and music:


KarmetiK Machine Orchestra - Live at REDCAT Walt Disney Hall - Los Angeles - Jan 27, 2010 from KarmetiK on Vimeo.


Information from the KarmetiK Machine Orchestra Vimeo page:
"On January 27th, 2010, KarmetiK and California Institute of the Arts brought together a group of interdisciplinary artists to perform in a revolutionary production. During this performance, The Machine Orchestra, a collective of musicians, engineers, dancers, and theatre designers, gave an audience at the Walt Disney Concert Hall's REDCAT performance space a glimpse of the future: one in which computers, robots, and humans join forces to make music.Featuring a cast of musicians, new musical interfaces, and musical robotics, The Machine Orchestra fused a wide array of musical styles ranging from free electronic improvisation to world dance music.This DVD features uninterrupted footage of The Machine Orchestra's debut concert, a performance exploring human interaction with KarmetiK's collection of musical robots: MahaDeviBot, GanaPatiBot, Tammy, Raina, and ReyongBot. Directed by Ajay Kapur and Michael Darling."
Music Director, Co-Creator: Ajay Kapur
Production Director, Co-Creator: Michael Darling
Guest Electronic Artists: Curtis Bahn & Perry Cook
World Music Performers: Ustad Aashish Khan, Pak Djoko Walujo, & I Nyoman Wenten
Multimedia Performer-Composers: Charlie Burgin, Dimitri Diakopoulos, Jordan Hochenbaum, Jim Murphy, Owen Vallis, Meason Wiley, and Tyler Yamin
Visual Design: Jeremiah Thies
Dance: Raakhi Sinha & Kieran Heralall
Lighting Design: Tiffany Williams
Sound Design: John Baffa
Production: Lauren Pratt
Editing: Meason Wiley
Filming: Benny Schuetze 
machineorchestra.com
Follow KarmetiK on Facebook and Twitter: 
facebook.com/​karmetik
twitter.com/​karmetik


Detailed information about this performance and Machine Orchestra:

Human-computer music performance uses system that links music and musical gestures

Lisa Zyga, Physorg.com 

MACHINE ORCHESTRA
KarmetiK Machine Orchestra

RELATED
Building a Hybrid Man/Machine Orchestra, Pt 1
Jordan Hochenbaum, Create Digital Music 1/25/10

Hybrid Man/Machine Orchestra:  Interfaces, Interaction and Keeping it Together

Jordan Hochenbaum, Create Digital Music 4/22/10

Direct links to the publications listed below, and more, on the  Publications: Refereed Journals and Conference Papers page of the Karmetik website.


Kapur, A. & M. Darling A Pedagogical Paradigm for Musical Robotics, Proceedings of the
International Conference on New Interfaces for Musical Expression, Sydney, Australia, June 2010.


Hochenbaum, J., Kapur, A., & M. Wright, Multimodal Musician Recognition, Proceedings of the International Conference on New Interfaces for Musical Expression, Sydney, Australia, June 2010


Vallis, O., Hochenbaum, J,, & A. Kapur, A Shift Towards Iterative and Open-Source Design for Musical Interfaces, Proceedings of the International Conference on New Interfaces for Musical Expression, Sydney, Australia, June 2010


Hochenbaum, J., Vallis, O., Diakopoulos, D., Murphy, J. & A. Kapur, On Designing Expressive Musical Interfaces for TableTop Surfaces , Proceedings of the International Conference on New Interfaces for Musical Expression, Sydney, Australia, June 2010


Murphy, J., Kapur, A., & C. Burgin, The Helio: A Study of Membrane Potentiometers and Long Force Sensing Resistors for Musical Interfaces, Proceedings of the International Conference on New Interfaces for Musical Expression, Sydney, Australia, June 2010

Juggling and Music: JAM meets the ReacTable

Enjoy!
Juggling and Music Meets the ReacTable, With Carles Lopez


I could play around with a ReacTable all day long!

RELATED
Need an 8 and 1/2 minute dance/exercise break?  Get up out of your chair and dance to this  video of the Brainwater ReacTable Live Performance 1

Reactable Mobile - Music Creation and DJ-ing On-the-Go : This is why I need an iPad!

A couple of weeks ago I posted about the Reactible Mobile,  a fun "DJ" electronic music generator for the iPod and iPad, created by Reactable Systems, makers of the Reactable.  The Reactable is a multi-touch, multi-user system for music creation, sharing, and DJ-ing, but it comes in the form of a table. Even though one version of the Reactable, the Reactable Live! is portable, it is still pretty big.

In our increasingly mobile world, it's nice to know that now we can have something similar to take everywhere we go, with the Reactable Mobile that runs on the iPhone, iPod Touch, and the iPad. You can download and buy the Reactable Mobile app for the iPhone and iPad from the iTunes Store:  http://itunes.apple.com/app/reactable-mobile/id381127666?mt=8

The following videos provide a quick overview of how the Reactable app runs on mobile devices. The first video is a demonstration of "Verde", Le Freak Selector. The second video demonstrates the Reactable app on the iPhone, iPod touch, and iPad.





Reactable Mobile (Reactable Systems SL) Description  (Taken from the iTunes website)

"Improvise and create music playfully with the Reactable mobile. Based on the award winning Reactable as used by Björk on her Volta Tour, this application brings the full creative power of the famous interactive instrument onto your mobile device."

"It uses concepts of modular synthesis, sampling, digital audio effects, DJing, and combines them with modern human computer interaction and multi-touch technology.Based on the very same audio and graphics engine as the Reactable, the mobile version brings a complete range of objects to multi-touch devices:"
A set of generator objects:
- loop players, with possibility to upload your own loops,
- synthesizers, with a large range of instruments to select from,
- oscillators, to synthesise pure and complex tones,
- input, to get audio directly from the device's microphone.
A set of effects to modify generated sounds:
- wave shapers: distortion, compression, and resampling,
- delays: reverb, feedback, and ping-pong,
- modulators: ring modulation, chorus, and flanger,
- filters: low pass, high pass, and band pass.
A set of controller objects to modify other objects parameters:
- sequencers, with step-by-step, matrix, or random modes,
- low frequency oscillators (LFO), with different waveforms,
- accelerometer, to fetch data from the movement of the device.
A set of global objects to modify the settings of the entire table:
- tempo, to change the speed of the table,
- volume, to lower or increase the loudness,
- tonalizer, to change the harmony of the melody.
"IMPORTANT NOTE: A graphic resolution issue with iPod Touch 4G prevents using the version 1.0.3 on this device. An update is on its way.  Note: as this application uses both graphic and audio resources extensively, recent devices will provide the best user experience."

RELATED
There is a Reactable in an exhibit at the Discovery Science Center in Charlotte, N.C.  I had a chance to play with it during my last visit, and it was... awesome.   For more information, including pictures and video,  take a look at this post:
Reactable Live! at Sonar Barcelona 2010 (You can play with one at Discovery Place in Charlotte, NC.)

Here is a very short video-clip of the Reactable at Discovery Place. (Because of the noise in the background, it is a bit difficult to hear what I was trying to play.)


If you are interested in tangible user interfaces, including the Reactable and other similar systems, take a look my previous posts on the topic:
Tangible User Interfaces Part 11: More Examples, Resources, and Use for TUI's in Education
Tangiblie User Interfaces, Part I: Siftables

Interesting blog roll and links on Christian Zoellner's TUI Blog by FORM+ZWECK; FILE Electronic Arts Festival

Christian Zoellner blogs about "tangible interaction & new interfaces" on the TUI Blog by Form + Zweck, a design magazine.  Christian is a designer and "presearcher" who lives and works in Berlin, Germany.  He teaches at the University of Fine Arts, Berlin. He maintains the Christian Zoellner website.

As I write this post, Christian is attending the FILE electronic arts festival, which focuses on "interactive art in public spaces, game design, and sonification".  It sounds like a fantastic conference! The festival is actually a group of events:  File Machinima, File Documenta, File Media Art, File Hypersonica, File Games, File Symposium, File Prix Lux, and workshops.


Christian Zoellner's Links:

• algorhythmics
• alt N research
• arduino
• bausteln
• cc
• christian zoellner
• createdigitalmotion.com
• createdigitalmusic.com
• de:bug
• doors of perception
• form+zweck
• fritzing
• groovemechanic
• hardhack
• interactiondesign.org
• interactivemultimediatechnology
• intuity.medialab
• M.I.T
• maker faire
• my future me
• nui-group
• openprocessing
• prado media lab
• processing
• salection
• smiles at strangers
• TEI conference
• the kitchen
• tinker
• tui.blog @ youtube

(I just noticed I'm on this list of links.)

RELATED

FILE SAO PAULO 2010:  July 27-August 29

"This year, besides the Centro Cultural FIESP - Ruth Cardoso programme, FILE launches FILE PAI (Paulista Avenue Interactive = Interactive Public Art), a project of digital public art that will occupy several spaces at Paulista Avenue with interactive works of art.

0 - Interactive Projection, 1 - Brigadeiro subway station, 2 - Electronic sound bus, 3 - Nomadic electronic graffiti, 4 - Paraíso subway station, 5 - Fnac store, 6 - FIESP Cultural Center - Ruth Cardoso, 7 - Trianon-Masp subway station, 8 - Conjunto Nacional building, 9 - Consolação subway station, 10 - Cervantes Institute, and 11 - São Paulo Art Museum - MASP compose the FILE PAI set, which intends to highlight the significance of interactive public art in order to understand and to absorb the new social phenomena provided by technology and, thus, to constitute strategies to interconnect with those new mass behaviors."

SOMEWHAT RELATED
Building Music at Spark Festival 09 (opens up to a full screen video of a musical building,from  Play the Magic.