Is the answer Voronoi? Looking for possible solutions to an art+dance+music+tech idea from a recurring dream....

If you are a long-time reader of this blog, you probably know that I sometimes have some unusual dreams about technology.  I don't blog about my dreams very often, but last night, I had another technology dream, a continuation of a dream I had one night last week.  
Voronoi Diagram (Wikipedia)

I'm pretty sure that the last two dreams were sparked by playing an online interactive demonstration of a Voronoi application before going to sleep one night, and also reading an article about "extracting ordered patterns from a triangular mesh surface" in the Novemeber/December IEEE Potentials magazine before turning in last night. 

The dance probably was influenced by my recent viewing of the North Carolina Dance Theater's performance of Innovative Works with my mom, someone who encouraged my  study of music, art, and dance at an early age.

Some of my tech dreams are sort of...practical. For example, in one recurring dream, I find myself coding for a flexible mesh/grid application. Sometimes the mesh/grid has something to do with wireless sensor networks on curved terrain, perhaps related to something like the Smart Grid, and sometimes I find myself working on an application that analyzes streaming data from a variety of sources, for security prediction purposes.  At other times, I'm coding for something more artistic, my preference.

Last night, my dream focused on creating a flexible mesh fabric that used in a multimedia dance/graphic arts/music performance.  I was coding for this performance using a Voronoi-like algorithm.


This is the best I can do to explain this: The fabric is carried by the dancers, and is both reactive and generative. In essence, the fabric is intertwined/embedded in the dance, the music, and the graphics. In my dream, everything looked/sounded/felt awesome and otherworldly, and the music that merged and morphed during the dance was so beautiful, not only the melodies, but the sounds.  (In a previous dream, the mesh contained a "nanotechnology" component, but I'll save that quest for the future.)


I thought I'd look at some of my web book marks and search a bit more for information related to this topic.  For now, here is the "brain dump".  I have more to add to this post, and plan to port it to a reference page for this blog in the future.  I hope that this post will be useful to some of my art/music/dance/tech readers!

RELATED AND SOMEWHAT RELATED


Update:  Right after uploaded this post, I came across a link to a WebGL demo for a 3D music video of pop singer Ellie Goulding's song, 'Lights', by HelloJoy.   Visitors to the webpage can click to interact with the environment. If you keep the button pressed, you fly faster.  If you tweet the link, you'll see your name crop up as you fly around in the soundspace.  For more information about the making of 'Lights' - take a look at  Behind the scenes of 'Lights": the latest WebGL sensation!  (Carlos Ulloa, 11/9/11)


After I watched the 'Lights' video, I recalled Radiohead's 'House of Cards' video, which I wrote about back in 2008: 
 "We were rolling computers all day"...The Making of Radioheads House of Cards using imaging and info visualization software.   The process behind the making of the House of Cards video was described in detail in Chapter 10 of the book, Beautiful Data.

Bradley, E., Capps, D., Luftig, J, & Stuart, J.M. Toward Stylistic Consonance in Human Movement Synthesis.(pdf)  The Open Artificial Intelligence Journal, 2010, 4, 1-19
Bradley, E., Stuart, J.  Using Chaos to Generate Variations on Movement Sequences (pdf) Chaos, 8:800-807 (1998)
Bradley, E., Stuart, J.  Learning the Grammar of Dance.(pdf)  Proceedings Fifteenth International Conference on Machine Learning, Madicson, WI, 1998
E. Bradley, D. Capps, and A. Rubin, "Can computers learn to dance?," Proceedings International Dance & Technology (IDAT), Tempe AZ, Feb 1999.
Chaotic Dance: Using mathematics to generate choreographic variations
Schedl, M., Hoglinger, C., Knees, P. Large-Scale Music Exploration in Hierarchically Organized Landscapes Using Prototypicality Information (pdf)
Fournel, N. Procedural Audio for Video Games: Are we there yet? (pdf) GDC 2010


Voronoi Cells, created by Nathan Nifong.  A version of this interactive work was used in a DanceDraw performance

Patterns in the Noise (Nathan Nifong's site - FYI, Nathan worked with Celine Latulipe with the Dance.Draw project while completing his bachelor degree in computer science at UNC-Charlotte)
Voronoi Dance (Christian Gross, using OpenFramework)
Voronoi art: Slow Trip (Oktalist/Mat)


The above video, by Mat/Oktal, was inspired by his viewing of Thomas Ruff's Substrat images. 
Scott Snibbe Studio (Intearctive art, music, and animation for iPhone, iPad, iPod, and Mac)

Interactive Voronoi Diagram Generator with WebGL (Alex Beutel)
 
The above video was found in Alex Beutel's blog post, "Interactive Voronoi Diagrams with WebGL"

Posts about DanceDraw and related work at UNC-Charlotte:
News from the HCI lab at UNC-Charlotte- Creative Interactions (Videos)
Exploring the Design Space in Technology-Augmented Dance at CHI 2010:  Celine Latulipe's team from UNC-Charlotte
Interactive Surveillance:  Live digital art installation by Annabel Manning and Celine Latulipe

SIGCHI  Digital Arts and Interaction Community:  Building Bridges
The Interdisciplinary World of Dance and Interactive Technology

HTML5Voronoi  (HTML5Code website)

HTML5 Voronoi, Live Version
Update to code to compute Voronoi diagrams (Raymond Hill, 5/22/11)
William Forsythe's "Synchronous Objects-One Flat Thing, Reproduced" - Multidisciplinary online interactive project: Translating choreography into new forms.
BCS HCI 2011 Workshop: When Words Fail:  What can Music Interaction tell us about HCI?
Woven Sound (Alex McLean)
Real DJs Code Live (Robert Andrews, Wired, 7/3/06)
Visualization of Live Code (Alex McLean)
Voronoi diagrams of music (pdf)  (Alex McLean, 2006)
WebGL
Sylvester: Vector and Matrix Math for JavaScript
Generative Art Links (Mikael Hvidtfeldt Christensen)
Schacher, J.C. Motion to Gesture to Sound:  Mapping For Interactive Dance (pdf) Proceedings of the 2010 Conference on New Interfaces for Musical Expression (NIME 2010), Sydney, Australia
Code & Form:  Computational Aesthetics (Marius Watz)
Werghi, N. Extracting ordered patterns from a triangular mesh surface.  IEEE Potentials, Nov/Dec 2011
Last night I dreamt about haptic touch-screen overlays


Hyun-Seok Kim's Voronoi Yacht

Visiting New (to me) Blogs: Learning WebGL

I'm interested in learning more about WebGL, and thought I'd share a link recommended by Rich White (of Edusim fame).


LEARNING WEBGL
Gile Thomas

Gile Thomas is co-creating a browser-based Python environment, PythonAnywhere, which might support WebGL in the (near?) future.










SOMEWHAT RELATED
I'd also like to share some exciting news. A group of Stevens Institute of Technology students are working on a year-long Senior Capstone project, a web-based conversation wheel/social skills game application.  The application will be accessed via interactive whiteboards (and other interfaces) for use with students at Wolfe School.  They will be using WebGL.


The application targets students with more significant disabilities, including those who have moderate to severe autism, and will help the students learn skills that will be useful on-the-job as they transition to adulthood.


Keep posted for updates about this project!  

Interactive Multimedia: Music, Videos and Ads -Choose your POV, Scenes, Timelines, and More -HTML5, JavaScript, WebGL

I thought I'd share some examples of interesting interactive multimedia sites on the web.  It seems that artists, musicians, and ad agency folks have been experimenting with tools such as HTML5, SVG, Canvas, and Web GL.  Some of this work is featured on Google's Chrome Experiments website, and other examples can be found on websites promoting Wrangler Jeans or Ikea furniture.   This sort of content is great on a larger display.

Take some time to watch the videos and explore the links below.  Enjoy!

ARCADE FIRE: "The Wilderness Downtown" - Interactive Video by Chris Black 

"Choreographed windows, interactive flocking, custom rendered maps, real-time compositing, procedural drawing, 3D canvas rendering... this Chrome Experiment has them all. "The Wilderness Downtown" is an interactive interpretation of Arcade Fire's song "We Used To Wait" and was built entirely with the latest open web technologies, including HTML5 video, audio, and canvas."

ROME:  "3 Dreams of Black", an Interactive Film by Chris Black (The link leads to the interactive site.)

WebGL Aquarium Site

WRANGLER

Blue Bell SS11 Collection Interactive Video

Wrangler: WE ARE ANIMALS GALLERY STUNTMAN (Interactive)

Wrangler: WORN ACROSS AMERICA (Interactive multimedia 

map - go to "Choose Scenes")

Wrangler Collection fw10(Interactive)

Demo of interaction on an iPad

Interactive Ad for IKEA: "A Better Sleep for Everyone"
The website for this amusing and creative ad campaign was created using HTML5. Here is a description of what you'll find if you follow the above link:


"IKEA is now launchig the Kokokaka produced A Better Sleep for Everyone campaign site, which features IKEA's bedding catalog.  6 different mattresses are shown by 6 Swedish artists, each interpreting a classic lullaby performed in a dreamy and surreal music video.  By scrolling up and down the user can change between the artist's music videos and the different mattresses.  Experience, for instance, a soulful Tingsek having problems falling asleep.  ust like the princess from the famous fairly tale he gets annoyed by something hard under the pile of mattresses.  But guess what? It's not a pea, it's Tingsek's band!  Let yourself fall asleep to beautifully performed lullabies!" 


Agency: Forsman and Bodenfors;  Film Production: Social Club;     Director: RBG6;   

Music: Music Super Circus;  Web Production: Kokokaka;   Photographer: Carl Nilsson

Below are videos of two of the lullabies featured in the interactive ad:





Here is a video from the interactive IKEA Come Into The Closet  website from a year or two ago:



"This is a campaign to promote IKEA's wardrobe solutions. IKEA wanted to show their huge range of styles and all the smart features on the inside. All the movements on the web site are controlled by sound and music. So change songs, upload your own music, play on your keyboard or sing into the microphone."


RELATED
Cacophony: An interactive video player in HTML5 and Javascript
"The basic elements of a Cacophony video are: An HTML5 Video on the base layer, a series of HTML5 Canvas layers above that, a timeline of effects to be triggered to the beat of the song, images and other elements to be used by the effects"