Your Palm as Remote Controller (Video and Links)

Traditional remote controls for televisions and home media centers are particularly frustrating to use, in my opinion.  There are too many buttons, the buttons are too small, and it is easy to mess it all up.  (I've touched on this topic in a number of blog posts, such as UX of ITV: The User Experience and Interactive TV (or Let's Stamp Out Bad Remote Controls) and others, linked at the end of this post.)


There is hope for the future!  I'm happy to share a video about a method of remote control that might make interacting with my TV less annoying...and possibly pleasant! 


Researchers at the Technical University of Darmstadt have been working on a way for people to control their TV without a remote or hand-held device.  I think they are on the right path. The video shows how UI elements, mapped to the hand, can be touched, with accuracy, to interact eyes-free with a large-screen TV.  This work won the Best Paper award at EuroITV'12.


For more details, see Leveraging the Hand Surface as and Eyes-free TV Remote Control and the references below:
Niloofar Dezfuli, Mohammadreza Khalilbeigi, Jochen Huber, Florian Müller, and Max Mühlhäuser. 2012. PalmRC: Imaginary Palm-based Remote Control for Eyes-free Television Interaction. In Proceedings of the 10th International Conference on Interactive Television (EuroITV '12). ACM, New York, NY, USA, to appear. 


Niloofar Dezfuli, Mohammadreza Khalilbeigi, Jochen Huber, Florian Müller, and Max Mühlhäuser. 2012. Leveraging the palm surface as an eyes-free tv remote control. In Proceedings of the 2012 ACM annual conference extended abstracts on Human Factors in Computing Systems Extended Abstracts (CHI EA '12). ACM, New York, NY, USA, 2483-2488. http://doi.acm.org/10.1145/2223656.2223823

SOMEWHAT RELATED
Oh! No! Sony's "Mother of Remote Controls" for Google TV
Interactive Multimedia Across Platforms and Screens: Adobe's Open Screen Project; MEX Mobile User Experience Manifesto... (Please don't annoy the user!)
Usability of a Remote Control
Quick Link:  The Remote - Death by Voice Control?

Quick Link: The Remote- Death by Voice Control?

Voice Control, the End of the TV Remote? Samsung, LG, and others are racing to bring voice control to the TV Set 
Peter Burrows and Cliff Edwards, Bloomberg Businessweek, 12/7/11


"Most consumers’ first opportunity to talk to their TVs—and have them listen—will be through voice-enabled apps for their smartphone or tablet."

I look forward to seeing if/how Apple TV uses Siri as a remote control!

Picture from Amazon.com

RELATED

Remote Control Anarchy

Jakob Nielsen, Alertbox, 6/7/04  

(A great read, still holds true in 2011!)

Oh! No! Sony's "Mother of Remote Controls" for Google TV

Lynn Marentette, Interactive Multimedia Technology, 10/12/10

Sony, Sony, what have you done?

Harry Brignull, 90 percent of everything, 10/6/10

Bad hospital TV remote control:

Instructions for my last remote control:

Usability of Touchable-Moveable Multimedia Content. Part I (carousels and more)

I've been revisiting the topic of interaction design/usability for "touchable/moveable" multimedia content for the past couple of weeks.    I recently got an iPad2, so I'm in the "app" exploration phase -  a new world, since I never had an iPhone, iPod touch, or iPad "1".  


I plan to share  experiences, thoughts, opinions, and suggestions related to this topic during the upcoming months.   I welcome input from IMT readers!


Getting back to my iPad2....In addition to using the device to run apps, like most people, I also use it to visit websites.  Many sites now appear to be optimized for mobile devices, which is great for my Android-based smartphone, but for my iPad, not so much.   I thought by now we'd have more touchy-feely content to play with.


I've noticed that  basic features that work OK with touch interaction, such as the carousel, 
seem to miss the target, as discussed in the following article:



5 Big Usability Designers Make on Carousels
Anthony, UX Movement, 8/2/11


Although the above article was written from a "click" rather than "touch" perspective, many of the comments Anthony brings up ring true for websites and apps that are accessed through tablets and other touch enable screens.



I'm disappointed.  Here are a few of my grumbles:

• Many designers/developers have been slow to look at the increasing number of people who are accessing websites and applications via touch-enabled screens - of any size.   
• Many designers/developers have been slow to catch on that a website or app might be viewed/used by more than one person at a time.  Websites for house-hunting, home furnishings, vacation/travel, and education are a few examples that come to mind.
• Too many "multimedia" apps/websites are flat. Many still have a power-pointy feel.
• Websites optimized for mobile devices often leave important features out.
• Designers/developers sometimes don't seem to think about the various scenarios in which their applications/websites might be used - while balancing a baby on one hip, preparing a meal,  working out,  walking the dog, trotting around the mall with a friends/kids/spouse,  eating, waiting at a stop light, etc.

Too many iPad apps are flat and power-pointy. Too many apps don't take full advantage multi-touch features.  Very few apps allow for efficient interaction between two people on the screen.  




SOMEWHAT RELATED
Carousel Design Patterns   I will add more related links in the future - not limited to carousels : )
Carousel Interaction #21 (Mike Highfield's Rapid eLearning Blog
Carousel (Interaction Patterns Wiki)
Carousel (Welie)
Mobile Implementation of a Web 3D Carousel with Touch Input
Bjorkskog, C., Jacucci, G., Lorentin, B., Gamberinit, L.



Photo Credit:  Ohad

Quicklinks: Cute video about need for Google+, Spielburg on 3D, Tactile Pixels, Touch Screen Steering Wheel, and More

Here are a few interesting links  and a couple of videos.  Enjoy exploring!


Comic-Con 2011: Steven Spielberg Gives His Thoughts on 3D
Jason Barr, Collider, 7/22/11

Tactile Pixels Will Make It Easy To Read Braille on Touchscreens

Kwame Opam, Gizmodo, 7/9/11

Gestural Input on a Touch Screen Steering Wheel in the Media

Albrecht Schmidt, User Interface Engineering Blog, 7/17/11

Potential to improve some user experiences, using HTML 5
HCI 596 Course Blog, Iowa State University, 7/11/11


PBS Launches LearningMedia, a New Digital Repository for Educational Content
Audrey Watters, Hack Education, 6/27/11


Link to Microsoft Surface 2.0 SDK and Resources
Luis Cabrera, Surface Blog, 7/12/11


AI (Artificial Intelligence) Demonstrates Natural Learning, Applies New Skills To Civilization
Devin Coldewey, Tech Crunch, 7/13/11


21 Google+ Privacy Tips: the Ultimate Guide
Craid Kanalley, Huffington Post, 7/21/11


iPad K-12 Sales Outpace Mac Products
Ian Quillen, Education Week, 7/20/11


Wearable lab coat TV packs thousands of LEDs, heads for Burning Man
Zach Honig, Engaget, 7/13/11

(I'd like a job where I can do tech experiments, silly ones, too!)

Best Practices For Designing Websites for Kids - Quick Link

Best Practices For Designing Websites for Kids
David Morrison, Smashing Magazine, 7/6/11


Here is a short quote from the article:


"According to Jacob Nielsen’s research on teenagers, interactive website features (such as forums, mini-games, polls, ranking systems, competitions and 3-D interfaces) are valued by kids if they build a sense of community and foster participation. Bolting such features on will not likely prove effective, because kids will soon see the gaps and re-evaluate the website, despite any initial interest."

Quick link: "14 burdens placed on the user" by Jeff Sauro, Measuring Usability Blog

It is 2011 and usability issues continue to plague us.


I am sure most of encounter many of the usability annoyances outlined in the following article several times a week, if not several times a day!


14 burdens placed on the user
Jeff Sauro, Measuring Usability, 7/5/11


via Smashing Magazine

A few interesting tech links from Experienta/Putting People First and HCI 596 Blog

No time for reflective blogging today, so here are a few interesting links!


The future of the TV Experience 
(Article discusses Blink, a media industry magazine)


Designing Pleasurable Products and Interfaces Conference (DPPI 11)

Cambridge Consultants: Patients Want More User-Friendly Medical Devices
Levent Ozler, Dexinger, 6/3/11

Internet of things blurs the lines of bits and atoms
Katia Moskvitch, BBC News, 6/2/11


NESTA: Hot Topics: Digital You-Discussion about Telepresence

Hot Topics - Digital You from NESTA UK on Vimeo.


NESTA is the UK's National Endowment for Science, Technology, and the Arts


RELATED
The following links provide a wealth of resources related to emerging technologies and human-computer interaction:
Experientia - Putting People First blog
HCI  596 blog 
"This blog is for the HCI 596 course being taught at Iowa State University through it's Human-Computer Interaction program."
NESTA 

Keeping up with Facebook complexity

Need I say more?

Why bother switching from GUI to NUI? - Asked and answered by Josh Blake; My 2-cents; Stevie B’s Microsoft Research Video; Marco Silva’s NUI-HCI presentation -and links

In Chapter 1 of Natural User Interfaces in .NET,  Josh Blake asks and answers a question posed by many people who have been under the spell of keyboard input and GUI/ WIMP interaction: 


Why bother switching from GUI to NUI?  The answer?  Read Chapter 1 (pdf) of the book - the chapter is free.


Here are a few of my personal reasons:  
1.  I want to buy the next version of the iPad or something like it.
2.  I want to buy a new large-screen Internet HD TV.

3.  I want to buy a Kinect.

4.  I do NOT want to interact with my new TV with a Sony remote.  Too many tiny buttons!

5. I do NOT want to interact with my new TV with a keyboard,  because it reminds me of...work.

6.  Most importantly: 

I want to design apps for the people I care about, and others with similar needs:

    My mom.  

    My grandson.

Moms and dads with kids in tow.
People with special needs and/or health concerns, and the people who care and guide them.
Knowledge sharers and (life-long) learners....

RELATED

"Smart" Interactive Display Research

How Natural User Interfaces are changing Human Computer Interaction

 

View more presentations from Marco Silva

My YouTube Playlist:

Post-WIMP Explorers' Club Videos

"Natural user interfaces, gesture interaction, multi-touch, natural interaction, post WIMP examples and more... "

Interactive Multimedia Technology Post:
Interactive Touch-Screen Technology, Participatory Design, and "Getting It": Revisited

RELATED - and somewhat related   

Moms with Apps

Design for Care

SparkOn

NUI Group

UX Matters      "Why don't usability problems get fixed"

Encyclopedia:  Human Computer Interaction, Interaction Design, User Experience, Information Architecture, Usability and More (Interaction-Design.org)

Josh Blake's Blog: Deconstructing the NUI    Book: Chapter 1 (pdf)  Free!
Blake.NUI

"Blake.NUI is a collection of helpful controls, utilities, and samples useful for multi-touch and NUI development with WPF, Surface, and Silverlight."

OpenKinect Wiki

OpenKinect Community Founder Joshua Blake Comes Up With His Own Version of Win 7 Multitouch on Kinect   (Madhav K, KinectHacks 1/14/11)

Kinect SDK Could Open New Worlds for Motion-Controlled Interfaces (Mike Martin, TechNews World, 2/22/11)

Seth Sandler

Seth Sandler's Blog

Seth Sandler's  Ultimate List of Interactive Tables

Dan Saffer

Bill Buxton

Johnny Chung Lee's Procrastineering Blog
Adam Greenfield's Speedbird Blog

 (This is not an inclusive list.)


GUI to NUI Post-WIMP Manifesto:  TBA

Wii Just Dance2 and Kinect Dance Central: UI and Usability Approaches; Challenges for Developing Accessible Games (revised)

I love to dance- I studied dance through college, and off and on as an adult.   I have a DDR (Dance Dance Revolution) game-floor pad somewhere in my attic gathering dust.  I'm ready for new challenges.


I'm planning on buying a couple new dance games for the Wii and the Kinect. There is more to this story, given my interest off-the-desktop, post-WIMP HCI (human-computer interaction), interactive multimedia and games, and a career as a school psychologist dedicated to young people with disabilities, I'm excited to see where new technologies, interfaces, and interactions will take us.


So what do the wise men of usability have to say about new ways of interacting with games and other applications?


"Kinect has many great design elements that clearly show that the team (a) knows usability, (b) did user testing, and (c) had management support to prioritize usability improvements, even when they required extra development work." -Jakob Nielsen


Jakob Nielsen, one of the godfathers of usability,  shared a few words of wisdom about the Kinect in his 12/27/10 Alertbox post: Kinect Gestural UI: First Impressions.  Although he did not review Dance Central, he concludes that the game he reviewed, Kinect Adventures, was fun to play, despite usability problems.


If this is a topic that interests you, I recommend you read Neilsen's post, and also take a look at which are outlined in the post.  Also take a look at recent essay Neilsen co-authored with Don Norman, another godfather of usability: Gestural Interfaces: A Step Backwards In Usability

Why is this topic important to me?

I have been involved in the Games for Health and Game Accessibility movement for many years.  Lately I've been exploring the OpenKinect project with an aim to create ways of making movement-oriented games accessible for young people with more complex disabilities.  For example, there is a need to have dance and movement games modified for students (and adults!) who need wheelchairs or walkers.  There are students who have milder mobility challenges who love to dance, and the current games don't address their needs.  Some of my students have vision or hearing impairments, too.  They deserve a chance to play things designed for the Kinect.

"OpenKinect is an open community of people interested in making use of the amazing Xbox Kinect hardware with our PCs and other devices. We are working on free, open source libraries that will enable the Kinect to be used with Windows, Linux, and Mac."

Note:  I currently work as a school psychologist with students up to age 22. My main office is adjacent to a large OT and PT room at Wolfe, a program for students who have special needs.   We just had a large interactive whiteboard installed in the room that is begging for us to connect it with the school's Wii,  and soon (we hope), a Kinect.   If we are going to use dance games to help promote healthy activities among our special students, the games need to be accessible for students with cognitive, motor, and other limitations.

FIRST STEPS
Although I can dance, I understand what the world is like through the eyes of many of the young people I work with who have motor coordination and sensory integration problems that interfere with their ability to  move and dance, let alone access fast-paced dance games on the Wii or Kinect. 


My initial plan is to look at what the new dance games might be like from the view of someone who doesn't know how to dance, and admits that they have "two left feet" - an perhaps, no sense of rhythm.  Where would I start?


Wii's Just Dance2 seems to offer some support for learning how to dance through the use of simple movement icons, in the form of outlined figures, that provide information about how to move with the dancer on the screen. As you can see from the video below, the gamer is provided with information about upcoming moves throughout the game.


I decided to take a look at Just Dance2's  MIKA "Big Girl" (You Are Beautiful) because some of the adolescent females I work with have weight concerns that interfere with their health. During the teen years, this can become a vicious cycle, resulting  in less movement, and less participation with peers in physical activities, such as playing dance games.  If a teen has depression as part of this mix, we know that exercise can help,  and a fun dance game might be a life-saver, in more ways than one.

The screen shots below show how the movement icons are used in the game:

I thought it would be useful to learn more about the story behind the making of JustDance2.
At 2:22, Alexia, the project's usability expert, makes her presence known. From what I can tell, she focused on aspects of the game that would make it more usable for non-dancers, including those with "two left feet", to play the game.  (I don't know if there was anyone consulted about accessibility concerns for the game.)


Kinect Dance Central
Dance Central uses a different approach when it comes to "teaching" people how to dance along through the game. It would be interesting to test out Dance Central and JustDance 2 with the same set of people to get a better feel for what works and what doesn't.  Below is a video that previews, in split-screen, the interaction that takes place in Dance Central:

Dance Central Full Motion Preview


In Dance Central, gamers are provided with information about the moves through icons that cycle up the right hand side of the screen.  The level of dance-coordination to keep up with the moves is challenging at times, even for people who are OK at dancing.  Players can select dances according to level of difficulty. 


Kinect Usability with Regular People

Steve Cable (CX Partmers) shared his team's look at usability issues related to the Kinect by testing several games, including Dance Central, with groups of people in his article, "Designing for XBox Kinect - a usability study".  The quote below is from the Steve's article:

"We’ve loved playing with the Kinect. There’s no doubt that the game play is lots of fun. In-game menus are a barrier to that fun. Kinect should allow players to move through menus quickly and compensate for inaccuracy.

We felt the Kinect would benefit from some standardised global controls – much like a controller uses the A button to select and the B button to move backwards. We also think it needs a more responsive pause gesture – one that doesn’t interfere with the user’s game play.

Most of our participants found the Dance Central menu to be more effective, more efficient and more satisfying to use. Here are our recommendations for designing a Kinect menu interface:

1. Allow users to make selections through positive gestures, rather than timed positions
2. Place options on a single axis to make them easier and quicker to select
3. Allow users to control menus with the game pad if they prefer
4. Use large easy to read text
5. Don’t make users scroll through options unnecessarily – it takes too long
6. Users will be distracted if used in a social setting – test your menus in a social context to see if they are prone to errors
7. Avoid the cursor metaphor, it’s not what gamers are used to seeing in game menus, and makes it harder to implement alternative joypad controls"

Below are screen shots that provide examples of how the movement icons are displayed in Dance Central:

RELATED
Just Dance 2 Review: Get your body moving. No, really. Give it a shot.

Kexa MacDonald, 10/19/10

Dance Central Website

The awesomeness of the Dance Central UI

Istran, 123 Kinect, 11/15/10

CX Partners
OpenKinect 
Alertbox
Game Access SIG -IGDA  
Game Accessibility
Games for Health

Accessible Games for Health and K-12 Education: Lessons from the Classroom

Quora Question:  What are the basic principles of NUI-Natural User Interface Design?    Question Followers

If you haven't seen it yet, here is a video of Microsoft Surface 2.0. A new usability feature in this version- you can sit with your legs under the table!

If you haven't seen it yet, here is a video of Microsoft Surface 2.0

"Where am I in this information space?" Bifocal Display Concept Video, 1982, via the Interaction Design.org Encylopedia

The following video about the Bifocal Display concept is an interesting look at how a set of ideas unfolded in the early 1980's. These ideas took flight and influenced many of the design concepts we view as "new" today.


The video was posted on the on-line Interaction Design Encyclopedia, a free, open source multimedia resource that includes a range of chapters written by leading researchers and practitioners from fields related to Human-Computer Interaction (HCI), Interaction Design (IXD), User Experience (UX), Information Architecture (IA), Usability, and more.



"People want to see context...." -Robert Spence

1980's Paper Prototype of  Carousel Concept (screen shot from video)

"Let's apply this idea to my in-tray,  an information that is quite varied, and often full of surprises..." -Robert Spence


For more information, see the article by Robert Spence and Mark Apperley on the Interaction -Design.org website.  Scroll down to "The Bifocal Display Explained" for great sketches of the concepts discussed in the video.  


Also take a look at the recent videos from InteractionDesign.org related to this topic.  What a great way to gain insight about the design and development process!


Video 1: Introduction to the Bifocal Display
Video 2. Main guidelines and future directions
Video 3: How the Bifocal Display was invented and launched


REFERENCE
Spence, Robert and Apperley, Mark (2011). Encyclopedia entry on Bifocal Display. Retrieved 9 February 2011 from Interaction-Design.org: http://www.interaction-design.org/encyclopedia/bifocal_display.html


RESOURCES
Interaction-Design.org YouTube Channel
Interaction-Design.org

"Undercover User Experience" by Cennydd Bowles and James Box: Charlotte UX Book Club Skype Session and Discussion

I just got back from a meeting of the Charlotte UX Book Club.  The group meets in the loft office of the Charlotte branch of Atlanta-based Macquarium,  and provides a great opportunity for like-minded folks to socialize and toss around informed ideas.


Tonight's meeting featured Cennydd Bowles and James Box, the authors of Undercover User Experience.  They joined the group via Skype, on a large screen TV.  After a minute or so, it was as if they were right in the room, even though they were in the UK.  What a delightful experience!


Cennydd and James both work at Clearleft, a team of designers and creative technologies that provide user-experience consulting.  They are bright, insightful, and know their stuff.  The book packs in a lot of good information in an easy-to-read, concise, and thought provoking manner.  The author's address tools of the trade within the context of how work gets done across disciplines in organizations.  


In my opinion, this book would be useful to people working on traditional projects as well as those who focus NUI design & development for emerging technologies.


BTW: Cennyd Bowles will be the closing plenary speaker at the IA Summit 2011 in March.




"Deliverables are a step on the journey, not the end of the line" -Undercover UX Design





Here are a couple of quotes from the Amazon.com editorial reviews:

  Product Description

"Once you catch the user experience bug, the world changes. Doors open the wrong way, websites don't work, and companies don't seem to care. And while anyone can learn the UX remedies usability testing, personas, prototyping and so on unless your organization 'gets it', putting them into practice is trickier. Undercover User Experience is a pragmatic guide from the front lines, giving frank advice on making UX work in real companies with real problems. Readers will learn how to fit research, ideation, prototyping and testing into their daily workflow, and how to design good user experiences under the all-too-common constraints of time, budget and culture."

  From the Back Cover

"Once you catch the user experience bug, the world changes. Doors open the wrong way, websites don't work, and companies don't seem to care. And while anyone can learn the UX remedies: usability testing, personas, prototyping and so on unless your organization 'gets it', putting them into practice is tricky. Undercover User Experience is a pragmatic guide from the front lines, giving frank advice on making UX work in real companies with real problems. Readers will learn how to fit research, ideation, prototyping and testing into their daily workflow, and how to design good user experiences under the all-too-common constraints of time, budget and culture." 

RELATED

Cennydd Bowles on user experience: The making of Undercover UX Design

Book Review from Johnny Holland

Interactive Display with QR Tag: Close Encounter at the Orlando Airport

I always like to try out technology in public spaces when I travel.  Imagine my excitement when I was presented with my first opportunity to try out my tag app on my Incredible while I was waiting for my baggage to arrive after my flight from Charlotte to Orlando...

On first glance, the location of this display near the rest rooms and baggage claim area makes sense. Travelers can access information about things to do in North Port from their Smartphones by scanning the QR tag with a tag reader app while they wait for their baggage.

To my dismay, my interaction with this display and my tag reader app was not enjoyable. 

Because the tag was located at the bottom of the display, I had to set aside my carry-on bags, purse,etc., and kneel front of it to center my smartphone precisely at the tag.  The display was on the wall between the restrooms, a convenient gathering point for people. Little kids darted around me, and parents used this spot to re-organize their family's "stuff".  As I tried to scan the tag, a dad had to ask his son to move back, "so the lady could take a picture". 

I was blocking the path of families and groups of travelers with all of their stuff.  I gave up trying after a while, much to the relief of my husband, who watched all of this as he waited for our baggage.