Team Members

Miles Yucht

David Dohan

Shubhro Saha

Andrew Cheong

Brainstorm

1. Digital flute powered by light sensors for people who have limited lung capacity but still would like to learn to play. Varying aperture can modulate tone volume, and it could also could teach you to play interactively.
2. For young, urban professionals who don’t carry mice with their laptops, one could have a credit-card format mouse that is thin enough to fit in your wallet.
3. I want a computer I can wear around my neck and interact with by holding up my fingers, a la Sixth Sense, if you work in the field where it’s difficult to set up your laptop.
4. For people who can’t control a mouse with their hand, they could move a ball with their feet to control their cursor.
5. For more effective group meetings, a giant electronic collaborative whiteboard with physical interface such that everyone could edit it simultaneously. At one time, everyone would have the same view.
6. If you want a copy of notes from today’s lecture if you couldn’t make it, a device that records a teacher’s notes on the blackboard and processes them into a PDF which would be available right after lecture
7. If I’m paralyzed or shopping from home, I want to be able to try on virtual clothing to see what I would look like without having to actually put the clothes on my physical body.
8. If I have no fingers, I could still control my TV with Kinect gesture/voice
9. Learning to jump rope is hard. It would be easier with a jump rope that gives you feedback on what you need to adjust to become better, and it could teach you new tricks and save scores/records.
10. Instead of hiring a personal trainer, you could instead buy a device that would record you exercising and give you feedback, such as squatting or golf swing, to improve your technique and lower your chance for injury
11. Markov-based model for predictive typing to guess the next word you’re going to type in your phone/sentence so you can text faster, for those of us that are horrible spellers.
12. If you want to learn how to dance, but the DDR style doesn’t appeal to you, pads on the ground could light up, playing back a dance step tutorial, to teach you and perhaps a partner how to dance.
13. Learning to skateboard is hard, so my skateboard could detect foot placement to give feedback when learning to ride the skateboard.
14. I want my mirror to sympathize with me. By analyzing my face, my mirror should give me words of encouragement if I look like I’m feeling down in the morning.
15. Use gestures to control the multitude of lights in large rooms or in rooms where light controls are not easily accessible, for handicapped people or those interested in making dramatic entrances/exits.
16. For parents who want to introduce their young children to instruments, one could use a plant as a musical instrument by measuring flexing in the plant. This would require minimal technical skill and would also have the performer interact with nature.
17. For those college students that have a hard time waking up in the morning, a wake up alarm that won’t reset unless subject to the most violent conditions, like throwing it or slamming it
18. Use Xbox Kinect to give feedback on how to improve your posture if you have posture-related health issues.
19. For someone who can hold objects but has trouble typing, one could use physical gestures or general input device motion as passwords, as opposed to a typed text string.
20. For someone who has no motor control in their hands, a phone-like device could speed-dial numbers and interact with the user according to patterns of blowing air.
21. Device should detect butt location to infer how well someone is paying attention in an audience. More complicated: body language inference from camera at front of room (the inference step might even be doable with the same seat sensors as well)
22. Authenticate based on a laser key based on uniquely-shaped objects in a 2D/3D laser field. Stick your hand in there if you want
23. Create a sensor in bed that turns the lights off when there’s someone laying down… or two. Could trigger many possible actions such as arming house alarm etc
24. Devise a sensor in the bathroom that makes you aware of the number of bacteria on your hands as you wash them
25. Flush a toilet by blowing air into a sensor, reduces germs on contact
26. Enable computers to teach and/or read sign language, perhaps with XBox Kinect
27. Create a system that detects facial emotions so they can be used in focus groups to more conveniently collect data
28. Blowing air into a sensor to create a beatbox drumkit that people with disabilities can use
29. An algorithm can analyze keyboard typing sound patterns to infer what type of activity is being performed, use to evaluate student attention levels in lecture
30. For people who can’t speak loudly, voice-interaction systems should try to read their lips
31. Rubbing your pocket to change tracks on iPhone on a cold day
32. Utensils/containers that tell you if your food is too hot to eat… alternatively, containers that automatically heat up food that is too cold
33. For people who use the same computer over the course of the day/night, a program that takes into account ambient light and current display (maybe even type) to calculate the best values for brightness and other display parameters (gamma, contrast, etc.)
34. Music playlist that automatically changes to suit you as you change tasks
35. 3D manipulation of models and visualizations (think molecules / proteins) with Leap, this is a much more natural gesture
36. Use LEAP motion as an effective, cost-effective way to scan faces for authentication
37. Play Rock, Paper, Scissors with LEAP to provide companionship for children
38. Direct a virtual live orchestra using baton movements captured in LEAP. This can be used to train amateur conductors
39. Integrate LEAP into clothing to make convenient computer gestures right in front of your body
40. Control a quadricopter with tongue movements so disabled individuals can go beyond joystick interaction
41. Violin that lights up on the frets to teach novices how to play songs
42. Computer in backpack with projector on chest to make a virtual piece of paper you can write on with a stylus (convenient, mobile notetaking)
43. Reconstruct ping pong game based on sounds from microphone (triangulate landing and where it is hit)
44. Control quadcopter or another electronic device with LEAP motion… it’s a far more convenient and natural gesture than joystick
45. When you’re working out and don’t want to change your music player for fear of covering it with sweat or taking time off of exercise, your music player could measure your heart rate and the speed of the repetition of the exercise and generate a playlist of appropriate songs.
46. Use Xbox Kinect to obviate human labor in semaphore training
47. Billiards table that visually augments your game interaction, suggests ball movements to make the game easier for novices
48. Control quadricopter by measuring movements in a 3D point cloud with an accelerometer… much more natural gesture than joystick systems.
49. Eye-tracking system will move a vehicle (quadricopter) to the desired location being looked… for people with limited limb movement
50. Teach children motor skills with a colored grid on the floor where they can play Simon Says with their feet

Sketches During Brainstorm

Project Choice Justification

LEAP-based Authentication

Dance Dance

Games are generally really fun to play but often have no real-life applicability, such as the Guitar Hero franchise, whereas some tasks in real life can be somewhat droll to learn. Enter the digital dance floor, which could teach you to dance by lighting up tiles for you to step on in time to music. Here, the idea is that you would stand on a dance floor, which is composed of a set of transparent, square tiles. Each tile would be controlled by a single light source, the set of which would be managed by a computational device. By storing and replaying a pattern of lights over time, one could effectively recreate the steps of many dances. Then, using force-sensitive resistors underneath each tile, the accuracy and timing of one’s responding dance steps could be measured and quantified into a score, which would then be recorded at the end of the game. Additionally, multiple panel colors could allow for more than one player to participate in the game at once. This idea is also extendable, as there are many other modes for operation one could conceive of: for instance, you could have the lights respond to pressure, creating a dance floor that tracks how people move along it and lights up squares beneath people on the dance floor, or you could play a full-body version of Simon Says.

Detailed Description

Problem Description & Context. Reliable user authentication has been a perennial problem in human-computer interaction. How can a system verify that the user is who he/she claims to be? The prevailing solution varies on and off the computer screen. Inside the web browser, username/passwords systems ensure that the desired user is the only individual who knows the correct combination of inputs. Outside the computer, locks, keys, and RFID cards dominate the physical world to open doors and grant physical access. These solutions are not without challenges of their own. For example, what happens when a user forgets his/her password? The password recovery process is prone to hacking by email and phishing attempts. In the real world, physical keys and cards are liable to misplacement. We’ve all had a time we lost a key during our day-to-day bustle. Finally, an underserved segment of our population are those disabled individuals who cannot easily use existing forms of authentication. Consider individuals who have difficulty typing– username/passwords are a nuisance. Similar challenges are presented to individuals who have difficulty with traditional locks and keys. Our overall goal is simple, fast, reliable, user authentication.

Target User Group. Our target user group boils down to two types:

1. Disabled Individuals– For reasons related to limited finger movement or arm motion, these individuals experience difficulty using locks/keys or username/password typed into web sites. They desire access to their favorite web sites and physical rooms behind locked doors.
2. Public Computer Users– Institutions like a university are full of public computers that require username/password authentication. The time spent authenticating by keyboard could be better spent serving another user, thus reducing the overall demand for computing resources over time. These institutional users desire speed– whether its university students trying to print a paper before class or a business professional trying to get a meeting started as quickly as possible.

Technology. Leap is a sensor that is capable of detecting 3D motion for 3D objects. While photos and videos are a 2D mapping of the 3D world, Leap is able to capture the full scope of 3D reality. This benefit allows our idea to become more viable. In the case for facial detection, the Kinect or a photo app would not suffice because authentication can be thwarted by simply placing a picture of a certain individual in front of the sensor. However, this wouldn’t be a problem for the Leap since the 3D would take into consideration the depth of the image. For a 3D object, due to the sensitivity and precision of a Leap, other systems may not be able to detect the subtleties of our authentication object.

Sketches.