Group Name: CAKE

We are not a lie.

Team members:

Connie Wan (cwan), Angela Dai (adai), Kiran Vodrahalli (knv), Edward Zhang (edwardz)

Brainstorming List:

1. 3D modelling with stereoscopic output overlapping with gestural input space for natural user interaction (perhaps a Maze game where you’re inside a cube)
2. Enhanced music stand for rehearsal and concerts, with gestural/automatic page turning, automatically jumping to sections based on conductor’s decisions, and easy part markup
3. Natural navigation in virtual worlds/virtual reality using gestural input.
4. Solve the problem of limited screen real-estate by projecting a virtual desktop on every surface in the room that you can interact with using pen, hands, and gesture.
5. 3D data exploration with visor – you put the visor on your head and you see streams of data everywhere like you’re in the matrix. just kidding, you should be able to interact with graphs with your hands, move them around, set axes differently, see different kinds of graphs for the same data, see Google maps with 3D cities, manipulate molecular models etc.
6. Hands-free transportation for the lazy – Segway chair
7. Make a enhanced web browser with a head-mounted display: instead of hidden tabs with no relationships, you can drag different pages around in the space surrounding you and organize them into your own mini-web (using gesture, of course).
8. Modern laser tag – have computers know where “portals” are, have gun-like thingies that have cameras, and you “fire” into portals which come out an associated portal to hit someone
9. Screenless navigation assistance in new environments using an audio and tactile interface – e.g. vibrating elements in clothing or gloves that warn of obstacles, audio signals that get louder or softer as you face places of interest, etc. Useful for the blind
10. Piano Tutor – projection onto actual piano keys that can light up keys or recognize when keys are played, synchronized with page turning.
11. living room aware computing — pointing at “TV”, turns on, knows where you are: we would use a model room and put hardware in the walls, chairs, etc. Simulate “The Living Room of 2040” or something like that
12. Smart food storage: refrigerator: texts you if it shuts off, keeps updated shopping list, keeps track of when food goes bad, organizes meals and moves ingredients to front of fridge if you need to use them, talks to your computer to get search history– what food have you been searching/ looking at recently, it compiles a list of ingredients and sends them for you to get
13. Smart food cooking: toaster, stove, microwave: control from your phone or computer, or via speech–it cooks it the way you want it (updated over time); alerts you if something burns, can be remotely shut off (like say you leave the house and realize you left the stove on, you could shut it off with an app, maybe with phone)
14. Smart dishwasher: control remotely, automatically knows when to wash dishes, rotating water jets that sense dish shapes, make sure everything is hit. Could integrate it with your calendar
15. Kitchen sink that automatically or hands-free turns on and off depending on when it’s needed, precisely filling up containers.
16. Smart closet and smart clothes hangers: keeps everything sorted for you, automatically sets out clothing for the day based on weather (thicker clothing for cold days) and calendar (bathing suit for beach days), keeps your clothing updated online so friends can like clothes, and maybe even suggest fashionable combos for you to impress everyone! (this may sort of exist?) also could be helpful as a shopping app–that way, you can automatically know if you have a certain type of clothing so you don’t buy too much of the same type
17. Smart mirror that allows you to virtually try on clothes (already exists?) – large screen w/ camera, can zoom in on parts, bring up clothing images, etc. update to fb/twitter if you think you’re looking pretty good today wink wink
18. Enhanced clothing, e.g. integrated music player and cellphone with speakers and microphone in hood, control system in the sleeves (combine with those t-shirts that have displays in the front).
19. Smart desk in schools – automatically sense who you are, directly integrated with your student account at a university. Personalized instructions in class/lab
20. Augmented window with touch and display capabilities – use to display widgets like weather, news, etc. Automatically raises and lowers blinds
21. Smart lamp (could double as projector — display internet pages on your desk, could be part of a desk UI (tabletop),  doubles as a disco ball for room parties), automatically turns off when you get in bed
22. Smart gym: dumbells that keep track of your weight-lifting abilities, prompts you sarcastically (“do u evn lft”) (“friends don’t let friends skip leg day”), all the devices talk to each other, give you suggestions as to what you should do next for an ideal workout
23. Smart bike: knows directions and your schedule, “automatic”, i.e. changes gears
24. Smart backpack: knows what’s inside, is connected to calendar so knows your schedule, so it can alert you if you need a book/notebook that you need for a class, reminders to pick up packages, lets you know if your water bottle is running low, interact with it there’s an interface you can strap on, or on strap, plug in Android or iPhone to give backpack computing power and access to calendar stuff, pop out stuff you need immediately, organizational system that lets you put stuff in only one compartment of the backpack
25. E-readers that track your gaze and turn pages automatically
26. Smart bookshelf (like the one in Google NY, but better) that links to a site like shelfari or goodreads (maybe tie to online living room) and allows you to browse books that you might like; you can insert an e-reader or tablet where a book is “displayed” and it downloads that book for you to read
27. Voice and camera on computer (like Jarvis). Framework that connects with your whole house and provides an interface to turn off the lights, make toast, open your garage, anything you might have to walk to do, you can do it from your computer/voice
28. smart toilet cubicle, with a projected display and hands free navigation — no more magazines/books, no more phones dropping in toilet.
29. sassy oven– tell it what to bake, it talks back (maybe it tells you that you’re getting a bit chubby, hold off on the cookies) — the point is to have even more fun with baking and cooking in general. Good for lonely grandparents who don’t have anything else to do
30. Go-go-gadget arm – Quadrocopter that fetches faraway things for you.
31. Sunburn warning (hat that has a timer and knows what the weather is like in your area (temp sensor, maybe some sort of UV sensor based on smart glass, maybe knows your skin color/type), and calculates when you’d be exposed to too much sun, and notifies you 10 minutes before by beeping)
32. Device that senses if you are falling asleep in class and subtly wakes you up – in clothing, vibrates? or a device in ear that makes a small beep straight into your ear
33. Gestural Video Editor – view video as rectangular prism, make operations such as cuts/transitions/timing control more natural than with mouse. You can stretch out a square prism to slow down the video, compress it to speed it up, color different parts with different music, slice the prism and rearrange it for scene editing, etc.
34. Gestural Music Editor – can be either for notation/composition or for mixing/mastering. Use gesture contours to input notation with pitch, time and dynamics. Or make cuts/EQ/effects more natural with gestures instead of mouse+dialogs.
35. Ultimate student: always helps you be working on something (uber productivity) based on how long tasks will take – brings up appropriate tasks on computer, pop out homework/problem sets from your binder, etc. (might be better as a desktop application?) — have a printer built in, new form factor for “laptop” — portable desk instead of tablet. (maybe it folds up)
36. Advanced video conferencing – 3D images, room-sized conference room space, locational audio.
37. Whisper/subvocalization translator (like in Ender’s Game sequels) – can understand sound without you having to vocalize, for private conversations. Have a sensor on throat or tongue that detects vibrations and translates to speech
38. Educational AI : speech, personality: teaches math to elementary school kids through interactive games, learns what sort of games work well for specific individuals on the fly, encourages people — could be through voice, games could be gesture/voice based
39. medical auto diagnosis: can tell what your heart rate is, cholesterol level, how “sick” you look–maybe you have the flu. tells you in advance if you’re catching a cold and tells you what to do to stop it, contacts doctor if detects something serious (would use computer vision to detect stuff, maybe other sensors as well)
40. Device that finds and removes split ends from hair, either using visual or physical processing of hair strands
41. Invisible mouse – instead of trackpad for laptops, just use your hand as a mouse (activated when you put your hand on the table).
42. make specialized pen better (write on any old paper, updates stuff in the cloud (write subject on top, it knows what document, edits that for you) (Already exists)
43. Artist’s pen that can change ink color / thickness / type / etc. with small gestures/indications
44. 2D character rigging with a drawing-based interface or an action figure.
45. Tactile interface – deeply integrate vibrating actuators into laptop/mouse/clothing for another output modality (like in game controllers or in phone games). E.g. use it in photo editors (indicating alpha value of pixels) or in web browsing.
46. Forcing you to do exercise – treadmill-activated computer that only runs while you run
47. 3D game that requires actual running to move
48. desk that categorizes any object you put on it and lights it up in a different color and/or actually moves it to gather with other similar objects
49. mood lamps/screens/etc. that change color/brightness/temperature/etc. with the environment’s temperature/sound/brightness/general color/etc.
50. touchpad that can sense when finger hovers above, gives 3rd dimension special functionality (e.g. switch between stack of applications/windows/tabs)
51. origami tutor: camera finds where the paper is on the table and projects dotted lines where folds should go directly onto the paper, plus a ghost of how the paper should be folded next
52. actual (new or invisible) instrument embedded in clothing (e.g. invisible violin, or instrument that depends on how your limbs/body are positioned)
53. virtual interior decorator that sees what is in a room and projects things/colors from a store’s inventory that would go with them onto walls/furniture/etc
54. Projector that can project onto any surface and still look flat
55. Virtual tape measure based on hands for distance or start/end points
56. Camera/projector combo that lets you highlight portions of a book with a finger, then can display your highlights by figuring out what page you’re on
57. Building blocks for small children with letters on them that say any word that they are stacked into
58. Taste-tester (can be used in cooking) – tells you how much seasoning to add, or whether or not you might enjoy something based on how similar it is to known foods. Does it really taste like chicken? http://www.digikey.com/us/en/techzone/sensors/resources/articles/The-Five-Senses-of-SensorsTaste.html
59. Roomba++ – knows when to do vacuuming, as well as mopping/sweeping. Also cleans up counters, and intelligently puts objects on the floor into appropriate places.
60. actual shadow boxing game: your shadows fight with each other (with tactile feedback)
61. smart socks: text with people in class without anyone noticing (use flex sensors and do a morse code style communication — could be marketed for elementary school students, teach students morse code!)
62. completely virtual board game playing and tutoring like chess (gesture/ 3d display based)
63. object recognition chess (camera identifies when you move a physical piece, updates internet board in real time)

Sketches

Segway Chair, Auto-flipping Music Stand, Exercise-powered Computer, Origami Teacher, Wearable electronics, Building Blocks that spell out words to toddlers, Gestural Internet Browser (see windows in 3D and move them around!), Invisible Instrument, Go-go-gadget Arm [with Quadrocopter], Eye-tracking e-reader, Toilet Cubicle Entertainment System

Project Description:  

The manipulation environment consists of a 3D stereoscopic monitor that makes objects “pop-out” and look like they are floating in mid air. A gesture camera (the Leap) has its input space aligned with the output space. We also use a webcam to track the primary user’s viewpoint so that users can move their heads and see different views of a scene. All in all, users are able to translate objects by pinching them and moving them directly to where they want to go, to scale objects by stretching our hands apart, to select objects by touching them, and so on.

Why we chose our idea:

For starters, the concept is super cool! We’ve all been inspired by this picture:

Tony Stark using gestures!

and since we have some pretty awesome hardware already (Leap, Stereoscopic Monitor, and a Kinect), we think it would be feasible to implement a neat prototype of a 3D gesture interface which interacts with virtual, floating 3D objects.

Also, our idea is very modular: it’s possible to create more specific subsets of this project by building more functionality on top of the core structure. Specifically, it might make sense to add speech recognition and processing later on (if we have time) to improve that naturalness of the user interface.

Furthermore, it’s just a really cool user interface that hasn’t been properly built yet! We envision that in the future, computers will move away from the screen paradigm and become more immersive.

Target User Group(s):  Artists, architects, designers, and animators who do 3D modelling, as well as scientists who manipulate 3D data, such as researchers in 3D capture who have to deal with point clouds or geneticists who manipulate and fold protein models.

Problem Description and Context: Current 3D modelling applications, such as Blender and Maya, though powerful, are unintuitive and awkward for navigation, relying on the 2D monitor and 2 degree-of-freedom mouse to represent and manipulate 3D objects. Because of these restrictions, it is very difficult to select arbitrary points in 3D space and perform rotations. By displaying objects stereoscopically, we can make them look like they are holograms, floating in the real world. Users could then interact with these objects as if they were real (as with Iron Man’s Jarvis), to make full use of our physical intuition. Although experimental gestural interfaces for 3D modelling exist, they have been shown in academic studies to be less efficient than mouse interfaces. By aligning the gestural input space with the stereoscopic output space, we make a fully natural interface for 3D modeling.

Justification of Technology Platform: The fundamental parts of our system will involve a stereoscopic display and a gesture camera. We will most likely use a standard stereoscopic monitor for output, although we are considering using a head-mounted display as well. For a gesture camera, we intend to use the Leap because of its low data latency. We also need an additional webcam to perform viewpoint tracking. These input and output technologies enable the fundamentally novel part of our system, which is colocated input and output spaces. We would also like to incorporate tactile feedback, namely a simple glove with vibrating actuators, to enhance the visual feedback provided by the 3D display (which for example vibrate when your hands intersect an object). These all add to the illusion that 3D objects are in fact “real” and thus are uniquely suited to our application of manipulating 3D objects.

Design Sketch:

A User manipulates a virtual cube in front of a screen! A Leap motion tracks the users hands, and the depth camera helps out.

Group Members

Colleen Carroll (cecarrol@)
Gabriel Chen (gcthree@)
Prakhar Agarwal (pagarwal@)

I. Brainstorm

1. Cell phone alarm clock extension, through maybe the speaker port or MicroUSB port, that uses muscle pulses to wake you up.
2. Kinect based alarm clock that will ring if it doesn’t recognize that a body has moved out of the bed
3. Alarm clock that vibrates the entire bed when it goes off (vibrations can be achieved by putting something like an eraser off center on a motor).
4. Proximity alarm that tests for human presence using a combination of different sensors (temperature, photo, visual shape?). This can be used as a homemade security system.

5. An AI “RCA” that complains if the noise levels in a room are too high by using volume sensors.
6. “RCA” system that uses a combination of temperature, noise, and light sensors (rave lighting?) to sense whether a party is going on in the room and complains.

7. FootMouse

7. *** Foot pedal based mouse so that one can use the keyboard and mouse simultaneously
8. Facial movement and gesture recognition system that lets you control different operations in a computer interface by actions such as winking or moving your mouth in different ways.

9. Automatically adjusting backpack straps with a force sensor to determine whether the backpack is being used and adjust to allow for best back support.
10. Backpack straps that open up to allow a user to get arms in easily when picked up and then go back to the original setting after the backpack is on.

11. There’s already hand-based controls for lights (snap your fingers to turn on/off the lights); what about blowing out lights in the same way you blow out candles?
12. Control the lights in a room by covering or uncovering a photo sensor to a certain extent. This would be intuitive and easy to do!

13. MatchMirror

13. A mirror that checks the coordination of your outfit to your skin color, body type and other parts of the outfit.
14. Use the kinect to help you cut your own hair. It shows the user how to cut their hair to a particular style and uses gesture recognition to correct the user as they go.
15. Use the Kinect to recognize your face shape and recommend hairstyles that are the most flattering.

17. SignGlove

18. RepGlove

16. *** “Secret handshake” glove that unlocks bikes, doors, laptops that can be programmed with certain gestures
17. *** Sign language glove that recognizes what is being signed by a person wearing it in order to transcribe it onto the computer.
18. *** Fingerless glove for lifting weights that records how many reps you do to an interface that records your workout progress and whether or not you are meeting your goals.
19. *** A sensor on your clothes (flex, etc.) that recognizes good form in exercise moves and stretches (push ups, planks, sit ups, running, swimming)
20. A glove that tells you which notes to play on the piano
21. A gesture-controllable glove to control household tasks (is the stove off? find your keys?).
22. Glove that recognizes gestures for different functions that may be useful while driving such as picking up the phone (move thumb to ear and pinky to mouth like a telephone).

23. Webcam attached to your face that, when you point or gesture to certain parts of a book, takes notes or snapshots of active areas.

24. SafeStove

24. A sensor on a dangerous object (stove, table saw) that detects the proximity of your finger and turns the object off when you’re too close.

25. An application that tells you which notes you’ve played and records them to a computer program that transcribes the music

26. HydrationBottle

27. DetoxFlask

26. Water bottle that keeps a log of how much water you intake during a day by sampling the weight of the water periodically. Lights up if at a given time, you haven’t had enough of the water.
27. Alcohol flask that keeps a log of how much alcohol you’ve had during a night by sampling the weight of the alcohol periodically. Estimates BAC and displays on flask.
28. A flex sensor on the ketchup bottle that lights up different amounts based on how much ketchup is in the bottle, the orientation of the bottle, and how hard you’re squeezing.

29. Pressure sensors on your bike brakes and handlebars that light up brake and turning signals (useful for drivers trying to avoid bikers).
30. Pressure sensors on your shoes that light up turning signals when you turn on your feet (useful for bikers trying to avoid pedestrians).

31. VolleyHelper

31. Accelerometer and pressure sensor on a volleyball that detects and lights up when you should hit a volleyball when it’s falling (useful for people trying to learn when to begin an approach to serve).
32. A device that detects your posture and helps you improve it using classical conditioning
33. A Kinect based application that helps you improve your presentation skills by looking at both physical cues (such as posture, nervous habits) and audio cues (number of “um”, “like”, etc)
34. Presentation tool to write or draw in the air and transfer to screen.

35. A system that can listen to users singing (often imperfectly) and use machine learning to determine and transcribe the notes they are actually trying to sing (or at least those that would musically make sense)

36. Using kinect, a waste bin that senses the waste item, classifies it as a particular type of trash and opens the appropriate hole for deposit

37. Pressure-sensing belt that tells you when to stop eating.

38. Application that recognizes human presence in a lecture hall and control the blinds to direct light towards locations where it won’t get in people’s eyes
39. Remote controlled vehicle for which the remote is based on one’s hand movements (point forward to move forward, rotate wrist to turn, etc)
40. A sensor that can detect the need for handicap facilities such as an automatic door by recognizing the image of a person on a wheelchair or crutches.

41. A better tape measure – use a fixed armspan as the base, put a sensor on each hand and use specific gestures.
42. 3d drawing like on a 2D tablet. Use a simple stylus that is easily detected and draw in the air to add the model to CAD.
43. Kinect in the hallway to direct and measure traffic during an emergency, and safely help people exit the building.
44. Vending machine where you point to the item that you want (using kinect).
45. Solve a puzzle on a gift box to make the gift opening experience more interactive! For example, a Christmas gift box that has a countdown on the box, with a new puzzle every day.
46. Stethoscope that lights up when there is a heartbeat and records and analyzes sound for listening to breathing.
47. A multi-head screwdriver that can look at the screw you are going to use and the correct size head lights up.
48. A tray that can measure the amount of food that you are getting to go and print a price tag. The tag can then be easily brought up to the cashier and paid for quickly.
49. A coffee mug that tells you when your drink is cool enough to drink.
50. Make passwords more secure by moving them from the keyboard to any chosen object where pressure sensor can sense a specific pattern to unlock the computer.
51. Attach an arduino to a dog’s collar that the dog can be trained to interact with (barks, licks, breath, light) and open the door or turn on a water dispenser.
52. A smart crib that can rock the crib when it senses too much noise, that alerts you when the baby does certain things, etc
53. Mount to skateboard that gives you points based on spins, acceleration, jumps (real life Tony Hawk).
54. Motivation for running – virtual reality chase game to keep you running while on treadmill and meet workout goal (think real life temple run).
55. Hands-free way of interacting with your phone using kinect to detect gestures that correspond to mechanical motion on the phone’s touch screen and accelerometer.

II. Idea Choices:

Kinect-based application:
Through the brainstorming process, we tried to come up with ideas by looking at inefficiencies we saw around us and by considering problems those that we or our friends recently had. The idea we have chosen is to develop a mechanical, hands-free means to control one’s phone and we actually came up with it after discussing how one of Gabe’s friends recently dropped and broke his phone as he tried to pick up a call while on a treadmill. With our incessant dependence on mobile communication, we thought that there was definitely room for a hands free means to control a phone (outside of Siri, which is pretty terrible at recognizing a lot of commands). Our proposed solution uses a Kinect device set on the dashboard of the treadmill. Motions of the user would be used to control a mechanical mount in which the phone could be set. This mount will consist of a mechanical “finger” that can be used to touch different parts of the screen, and ideally, we will be able to move the entire mount in order to control accelerometer related functions of the phone. Through this hybrid, mechanical solution, we can translate our own full body actions to actions on the phone without actually touching the device. One could imagine waving to pick up a phone call or pretending to bob one’s head to music to start iTunes.

55. OnTheRun

Our second best idea:
Our second idea came from realizing the limitations in existing password systems. Most are based on a string, comprised of basic, finite character sets. This puts users in the position of either memorizing a complicated password or risk being vulnerable to simple attacks. Users tend to opt for convenience and choose passwords that are susceptible even to simple, brute-force attacks. We imagine a system for “passwords” that is virtually infinite, by taking gestures as input. A glove, when hooked up to a computer, can sense the gesture of a user’s hand and a software program can memorize the sequence of gestures that the user creates and then check future inputs against it. Each gesture, or “character”, is a product of the user’s imagination and thus is much harder for a hacker to guess.

III. Project Description

Target User Group:
Our target user group for this project consists of those who, like Gabe’s friend, are very plugged into their electronic communication and media devices and want to use them while exercising. With cellphones becoming more and more advanced and commonplace, it is becoming necessary for many of us to interact with and respond to them on an immediate basis. At the same time, though, they are becoming increasingly expensive and fragile. As such, one of the needs of the target group is to ensure that their phones are safe and not in fear of falling and having a cracked screen. Young users especially, who are used to always being plugged in, also want to be able to perform simple tasks like taking calls, sending texts, and maybe changing the music playing while they are working out. A gesture based, hands free manner to interact with their devices is especially useful for these users because it is simple and would allow them perform a task without requiring the exact and careful motor skills that touching the screen does.

Problem Description and Context:
As stated earlier, we first thought of the problem when a friend of Gabe’s experienced it himself. Those who exercise, especially in the gym, often multitask while doing so, but risk getting distracted and damaging their phone or themselves. On treadmills in particular, users often need some external stimulation besides staring at the wall ahead. We want to design a solution that allows the user to interact with their phone in a safe, fun way. We believe that a major cause of difficulties with using phones on a treadmill is that phones have a relatively small interactive surface, which requires more attention to get right, especially while moving quickly. Our system would need to have simple interactions and a large interactive surface to resolve these issues. It would also need to function in some of the most commonly attempted activities while running, for example, talking on the phone and reading. The interface could then be safe enough for use while running, but also give the user the entertainment or productivity that they are looking for.

Technology platform:
We chose to use a hybrid platform of both Arduino and Kinect for our application. We felt that Kinect was the best form of detection for the touch-free interaction with a large interface and variety of simple interactions, which is key to the solution of the problem described above. Additionally, from lab, we have discovered that Arduino has sufficient means of recognizing analog input. We need to be able to process the signals detected by Kinect in some way; Arduino is an appropriate means of translating this into the mechanical interactions on a touch-screen.

Members

Keunwoo Peter Yu
Green Choi
Dylan Bowman
Tae Jun Ham

Ideas

1. Motion Based Coding environment  (Kinect or Leapmotion)
– Uses gestures to reach, move, and otherwise manipulate code.

2. Physical Therapy (Kinect)
– Shows correct form on screen and allows user to try and mimic it
– Records best match, progress in getting better, warnings to rest when getting worse

3. Coffee Order Counter from Line (Arduino, etc.)
– Have switches that can be accessed from line that allow multiple orders to be placed
– Could be used by coffee store customers (Starbucks, etc.)

4. Security system based on gestures (Kinect)
– Use a special gesture as the key to the system.

5. Outfit recommendation system based on weather (Arduino)
– Detect the current temperature, wind, etc. and make recommendations on the day’s outfit.
– Could be used to help senior citizens go through their daily dressing routines

6. Music recommendation based on facial expressions (Kinect or webcam)
– Choose music to play based on the facial expression the user makes

7. Refrigerator camera to recommend recipes based on contents of fridge
– Recognizes ingredients in fridge and suggests recipes that could be made from those

8. Refrigerator compartmentalized tracks number of accesses to warn against obesity (Arduino)
– Could even apply to non-refrigerator – just use it where you store the most unhealthy items in your kitchen

9. Door control based on pattern or face recognition, primarily for cats and dogs (Kinect)
– Guards against strays and dirty inferior creatures like raccoons and possums

10. Attachment to measure acceleration of punches or kicks. (Wii controler)
– Measure the power and speed of punches or kicks using the Wii controler for entertainment or training purposes.

11. A device  to keep you awake in class based on variable resistor.
– Requires constant change in resistance to not shock you.
– No change indicates falling asleep.

12. Presentation aide based on gestures. Kinect or accelerometer.
– Pointing causes laser on screen that observers can follow
– Swiping allows next slide or previous slide to come up

13. Presentation aide based on voice. (Arduino)
– “Next” or “Back” moves slides forward or back
– “2nd point” highlights second point on slide

14. Voice or gesture combination for pet doors or food dispensers
– Teach cat or dog certain combination of actions that allows pet-door to unlock or food to dispense

15. Garbage can attachement to detect fullness and make alarm (Arduino)
– Once garbage in the bin passes certain height, sounds an annoying alarm so that people actually empty the bin.

16. Webcam device that tells you optimal chiropractic chair height.
– Takes a picture with the webcam and see if the chair is at the optimal height.

17. Device to measure motion, lack of which will trigger alarm to keep you awake.
– Security Guards: Administers caffeine or stimulants.
– At Home: Alarm sounds.

18. LED flash light on your glasses that adjusts brightness according to environment’s light (Arduino)
– Button to turn function on and off.

19. Wireless motor to attach to existing switches using remote.
– Remote control lighting, etc. to existing switches.

20. Auto-spotlight (or fan) that tracks moving objects (Kinect)
– Pre-game functionality

21. Autofan for temperature changes or humidity changes.
– Sensor outside window tracks changes in temp/humiditiy/possibly others (precipitation)
– Changes fan speed/power in room to adjust

22. Sensor in shoes to measure how high you can jump.
– Measures the weight and the force the person exerts to calculate how high he can jump
– No need for a wall with a measurement tape.

23. Light inside bag turns on when a girl opens it and there’s no environmental light.

24. Toothbrush holder that lights up brighter and makes sound warnings if you don’t take it out for too long (like longer than a day or something). (Arduino)
– Light will get brighter, sounds get brighter and more frequent.

25. Toothbrush attachment with accelerometer that recognizes proper brushing technique and length (Arduino)
– Good for kids. Proper recognition prevents false brushing.

26. Toilet attachment that vibrates when the user has been sitting for too long. (Arduino)
-Time management for toilet gamers.

27. Bathroom hygiene circuit that makes sure faucet is turned on after flushing.
– If you flush and then don’t turn on faucet in next 30 seconds, starts buzzing annoyingly, or locks the door.

28. Security system that tracks incoming and outgoing people in a building.
– Ensures that number is even at end of day, or just # of people in building.

29. Laundry Security (Arduino, we picked this idea)
– Can tell if washer/dryer has been opened before it is done (people moving your laundry because nothing else is open)
– Possibly takes a video if it does, or buzzes alarmingly to warn them.

30. Vending machine add-on ensures product dispenses after purchase with motion sensor at the bottom.
– Prevents vending machine damage from people kicking it after stuck items.

31. RC car that follows you around with a flashlight in dark areas.
– Can help certain jobs like plumbers, etc. explore small compact areas like pipes or under houses
– Can possibly turn a different color if sensing hear or light to alert operator of its presence

32. Self-organizing chairs or other furniture objects.
– By remote positioning

33. Chair mat with sensors that tell you whether your posture is okay or not.
– If uneven left or right it beeps to tell you scholiosis alert.

34. Motion sensing screensaver functions for TV. (Kinect)
– Can tell if you fell asleep, and then turns off TV

35. Simple hand/fingertip scanning device. Paper to digital.
– Scan finger over lines of text and it comes up on the screen

36. Book reader that reads out loud as your scan in pages.

37. Remote Controller Finder (Arduino)
– A pair of devices. One is attached to the remote controller, and the other to the actual device (e.g. TV)
– When the remote controller is lost, press the button on the device.

38. Coffee/Tea Temperature Assistant
– A device you can attach to the cup of your favorite hot beverage.
– Tells you when the optimal temperature has been reached.
– AKA Thermometer 😉

39. Measuring system with fridge that measures BMI and tells you when you’re being too unhealthy.
– Locks the fridge door if you’re gaining obesity points.

40. Video game controller that has biometric ID-ing device that locks the game if you’re playing too much.
– Parents can control how much their kids play video games

41. Toilet device that measures the contents of your urine and warns you with a light if you’re dehydrated or have some kind of problem.
– Too high blood sugar, cholesterol, etc.
– Activated upon flush

42. Public laundry machine that requires your phone number or NETID to activate.
– Texts you a reminder it’s done or before it’s done
– Phone number needed to open the door until after 15 minutes after cycle
– Bills the number a late fee if you don’t show up on time!!!!

43. 3D space explorer (Kinect or leapmotion)
– Can view the solar system and beyond using hand gestures.
– Great for science classes.

44. Biometric sensor smartphone attachment that calculates levels of alcohol or other drugs and locks certain features accordingly.
– Can lock text messaging or calling for certain predetermined numbers. (Family, girls, etc.)

45. Motion based movie player. (leapmotion or Kinect)
– Use intuitive gestures to play/fast forward/rewind/pause movies or any kind of videos.
– can change volumes and stuff

46. Device that uses motion detection or accelerometer to determine when you are dozing off and begins recording sound and/or video.
– Allows sleepy people to review what they would have missed.

47. Thermostat that only opens when a hand is touching it.
-Use variable resistor; it it’s at neutral state, don’t open the lid.

48. Kinect sensor in car that recognizes specific hand motion to start the engine.
– Could be extended to locking/unlocking the doors.

49. Radio sensor that flashes warning light to indicate when police or public safety have entered the building
– Could also use facial recognition to scan for faces older than college age.

50.  App that allows scientists to examine the 3-D structure of atoms or molecules, etc.  (Leap Motion)
– Great learning tool, would be awesome for classes

51.  App for a new textbook interface on the Leap Motion
– Can turn pages, highlight, write in margins, mark favorite pages etc.
– No longer have to buy textbooks, can store all of them on your computer
– Could make it more general as a pdf reader, since most notes and textbooks come as pdfs.

Why We Picked Idea #29

We the Backend Cleaning Inspectors believe in a better world in which everyone can focus on the important things without the distraction and stress from mundane chores. This is why we chose idea #29: stressing over laundry is by far one of the most annoying chores, and we’d like to rescue people from that annoyance. With our new “Clean Safe” laundry security system, people will be able to work without having to worry about the safety of their laundry.

Project Description

• Target User Group: Our broad target user group will be anyone that uses a public laundry machine.  Our product will allow them to have the peace of mind that their laundry will not be disturbed.  However, narrowing our focus to users that we have access to, we can say that our specific target user group for this project will be students at Princeton University who use Princeton laundry services, especially those who have had problems with their laundry being messed with (either taken out before it was done to make room for someone else’s laundry or something similar) while they were not in the laundry rooms.  Our target user group wants a product that will allow them to have confidence that their laundry will not be disturbed when they leave the room.  Very few people who use the laundry services at Princeton stay there and watch their laundry until it is finished. Our product will essentially do that for them.
• Problem Description & Context: Our idea addresses the common problem at Princeton of strangers tampering with, moving, or even stealing other peoples’ laundry that is left in the machine after the cycle. The context is the public laundry system at Princeton in which users do not necessarily know when washed or dried laundry will be removed by its owner. This leads to people moving the laundry to potentially dirty places like the floor, where in many cases it is lost or even stolen. This problem may be affected by the time and date, as many students do their laundry before or after classes or on weekends, as well as location, as areas with fewer laundry rooms will be more frequently in demand. These factors can be accounted for statically or dynamically with machine usage data. We propose to solve this problem with an system that will enhance security, responsibility, and efficiency in the laundry room. We hope to take identifying input from the users of the laundry/dryer machines, such as Princeton NetID, phone number, etc. and use it to 1.) send them SMS, email, or other warnings at time intervals and 2.) require this information to unlock the laundry machine until a certain grace period after the cycle has completed (intervals will be calculated based on time, date, and location). This idea may be expanded or refined depending on our capabilities and other environmental factors we encounter in the process, such as adding a fine to late laundry owners.
• Why Arduino? Arduino offers a good solution because our users will need to input identifying information(netid, phone number, passcode, etc.). We can accomplish this by programming necessary buttons on the Arduino platform. The attachable LCD displays will also come in handy to display remaining times, warnings, etc. Also, Arduino and its supporting programming languages allow us to send SMS and email notifications to our users. In general, Arduino’s compact size and simple construction will allow us to program a clean, user-friendly interface device to attach to existing laundry machines with minimal modifications.

Sketch

Group

• Avneesh Sarwate (asarwate@)
• John Subosits (subosits@)
• Joe Turchiano (jturchia@)
• Kuni Nagakura (nagakura@)
• Yaared Al-Mehairi (kyal@)

Brainstorming List

1. Spare Prox/Key Shooter (SPK) – SPK shoots a spare prox or key under your door when triggered by an iPhone.
2. hiLight – A highlighter that reads text that you highlight and stores it on your computer.
3. Snoozy – An alarm clock that uses motion sensors for snoozing capabilities (helps you control an alarm clock that is out of reach).
4. MotherClocker – An alarm clock that senses if you’re levels of motion (e.g. stirring or dead silent) and adjusts volume accordingly.
5. myBar – An interactive bar counter that lights up when you want a drink and visually notifies the bartender (kind of like in airplanes).
6. hiRoom – A system that tracks motion and heat in rooms and keeps track of the availability of rooms.
7. HandBoard – A new type of extended keyboard that makes full use of gesture control.
8. AbsolutelyRotten! – An alarm that detects rotten food in your fridge.
9. MatchFit – A color sensing camera app that detects if your clothes match or not.
10. HowToDressWell – An application that learns your fashion tastes and sensibilities.
11. GComp – An immersive gesture control computer.
12. GTV – An immersive gesture control TV.
13. Grillo – A streamlined, interactive grill ordering system for Terrace/Tower that also tells you what orders have finished being prepared.
14. PencilMouse – Pencil shaped mouse that you can use on your screen.
15. HotDamn! – Heat sensor that notifies when you’re overheating or sweating in formalwear.
16. mobPage – Pager for smartphone that notifies if a special person is trying to contact you (sketch below).
    

    Pager for mobile when you are out

17. myMix – A system that makes your mixed drink.
18. ArchScan – A scanner for 3D prototypes that translates them into CAD or Rhino.
19. Tabelisk – A desk that incorporates a tablet computer.
20. PetPet – A pet feeder that can feed your pets on a timer or detect when they approach.
21. Scarecrow – Scarecrow that senses unwanted animals and scares them off.
22. SketchIt – Etch-a-sketch controller/app that works on computers.
23. PubPort – Interactive system for optimizing public transportation hubs/stations.
24. MCVolume – Motion-controlled volume controller (possibly for TV).
25. Rise – Light-sensor alarm clock that wakes you up when the sun comes up.
26. DrinkUp – A system that uses pressure sensors to detect when your drink is empty and notifies the bartender/waiter to get a new one.
27. EyeComp – Eyepiece computer with facial expression and voice recognition software.
28. DifPic – A camera that compares a picture of your room to a picture of it when clean (sketch below).
    

    Before versus after

     

29. E-Cig – An electronic cigarette measuring smoke throughput that gives you your stats.
30. GGPS – A GPS system that notifies trafic patterns and indicates most efficient routes.
31. UrbOp – An urban system in which road and street sensors optimize traffic lights.
32. CarMap – A density map that shows you where cars are in a selected area.
33. Radar – Laser tag radar system with map (kind of like how video games display enemy position).
34. Empto – System for detecting decreasing drinks in restaurants.
35. DJ Light – A gesture controlled light show that allows DJs to use their movements for a light show (sketch below).
    

    Gesture controlled light show

     

36. MIDI Light – A system that allows DJs to have a MIDI controlled light show.
37. Sound Light – A system that allows DJs to have a live-sound controlled light show.
38. Gestar – An enhanced guitar incorporating extra sensors to trigger effects without extra gestures.
39. DJ Mat – Rollout DJ mat that uses smaller sensors instead of big 3D hardware (sketch below).
    

    Roll out DJ mat

     

40. MoodMix – A song mixer that plays music depending on mood lighting, temperature, and room humidity.
41. Posturrector – A posture correcting system that beeps when you slouch.
42. SenseForm – A mechanism that assesses the form of your squats/lifts against ideal static finish positions using flex sensors (or Kinect).
43. PressBox – A box squat device with pressure sensors to stop you from cheating (i.e. sitting on the box).
44. Juggo – Learn how to juggle a soccer ball using Kinect.
45. GloveForm – Martial arts glove that teaches proper form by detecting hit knuckles and wrist angle (sketch below).
    

    Hit knuckles and wrist angle

     

46. ForceForm – Force/accuracy sensor to determine the effectiveness of your punch/kick on a bag.
47. MemPic – A smartphone app for students (mostly freshman) that can take pictures of buildings and know their campus location and department/facility associations.
48. VirtualNote – A system that incorporates a pencil that integrates handwritten notes to a virtual notepad (so it is okay to lose your hard copy).
49. JuggoMotion – Special juggling balls that teach you how to juggle by displaying the motion of the balls on the screen.
50. MindMouse – A mind-controlled mouse, using EEG caps that measure brain waves.
51. EnviroSense – An environment (e.g. room) with sensors that measure your sweat and body temperature and adjusts the humidity and temperature of the room accordingly.

Idea Choice

Prime Ideas

• DrinkUp (#26)
• SenseForm (#42)
• VirtualNote (#48)
• DJ Light (#35)

Our first choice for ideas (non-Kinect based) is DrinkUp – a system that notifies staff, waiters, and bartenders in a bar or restaurant if a customer’s drink needs to be refilled. Our second choice for ideas (Kinect based) is SenseForm – a system that serves as a personal weight-lifting trainer that makes use of Kinect to direct and teach users how to properly lift weights. These ideas have clearly defined problems and the ideas in themselves are not overly complicated, therefore leading to simple solutions. Below we expand on both of these ideas (however we have left out designs and sketches for our second choice).

DrinkUp: Idea #1

Target User Group

The target user groups for this technology are both patrons and staff of restaurants, bars, or virtually any other establishment where a drink might be served. The idea driving our project is to remove any active participation on the part of the patron for requesting refills. We imagine the drink and table sensors being used by patrons, who might want to let their waiter know they need a refill on their drink, or by the waiters themselves, who might want to keep track of all the drinks in the restaurant and refill them when they are empty.

Problem Description and Context

Imagine yourself in a crowded bar or restaurant. You’ve just finished your drink of choice, and your glass is now empty. However, you are still eating food of some sort, or perhaps for some reason you are just very thirsty today. Maybe you’re enjoying a conversation with a friend or a colleague and you can’t be bothered to scan the restaurant for a waiter. Unfortunately, no matter what you do, you’re having difficulty getting a refill! Maybe the bartender is at the other end of the bar at the moment and well-occupied, or perhaps your waiter at the restaurant is just nowhere to be seen. Has this ever happened to you? Our system will let you stay “in the moment” with whatever is engaging you at your eatery of choice, and stop service lags from breaking your flow of conversation. Experience the restaurant as it should be.

Justification of Technology Platform

Fear not! Imagine this scenario: there is a small touchpad on your table/bar seat. When you tap the touchpad, it sends a signal to the restaurant staff or the bartender to let them know that you’d like a refill for your drink! But let’s say that the restaurant is crowded at the moment. Even if the waiters wanted to come by and refill your drink, they are just too busy to do so. Well, it just so happens that in addition to that manual touch sensor on your table, there is a pressure sensor directly underneath your cup. This pressure sensor took note of when your drink was full, and it’s able to tell when the weight of your cup has decreased enough for it to be empty. When it detects that your cup is empty, it automatically sends a signal to the staff to let them know – and the staff maintains a computerized list of which cups in the restaurant are empty at the moment. Now the restaurant staff has a more effective method of managing empty drinks, and as a result you get yours refilled within moments rather than having to wait as long as a half hour for a waiter to notice your empty glass!
Our system is arduino-based, making use of pressure sensors to detect whether a glass is full or not, and then sending the table location to the staff computer. The system needs to be portable to accommodate different table locations as well as be able to interpret the data returned from a pressure sensor, so the arduino is ideal for that purpose. In order to actually be useful, the data must then be sent to a central location, which is why a computer application is necessary.

Designs and Sketches

Empty drinks, busy waiters/bartenders, unhappy restaurant/bar-goers

When below minimum pressure tolerance, signal is sent to central location

Central location gathers data for all drinks on all tables in restaurant/bar

SenseForm: Idea #2

Target User Group

This project is targeted towards anyone performing standard power lifting lifts in the gym. It will work best for lifts that have a static final position (i.e., the low-holding position in a squat). The system will tell them if they are performing the lift properly or not.

Problem Description and Context

Often, when people go to the gym to lift, they have a “gym buddy” that they can go with, one who can watch them and tell them if they are properly performing the lifts. For power lifting lifts in particular, the technique of the lift is of utmost importance. If a person goes to the gym alone, it becomes difficult to effectively determine whether one is correctly performing a lift or not. Mirrors are sometimes not available, and even when they are it is difficult to properly evaluate your form simply from a mirror view. This system could help alleviate that problem.

Justification of Technology Platform

A kinect, wekinator, and a laptop could be used to implement this system. The user could hold the final position and use these training examples to train the “finished” position on the wekinator, and then go through the motions before finishing and use these examples to train the “not finished” positions. The kinect’s skeletal tracking would be ideal for sensing the body positions. A simple footswitch could be used to turn example selection from the kinect on or off.

Group 21 – Epple

Andrew Boik (aboik@)
Brian Huang (bwhuang@)
Kevin Lee (kevinlee@)
Saswathi Natta (snatta@)

Brainstorming

1. Head tracking for use in dynamic crosstalk cancellation for 3D binaural audio
   
2. [Expansion on previous idea] Moving speakers and display to sweet spot you as you move around the room
3. Sign language to English translator
   
   1. can translate to different languages, not just english
   2. maybe as a design plan, put indicators on fingertips to make it easier to identify and track different finger positions ← can use this as a design alternate
   3. May also do flag semaphores.
      
      1. Could actually be really useful for ships
4. [Expansion on previous idea] Sign language teacher ← similar to above but with error correction!
5. Chair that alerts you when you have bad posture
   1. design would need lots of pressure sensors embedded in a chair – might be difficult mechanically for our group
6. Table on wheels that follows you
7. Take a picture from far away with a gesture instead of timer
8. Device to help you read in bed
   1. Gestures or button to flip a page
   2. One hand controller might be easier and more comfortable
   3. Extensible to music (idea is basically for seeking through book without hands)
9. Alarm for if you look like you fell asleep while driving (or in office)
10. [Expansion on previous idea] Alarm clock that will ring if you stay or go back to bed
11. [Expansion on previous idea] Alarm that warns you if you are about to bump into something in the dark
12. [Expansion on previous idea] Alarm that sounds when a person falls, aimed at saving elderly
13. Cheap paper keyboard and use device to actually detect keystrokes
14. [Expansion on previous idea] Play air piano and actually generate sound…
    1. a potential design for this would require a projector, maybe like a table top computer
15. Device to automatically turn off stove (or electronic device) when you are not present for a long time
    1. default: maybe after 30 minutes, but have a button or something that would allow you to disable it if you decide to, say, simmer something for several hours
16. [Expansion on previous idea] Device to detect if you left the garage door/car door on/unlocked (or the toilet seat up!)
17. Use gestures as a universal remote control for your entertainment center
18. Use gestures to smoothly move through pdf or slides
19. Detector for raised hands used for fast polling (or alert professor that someone has a question)
    
20. Use gestures to conduct a virtual orchestra that will play related music
    
21. Create a robot servant that will identify different objects and bring them to you based on your command
    1. Design: involves a camera, a claw to grab object, possibly an arm to move objects out of the way and a motor as well as movement vehicle that is stable enough to pick up simple objects
22. Scanning system to detect dirt on a surface
23. Object finding device
    1. plays a sound to show location
    2. shows on a screen where object is in the room
    3. Design: would require a tracker to be placed on each object and involves wireless technology
24. Digital golf swing critiquer.
25. [Expansion on previous idea] Augmented tennis ball machine that can identify and drill your weakest strokes to improve practice.
    1. Use similar idea to help you improve your baseball skills
26. [Expansion on previous idea] A digital personal trainer to watch your weight lifting form (and give encouragement! RAH!)
27. [Expansion on previous idea] A tracking digital coach that can critique swimming stroke form (would have to move with swimmer).
28. [Expansion on idea 26] Digital coach to watch you exercise,  and then aggregate actions to determine what muscles might have experienced the most strain, and recommend a battery of stretches that target it.
    1. Fancy treadmill or other exercise machine that, in addition to having a specialized routine that targets a certain group of muscles, also recommends stretches for it.
    2. Design: would need complex systems of determining movement if it is kinect based. otherwise, would need complex systems of pressure sensors in exercise equipment to determine which muscles were used and need to be stretched
29. [Expansion on previous idea] An alternative to prescribing stretches is to use systems of pressure sensors to see if the user is doing the exercise correctly depending on where there is more or less pressure on the exercise equipment.
30. An instrument monitor that would provide feedback on when it has warmed to room temperature (helps a great deal for tuning, especially woodwind and brass instruments), or at least a stable temperature (if playing an instrument in a cold place, for example).
31. [Expansion on previous idea] An instrument tuner that would dynamically tune certain instruments (like a trumpet) using a tuner and robotic hand to manually move tuning slide.
32. Prevent cat from scratching sofa (more generally, prevent a pet from doing something bad) by detecting the pet scratching the furniture based on patterns of movement and sounding an alarm or sending a text to the owner.
33. [Expansion on previous idea] would detect if pet has a certain pattern of movement to signify that it needs/wants its owner back if they are outside the house and signals the owner of this. Similar to a baby monitor, or like a cellphone idea for pets to use
34. Child/pet/prisoner detector that detects how “lost” target is based on distance/obstacles between target and device.
    1. Design: would have two modules. One is the base device with an alarm and a wireless receiver and the other would be the module that goes with the child/pet/prisoner with a transmitter and a circuit to calculate how far away it is from the base. If we want large distances, we can use GPS based technology, for small distance we can use signal strength from base or proximity to some preset border.
35. Detect how long the line outside a restaurant is.
36. Use a temperature sensor to make doors that are not double doors lock if temperature is below a certain level
    1. Idea inspired by observing students being annoyed by the equad single doors being used in the wintertime, letting in cold wind instead of taking two steps to open the double doors
    2. Design: detect temperature and wind level outside and enable automatic door locking mechanism
37. A group tracker and interface that will keep a tour group together – maybe point in direction of the tour guide if the person gets lost or play a noise.
38. A noise detector that plays an annoying alarm if a roommate plays music too loudly.
39. A student tracker for detecting whether students are bored in class – note nodding heads, facebook checking, etc.  Mainly intended for use in large classes.
40. Pseudo-GPS system that uses Wi-fi and sensors like accelerometers to provide position tracking via dead-reckoning
    1. Map new areas with sensors, and record relative positioning.  Never get lost in a new place again!
41. Use level bar and adjustable legs to automatically level a pool table
42. Stalker detector that tells you if it looks like you’re being followed
43. Robot arms controller that looks at your arm motions and replicates it with a pair of robot arms
44. 
    1. can be used for long distance control (like surgery from a different hospital)
    2. or perhaps scale down movement so that large actions translate to small ones (for microscopic work?)
    3. or scale up actions?  Sounds like big robots now…
    4. maybe try reversing actions to help, say, put on makeup
45. Use kinect to rotate virtual 3d objects
    
    1. Use similar system to reorganize/redecorate a room
46. Device to check if a person limit is exceeded
47. Virtual tour where your leg movement is tracked and used to move through a displayed virtual 3d space
48. Head movement tracker that will allow for exploration of a remote environment according to your head movements and displays the image in that direction on a display.
49. Presentation trainer that will warn you if you are fidgeting or have other bad habits
50. Suit for providing remote tactile feedback
51. Smart window blinds controller that raises and lowers based on detected ambient light (may need an array to be accurate and to keep unscrupulous types from seeing inside by shining a flashlight at the window).
52. Motile scarecrow (or scaredog for those annoying Canada Geese?) that will move suddenly and/or make noise if pests get too close.
    
53. Recording device for creating beats from ambient noise.
54. Karaoke song selector (takes input from ambient noise/sound to select songs based on “mood”).
55. Automatic gardener: water plants when necessary (if it’s been very sunny and dry, or plants look dehydrated)
56. Virtual environment simulator that will map movements and show on screen a corresponding superpower. Such as a flick of the wrist will show your image on screen with a spiderman type web. (inspired by the movie BOLT) (this has practical use.  We promise)
57. Virtual reality for providing virtual friends for lonely people (while pretending it’s a multiplayer game?).

Project Description

Choice 1:
Idea 47: Kinect based head movement tracker that will allow for exploration of a remote environment according to your head movements and pans around to display the image in that direction on a display.

We picked this amongst all the possibilities in the list because it has a clear application for us and what we are studying in terms of using HCI to break past the stagnant interface that is the mouse and keyboard.  It involves a more natural way of computer interaction as it allows for humans to intuitively turn their heads and have an imaginary camera replicate this movement to show what one wants to see in a remote environment. This can be accomplished through panning around a panoramic image or actually controlling a rotating video camera through Kinect.  It also involves a complex enough challenge in head tracking that it is not trivial.

Target users: Our project may be broadly aimed towards anyone who wants to remotely visualize an environment. However, for purposes of our project, we specifically identify people who wish to travel but do not have the time or funds to do so.  These users seek a means for exploring a space as naturally and intuitively as possible (the stronger the sense of immersion, the better) and need a system that can bring the sights and sounds to the area to them with the least sense of artifice possible. Possible applications that we could apply this to include Google Streetview and virtual tours.

Problem description and context: Currently, exploring remote environments through applications such as Google Streetview rely on the clunky interface that is the mouse and keyboard.  Unfortunately, this is an unintuitive tool that often causes people to awkwardly explore an environment through clicking arrows and pressing keys.  This makes true immersion impossible.  The goal of our project is to create an interface to make remote viewing of an environment much more intuitive.  We aim to improve the situation by replacing the mouse and keyboard with Kinect-based head tracking used to pan around the environment.  The image of the environment will then be displayed on a mobile display so as to always keep the display in view of the user.  Aspects that may influence the problem solution include the demand for viewpoints; that is, each user will require a unique viewer of the environment that will respond only to his/her movements.  If there is high demand, then an easily scaled solution may be necessary, such as virtualizing the experience into an exploration of a panoramic image.  Panoramic images are easily supplied as anyone can take a panoramic image and upload it for multiple users to concurrently use.  By having others upload panoramic images, this also allows us to solve a time and location problem, as we will not have to actually travel to the viewpoint to get the image.  Our user group is characterized by little available time and money.  We thus will want to aim to keep the cost of accessing our tool low.  Related solutions to this problem include the Oculus Rift; however, the Oculus Rift is at this point an unproven technology and is far from being easily obtainable.

Technology platform choice: We choose to use the Kinect as it offers the necessary support for head tracking which we need to control the exploration of our remote environment with. We also choose to display the scenery on a mobile display such as a laptop screen, iPad, or smartphone held by the user as it allows us to change the view of the scenery while keeping the display in view of the user.

Sketches:

Choice 2 (non-Kinect backup):
Idea 10: Intelligent alarm clock controlled by pressure sensors on a bed, refusing to turn off until the owner gets up off the bed.

Although this is a simple idea, we think it would be very practical.  Actually, we would really like to have this and wonder why no one has done this before.  Our target user group is people who have trouble waking up in the morning.  We’re specifically interested in users who frequently catch themselves sleeping through alarms and/or turning them off without remembering doing so.  Conventional alarm clocks are not sufficient for these users because they are so easy to ignore or turn off.  What they need is something more persistent, but intelligently so.  An alarm that is excessively difficult to turn off will indeed force these users to get up in the desired situations, but in others (for example, if the user has been up all night, has already woken up, or is sleeping elsewhere), the system is an annoyance.  These users want an alarm that is persistent only when it needs to be–that is, have sensitivity for context.  We can always add features to enhance this device in case it is too simple (automatic prompt for alarm when body detected, context-aware alarm types, or daylight tracking).

Brian Matejek
Matt Dolan
Ed Kelley
Joshua Prager

Team Name: Name Redacted

Assist the Elderly

1. Device for assisting elderly with limited use of sense to navigate early morning tasks.
   

2. “Life-alert” style device built into hearing aid. Can register a fall and connect to representative.

Games

3. Interactive game that teaches fundamentals of computer science to young children.
   

4. Projection based floor game. (probably not within budget)

5. Interactive board game with lights and sensors that add complexity and randomness to the game.

6. Attach flex sensors to trampolene or bouncing castle to provide visual feedback and make it into a game.

7. Augmented visual deprecation exploration exhibit by creating highly intuitive feedback devices. (See this exhibit)

8. Create a collaboration based game in which a group of people are in a room and are given choices and prompts and must make collaborative choices through real world interaction.

Tools

9. Use a sensor (like kinect) to help people interact with objects in 3D and 2D virtually modeled environments or video games more naturally.

10. Teach knot tying with interactive ropes (flex sensors plus led strips).

11. Explore different ways of created faster virtual keyboards.

12. Ways to input text that are not a keyboard.

13. Automating fencing bouts start and end times to help reduce timing uncertainty.

14. Have sensors on a tennis court to determine if the ball is in or out.

15. Interacting with video replays for editing, youtube, etc.

16. Create dynamic music composition layouts for iPad (e.g. Bjork used to create album)

17. Make suitcases that automatically tell you how much they weigh.

18. Measuring concussions using sensors in equipment.

19. GPS that gives you directions through vibration rather than audio/visual signals.

    

20. Create a better calculator that allows you to be more accurate and efficient.

21. Control devices through sensors in your shoe.

22. Interactive way to provide feedback on your CPR technique or how to use a defibrillator.

23. Create a sudo-3D display that tracks a person’s head and adjust the display accordingly.

24. Make bike/vehicle tires that automatically warn you when they’re getting low on pressure.

25. Method for preventing DUIs/tracking alcohol consumed.

Improve Home Environment

26. Make a motion based tv remote.

27. Interactive bathroom mirror for getting news and information while you are brushing your teeth.

28. Flower pot that lets you know when it needs to be watered.

29. Interactive chopping board that gives you feedback on what you are cutting.

30. Sensor in backpack to determine weight. Help reduce back injuries in kids. Could quickly tell you if you have everything you need for the day.

    

31. Use motion sensor to go through motions of a sport action and have the program provide advice and feedback (like personalized coaching without a coach).

32. Interactive movies that bridge the gap between passive entertainment and extremely active video games.

33. Controlling different aspects of fish tank care (feeding, lights, water quality, etc…).

34. Smart shower that lets you know how much water you’ve used and temperature.

35. Wardrobe/pantry that helps you select, locate, clothes/food that you want to wear and put them back.

36. Interactive poker table that deals the cards and calculates bets and the pot.

Services

37. Quick way for waiters to digitally take orders in a restaurant through a handheld device.

38. User interface for virtual restaurant table.

39. Better way of interacting with a vending machine.

40. Faster/better/increased functionality for campus print release stations.

41. Design a cup that tells you how hot your drink is.

Improved Computer UI

42. Create the next generation of computer desktop (essentially rethink the choices that were made with Windows 8).

43. Make an “invisible” keyboard that senses which key you press based only on your hand movements

44. Change the way we download and update by creating a lightweight dynamic game that either helps the computer in some way personalize the information or entertains the user during the process (or both).

45. Create an input device that learns based on your movements and habits and adapts to become more accurate and effective.

School/Work

46. Better way for students to interact with/receive information about class assignments. (Not 100% in the scope of this class)

47. Create a teacher feedback device that shows teachers in real time how interested or confused students are, so they know what teaching techniques worked and what topics need to be more elaborated.

    

48. Better iClicker (ability to have questions other than multiple choice).

49. Create a type of non-linear presentation which is organized in a more natural mental model.

50. Controlling class/work presentations or lectures using gestures.

Shortlist

1. Device for assisting elderly with limited use of sense to navigate early morning tasks.

2. Interactive game that teaches fundamentals of computer science to young children.

3. Generic keyboard thing.

4. Interacting with video replays for editing, youtube, etc.

5. Smart bag

6. More intuitive GPS

7. Teacher feedback

8. Controlling different aspects of fish tank care (feeding, lights, water quality, etc…).

Project Description

The project that we chose is to create an interactive tactile game or experience that facilitates the early understanding of fundamental computer science topics.  Essentially what this means, is some way in which we could expose people, especially children at a young age, to some of the metaphors and early concepts of CS and make it accessible enough.  Technologically, we think that the best way to achieve this is an extension of the white board.  We hope to combine computer vision with a projector.  Students would go up to the board and place tiles up.  These tiles would be then read in, executed like any other program, and results would be fed back to the user with visual feedback from the projector.

We chose this project for a few reasons.  Firstly, we think that if solves a fundamental problem.  In the past decade, there has been an increased push towards understanding and innovating on education.  It has long been known that education is the driving force of the future success of not only nations but the world.  But still today, many fewer students graduate from high school or college with the skills necessary to work in the tech industry than there are job openings.  This tech gap needs to be closed.  Learning a programming language has become just as important as learning a second spoken language (if not more important).  But besides the importance of the problem, we also chose this project because it is extremely well suited to the skills of the group.  We have experience with computer vision and graphics but more importantly we are all recent students in CS.  We know what it was like to learn our first languages and we feel like that puts us in a good position to improve the experience.

Target User Group:

The target user group for our project would primarily be younger students who have little to no knowledge of computer science or programming, though it could be generalized to people who lack knowledge of computer science in general. The barrier of entry to learning how to program is perceived as being very high by those unfamiliar with computer science and programming techniques. Sitting down in front of a computer and writing programs in a language completely foreign to them can seem like a very daunting task. Our goal is to lower this perceived barrier by making basic principles of computer science intuitive, engaging, and fun for the learning audience. People gravitate towards and are more comfortable with mediums they are familiar with. A whiteboard is something students are very accustomed to and also provides an easy way to display information visually. Being able to formulate a picture of what is going on inside a block of code instead of just staring at a wall of foreign text is something we think would be very useful in helping people understand what code does and how if functions.

Problem Description & Context:

Currently, there is very little computer science education in middle school classrooms.  Students who eventually do learn to code generally start in high school or college.  Our goal is to create an interactive, intuitive, and fun way  to teach the fundamentals of computer science to middle school students, and others with no formal computer science schooling.  Many middle schools are not equipped to teach computer science; they do not have the teachers and/or computer software to teach students how to code in Java or C.  Although middle school teachers might understand the fundamentals of computer science, they may not know enough about individual programming languages to teach their students.  All of these factors create an environment where computer science is simply not taught.  However, it is possible to teach students the fundamentals of computer science without relying on one particular language and fancy computer software.  In terms of the availability of the user group, it is entirely possible that students could devote part of class to learning computer science in this fun and interactive way.

Why We Chose this Platform:

Most middle school classrooms do not have computers for every student.  However, most have a projector, and most schools have some video cameras.  Almost every teacher also has a computer in class, but not every student has their own computer.  By connecting the video camera feeds and projector to the computer of the professor, we can run software on the computer that simultaneously parses the video input and outputs the corresponding images that we want.  Students will be able to interact with the video camera without having to interact with computers.  The projector offers an ideal way to display information to the students and change output simultaneously with the video camera.  But beyond practical logistics, we wanted to bring the experience of coding into the physical world.  One of the main barriers to entry for early CS students is often the intangibility of the virtual world.  It can be hard for students to wrap their head around the ideas of pointers or virtualization.  We hope that using physical types and real-time visual feedback, we can target these concepts and make them look and feel more real.  Imagine, for example, allocating memory in a virtual heap and then seeing a physical representation of that memory block instantly appear on the board next to the code.  Or even imagine how placing tiles in a for loop might help a student understand how iteration works.

Sketches:

Group Name: The Elite Four

Members:
Clay Whetung (cwhetung@)
Jae Young Lee (jyltwo@)
Jeff Snyder (jasnyder@)
Michael Newman (menewman@)

Brainstorming Ideas:

1. Create an interface for Magic: The Gathering or a similar game that maintains an internal representation of the game, enforces rules, and provides graphical feedback on game state to users.
2. Use a Kinect to train users in martial arts, dance, yoga, tai chi, etc. at a low cost by tracking the user’s skeleton and comparing their forms to those of expert users. The application can give them feedback on exactly where they’re going wrong.
3. Use pitch detection and score following to help musicians/singers know which notes they play out of tune while practicing and give them graphical feedback including direction and pitch distance.
4. An automated metronome with score following for practicing — when a musician plays a section correctly, the metronome automatically increases speed. If they play it incorrectly, the metronome shows them their mistakes and slows speed down.
5. A transparent device that can be overlaid on computer monitors (or televisions) of multiple sizes and transforms them into a pseudo-tablet so that artists using the system in Photoshop etc. get immediate visual feedback.
6. A drum pad for percussion practice in low-noise situations that uses a practice pad with piezo sensors inside to trigger drum sounds through headphones. The device would be similar to a practice mute for brass instruments and could be customized for varying sensitivities and with a range of sounds.
7. Interface to use basic functions of a computer while prone (i.e. in bed) — the device would project the interface onto the ceiling or wall, and use gestural control via Kinect or Wiimote. Avoids the common problems of using a laptop in bed – neck strain, hot surfaces, sharp corners, no surface to mouse on, etc.
8. A silent alarm clock that raises your shade for those who prefer to wake up with natural light, but want to do so later than the sun rises.
9. Voice recognition system to open doors. Allows for secure access by multiple people.
10. Make a self-control type device using the Kinect that locks users out of banned applications (i.e. World of Warcraft) or specific websites until they perform some physical task, for example a specific number of push-ups or a yoga routine.
11. A frisbee-throwing robot to help users practice Ultimate Frisbee skills. This allows users to practice catching when another human isn’t available, and it can throw consistently at specific speeds, heights, etc.
12. A system that actuates a laser or other toy to exercise your cat or another pet in your absence.
13. A similar device could use a treadmill to exercise your dog or other larger pet. For motivation, the animal could be receive a treat for running for a certain time or distance.
14. A voice-controlled kitchen helper that can automatically measure and dispense frequently-used ingredients.
15. A system that listens on a certain phone/Google Voice number for text messages and calls and allows you to remotely start arbitrary electronic appliances by connecting them to mains, for example to start coffee brewing 30 mins before you arrive home.
16. A device that aids disabled users in basic computer use by automatically scrolling based on the position of your eyes and having activated eye gestures for basic commands.
17. An Arduino-based system that makes the entire bathroom process hands-free (turning on the sink, dispensing soap, opening the door, etc.) — people don’t want to get their hands dirty again.
18. Go-to-sleep button: Press it when you want to sleep and it does everything you should before bed (i.e. hibernates computer, turns lights off, sets alarm clock).  Then when your alarm goes off everything turns back on!
19. Automatic bike lock — when you put the kickstand down, the bike automatically locks itself. Alternatively, a coiled lock system that automatically retracts the lock when you get close to it.
20. Mixology robot: When you go up to it and place an order verbally, it mixes the drink for you (makes much more complicated drinks for parties without hiring a bartender).
21. Heart rate sensor that controls the speed of a treadmill and plays ambient music with appropriate (and motivational) BPM.
22. Proximity sensor that can be attached to objects, so if you lose it then you can use the range sensor to find them.
23. Automatic transcription of music — you sing something and the device outputs the pitches and rhythms that you sang.
24. An interface that personalizes your entire house — scan your fingerprint and it does things like set the temperature, turn on certain lights in the house, etc.
25. System that uses RFID tags to track small informal transactions between friends so that money is kept even (i.e. Joe buys Frank a coffee and it gets logged, so maybe next time Frank will know to pay for Joe).
26. Smoke-controlled music player that plays progressively more progressive/alternative music as the amount of smoke in the room increases.
27. A weather sensor that detects brightness or dryness of the outside and dispenses sunscreen if it is very bright or lotion if it is very dry.
28. Alarm clock that syncs with your Google Calendar and wakes you up before your first appointment of the day, even if you forget to manually set an alarm. Also detects if you are awake already and won’t wake you up.
29. Freestyle rapping/poetry companion device that automatically performs voice recognition (very quickly) and suggests rhymes and/or insults based on previous words.
30. Musical routing system that automatically routes instrument inputs to amplifiers based on a performance schedule without the need to plug/unplug.
31. Silent band practice system with individual headphone mixes.
32. Party robot – allows attendees to vote by text on which of a few songs to play next, displays live tallies and automatically beat-matches and crossfades between songs to ensure that there are never silent moments.
33. Biometric sensor that detects which finger is being held to the sensor and opens certain applications and/or performs particular actions associated with each finger (e.g., different workspaces, a “gaming” finger, a multimedia finger…)
34. Alarm clock that is somehow synced with your sleep cycle (through movement, for example) in order to wake you up at the end of a sleep cycle before a certain time.
35. Fingerprint system that protects food/resources from roommates. Can also be used to “book” shared appliances like the oven or washing machine.
36. A credit-card or key fob form-factor sensor system that detects if you try to leave your room without your keys, wallet, or phone. If you forget them, it beeps and/or flashes a light to alert you.
37. A better way to flip through channels. Could use a Kinect and certain gestures, or a trackpad to scroll or detect blinking. Much easier than pressing the channel up/down buttons on a  remote.
38. Chair that detects your posture (e.g., how much you’re slouching, where your shoulders are, how much of your back is in contact with your chair) and automatically adjusts itself (with motorized cushions) to accommodate you.
39. Use a Kinect and projector to make an interactive desktop (like Jarvis from Iron Man)
40. Fingerprint scanning bike lock.
41. Shoes that inflate (or let in more air, like in those Nike air pockets) when you’re higher up in the air (when you jump) to cushion your landing.
42. Smartphone keyboard that adjusts based on how you’re holding it — if you’re holding it with one hand, for example, then the keyboard will automatically adjust to make it easier to type.
43. Television device that can sense when a laptop or other device with a screen is in front of it, then gives you the option of projecting that screen onto the TV, almost like a wireless external monitor.
44. External monitors that have a sense of physical location relative to other monitors that they are hooked up to — so if you have one monitor on the left and one on the right, you can swap their position and the mouse would still move correctly from the left screen to the right screen.
45. A device that enables any screen to be projected into a bigger size while maintaining any special properties it has (such as a touch screen) — a smartphone screen could be projected onto a table and essentially turned into a tabletop tablet.
46. A device in your shoes that measure how straight your steps are and sends the results to your phone — might be helpful for detecting when you’ve had too much to drink.
47. A glove that that can measure body temperature and control the room temperature accordingly. Additionally, gestures can be used to turn on speakers, turn lights off, etc.
48. Glasses/goggles with an infrared sensor that provide a heads-up display identifying living (and other warm) things in your field of view, classifying them based on their general shape and size (could be useful for detecting wildlife or stalkers).
49. Toilet that keeps track of the size/consistency/color of your stools, rating them on the Bristol stool scale and letting you know of potential health problems.
50. Either sensing gloves or a flat keyboard without a display that can be used to perform typing motions in the absence of a laptop, display, or proper keyboard — useful for situations where it’s not feasible to have a laptop out, or (for example) in a class where laptops are banned.

Brainstorming Sketches:
1) Interface for Magic: The Gathering or a similar game that maintains an internal representation of the game, enforces rules, and provides graphical feedback on game state to users.

7) Interface to use basic functions of a computer while prone.

10) Make a self-control type device using the Kinect that locks users out of banned appli­ca­tions or spe­cific web­sites until they per­form some phys­i­cal task.

19) Automatic bike lock — A coiled lock system that automatically retracts the lock when you get close to it.

26) Smoke-controlled music player.

32) Party robot – allows attendees to vote by text on which of a few songs to play next.

36) A credit-card or key fob form-factor sensor system that detects if you try to leave your room without your keys, wallet, or phone.

50) Sensing gloves that can be used to perform typing motions in the absence of a laptop, display, or proper keyboard.

Our idea:
We chose to work on the key alert system that will alert users if they try to leave their house/room without their keys/wallet/phone. We chose this idea because it solves a common problem: leaving home unprepared and getting locked out or worse. The proximity sensor will also allow users to find their lost necessities with ease. Usually this is done only for cell phones through a desktop or web app that will make the phone ring or give its GPS coordinates. With our system, however, all a user needs to do is grab the proximity finder that is hanging nearby and walk through her house until she finds the missing item(s). This project is somewhat open-ended; for example, the proximity technology could be carried outside of one’s domicile and used to find lost objects anywhere. This project also allows us to explore making user interfaces that have minimal use of traditional interfaces.

Target User Group:
This project is aimed primarily at people who have self-locking doors — a type of lock extremely common on college campuses. The primary issue with these doors is that if you leave your room without your key, you will promptly be locked out, often requiring a call to security and having to wait for their arrival. This is a massive inconvenience that costs the user time, comfort, and — in some cases — money. With our system, students (or even the university itself) would be able to install a very simple system and feel secure in their preparedness. This user group can quite often be disorganized, hurried, and stressed; these issues can compound to make remembering the little things, such as one’s keys, quite difficult.

Problem Description & Context:
The high level goal of this project is to ensure that users can feel confident that they haven’t forgotten anything when they leave the house — in particular, we are addressing the problem of forgetful students getting locked out by self-locking doors. A technical solution could improve the situation by alerting students when they try to leave their room without their keys. This is superior to a non-technical solution, such as simply leaving a reminder note, because the automated system will never “forget” to alert the user, whereas a student might forget to leave/read a note. Thus, it is important that the system be as automatic and simple as possible — after all, the goal of this project is to make users’ lives easier, not more difficult. In addition, we need to consider that these projects will be implemented in dorms or other places where the user does not have actual ownership of the building. As such, our solution must not require a destructive installation. Rather, it should be simple to install and remove. Since our target user group lives in a busy, high-stress environment, we do not want the solution to require much maintenance on their part. Instead, it should be more of an “install-once-and-forget-about-it” solution. A similar problem has been solved by using a beeper system that helps users find keys. But our goal here is not just to help users keep track of their keys, but to protect them from absentmindedness in general.

Platform:
We intend to build our device around the Arduino platform. This platform seems appropriate for an automated system that is meant to be small, simple, and generally unobtrusive. Relying on a desktop/laptop/mobile app would make less sense, since such devices would be useless whenever powered down or (in the case of mobile devices) lost or left elsewhere. For sensors, RFID tags or similar devices may be suitable for short-range proximity detection. For example, the system could detect when the door is opened but an RFID tag (associated with keys/wallet) is not present.

More Sketches:

Group Mem­bers: Sam Payne, Pre­rna Ramachan­dra, Dil­lon Reis­man, Abbi Ward

Ideas for a Better World:

1. A device that can notify you when people open/close doors or enter/leave your room so you can track room occupancy remotely, for the benefit who cohabitate with someone and want privacy.
2. Track the number of people in dining halls with sensors at the door – trigger a movie or TV show or music when enough people have entered the room or when they have exited.
3. Put sensors on chairs so that dining hall table layout can be tracked to determine where there are free seats.
4. Make a device which tracks when people start to fall asleep by sensing when their head slouches and wake them up using a vibration or light
5. (Sketched by Sam) A jacket to help posture – senses when someone is slouching and gives them a vibration or stimulus to correct posture.

6. A set of utensils that give feedback when a user has brought food to their mouth too quickly (vibrate, retract head of utensil etc.)
7. When reading a book on the treadmill, a device that bobs the book up and down to match your head movement to make reading in motion easier.
8. Gloves that give feedback to adults (or children) learning how to type by offering feedback (vibration) when an adult types with the wrong finger, or incorrectly otherwise.
9. A ‘personal clipboard’ that a user wears so that the user has memory which can interact with the keyboard (metal contacts?) to quickly copy data onto the memory, which the user can then paste onto other computers.
10. A jacket which allows users to feel music using vibrations (helpful for the deaf, or people who want to feel the bass but want to avoid noise pollution).

11. Motorized robot which can create hand made soft serve ice cream cones – that way a person doesn’t have to do it (and it’s perfect every time!)
12. When ice cream is running low, people running the store don’t know – add a sensor to determine if ice cream in a container is low and a new bag of soft serve/container of ice cream must be retrieved so that not as much time is wasted.
13. Camera on glasses which can track eyelid movement and be controlled to take pictures hands free for people who want to take pictures while driving, rock or mountain climbing, doing other activities which do not leave their hands free to work a camera.

14. Train movements for walking after leg injury (give feedback on bad walking form) – can also be used for activities such as sports.
15. Glasses whose lenses go darker when bright flashes of light are detected to prevent seizures in epileptic patients.
16. Small modules which interact with each other for adolescent learning – simulating chemical reactions, abstract physics concepts such as force, etc.
17. Dynamic mood lighting based on frequencies in music – fast movement generates warm colors and slow music generates cool colors (hear the rainbow!), for people who are hearing impaired and can’t hear music.
18. Dogs need companionship – make a robotic trainer which is able to reward dogs but still provide remote interaction with owner, if dog is alone at home and owner is away.
19. Long distance communication with a dog via a robot that shows your face, has pre-programmed audio, a receptacle and thrower for a ball/biscuits – for people who leave dogs alone at home or with do sitters.
20. Play instruments remotely using flex sensors on your hands — elbow positions for trombone, finger positions for guitar/violin.
21. Use Kinect gestures to communicate and interact with a robot dog – for people who can’t own a real dog!
22. Detect when babies roll over in their cribs and sleep in wrong positions by using sensors in the crib.
23. Play frogger in real life! Speed sensors on body to warn you when crossing the street. Maybe for young children, or crossing mall highways which don’t have actual crosswalks.
24. Find your car in a parking garage and leads you to it using light or sound or some kind of directional compass.
25. Calorie counter of food, as a function of weight of plate at start of meal, that can notify you if you’re eating too much or that can be integrated with a mobile app to help you monitor your food consumption.
26. Embedded sensors in glasses so waiters know when to refill your drink
27. Sensors that warn students about backpacks that are too heavy.
28. Sensors that warn when music is too loud.
29. Backpack that senses when you are stressed and becomes a back massager

30. Put out signal to a room and figure out where you should put dampeners for the best acoustics.
31. Use detectors on tongue to talk with your mouth closed.
32. A system that gives you feedback about your speed of speech or time left during presentations.
33. Use dancing to produce music, use Kinect to match beat to dancing.
34. Correlate good dancing to good music- suit of body sensors that detect movement, evaluate dance performance, produce music based on ‘quality’ of dance.(Variation on 33)
35. Sensing for epileptic seizures to notify those around you that you are at a health risk (detect the early signs that a seizure may start).
36. Automatic nail filer that files nail to the perfect level.
37. Device that monitors your alchol consumption, such as by tracking your breath or rate of drinking, and tells you to slow down your drinking privately.
*38. Tune your trumpet in real time by attaching a device to your tuning slide.
39. Feedback for learning instrument

• trombone
• violin

40. Track body movement to create art – dancing can become digital/visual art. Could also be built for the handicapped to make art.
41. Convert gesture to text in real-time for better remote interaction. Possible applications include ASL or understanding exorcisms.
42. Tune brass and other sections remotely so the conductor can easily tune the entire band
43. Sustainability: reinforce sustainable habits with rewards – anything from a text message saying thank you  to something more advanced
44. Robotic thermostat for students who don’t have access to actual thermostat. (Mechanically turns the knob)
45. Something that warns you of bad behavior that will affect you in the long term

• biting nails
• picking your nose
• keep your legs closed while sitting
• grinding your teeth
• snoring

46. An umbrella that responds to wind and tilts to give you the most coverage. It tells you which way to tilt using an array of LED’s.
– It could also light up and play music when it’s raining to cheer you up!

47. Use head rotation as a method for computer navigation for those who cannot use traditional methods of navigation
48. Use sensors for an interactive lifting trainer that will detect and correct your form and motivate you as the workout gets difficult.
49. Use the Kinect to watch where you are and only turn on lights in that direction.

• Use for spotlights in theater
• When you and your hot date are getting close, the lights dim

50. Create an iris scan for your entertainment system or computer so only authorized people can turn it on (parental control)

51. Make brushing your teeth easier for people who have limited range of arm motion. They can control with their hand where the brush goes using a map of the teeth they can hold in their lap.

52. A projector capable of projecting flat images onto surfaces which are not flat.

Final Project Chosen: Feeling your music

Problem Description and Context

Many people like to listen to their music at a high volume, not only because it is loud, but also because they can feel the lower frequencies. Many home theaters come with larger subwoofers that provide frequencies below hearable frequencies so that people can feel the effects of their movie. However, subwoofers of this kind are expensive, and playing music with loud bass tones can be irritating to neighbors. We wish to provide a solution to the problem of requiring high volume or expensive equipment to feel lower frequencies. This product would conveniently integrate with users’ listening habits. Solutions exist but are not widespread

Target User Group

Our target user group is people who want to feel low bass frequencies in dorms, apartments, or other locations which do not allow noise pollution. We are also targeting audiences which cannot afford expensive audio equipment – very often these are the same people who live in dorms and apartments. Specifically, our target audience would attend concerts which play for audiences who enjoy feeling the music. Electronic music festivals are especially attractive for those who like the feel music. These days, more electronic music genres such as Dubstep stress heavy basslines. To summarize in a few words: we are looking for people who live in apartments or dorms who do not have resources to purchase expensive sound equipment who like to listen to music with heavy basslines.

Tech platform

We’ve chosen to use the Arduino because it can perform Fast Fourier Transforms for analyzing audio frequencies quickly and we can easily integrate it with the hardware we will need to create vibrations. It is also portable (when combined with a battery pack) which will be critical to creating a wearable product.

Why we chose this idea

We chose this idea because the user group is accessible. There have been other trends in technology to match this same idea (approached in different ways) Skull Candy’s Skull-Crusher headphones attempt to vibrate the skull to provide concert-like experiences. We were torn between several ideas, and this project seems to match our group’s interest as the most exciting.

Team Members

Miles Yucht

David Dohan

Shubhro Saha

Andrew Cheong

Brainstorm

1. Digital flute powered by light sensors for people who have limited lung capacity but still would like to learn to play. Varying aperture can modulate tone volume, and it could also could teach you to play interactively.
2. For young, urban professionals who don’t carry mice with their laptops, one could have a credit-card format mouse that is thin enough to fit in your wallet.
3. I want a computer I can wear around my neck and interact with by holding up my fingers, a la Sixth Sense, if you work in the field where it’s difficult to set up your laptop.
4. For people who can’t control a mouse with their hand, they could move a ball with their feet to control their cursor.
5. For more effective group meetings, a giant electronic collaborative whiteboard with physical interface such that everyone could edit it simultaneously. At one time, everyone would have the same view.
6. If you want a copy of notes from today’s lecture if you couldn’t make it, a device that records a teacher’s notes on the blackboard and processes them into a PDF which would be available right after lecture
7. If I’m paralyzed or shopping from home, I want to be able to try on virtual clothing to see what I would look like without having to actually put the clothes on my physical body.
8. If I have no fingers, I could still control my TV with Kinect gesture/voice
9. Learning to jump rope is hard. It would be easier with a jump rope that gives you feedback on what you need to adjust to become better, and it could teach you new tricks and save scores/records.
10. Instead of hiring a personal trainer, you could instead buy a device that would record you exercising and give you feedback, such as squatting or golf swing, to improve your technique and lower your chance for injury
11. Markov-based model for predictive typing to guess the next word you’re going to type in your phone/sentence so you can text faster, for those of us that are horrible spellers.
12. If you want to learn how to dance, but the DDR style doesn’t appeal to you, pads on the ground could light up, playing back a dance step tutorial, to teach you and perhaps a partner how to dance.
13. Learning to skateboard is hard, so my skateboard could detect foot placement to give feedback when learning to ride the skateboard.
14. I want my mirror to sympathize with me. By analyzing my face, my mirror should give me words of encouragement if I look like I’m feeling down in the morning.
15. Use gestures to control the multitude of lights in large rooms or in rooms where light controls are not easily accessible, for handicapped people or those interested in making dramatic entrances/exits.
16. For parents who want to introduce their young children to instruments, one could use a plant as a musical instrument by measuring flexing in the plant. This would require minimal technical skill and would also have the performer interact with nature.
17. For those college students that have a hard time waking up in the morning, a wake up alarm that won’t reset unless subject to the most violent conditions, like throwing it or slamming it
18. Use Xbox Kinect to give feedback on how to improve your posture if you have posture-related health issues.
19. For someone who can hold objects but has trouble typing, one could use physical gestures or general input device motion as passwords, as opposed to a typed text string.
20. For someone who has no motor control in their hands, a phone-like device could speed-dial numbers and interact with the user according to patterns of blowing air.
21. Device should detect butt location to infer how well someone is paying attention in an audience. More complicated: body language inference from camera at front of room (the inference step might even be doable with the same seat sensors as well)
22. Authenticate based on a laser key based on uniquely-shaped objects in a 2D/3D laser field. Stick your hand in there if you want
23. Create a sensor in bed that turns the lights off when there’s someone laying down… or two. Could trigger many possible actions such as arming house alarm etc
24. Devise a sensor in the bathroom that makes you aware of the number of bacteria on your hands as you wash them
25. Flush a toilet by blowing air into a sensor, reduces germs on contact
26. Enable computers to teach and/or read sign language, perhaps with XBox Kinect
27. Create a system that detects facial emotions so they can be used in focus groups to more conveniently collect data
28. Blowing air into a sensor to create a beatbox drumkit that people with disabilities can use
29. An algorithm can analyze keyboard typing sound patterns to infer what type of activity is being performed, use to evaluate student attention levels in lecture
30. For people who can’t speak loudly, voice-interaction systems should try to read their lips
31. Rubbing your pocket to change tracks on iPhone on a cold day
32. Utensils/containers that tell you if your food is too hot to eat… alternatively, containers that automatically heat up food that is too cold
33. For people who use the same computer over the course of the day/night, a program that takes into account ambient light and current display (maybe even type) to calculate the best values for brightness and other display parameters (gamma, contrast, etc.)
34. Music playlist that automatically changes to suit you as you change tasks
35. 3D manipulation of models and visualizations (think molecules / proteins) with Leap, this is a much more natural gesture
36. Use LEAP motion as an effective, cost-effective way to scan faces for authentication
37. Play Rock, Paper, Scissors with LEAP to provide companionship for children
38. Direct a virtual live orchestra using baton movements captured in LEAP. This can be used to train amateur conductors
39. Integrate LEAP into clothing to make convenient computer gestures right in front of your body
40. Control a quadricopter with tongue movements so disabled individuals can go beyond joystick interaction
41. Violin that lights up on the frets to teach novices how to play songs
42. Computer in backpack with projector on chest to make a virtual piece of paper you can write on with a stylus (convenient, mobile notetaking)
43. Reconstruct ping pong game based on sounds from microphone (triangulate landing and where it is hit)
44. Control quadcopter or another electronic device with LEAP motion… it’s a far more convenient and natural gesture than joystick
45. When you’re working out and don’t want to change your music player for fear of covering it with sweat or taking time off of exercise, your music player could measure your heart rate and the speed of the repetition of the exercise and generate a playlist of appropriate songs.
46. Use Xbox Kinect to obviate human labor in semaphore training
47. Billiards table that visually augments your game interaction, suggests ball movements to make the game easier for novices
48. Control quadricopter by measuring movements in a 3D point cloud with an accelerometer… much more natural gesture than joystick systems.
49. Eye-tracking system will move a vehicle (quadricopter) to the desired location being looked… for people with limited limb movement
50. Teach children motor skills with a colored grid on the floor where they can play Simon Says with their feet

Sketches During Brainstorm

Project Choice Justification

LEAP-based Authentication

Dance Dance

Games are generally really fun to play but often have no real-life applicability, such as the Guitar Hero franchise, whereas some tasks in real life can be somewhat droll to learn. Enter the digital dance floor, which could teach you to dance by lighting up tiles for you to step on in time to music. Here, the idea is that you would stand on a dance floor, which is composed of a set of transparent, square tiles. Each tile would be controlled by a single light source, the set of which would be managed by a computational device. By storing and replaying a pattern of lights over time, one could effectively recreate the steps of many dances. Then, using force-sensitive resistors underneath each tile, the accuracy and timing of one’s responding dance steps could be measured and quantified into a score, which would then be recorded at the end of the game. Additionally, multiple panel colors could allow for more than one player to participate in the game at once. This idea is also extendable, as there are many other modes for operation one could conceive of: for instance, you could have the lights respond to pressure, creating a dance floor that tracks how people move along it and lights up squares beneath people on the dance floor, or you could play a full-body version of Simon Says.

Detailed Description

Problem Description & Context. Reliable user authentication has been a perennial problem in human-computer interaction. How can a system verify that the user is who he/she claims to be? The prevailing solution varies on and off the computer screen. Inside the web browser, username/passwords systems ensure that the desired user is the only individual who knows the correct combination of inputs. Outside the computer, locks, keys, and RFID cards dominate the physical world to open doors and grant physical access. These solutions are not without challenges of their own. For example, what happens when a user forgets his/her password? The password recovery process is prone to hacking by email and phishing attempts. In the real world, physical keys and cards are liable to misplacement. We’ve all had a time we lost a key during our day-to-day bustle. Finally, an underserved segment of our population are those disabled individuals who cannot easily use existing forms of authentication. Consider individuals who have difficulty typing– username/passwords are a nuisance. Similar challenges are presented to individuals who have difficulty with traditional locks and keys. Our overall goal is simple, fast, reliable, user authentication.

Target User Group. Our target user group boils down to two types:

1. Disabled Individuals– For reasons related to limited finger movement or arm motion, these individuals experience difficulty using locks/keys or username/password typed into web sites. They desire access to their favorite web sites and physical rooms behind locked doors.
2. Public Computer Users– Institutions like a university are full of public computers that require username/password authentication. The time spent authenticating by keyboard could be better spent serving another user, thus reducing the overall demand for computing resources over time. These institutional users desire speed– whether its university students trying to print a paper before class or a business professional trying to get a meeting started as quickly as possible.

Technology. Leap is a sensor that is capable of detecting 3D motion for 3D objects. While photos and videos are a 2D mapping of the 3D world, Leap is able to capture the full scope of 3D reality. This benefit allows our idea to become more viable. In the case for facial detection, the Kinect or a photo app would not suffice because authentication can be thwarted by simply placing a picture of a certain individual in front of the sensor. However, this wouldn’t be a problem for the Leap since the 3D would take into consideration the depth of the image. For a 3D object, due to the sensitivity and precision of a Leap, other systems may not be able to detect the subtleties of our authentication object.

Sketches.