P1: Team Chewbacca

Posted on February 22, 2013 by jeanchoi

Brainstormed Ideas:

1. Posture helper—Detects and notifies you when you do not have the right posture.

2. Push-up monitor—detects motion along your body as you do push-up and ensures that your push-up form is at tip-top shape; can also add some sort of motivational game that helps improve your form

3. Built-in shoe sensor—help improve the monitoring of a runner; detects how many steps you have taken, how fast you have taken them, distance, and impact, and compares to previous runs/makes suggestions about how to avoid injury

4. Alarm clock stimulator—Uses a combination of features: tactical, sound, touch, and smell to guarantee that user wakes up

5. Sleep-movement regulator— detects lapses in the REM cycle; plays soothing sounds when movement is detected to help user sleep better

6. Classroom alarm clock— detects when a student is beginning to fall asleep and uses sensory stimulation such as vibration to keep the student awake

7. Pen that does not write, but senses pressure and shapes and identifies writing so you can write on any surface and it will compile it into a digital document.

8. Proximity reminder—signals to you what types or errands or tasks you should do when you’re within a certain area (or within proximity of something like the grocery store)

9. Office-hour scheduler—helps coordinate meetings with other people (allow them to see your location)

10. Parking-locator— Indicates where you have parked and sends you the location; also can issue an alarm that emits from the car if button is pressed (alarm button attached to key only works within small distance)

11. Item-locator—helps locate most valuable misplaced items simultaneously (cell-phone, wallet, keys, etc.)

12. Smart shower: before shower, collects data from skin contact and determines water
temperature and duration needed for skin’s dryness and dirtiness

13. Expanding on #12, during the shower, skin sensor that prevents skin from becoming too hot (proven to be unhealthy), gives you/shower feedback)

14. Small wearable sensor worn on skin that detects skin’s dryness and suggests when lotion should be applied.

15. Better sensing techniques for a blind person—modify a cane with additional detection of distance and sound/touch feedback

16. Breath detection—resolves the problem of bad breath; detects the chemical composition of breath and warns user whether or not they need mints or breath-refreshers

17. Hydration detection— building off of #16, resolves the problem of bad dehydration; detects the chemical composition of breath and warns user whether or not they need to drink water

18. Bacteria detecting utensils—prevent user from ever getting stomach-viruses by warning user whether the food they are picking up with their utensil has potentially harmful bacteria.

19. Interactive learning game (e.g. for multiplication tables) with sensory feedback such as vibration, sounds, or smells that correspond to the associations (helps user remember by using multiple senses)

20. Visual note-taking: Records what you write, detects circled terms, and creates a digital version of your notes with images of the terms you circled. Could also automatically detect keywords in your notes and create a “collage” for visual learners/quick refreshing.

21. Glasses that display notifications on the lenses, could be used for emails, schedule notifications.

22. Discreet texting device: personal projector/video camera that projects a keyboard onto the surface in front of you, detects which letters you press, and allows you to send texts discreetly during classes, meetings, etc.

23. Discreet device attached to arm or clothing that warns you when your stocks are going down with vibrations, includes a quick sell button or a more complex interface based on personal projector (see texting device).

24. An accelerometer or flex sensor for hand and arm motions that senses and remembers if you’ve done certain actions (locking doors, closing door, etc)

25. Commute to-do list — user uploads to-do list (quick calls, small articles/memos to read), and device detects traffic/stop-and-go motion of car and suggests appropriate to-do list for commute (may give warning/turn off when car is in motion)

26. Shoe/step sensor — worn inside shoe, gives information about step style, detects pronation and supination and recommends appropriate shoes/insoles

27. Smart closet — takes data about weather and suggests appropriate clothing (if complete integrated system, might move those pieces of clothing to front/middle of closet)

28. Habit-fixer — a small accelerometer device that is affixed to appropriate part of body and detects habits that you want to fix (biting nails, jiggling legs)

29. Cell phone as Mouse – attach your cell phone to your computer, and then use that as a mouse / menu options to reduce clutter on the screen, use ultrasonic information too.

30. Calorie detector – probes food items and determines the nutrition information.

31. Smart Room – changes music and TV upon entering a room according to predetermined tastes of the users who enter (using device carried by user or mobile data). If more than one user enters, it will figure out the common interests among them all.

32. Athletic-form helper – for any type of athletic activity, a certain technique is necessary; use some sensors that detect the motion of the user, and the movement can be shown through a video feed, and thus, the form of the athlete can be improved (ice skating, pitching a baseball, etc.)

33. Hand sensor for playing violin, guitar, etc, that detects hand posture and gives advice on improvement, also can download specific pieces and it can tell you which finger intervals were wrong

34. Orchestra section communication device: attaches to music stands, detects when changes are made to music of principal stand, and notifies rest of stands about type of change (bowing, etc) and location (measure number)

35. Building off of #34, Kinect-type device attached to music stand that detects when bowings within section are not coordinated.

36. Contextual music page-turner: detects the notes that are being played or sung and lights up the corresponding notes on the sheet music; mechanically flips the pages when the end of the sheet music page is detected

37. Instrument upkeep device — small device that can be run over stringed instruments to detect moisture of wood, rosin/oil buildup, and state of varnish to detect when humidifier should be used or cleaning/varnishing should be done (already analog devices fro humidity, but need more exact/holistic sensor)

38. Urine characterization – takes input of color, chemicals, etc. and determines certain characteristics about your healthiness, and suggests improvements.

39. Virtual mouse – acts like a mouse, you can move around empty space to control the mouse on a computer.

40. Smart baking pan — detects which areas have been greased, so you can cover whole surface and grease enough

41. Smart beat – Creates music based on your interactions with the environment (if you are walking, takes up the beat of your steps)

42. Remote button pusher – you can have a button that will adapt to whatever device you want to click (camera so you don’t need a self-timer, turn off a light from far away, etc.)

43. Handheld scanner — scan a physical page (notes, etc), and uploads it with good design (checks alignment, spacing, etc)

44. Interactive scanner for physical books — looks like (and may act as ) clip-on reading lamp, but detects when words are underlined or notes are taken, and scans those pages for easy recap of significant passages

45. Individual temperature-sensing thermostat — has infrared sensor that detects the temperature of individuals inside the room and adjusts (if someone is exercising within the room, the heater would turn off/air conditioner would come on). Right now thermostats only detect the overall temperature of the whole room, which adjusts very slowly to individuals’ movement or activities.

46. Instant feedback for lecturers — projector-like device that can be mounted in front of blackboard, is given information about dimensions of lecture hall, and immediately gives feedback to instructor about whether they are writing too small for those in farthest seats. Could also be implemented with device that measures microphone input/speaker output to tell if lecturer is speaking loud enough to be heard

47. Light-detecting window blinds — detects brightness of sunlight and opens/closes automatically in order to create appropriate amount of light in room and prevent overheating in room (saves energy)

48. Kinect-like device in doorway of retail stores, connects to mobile device to tell customers as they walk in what size they should look at for different articles of clothing (shirt, pants, etc), specific garments or styles that would be flattering to their body type (sizing for different stores vary greatly, would reduce time trying on different sizes, helps marketing for each store)

49. Accompaniment-producer: Sound-detecting device that listens to the piece you are playing, identifies it, detects the speed at which you are playing it, and prepares/plays accompaniment at appropriate speed. Works for singing (karaoke soundtrack) or instrumental pieces/ concertos (orchestral or piano accompaniment recording), and can speed up/slow down recording to match your speed.

50. Plates/cups that detect content volume and give information to central device (can be used by restaurants to detect which diners need drink refills/are done with their courses)

51. Alternative to #50: Proximity sensors in restaurant tables that detect when devices worn/carried by waiters stop near the table for an extended period of time, gives data about which tables are being neglected/allows waiters to see which tables have already been visited by waiters who were not assigned that specific table

52. A wearable device that creates your own personal soundtrack – all of your movements are converted to sounds. For example, as you’re walking, your rate may be converted to the tempo of the song.

Selected Project:

Target Users:
Our target users are musicians, who often sheet music but have difficulties turning the pages at the appropriate times. This is particularly true for pianists, who often have people turn their pages for them during performances. In addition, page turns often do not occur at appropriate times in the music, and are often during complex or fast passages that require a player’s full attention and do not lend themselves well to pauses. Turning pages by yourself or using a human page-turner can introduce error into musical performances. Musicians would therefore benefit from an automated page-turning device, to eliminate the need for extra people during performances and make the page-turns more accurate and dependable.
Problem Description:
The problem is the lack of an automated way to reliably and robustly turn pages of sheet music. This problem occurs most often for solo performers, who are currently forced to break the music’s rhythm and their concentration in order to flip these pages. Because this problem is most important for live performances, the solution must be discreet, and not disrupt the performance (quiet, be able to detect a single user’s instrument even when playing with accompaniment, and function even if performer does not follow music exactly). Currently, there are some mechanical page-turners implemented primarily for organ players that respond to mechanical input (pushing pedal), but the process has not been completely automated.
Why the chosen platform/technology?
An Arduino-based solution would be ideal for this type of problem. Beforehand, the Arduino must have access to an online version of the sheet music, where it can read in the notes and know where the page breaks are for the given sheet music. The sound played by the musician would then be detected by an Arduino’s microphone, so that the Arduino could detect different pitches and speeds of notes played. It would then compare the given note to the sheet music stored in its online database, and identify where the user is currently playing the music. When the user reaches a note that is near the end of the current page, it will automatically turn the page for the musician.The Arduino is a good platform for these functionalities–specifically for powering the motor to mechanically flip pages, and also detecting analog inputs from sound changes.

P1

Posted on February 22, 2013 by Dale Markowitz

P 1

TFCS

Collin Stedman, Raymond Zhong, Farhan Abrol, Dale Markowitz

Ideas

NFC device that allows user to “transport” documents and websites from one computer to another.
Habit-acquisition system. Tag physical objects like dumbells, medicine bottles, etc with RF tags or microcontrollers, and use them to detect and log users’ interactions with these objects.

Forget-me-not backpack. If it is zipped without an NFC-tagged laptop, it beeps to alert its user he/she has left something behind.

backpack

Portable 3D pointer/mouse which consist of a string whose tension is measured to determine its position in 3D space.

Interface for navigation through multiple degrees of graphs (such as WordNet) with gestures using Kinect/Leap Motion.
Real-time tracking of facial microexpressions using a wearable camera, to provide instant feedback on social interactions and crowds.
Real-time tracking of facial microexpressions using a stationary webcam, to track energy and dynamics in a room during meetings, lectures, etc.
Intelligent conference tables that transfers powerpoints/handouts to and from users who place their devices on it.
Augmented-reality phone apps that overlay location data from Facebook, Linkedin, etc on the real world, like Yelp Monocle but with open data integrations.
Panoramic photography application for iPhone, which allows tagging things around you and immersively recreating the place where you took the photo.
Pen peripheral device for iPhone, which has a pencil tip and can write on paper, but also saves digital copies of what users have written.
Add electrical/magnetic field sensing to an iPhone or Pebble, and use it to provide an additional sense through vibration, tension, etc.
iPhone peripheral multimeter attachment

Geo-aware reminder app: App that reminds you to do tasks when you are in the appropriate location, sensed using GPS.
Public computer setup that senses’ users presence using face recognition and/or NFC, and automatically loads their personal desktop from the cloud.
An audio looper that uses Leap/Kinect to control dynamics, pitch, etc.
Virtual plants and creatures that simulate the health of real things, like work, schedules, or relationships (as measured by online communication). Best if a clear interface for tracking many different things.
Interactive canvas that can be touched or walked on to draw stories or animated art/videos/slideshows.
SMS enabled door lock – users text their friends keys which enable the friends to open the lock.
A tangible interface for tangibly representing and manipulating object-oriented programming constructs.
Input/output integrations between two computers that allow them to work together seamlessly.
Keyboard gloves. Design a different keyboard that allows typing from memory.
Digital business cards that are exchanged by phone via NFC.
A game played on a digital surface where the physical location and orientation of game pieces upon the board causes different game behaviors, events, and outcomes.
Photo frames whose photos are static until users stop to look at them, at which time the photos become videos and possibly react to the observer(s).
3D scanner which reads and interprets braille.

A petlike robot which is meant to be a dynamic presence in the homes of people living alone.
Wifi-detector necklace/keychain, which glows when open wi-fi is available, or for important calls.

Kinect program which allows users to “walk through” panoramic google maps.
NFC-key bracelet, which stores NFC keys like a virtual keyring would.
Multiuser NFC laptop lock screen which tracks recent users of the computer, retaining traces of their presence and usage of the computer. (Useful for making computer applications more discoverable, if people can see what software is used at a certain workstation.)
Flash drives that plug into a computer and download content from Internet livestreams, and replay them until they are exhausted and must be recharged.
Lights that automatically turn off when user gets into bed (detects this with pressure sensor)
Chandelier that displays when user receives an email, Tweet, etc with different color LEDs.

Persistence of Vision machine that sits on users desks, displays tweets, email titles, etc.
Lecture attendance taker using NFC login.
Headband that measures user’s concentration and vibrates to alert them when they lose concentration (for use in lecture, studying, etc).
Posture monitor using a stretch sensor, which vibrates to alert user when he/she is slouching.
Musical device that consists of streams of water (water harp, for example). Perhaps for hearing-impaired users to visualize and create music.
Controllers for complex theater lighting configurations using gestures/tactile interfaces.
Redesigned controls for mixers and synthesizers, perhaps gesture-based using Kinect.
Interactive table that can be drawn on and that makes noise when users touch it, also responding to pressure of touch.
A set of triggers for safely receiving a delivery carried by a small quadrotor/helicopter.
A gestural user interface used to provide navigation instructions for a robotic reconnaissance drone before it is deployed.
Intelligent teapot that brews tea when it reaches the right temperature.
Canvases that interactively display text/poems/etc through gestural input. Think interactive museum exhibit but with text.
Journal that automatically logs daily temperature, pulse, location based on data taken by phone, fitbit, etc.
Directional sensing of wifi or cell signals – overlay where wifi or cell signals are coming from on a webcam view or Google Glass.
Virtual spatial interface across multiple computer screens: ability to throw virtual objects from one screen or computer to another in physical space using gestures.
Object oriented programming language which is embodied by physical lego-style blocks. Each block corresponds to an object with a series of functions. Plugging blocks together in different ways creates complete programs.
Shoes that sense and warn athletes of poor posture in real-time and historically.
Using a sheet as a screen. Video camera detects indents or ripples in sheet, interprets command and projects new image on sheet.
3D scanner, and interface for virtually rearranging scanned objects in e.g. a room.

Our Choice

Habit Tracking for Object Interactions

We will create a variety of sensors to be attached to everyday objects like medicine boxes and exercise equipment, which detect when they are disturbed and update a habit-tracking application on your phone or computer.

We thought of many physical computing ideas, and chose the one that enabled the most interesting interaction cycle between the user and computer. Dale and Collin were interested in using NFC tags to remind themselves when they forgot to bring things with them. Raymond noticed he left things untouched that he wanted to use, like vitamins, whey, and dumbbells. We realized that it’s hard to profile what objects should be on someone at all times, but it is much easier to couple everyday interactions with objects like medicine bottles and dumbbells to a computer. What if those objects could remind us when they were left unused? This is useful; it would solve problems for anyone on a medical regimes or trying to build a new habit. It also enables some of the most interesting human-computer interactions among all the ideas we considered, since it it imbues common objects with intelligence which can go far beyond sensing disturbances, enabling a whole class of interactions between people and things around them.

Target Users and Context

Habit tracking serves strong needs for a few user groups. The elderly and those suffering from Alzheimer’s would benefit massively from reminders for their daily habits. For both the elderly and the infirm, daily medication schedules are vital; both overdosage and underdosage are unacceptable. Elderly individuals frequently use pill boxes labeled by day in order to remember which medicines to take on a particular day of the week. While this humble solution is simple and reliable, it does not provide active reminders and is only effective when used regularly. Even more importantly, the elderly population is provided with very few tools for remembering non-medical habits or behaviors, such as calling a family member, attending a scheduled appointment, feeding a pet, or checking the mail. This demographic is likely to have few but more significant uses for mobile or desktop computers, like calling and messaging relatives or reading news.

Another group of people who would benefit from habit tracking is athletes. In order to prepare for a marathon, be in peak shape for a championship, or simply maintain physique, athletes must be diligent in their workout routines. They need to remember to workout with a certain frequency, or want to make sure they they work out at a certain time of the day and for a certain amount of time. Most importantly of all, they need to make sure that they manage to maintain whatever changes they make to their established routines. The category of athletes extends to larger groups like students or the general population, as we are all too familiar with making resolutions to go to the gym or floss every day. Many people are interested in either acquiring new habits or reinforcing existing ones.

Problem Description

Our product solves two general problems. The first is struggling to acquire habits, which requires a solution that is reliable and easy to use. A reminder system must not fail upon losing Internet connectivity, although it could expect a phone to stay powered on, and it should not disrupt existing non-technological solutions like pill boxes. The habit tracker will be used mostly at home, or perhaps at work, by people with limited experience with technology. The system should require little maintenance, although users are not likely to be busy.

Another user segment is those who would like to acquire new habits, like flossing, taking vitamins, or exercising. These users currently use Seinfeld-style calendars where they cross out days when they have accomplished their goals, or habit tracking apps that are general-purpose (like Lift or Beeminder) or specific (like RunKeeper). Calendars are limited in how many habits they can track and do not provide reminders. Habit tracking apps require a superfluous interaction beyond the actual activity of the habit. Since these users are likely to have limited motivation during most times (but high motivation occasionally), a lower-friction interaction is beneficial for them.

Choice of Technology

Our system will use a combination of platforms to sense and track object interactions. The frontend of the app will be a smartphone or web app which lets you identify and track the habits and objects that are being tracked. It generates reminders and alerts for objects that haven’t been disturbed by the user in a set time period, and provide reminders to enforce important actions/habits the user is trying to follow.

The sensors must detect usage of the object; since the interaction varies between object, we would use a number of sensor integrations.

Weight-sensing pads for: pill boxes, dumbbells, laundry bags, sports equipment
Tilt/shake sensors for: dental Floss, hygiene items, tilt-sensors for pill boxes
Light sensors for: unused rooms
Electrical sensors for: windows, doors, lights

More than just force-sensors, these should sense time, so we can also track how long the object in question was used for. These sensors could be connected to microcontrollers which broadcast a signal. For a more polished system, we could use RF (radio frequency) tags and transmitters to reduce cost and size. The information could then be relayed to a hub or an Internet connection.

NFC-based sensors and NFC-reader phones are one other feasible implementation for the system. NFC tags are cheap and easy to tag an object with, and the data error rates are close to zero. They are also passive, requiring no power supply, which is ideal for sensors that you would attach to objects of daily use. The caveat of using an NFC based system is that the reader (phone) needs to be physically close (3-4 cm) to the tag in order to detect it and get data. This limits the kinds of information that we can get about patterns of usage of objects to only situations where you would physically touch the object with your phone to register a reading. Another implementation choice following in the same vein is using RFID-tags, active or passive as the sensors, and an RFID reader for receiving data.

P1: Runway

Posted on February 22, 2013 by Kiran Vodrahalli

Group Name: CAKE

We are not a lie.

Team members:

Connie Wan (cwan), Angela Dai (adai), Kiran Vodrahalli (knv), Edward Zhang (edwardz)

Brainstorming List:

3D modelling with stereoscopic output overlapping with gestural input space for natural user interaction (perhaps a Maze game where you’re inside a cube)
Enhanced music stand for rehearsal and concerts, with gestural/automatic page turning, automatically jumping to sections based on conductor’s decisions, and easy part markup
Natural navigation in virtual worlds/virtual reality using gestural input.
Solve the problem of limited screen real-estate by projecting a virtual desktop on every surface in the room that you can interact with using pen, hands, and gesture.
3D data exploration with visor – you put the visor on your head and you see streams of data everywhere like you’re in the matrix. just kidding, you should be able to interact with graphs with your hands, move them around, set axes differently, see different kinds of graphs for the same data, see Google maps with 3D cities, manipulate molecular models etc.
Hands-free transportation for the lazy – Segway chair
Make a enhanced web browser with a head-mounted display: instead of hidden tabs with no relationships, you can drag different pages around in the space surrounding you and organize them into your own mini-web (using gesture, of course).
Modern laser tag – have computers know where “portals” are, have gun-like thingies that have cameras, and you “fire” into portals which come out an associated portal to hit someone
Screenless navigation assistance in new environments using an audio and tactile interface – e.g. vibrating elements in clothing or gloves that warn of obstacles, audio signals that get louder or softer as you face places of interest, etc. Useful for the blind
Piano Tutor – projection onto actual piano keys that can light up keys or recognize when keys are played, synchronized with page turning.
living room aware computing — pointing at “TV”, turns on, knows where you are: we would use a model room and put hardware in the walls, chairs, etc. Simulate “The Living Room of 2040” or something like that
Smart food storage: refrigerator: texts you if it shuts off, keeps updated shopping list, keeps track of when food goes bad, organizes meals and moves ingredients to front of fridge if you need to use them, talks to your computer to get search history– what food have you been searching/ looking at recently, it compiles a list of ingredients and sends them for you to get
Smart food cooking: toaster, stove, microwave: control from your phone or computer, or via speech–it cooks it the way you want it (updated over time); alerts you if something burns, can be remotely shut off (like say you leave the house and realize you left the stove on, you could shut it off with an app, maybe with phone)
Smart dishwasher: control remotely, automatically knows when to wash dishes, rotating water jets that sense dish shapes, make sure everything is hit. Could integrate it with your calendar
Kitchen sink that automatically or hands-free turns on and off depending on when it’s needed, precisely filling up containers.
Smart closet and smart clothes hangers: keeps everything sorted for you, automatically sets out clothing for the day based on weather (thicker clothing for cold days) and calendar (bathing suit for beach days), keeps your clothing updated online so friends can like clothes, and maybe even suggest fashionable combos for you to impress everyone! (this may sort of exist?) also could be helpful as a shopping app–that way, you can automatically know if you have a certain type of clothing so you don’t buy too much of the same type
Smart mirror that allows you to virtually try on clothes (already exists?) – large screen w/ camera, can zoom in on parts, bring up clothing images, etc. update to fb/twitter if you think you’re looking pretty good today wink wink
Enhanced clothing, e.g. integrated music player and cellphone with speakers and microphone in hood, control system in the sleeves (combine with those t-shirts that have displays in the front).
Smart desk in schools – automatically sense who you are, directly integrated with your student account at a university. Personalized instructions in class/lab
Augmented window with touch and display capabilities – use to display widgets like weather, news, etc. Automatically raises and lowers blinds
Smart lamp (could double as projector — display internet pages on your desk, could be part of a desk UI (tabletop), doubles as a disco ball for room parties), automatically turns off when you get in bed
Smart gym: dumbells that keep track of your weight-lifting abilities, prompts you sarcastically (“do u evn lft”) (“friends don’t let friends skip leg day”), all the devices talk to each other, give you suggestions as to what you should do next for an ideal workout
Smart bike: knows directions and your schedule, “automatic”, i.e. changes gears
Smart backpack: knows what’s inside, is connected to calendar so knows your schedule, so it can alert you if you need a book/notebook that you need for a class, reminders to pick up packages, lets you know if your water bottle is running low, interact with it there’s an interface you can strap on, or on strap, plug in Android or iPhone to give backpack computing power and access to calendar stuff, pop out stuff you need immediately, organizational system that lets you put stuff in only one compartment of the backpack
E-readers that track your gaze and turn pages automatically
Smart bookshelf (like the one in Google NY, but better) that links to a site like shelfari or goodreads (maybe tie to online living room) and allows you to browse books that you might like; you can insert an e-reader or tablet where a book is “displayed” and it downloads that book for you to read
Voice and camera on computer (like Jarvis). Framework that connects with your whole house and provides an interface to turn off the lights, make toast, open your garage, anything you might have to walk to do, you can do it from your computer/voice
smart toilet cubicle, with a projected display and hands free navigation — no more magazines/books, no more phones dropping in toilet.
sassy oven– tell it what to bake, it talks back (maybe it tells you that you’re getting a bit chubby, hold off on the cookies) — the point is to have even more fun with baking and cooking in general. Good for lonely grandparents who don’t have anything else to do
Go-go-gadget arm – Quadrocopter that fetches faraway things for you.
Sunburn warning (hat that has a timer and knows what the weather is like in your area (temp sensor, maybe some sort of UV sensor based on smart glass, maybe knows your skin color/type), and calculates when you’d be exposed to too much sun, and notifies you 10 minutes before by beeping)
Device that senses if you are falling asleep in class and subtly wakes you up – in clothing, vibrates? or a device in ear that makes a small beep straight into your ear
Gestural Video Editor – view video as rectangular prism, make operations such as cuts/transitions/timing control more natural than with mouse. You can stretch out a square prism to slow down the video, compress it to speed it up, color different parts with different music, slice the prism and rearrange it for scene editing, etc.
Gestural Music Editor – can be either for notation/composition or for mixing/mastering. Use gesture contours to input notation with pitch, time and dynamics. Or make cuts/EQ/effects more natural with gestures instead of mouse+dialogs.
Ultimate student: always helps you be working on something (uber productivity) based on how long tasks will take – brings up appropriate tasks on computer, pop out homework/problem sets from your binder, etc. (might be better as a desktop application?) — have a printer built in, new form factor for “laptop” — portable desk instead of tablet. (maybe it folds up)
Advanced video conferencing – 3D images, room-sized conference room space, locational audio.
Whisper/subvocalization translator (like in Ender’s Game sequels) – can understand sound without you having to vocalize, for private conversations. Have a sensor on throat or tongue that detects vibrations and translates to speech
Educational AI : speech, personality: teaches math to elementary school kids through interactive games, learns what sort of games work well for specific individuals on the fly, encourages people — could be through voice, games could be gesture/voice based
medical auto diagnosis: can tell what your heart rate is, cholesterol level, how “sick” you look–maybe you have the flu. tells you in advance if you’re catching a cold and tells you what to do to stop it, contacts doctor if detects something serious (would use computer vision to detect stuff, maybe other sensors as well)
Device that finds and removes split ends from hair, either using visual or physical processing of hair strands
Invisible mouse – instead of trackpad for laptops, just use your hand as a mouse (activated when you put your hand on the table).
make specialized pen better (write on any old paper, updates stuff in the cloud (write subject on top, it knows what document, edits that for you) (Already exists)
Artist’s pen that can change ink color / thickness / type / etc. with small gestures/indications
2D character rigging with a drawing-based interface or an action figure.
Tactile interface – deeply integrate vibrating actuators into laptop/mouse/clothing for another output modality (like in game controllers or in phone games). E.g. use it in photo editors (indicating alpha value of pixels) or in web browsing.
Forcing you to do exercise – treadmill-activated computer that only runs while you run
3D game that requires actual running to move
desk that categorizes any object you put on it and lights it up in a different color and/or actually moves it to gather with other similar objects
mood lamps/screens/etc. that change color/brightness/temperature/etc. with the environment’s temperature/sound/brightness/general color/etc.
touchpad that can sense when finger hovers above, gives 3rd dimension special functionality (e.g. switch between stack of applications/windows/tabs)
origami tutor: camera finds where the paper is on the table and projects dotted lines where folds should go directly onto the paper, plus a ghost of how the paper should be folded next
actual (new or invisible) instrument embedded in clothing (e.g. invisible violin, or instrument that depends on how your limbs/body are positioned)
virtual interior decorator that sees what is in a room and projects things/colors from a store’s inventory that would go with them onto walls/furniture/etc
Projector that can project onto any surface and still look flat
Virtual tape measure based on hands for distance or start/end points
Camera/projector combo that lets you highlight portions of a book with a finger, then can display your highlights by figuring out what page you’re on
Building blocks for small children with letters on them that say any word that they are stacked into
Taste-tester (can be used in cooking) – tells you how much seasoning to add, or whether or not you might enjoy something based on how similar it is to known foods. Does it really taste like chicken? http://www.digikey.com/us/en/techzone/sensors/resources/articles/The-Five-Senses-of-SensorsTaste.html
Roomba++ – knows when to do vacuuming, as well as mopping/sweeping. Also cleans up counters, and intelligently puts objects on the floor into appropriate places.
actual shadow boxing game: your shadows fight with each other (with tactile feedback)
smart socks: text with people in class without anyone noticing (use flex sensors and do a morse code style communication — could be marketed for elementary school students, teach students morse code!)
completely virtual board game playing and tutoring like chess (gesture/ 3d display based)
object recognition chess (camera identifies when you move a physical piece, updates internet board in real time)

Sketches

Segway Chair, Auto-flipping Music Stand, Exercise-powered Computer, Origami Teacher, Wearable electronics, Building Blocks that spell out words to toddlers, Gestural Internet Browser (see windows in 3D and move them around!), Invisible Instrument, Go-go-gadget Arm [with Quadrocopter], Eye-tracking e-reader, Toilet Cubicle Entertainment System

Our Chosen Idea!:

Project Description:

The manipulation environment consists of a 3D stereoscopic monitor that makes objects “pop-out” and look like they are floating in mid air. A gesture camera (the Leap) has its input space aligned with the output space. We also use a webcam to track the primary user’s viewpoint so that users can move their heads and see different views of a scene. All in all, users are able to translate objects by pinching them and moving them directly to where they want to go, to scale objects by stretching our hands apart, to select objects by touching them, and so on.

Why we chose our idea:

For starters, the concept is super cool! We’ve all been inspired by this picture:

Tony Stark using gestures!

and since we have some pretty awesome hardware already (Leap, Stereoscopic Monitor, and a Kinect), we think it would be feasible to implement a neat prototype of a 3D gesture interface which interacts with virtual, floating 3D objects.

Also, our idea is very modular: it’s possible to create more specific subsets of this project by building more functionality on top of the core structure. Specifically, it might make sense to add speech recognition and processing later on (if we have time) to improve that naturalness of the user interface.

Furthermore, it’s just a really cool user interface that hasn’t been properly built yet! We envision that in the future, computers will move away from the screen paradigm and become more immersive.

Target User Group(s): Artists, architects, designers, and animators who do 3D modelling, as well as scientists who manipulate 3D data, such as researchers in 3D capture who have to deal with point clouds or geneticists who manipulate and fold protein models.

Problem Description and Context: Current 3D modelling applications, such as Blender and Maya, though powerful, are unintuitive and awkward for navigation, relying on the 2D monitor and 2 degree-of-freedom mouse to represent and manipulate 3D objects. Because of these restrictions, it is very difficult to select arbitrary points in 3D space and perform rotations. By displaying objects stereoscopically, we can make them look like they are holograms, floating in the real world. Users could then interact with these objects as if they were real (as with Iron Man’s Jarvis), to make full use of our physical intuition. Although experimental gestural interfaces for 3D modelling exist, they have been shown in academic studies to be less efficient than mouse interfaces. By aligning the gestural input space with the stereoscopic output space, we make a fully natural interface for 3D modeling.

Justification of Technology Platform: The fundamental parts of our system will involve a stereoscopic display and a gesture camera. We will most likely use a standard stereoscopic monitor for output, although we are considering using a head-mounted display as well. For a gesture camera, we intend to use the Leap because of its low data latency. We also need an additional webcam to perform viewpoint tracking. These input and output technologies enable the fundamentally novel part of our system, which is colocated input and output spaces. We would also like to incorporate tactile feedback, namely a simple glove with vibrating actuators, to enhance the visual feedback provided by the 3D display (which for example vibrate when your hands intersect an object). These all add to the illusion that 3D objects are in fact “real” and thus are uniquely suited to our application of manipulating 3D objects.

Design Sketch:

A User manipulates a virtual cube in front of a screen! A Leap motion tracks the users hands, and the depth camera helps out.

Life Hackers P1

Posted on February 22, 2013 by Gabriel Chen

Group Members

Colleen Carroll (cecarrol@)
Gabriel Chen (gcthree@)
Prakhar Agarwal (pagarwal@)

I. Brainstorm

Cell phone alarm clock extension, through maybe the speaker port or MicroUSB port, that uses muscle pulses to wake you up.
Kinect based alarm clock that will ring if it doesn’t recognize that a body has moved out of the bed
Alarm clock that vibrates the entire bed when it goes off (vibrations can be achieved by putting something like an eraser off center on a motor).
Proximity alarm that tests for human presence using a combination of different sensors (temperature, photo, visual shape?). This can be used as a homemade security system.

An AI “RCA” that complains if the noise levels in a room are too high by using volume sensors.
“RCA” system that uses a combination of temperature, noise, and light sensors (rave lighting?) to sense whether a party is going on in the room and complains.

7. FootMouse

*** Foot pedal based mouse so that one can use the keyboard and mouse simultaneously
Facial movement and gesture recognition system that lets you control different operations in a computer interface by actions such as winking or moving your mouth in different ways.

Automatically adjusting backpack straps with a force sensor to determine whether the backpack is being used and adjust to allow for best back support.
Backpack straps that open up to allow a user to get arms in easily when picked up and then go back to the original setting after the backpack is on.

There’s already hand-based controls for lights (snap your fingers to turn on/off the lights); what about blowing out lights in the same way you blow out candles?
Control the lights in a room by covering or uncovering a photo sensor to a certain extent. This would be intuitive and easy to do!

13. MatchMirror

A mirror that checks the coordination of your outfit to your skin color, body type and other parts of the outfit.
Use the kinect to help you cut your own hair. It shows the user how to cut their hair to a particular style and uses gesture recognition to correct the user as they go.
Use the Kinect to recognize your face shape and recommend hairstyles that are the most flattering.

17. SignGlove

18. RepGlove

*** “Secret handshake” glove that unlocks bikes, doors, laptops that can be programmed with certain gestures
*** Sign language glove that recognizes what is being signed by a person wearing it in order to transcribe it onto the computer.
*** Fingerless glove for lifting weights that records how many reps you do to an interface that records your workout progress and whether or not you are meeting your goals.
*** A sensor on your clothes (flex, etc.) that recognizes good form in exercise moves and stretches (push ups, planks, sit ups, running, swimming)
A glove that tells you which notes to play on the piano
A gesture-controllable glove to control household tasks (is the stove off? find your keys?).
Glove that recognizes gestures for different functions that may be useful while driving such as picking up the phone (move thumb to ear and pinky to mouth like a telephone).

Webcam attached to your face that, when you point or gesture to certain parts of a book, takes notes or snapshots of active areas.

24. SafeStove

A sensor on a dangerous object (stove, table saw) that detects the proximity of your finger and turns the object off when you’re too close.

An application that tells you which notes you’ve played and records them to a computer program that transcribes the music

26. HydrationBottle

27. DetoxFlask

Water bottle that keeps a log of how much water you intake during a day by sampling the weight of the water periodically. Lights up if at a given time, you haven’t had enough of the water.
Alcohol flask that keeps a log of how much alcohol you’ve had during a night by sampling the weight of the alcohol periodically. Estimates BAC and displays on flask.
A flex sensor on the ketchup bottle that lights up different amounts based on how much ketchup is in the bottle, the orientation of the bottle, and how hard you’re squeezing.

Pressure sensors on your bike brakes and handlebars that light up brake and turning signals (useful for drivers trying to avoid bikers).
Pressure sensors on your shoes that light up turning signals when you turn on your feet (useful for bikers trying to avoid pedestrians).

31. VolleyHelper

Accelerometer and pressure sensor on a volleyball that detects and lights up when you should hit a volleyball when it’s falling (useful for people trying to learn when to begin an approach to serve).
A device that detects your posture and helps you improve it using classical conditioning
A Kinect based application that helps you improve your presentation skills by looking at both physical cues (such as posture, nervous habits) and audio cues (number of “um”, “like”, etc)
Presentation tool to write or draw in the air and transfer to screen.

A system that can listen to users singing (often imperfectly) and use machine learning to determine and transcribe the notes they are actually trying to sing (or at least those that would musically make sense)

Using kinect, a waste bin that senses the waste item, classifies it as a particular type of trash and opens the appropriate hole for deposit

Pressure-sensing belt that tells you when to stop eating.

Application that recognizes human presence in a lecture hall and control the blinds to direct light towards locations where it won’t get in people’s eyes
Remote controlled vehicle for which the remote is based on one’s hand movements (point forward to move forward, rotate wrist to turn, etc)
A sensor that can detect the need for handicap facilities such as an automatic door by recognizing the image of a person on a wheelchair or crutches.

A better tape measure – use a fixed armspan as the base, put a sensor on each hand and use specific gestures.
3d drawing like on a 2D tablet. Use a simple stylus that is easily detected and draw in the air to add the model to CAD.
Kinect in the hallway to direct and measure traffic during an emergency, and safely help people exit the building.
Vending machine where you point to the item that you want (using kinect).
Solve a puzzle on a gift box to make the gift opening experience more interactive! For example, a Christmas gift box that has a countdown on the box, with a new puzzle every day.
Stethoscope that lights up when there is a heartbeat and records and analyzes sound for listening to breathing.
A multi-head screwdriver that can look at the screw you are going to use and the correct size head lights up.
A tray that can measure the amount of food that you are getting to go and print a price tag. The tag can then be easily brought up to the cashier and paid for quickly.
A coffee mug that tells you when your drink is cool enough to drink.
Make passwords more secure by moving them from the keyboard to any chosen object where pressure sensor can sense a specific pattern to unlock the computer.
Attach an arduino to a dog’s collar that the dog can be trained to interact with (barks, licks, breath, light) and open the door or turn on a water dispenser.
A smart crib that can rock the crib when it senses too much noise, that alerts you when the baby does certain things, etc
Mount to skateboard that gives you points based on spins, acceleration, jumps (real life Tony Hawk).
Motivation for running – virtual reality chase game to keep you running while on treadmill and meet workout goal (think real life temple run).
Hands-free way of interacting with your phone using kinect to detect gestures that correspond to mechanical motion on the phone’s touch screen and accelerometer.

II. Idea Choices:

Kinect-based application:
Through the brainstorming process, we tried to come up with ideas by looking at inefficiencies we saw around us and by considering problems those that we or our friends recently had. The idea we have chosen is to develop a mechanical, hands-free means to control one’s phone and we actually came up with it after discussing how one of Gabe’s friends recently dropped and broke his phone as he tried to pick up a call while on a treadmill. With our incessant dependence on mobile communication, we thought that there was definitely room for a hands free means to control a phone (outside of Siri, which is pretty terrible at recognizing a lot of commands). Our proposed solution uses a Kinect device set on the dashboard of the treadmill. Motions of the user would be used to control a mechanical mount in which the phone could be set. This mount will consist of a mechanical “finger” that can be used to touch different parts of the screen, and ideally, we will be able to move the entire mount in order to control accelerometer related functions of the phone. Through this hybrid, mechanical solution, we can translate our own full body actions to actions on the phone without actually touching the device. One could imagine waving to pick up a phone call or pretending to bob one’s head to music to start iTunes.

55. OnTheRun

Our second best idea:
Our second idea came from realizing the limitations in existing password systems. Most are based on a string, comprised of basic, finite character sets. This puts users in the position of either memorizing a complicated password or risk being vulnerable to simple attacks. Users tend to opt for convenience and choose passwords that are susceptible even to simple, brute-force attacks. We imagine a system for “passwords” that is virtually infinite, by taking gestures as input. A glove, when hooked up to a computer, can sense the gesture of a user’s hand and a software program can memorize the sequence of gestures that the user creates and then check future inputs against it. Each gesture, or “character”, is a product of the user’s imagination and thus is much harder for a hacker to guess.

III. Project Description

Target User Group:
Our target user group for this project consists of those who, like Gabe’s friend, are very plugged into their electronic communication and media devices and want to use them while exercising. With cellphones becoming more and more advanced and commonplace, it is becoming necessary for many of us to interact with and respond to them on an immediate basis. At the same time, though, they are becoming increasingly expensive and fragile. As such, one of the needs of the target group is to ensure that their phones are safe and not in fear of falling and having a cracked screen. Young users especially, who are used to always being plugged in, also want to be able to perform simple tasks like taking calls, sending texts, and maybe changing the music playing while they are working out. A gesture based, hands free manner to interact with their devices is especially useful for these users because it is simple and would allow them perform a task without requiring the exact and careful motor skills that touching the screen does.

Problem Description and Context:
As stated earlier, we first thought of the problem when a friend of Gabe’s experienced it himself. Those who exercise, especially in the gym, often multitask while doing so, but risk getting distracted and damaging their phone or themselves. On treadmills in particular, users often need some external stimulation besides staring at the wall ahead. We want to design a solution that allows the user to interact with their phone in a safe, fun way. We believe that a major cause of difficulties with using phones on a treadmill is that phones have a relatively small interactive surface, which requires more attention to get right, especially while moving quickly. Our system would need to have simple interactions and a large interactive surface to resolve these issues. It would also need to function in some of the most commonly attempted activities while running, for example, talking on the phone and reading. The interface could then be safe enough for use while running, but also give the user the entertainment or productivity that they are looking for.

Technology platform:
We chose to use a hybrid platform of both Arduino and Kinect for our application. We felt that Kinect was the best form of detection for the touch-free interaction with a large interface and variety of simple interactions, which is key to the solution of the problem described above. Additionally, from lab, we have discovered that Arduino has sufficient means of recognizing analog input. We need to be able to process the signals detected by Kinect in some way; Arduino is an appropriate means of translating this into the mechanical interactions on a touch-screen.

Team Deep Thought

Posted on February 22, 2013 by Elizabeth Qu

GROUP MEMBERS:

Neil Chatterjee (neilc@)
Alan Thorne (athorne@)
Vivian Qu (equ@)
Harvest Zhang (hlzhang@)

PART I:

Kinect DJ system: Uses gestures and body motion from skeleton data to control music mixing.
Air guitar: a flex-sensor based air guitar for Xbox games.
Automated third arm: Kinect or Android-based robotic arm for assisting in building, soldering, etc. A functional helping hands tool.
Kinect Haptic Vision: Vibration based on Kinect’s forward vision to help blind people see by feel from depth sensor data.
Shower rave: LED filter through water streams to color the stream optically; add music and make it into a whole dance system.
Posture Adjusting Shoes: using weight sensors and motors, readjust the inside bottom surface of the shoes to improve posture.
Sychronized Firework Launcher: Arduino and smartphone controlled synchronized fireworks launcher for private fireworks shows
NFC Information Platform: NFC controlled information booths throughout a building, accompanied by a database on an app and a map. Get close to get info
Kinect Gesture Analyzer: Observe which physical tics people use while giving talks and critique.
Turn Signal Clothing: Bicycle shirts that have LED turn signals for safety.
Kinect room analyzer + home decorator: Kinect scans a room and then uploads a mesh of the surrounding area to alter in photoshop.
Smart Clothing — Breathalyzer: measures people’s health; Drunkard’s shirt that measures pulse, temperature, and BAC and reports safety.
Laser tag T-Shirts to create a more visual experience.
Real-life angry birds with tracking data and projectile’s simulated with Kinect motion.
Symmetric face properties: Analysis of a person’s face to reveal how symmetrical they really are.
Kinect-controlled quadcopter/RV car. Kinect motions feed data to an Arduino to remotely control these electronics.
3D Kinect scanner that rotates an object on a platform to create a point cloud of the object.
Reflex games that use an arduino to interpret response time.
Triangulating gunshots using Raspberry Pis and sound sensors spaced throughout a city, room, building, etc.
Arduino swiffer/roomba that automatically navigates the floor of a room to clean.
Arduino door opener that contains a security presence to allow access for some people
Nest like system that allows general control of the homeostasis of a house.
Security cam from bike stealers that texts you if your bike has been stolen.
Arduino oscilloscope that measures voltage and current by using variable resistors. Additionally, Processing would create a plot.
Fire safety ALERT button that sends texts to everyone in a hall.
Backpack dog follows you around by tracking your motion with a Kinect.
Predicting lightning strikes using ozone sensors.
Methane Interpolator: determine where cows are or a fart detector that uses an assembly of these in an enclosed area.
Automatic garbage can: to follow drunk people around in case they can no longer handle themselves.
Electric Bike Conversion optimizes power usage during a bike and can provide uphill assist.
Laptop controller for when you’re lying down (hard to see / use keyboard / balance weight).
Automatic page turner for books.
Planetarium (lasers) that uses servos to project images onto a screen.
Transparent screens (lasers) that uses servos to project lasers onto an opaque glass
Keyboard pants that allow anyone to type on themselves.
Giant touchscreen made out of an assembly of smaller phones.
Phone tag: Mobile game that allows you to play tag with your cellphones.
Bluetooth phone glove on your hand so that you can talk with your gloves.
Measure foot pressure and adjust for correct walking position to improve posture.
Theremin instrument simulated using the Kinect.
Bluetooth backup camera to be implemented in cars for a cheap solution to alternatives.
Auto-instagram shots uses Arduino to easily and quickly upload certain images to Instagram (ethernet shield).
Home automation wake up system to integrate a morning routine into an electrical system
Nightmare detector that monitors respiration, pulse, and temperature to isolate REM cycles and interpret the presence of a nightmare.
Face 3d models with pin array uses an array of FSRs to create a point cloud of someone’s face.
Hug meter (emotion tracker with Kinect) lets you know who needs a hug in a room.
Lighting based on music input (visualizer)
Lie detector based on voice. Arduino does FFT to isolate irregularities in the person’s voice.
Underground meal swipes market uses an Arduino and a magnetic card reader to reprogram a prox.
Covert t-shirts pay attention to vibrations and secret gestures to convey information without speaking.
Fingerprint replication uses an Arduino and a fingerprint scanner to create a fingerprint mesh for replication.
Sound detecting system that generates mood/ambiance music to rhythm of sex (or any other rhythmic activity)
Using a Kinect or raspberry pis to track a speaker on stage (example application — spotlight).
Using a Kinect to scan someone’s BMI, offering customized health information (ex. safe amount of alcohol one individual could imbibe)
Leap motion controller virtual keyboard that allows typing on any surface.
Kinect a cappella system which uses spatial data and skeleton data to allow a person to create synthesized songs by themselves (or with others!), recording multiple music tracks and overlaying them.

PART 2:

A Cappella Solo Live (#56): Mainstream computer-based electronic instruments and music making machines are still few and far between; there are custom setups designed for particular artists and there are non-programmable, dedicated hardware devices that accomplish much of the same functionality, but they are usually better left to recording engineers instead of performers and certainly are an obstruction while performing on stage in front of a live audience. As several of our group members are musicians and performers, this is also something we are personally quite enthusiastic about. It is also modular enough that a sizeable chunk of functionality can be implemented in this course, and further features can be integrated later.

Alternative: Triangulation of sounds using multiple Raspberry Pis and sound sensors (#19): Initially, the idea was developed to have real-world significant applications, like triangulating gunshots in warzones or other critical sounds throughout a city, room, building, etc. It’s really interesting because, even though the technology already exists, using raspberry pis would effectively make it the cheapest available technology of its kind. This would allow the product to be used in a wide-range of applications previously not possible!

PART 3:

Target audience: professional musicians, amateur musicians, “Youtube artists,” and other Kinect owners who want a live, interactive synthesizer and looper for experimentation. With an unobtrusive, floor-mounted Kinect connected to a small laptop (that can be used as a monitoring screen) in front of the user, the system can be used in concert halls as a live performance tool. Alternatively, the same system set up in a living room (perhaps connected to a home theatre PC) allows for an entertaining quasi-game, and an easy way to record and create music for publishing online. This could gain traction among the many talented Youtube artists who incorporate various technologies to create great covers and original music.

Problem description/context: Looper-synthesizers (usually used with guitars or synth tracks) exist in several forms today, including iOS apps and hardware devices, but the iOS take on loopers (VoiceJam) is very much an experimentational and maybe recording tool instead of a live performance capable setup, and hardware-based loopers are specialized equipment that present a pretty steep learning curve for those not well versed in how they work; they are also quite expensive. However, they are great fun, and we expect that many people would love to have the functionality if presented in an intuitive way that requires little specialized equipment. Thus the barrier to entry is lowered by using the Kinect + computer based approach, and the experience of interacting with the machine is less a distraction from the performance and more an integral part of the choreography itself.

Appropriate technology: The modularity and flexibility of this system means that it can be used to perform live (laptop + Kinect), practice and experiment in a living room (HTPC + TV + Kinect), or any of a variety of other configurations. Kinect is small and highly portable, making it trivially easy to set up at performance venues (especially compared to things like speakers, stage extensions, etc). Implementing all of the backend logic in software allows us to add functionality as we choose, and the Kinect is a relatively inexpensive I/O device that is also perfectly suited to the job; it can track skeleton movements as well as z-axis movement for easier and more precise gesture analysis, which we wouldn’t get from a normal camera. Finally, we can include Arduino-powered elements that also interface with the same computer running the backend software, which would allow us to add functionality that the Kinect can’t handle, such as individual finger flex movements.

Possible sketches:

Kinect vision — what the kinect sees, multiple areas of the screen correspond to multiple voices/modulations.
One person can sing with themselves; the system records the sounds at each section of the screen and modulates based on the body movements, overlaying each section — jumping, kicking, arm movements, different voice sounds, more dancing and singing

Don’t Worry About It

Posted on February 22, 2013 by Amy Zhou

Members: Dan Chyan, Thomas Truongchau, Jonathan Neilan, Krithin Sitaram, Amy Zhou

Team Name: Don’t Worry About It

Here’s our list of ideas and problems to solve. Ideas that were the result of building on a previous idea are listed as subbullets.

Losing items (tags, training, warning for ‘out of order.) / forget proxes, keys, etc.
No date for Valentine.s Day/Speed Dating (compatibility tools, remotely trigger incidental things) (note: we compiled this list on Feb. 13)
Not knowing how to approach a girl (text about relevant details)
Don’t know when there is no reception/signal on a phone, etc.
Don’t know when your phone has vibrated
Temperature control
Not knowing point A to point B, or the travel time of point A to point B
- arrows on brim of hat, accustoms to your avg. walking speed)
Waking up snoring kids in lecture
Non first-world-problems:

Monsoons destroying farms (either prevention OR facilitating repairs)
Political corruption
Transporting water over large distances
- Embedded water quality sensor

Waiting for elevators/Call elevators earlier
Folding clothes
Typing forces you to find a flat surface and use both hands — egg-shaped keyboard in your pocket?
Kinect teaching clothes matching/right fashion
Printing/reminders to do tasks in certain areas as you pass by them
Measuring awkwardness level
Discovering who’s a mean person
Selfadapting clothes (to temperature, i.e. ski pants with auto venting)
Lying down but laptop doesn’t go sideways
Detect when the refrigerator/cupboard/sock drawer is empty
- for that matter, it’s sometimes unclear if the fridge is working at all
Bad singers are really annoying — it’d be nice to automatically autotune everyone around you
- Getting a stuck song out of head
Not knowing when subway/transportation shows up
How much money left on a metro card
Clothes online: don’t fit (kinect, shoes) or DIY measuring of your body
Staying awake in class head nod/accelerometer, need info on how people fall asleep, have the target user be the lecturer?)
Waking up for lecture (sleep cycles?)
Starting to rain, leave now or 5 minutes from now?
- not knowing whether to bring an umbrella — have the umbrella stand flash when it’s about to rain?
Bad drivers/ anything to dissipate road rage
Reading a book on a treadmill (have book bob up and down with you/display 1 word at a time)
Remembering people’s names
Way to trigger something remotely — ChemE friend needs to walk all the way to the E-Quad just to turn a knob
Timers on events
Knowing when things are done baking — keep jabbing until food is done? thermometers? scent measuring?

Microwaves too — alert you when your meal is heated all the way through
Don’t want to burn the house down by accidentally leaving the oven on overnight(every 1 hour the oven is on, alert?)
Drunk people (noise too loud call Psafe)
Watch/clock changes (emoticon faces?) based on time of day or if you’re late for scheduled events (programmed into it, or it is synced with your google calendar?)
New GPS system interface involving glove with LED lights (as opposed to using your smart phone)
Telemedicine/using a machine for testing physical ailments (i.e. arm sock + pressure sensors)
Wanting hugs
Maintaining longdistance relationships in general
Health issue, avoid germs spread via contact by using gestures to open doors, windows, cabinets/Specific gestures as a password. ESPECIALLY BATHROOMS!!!
Dance to turn off alarm (shake your booty, oh yeah)
For dance groups, being able to mark lines and formations (especially moving ones) with something other than tape on the floor (maybe LEDs embedded in the floor you can activate somehow, or projector on ceiling)
For joggers/cyclists, being able to trace out circular routes for training (current map apps just calculate quickest or shortest route from point A to point B.
For sports teams, training glasses(?) used as a screen that show the proper form/followthrough for certain actions (i.e. being able to view a shadow or outline of someone doing a proper golf swing or tennis swing) using a kinect perhaps?
A fridge that you can program what groceries go in, and based on how often you buy certain items, will inform you of where/what nearby stores those foodstuffs are on sale at!
A phone with basic functions controlled by eye movement for when you.re in class or wearing gloves
For dance groups or any interested party, all dancers have earphones/headphones and being able to sync up all the dancers with the same music; distributed realtime playback of music/videos/etc. on personal, discrete device
Sharing multimedia experiences discreetly on discrete devices. (i.e. a series of tables have ‘domes’ or ‘transmitters’, as you get closer to a ‘dome’ or table, your personal device ‘tunes in’ to whatever video/channel/music that table is currently playing or set to. If you like it, you stay, if you don’t, you can jump to another table; alternatively you could share material and view shared material with all users within a dome area)

Flashing words one-at-a-time at very large font size on treadmill, to work around bouncing problem

Chorded keyboard to use one-handedly without looking and without flat surface, eg texting from inside pocket

Occasionally, students fall asleep in lecture and snore. Wouldn’t it be great if we could detect that and zap them awake automatically?

Project lines and formations onto the floor, because anyone in a dance group knows that marking up the floor with masking tape is a pain

Direction-finding without looking at a map! We’ve illustrated several potential ideas: a “magic compass” on your watch, arrows on the underside of the brim of your hat, a vibrating belt…

This pod broadcasts media to everyone around it, on their own screen.

The idea that we chose to work on was a hands-free method of providing directions. We think this is something that has wide applicability and is also a problem that we have personally experienced. In addition, the problem seemed of an appropriate size for a semester project, and had the benefit that after we framed the problem we saw several very different approaches to solving it that seemed viable, so there is room to explore.

Target User Group:

Our target users are people using smartphones to find their way around an unfamiliar city while their hands are busy.

Problem description:

The normal way people use smartphones to find directions raises two problems. Firstly, the user needs to hold the phone in his hand, which makes it difficult to perform tasks that need the use of both hands, and might also be uncomfortable if the user is outdoors in a cold climate. Secondly, the phone also demands the user’s visual attention, which could be distracting at best, preventing him from enjoying the sights of the place he is visiting, and possibly even dangerous, since pedestrians using it might not notice hazards like oncoming vehicles while crossing a road. In addition, obvious displays of wealth (such as a smartphone) are bad ideas when traveling, if the user is worried about theft. Our solution will need to be lightweight, because it will often be used when traveling in a foreign place, possibly having arrived via airplane; it will have to stand up to reasonable amounts of jostling, moisture, and pressure; it needs to be as close to absolutely failproof as possible or at the very least fail gracefully in a way such that the user can still find his way around; it needs to be intuitive to use, since users will probably not want to put in substantial learning time to use a system that they may not use very often, and the entire point is to put direction-finding in the background of the user’s mind, so that they can better enjoy the place they’re exploring; and it will need to be cheap and easy enough to make that we can iterate quickly — anything requiring large amounts of custom hardware that we cannot make ourselves will not work.Previous solutions to the problem include audio prompts (which are unintuitive, slow, and can block out outside sounds).

Platform

The key element of our proposed solution to this problem is a belt with many small buzzers embedded along its length, inspired by the haptic belt from Stanford’s AI lab. The buzzers along the belt will be controlled by an Arduino which in turn receives direction information from the user’s smartphone over a wireless link. This will be paired with an app in the user’s phone that will determine the user’s current location and bearing using GPS and then obtain routes from a publicly available mapping service and send signals to the Arduino to activate the appropriate buzzer on the belt to indicate the direction the user needs to move in. This is an appropriate solution because it is lightweight and simple, and an Arduino is really the only feasibly way to provide tactile output, which seems to be the best way to convey this information unobtrusively. Because the Arduino can’t download directions on its own, a smartphone is the best way to add extra CPU power, internet access (including access to an online direction-finding API), and GPS coordinates. We are also considering a number of other possible places to position the vibrating motors on the user’s body — an ankle bracelet, around the rim of a cap, one on each foot — but a belt is easy to use, much less dorky-looking than the other options, and it is in contact with the user’s body all the way around; it is also less prone to being knocked off or stepped on than other parts of the body.

The Backend Cleaning Inspectors

Posted on February 22, 2013 by Keunwoo Yu

Members

Keunwoo Peter Yu
Green Choi
Dylan Bowman
Tae Jun Ham

Ideas

1. Motion Based Coding environment (Kinect or Leapmotion)
– Uses gestures to reach, move, and otherwise manipulate code.

2. Physical Therapy (Kinect)
– Shows correct form on screen and allows user to try and mimic it
– Records best match, progress in getting better, warnings to rest when getting worse

3. Coffee Order Counter from Line (Arduino, etc.)
– Have switches that can be accessed from line that allow multiple orders to be placed
– Could be used by coffee store customers (Starbucks, etc.)

4. Security system based on gestures (Kinect)
– Use a special gesture as the key to the system.

5. Outfit recommendation system based on weather (Arduino)
– Detect the current temperature, wind, etc. and make recommendations on the day’s outfit.
– Could be used to help senior citizens go through their daily dressing routines

6. Music recommendation based on facial expressions (Kinect or webcam)
– Choose music to play based on the facial expression the user makes

7. Refrigerator camera to recommend recipes based on contents of fridge
– Recognizes ingredients in fridge and suggests recipes that could be made from those

8. Refrigerator compartmentalized tracks number of accesses to warn against obesity (Arduino)
– Could even apply to non-refrigerator – just use it where you store the most unhealthy items in your kitchen

9. Door control based on pattern or face recognition, primarily for cats and dogs (Kinect)
– Guards against strays and dirty inferior creatures like raccoons and possums

10. Attachment to measure acceleration of punches or kicks. (Wii controler)
– Measure the power and speed of punches or kicks using the Wii controler for entertainment or training purposes.

11. A device to keep you awake in class based on variable resistor.
– Requires constant change in resistance to not shock you.
– No change indicates falling asleep.

12. Presentation aide based on gestures. Kinect or accelerometer.
– Pointing causes laser on screen that observers can follow
– Swiping allows next slide or previous slide to come up

13. Presentation aide based on voice. (Arduino)
– “Next” or “Back” moves slides forward or back
– “2nd point” highlights second point on slide

14. Voice or gesture combination for pet doors or food dispensers
– Teach cat or dog certain combination of actions that allows pet-door to unlock or food to dispense

15. Garbage can attachement to detect fullness and make alarm (Arduino)
– Once garbage in the bin passes certain height, sounds an annoying alarm so that people actually empty the bin.

16. Webcam device that tells you optimal chiropractic chair height.
– Takes a picture with the webcam and see if the chair is at the optimal height.

17. Device to measure motion, lack of which will trigger alarm to keep you awake.
– Security Guards: Administers caffeine or stimulants.
– At Home: Alarm sounds.

18. LED flash light on your glasses that adjusts brightness according to environment’s light (Arduino)
– Button to turn function on and off.

19. Wireless motor to attach to existing switches using remote.
– Remote control lighting, etc. to existing switches.

20. Auto-spotlight (or fan) that tracks moving objects (Kinect)
– Pre-game functionality

21. Autofan for temperature changes or humidity changes.
– Sensor outside window tracks changes in temp/humiditiy/possibly others (precipitation)
– Changes fan speed/power in room to adjust

22. Sensor in shoes to measure how high you can jump.
– Measures the weight and the force the person exerts to calculate how high he can jump
– No need for a wall with a measurement tape.

23. Light inside bag turns on when a girl opens it and there’s no environmental light.

24. Toothbrush holder that lights up brighter and makes sound warnings if you don’t take it out for too long (like longer than a day or something). (Arduino)
– Light will get brighter, sounds get brighter and more frequent.

25. Toothbrush attachment with accelerometer that recognizes proper brushing technique and length (Arduino)
– Good for kids. Proper recognition prevents false brushing.

26. Toilet attachment that vibrates when the user has been sitting for too long. (Arduino)
-Time management for toilet gamers.

27. Bathroom hygiene circuit that makes sure faucet is turned on after flushing.
– If you flush and then don’t turn on faucet in next 30 seconds, starts buzzing annoyingly, or locks the door.

28. Security system that tracks incoming and outgoing people in a building.
– Ensures that number is even at end of day, or just # of people in building.

29. Laundry Security (Arduino, we picked this idea)
– Can tell if washer/dryer has been opened before it is done (people moving your laundry because nothing else is open)
– Possibly takes a video if it does, or buzzes alarmingly to warn them.

30. Vending machine add-on ensures product dispenses after purchase with motion sensor at the bottom.
– Prevents vending machine damage from people kicking it after stuck items.

31. RC car that follows you around with a flashlight in dark areas.
– Can help certain jobs like plumbers, etc. explore small compact areas like pipes or under houses
– Can possibly turn a different color if sensing hear or light to alert operator of its presence

32. Self-organizing chairs or other furniture objects.
– By remote positioning

33. Chair mat with sensors that tell you whether your posture is okay or not.
– If uneven left or right it beeps to tell you scholiosis alert.

34. Motion sensing screensaver functions for TV. (Kinect)
– Can tell if you fell asleep, and then turns off TV

35. Simple hand/fingertip scanning device. Paper to digital.
– Scan finger over lines of text and it comes up on the screen

36. Book reader that reads out loud as your scan in pages.

37. Remote Controller Finder (Arduino)
– A pair of devices. One is attached to the remote controller, and the other to the actual device (e.g. TV)
– When the remote controller is lost, press the button on the device.

38. Coffee/Tea Temperature Assistant
– A device you can attach to the cup of your favorite hot beverage.
– Tells you when the optimal temperature has been reached.
– AKA Thermometer 😉

39. Measuring system with fridge that measures BMI and tells you when you’re being too unhealthy.
– Locks the fridge door if you’re gaining obesity points.

40. Video game controller that has biometric ID-ing device that locks the game if you’re playing too much.
– Parents can control how much their kids play video games

41. Toilet device that measures the contents of your urine and warns you with a light if you’re dehydrated or have some kind of problem.
– Too high blood sugar, cholesterol, etc.
– Activated upon flush

42. Public laundry machine that requires your phone number or NETID to activate.
– Texts you a reminder it’s done or before it’s done
– Phone number needed to open the door until after 15 minutes after cycle
– Bills the number a late fee if you don’t show up on time!!!!

43. 3D space explorer (Kinect or leapmotion)
– Can view the solar system and beyond using hand gestures.
– Great for science classes.

44. Biometric sensor smartphone attachment that calculates levels of alcohol or other drugs and locks certain features accordingly.
– Can lock text messaging or calling for certain predetermined numbers. (Family, girls, etc.)

45. Motion based movie player. (leapmotion or Kinect)
– Use intuitive gestures to play/fast forward/rewind/pause movies or any kind of videos.
– can change volumes and stuff

46. Device that uses motion detection or accelerometer to determine when you are dozing off and begins recording sound and/or video.
– Allows sleepy people to review what they would have missed.

47. Thermostat that only opens when a hand is touching it.
-Use variable resistor; it it’s at neutral state, don’t open the lid.

48. Kinect sensor in car that recognizes specific hand motion to start the engine.
– Could be extended to locking/unlocking the doors.

49. Radio sensor that flashes warning light to indicate when police or public safety have entered the building
– Could also use facial recognition to scan for faces older than college age.

50. App that allows scientists to examine the 3-D structure of atoms or molecules, etc. (Leap Motion)
– Great learning tool, would be awesome for classes

51. App for a new textbook interface on the Leap Motion
– Can turn pages, highlight, write in margins, mark favorite pages etc.
– No longer have to buy textbooks, can store all of them on your computer
– Could make it more general as a pdf reader, since most notes and textbooks come as pdfs.

Why We Picked Idea #29

We the Backend Cleaning Inspectors believe in a better world in which everyone can focus on the important things without the distraction and stress from mundane chores. This is why we chose idea #29: stressing over laundry is by far one of the most annoying chores, and we’d like to rescue people from that annoyance. With our new “Clean Safe” laundry security system, people will be able to work without having to worry about the safety of their laundry.

Project Description

Target User Group: Our broad target user group will be anyone that uses a public laundry machine. Our product will allow them to have the peace of mind that their laundry will not be disturbed. However, narrowing our focus to users that we have access to, we can say that our specific target user group for this project will be students at Princeton University who use Princeton laundry services, especially those who have had problems with their laundry being messed with (either taken out before it was done to make room for someone else’s laundry or something similar) while they were not in the laundry rooms. Our target user group wants a product that will allow them to have confidence that their laundry will not be disturbed when they leave the room. Very few people who use the laundry services at Princeton stay there and watch their laundry until it is finished. Our product will essentially do that for them.
Problem Description & Context: Our idea addresses the common problem at Princeton of strangers tampering with, moving, or even stealing other peoples’ laundry that is left in the machine after the cycle. The context is the public laundry system at Princeton in which users do not necessarily know when washed or dried laundry will be removed by its owner. This leads to people moving the laundry to potentially dirty places like the floor, where in many cases it is lost or even stolen. This problem may be affected by the time and date, as many students do their laundry before or after classes or on weekends, as well as location, as areas with fewer laundry rooms will be more frequently in demand. These factors can be accounted for statically or dynamically with machine usage data. We propose to solve this problem with an system that will enhance security, responsibility, and efficiency in the laundry room. We hope to take identifying input from the users of the laundry/dryer machines, such as Princeton NetID, phone number, etc. and use it to 1.) send them SMS, email, or other warnings at time intervals and 2.) require this information to unlock the laundry machine until a certain grace period after the cycle has completed (intervals will be calculated based on time, date, and location). This idea may be expanded or refined depending on our capabilities and other environmental factors we encounter in the process, such as adding a fine to late laundry owners.
Why Arduino? Arduino offers a good solution because our users will need to input identifying information(netid, phone number, passcode, etc.). We can accomplish this by programming necessary buttons on the Arduino platform. The attachable LCD displays will also come in handy to display remaining times, warnings, etc. Also, Arduino and its supporting programming languages allow us to send SMS and email notifications to our users. In general, Arduino’s compact size and simple construction will allow us to program a clean, user-friendly interface device to attach to existing laundry machines with minimal modifications.

Sketch

Grupo Naidy – P1 Brainstorming

Posted on February 22, 2013 by Khalid Al-Mehairi

Group

Avneesh Sarwate (asarwate@)
John Subosits (subosits@)
Joe Turchiano (jturchia@)
Kuni Nagakura (nagakura@)
Yaared Al-Mehairi (kyal@)

Brainstorming List

Spare Prox/Key Shooter (SPK) – SPK shoots a spare prox or key under your door when triggered by an iPhone.
hiLight – A highlighter that reads text that you highlight and stores it on your computer.
Snoozy – An alarm clock that uses motion sensors for snoozing capabilities (helps you control an alarm clock that is out of reach).
MotherClocker – An alarm clock that senses if you’re levels of motion (e.g. stirring or dead silent) and adjusts volume accordingly.
myBar – An interactive bar counter that lights up when you want a drink and visually notifies the bartender (kind of like in airplanes).
hiRoom – A system that tracks motion and heat in rooms and keeps track of the availability of rooms.
HandBoard – A new type of extended keyboard that makes full use of gesture control.
AbsolutelyRotten! – An alarm that detects rotten food in your fridge.
MatchFit – A color sensing camera app that detects if your clothes match or not.
HowToDressWell – An application that learns your fashion tastes and sensibilities.
GComp – An immersive gesture control computer.
GTV – An immersive gesture control TV.
Grillo – A streamlined, interactive grill ordering system for Terrace/Tower that also tells you what orders have finished being prepared.
PencilMouse – Pencil shaped mouse that you can use on your screen.
HotDamn! – Heat sensor that notifies when you’re overheating or sweating in formalwear.
mobPage – Pager for smartphone that notifies if a special person is trying to contact you (sketch below).
Pager for mobile when you are out
myMix – A system that makes your mixed drink.
ArchScan – A scanner for 3D prototypes that translates them into CAD or Rhino.
Tabelisk – A desk that incorporates a tablet computer.
PetPet – A pet feeder that can feed your pets on a timer or detect when they approach.
Scarecrow – Scarecrow that senses unwanted animals and scares them off.
SketchIt – Etch-a-sketch controller/app that works on computers.
PubPort – Interactive system for optimizing public transportation hubs/stations.
MCVolume – Motion-controlled volume controller (possibly for TV).
Rise – Light-sensor alarm clock that wakes you up when the sun comes up.
DrinkUp – A system that uses pressure sensors to detect when your drink is empty and notifies the bartender/waiter to get a new one.
EyeComp – Eyepiece computer with facial expression and voice recognition software.
DifPic – A camera that compares a picture of your room to a picture of it when clean (sketch below).
Before versus after
E-Cig – An electronic cigarette measuring smoke throughput that gives you your stats.
GGPS – A GPS system that notifies trafic patterns and indicates most efficient routes.
UrbOp – An urban system in which road and street sensors optimize traffic lights.
CarMap – A density map that shows you where cars are in a selected area.
Radar – Laser tag radar system with map (kind of like how video games display enemy position).
Empto – System for detecting decreasing drinks in restaurants.
DJ Light – A gesture controlled light show that allows DJs to use their movements for a light show (sketch below).
Gesture controlled light show
MIDI Light – A system that allows DJs to have a MIDI controlled light show.
Sound Light – A system that allows DJs to have a live-sound controlled light show.
Gestar – An enhanced guitar incorporating extra sensors to trigger effects without extra gestures.
DJ Mat – Rollout DJ mat that uses smaller sensors instead of big 3D hardware (sketch below).
Roll out DJ mat
MoodMix – A song mixer that plays music depending on mood lighting, temperature, and room humidity.
Posturrector – A posture correcting system that beeps when you slouch.
SenseForm – A mechanism that assesses the form of your squats/lifts against ideal static finish positions using flex sensors (or Kinect).
PressBox – A box squat device with pressure sensors to stop you from cheating (i.e. sitting on the box).
Juggo – Learn how to juggle a soccer ball using Kinect.
GloveForm – Martial arts glove that teaches proper form by detecting hit knuckles and wrist angle (sketch below).
Hit knuckles and wrist angle
ForceForm – Force/accuracy sensor to determine the effectiveness of your punch/kick on a bag.
MemPic – A smartphone app for students (mostly freshman) that can take pictures of buildings and know their campus location and department/facility associations.
VirtualNote – A system that incorporates a pencil that integrates handwritten notes to a virtual notepad (so it is okay to lose your hard copy).
JuggoMotion – Special juggling balls that teach you how to juggle by displaying the motion of the balls on the screen.
MindMouse – A mind-controlled mouse, using EEG caps that measure brain waves.
EnviroSense – An environment (e.g. room) with sensors that measure your sweat and body temperature and adjusts the humidity and temperature of the room accordingly.

Idea Choice

Prime Ideas

DrinkUp (#26)
SenseForm (#42)
VirtualNote (#48)
DJ Light (#35)

Our first choice for ideas (non-Kinect based) is DrinkUp – a system that notifies staff, waiters, and bartenders in a bar or restaurant if a customer’s drink needs to be refilled. Our second choice for ideas (Kinect based) is SenseForm – a system that serves as a personal weight-lifting trainer that makes use of Kinect to direct and teach users how to properly lift weights. These ideas have clearly defined problems and the ideas in themselves are not overly complicated, therefore leading to simple solutions. Below we expand on both of these ideas (however we have left out designs and sketches for our second choice).

DrinkUp: Idea #1

Target User Group

The target user groups for this technology are both patrons and staff of restaurants, bars, or virtually any other establishment where a drink might be served. The idea driving our project is to remove any active participation on the part of the patron for requesting refills. We imagine the drink and table sensors being used by patrons, who might want to let their waiter know they need a refill on their drink, or by the waiters themselves, who might want to keep track of all the drinks in the restaurant and refill them when they are empty.

Problem Description and Context

Imagine yourself in a crowded bar or restaurant. You’ve just finished your drink of choice, and your glass is now empty. However, you are still eating food of some sort, or perhaps for some reason you are just very thirsty today. Maybe you’re enjoying a conversation with a friend or a colleague and you can’t be bothered to scan the restaurant for a waiter. Unfortunately, no matter what you do, you’re having difficulty getting a refill! Maybe the bartender is at the other end of the bar at the moment and well-occupied, or perhaps your waiter at the restaurant is just nowhere to be seen. Has this ever happened to you? Our system will let you stay “in the moment” with whatever is engaging you at your eatery of choice, and stop service lags from breaking your flow of conversation. Experience the restaurant as it should be.

Justification of Technology Platform

Fear not! Imagine this scenario: there is a small touchpad on your table/bar seat. When you tap the touchpad, it sends a signal to the restaurant staff or the bartender to let them know that you’d like a refill for your drink! But let’s say that the restaurant is crowded at the moment. Even if the waiters wanted to come by and refill your drink, they are just too busy to do so. Well, it just so happens that in addition to that manual touch sensor on your table, there is a pressure sensor directly underneath your cup. This pressure sensor took note of when your drink was full, and it’s able to tell when the weight of your cup has decreased enough for it to be empty. When it detects that your cup is empty, it automatically sends a signal to the staff to let them know – and the staff maintains a computerized list of which cups in the restaurant are empty at the moment. Now the restaurant staff has a more effective method of managing empty drinks, and as a result you get yours refilled within moments rather than having to wait as long as a half hour for a waiter to notice your empty glass!
Our system is arduino-based, making use of pressure sensors to detect whether a glass is full or not, and then sending the table location to the staff computer. The system needs to be portable to accommodate different table locations as well as be able to interpret the data returned from a pressure sensor, so the arduino is ideal for that purpose. In order to actually be useful, the data must then be sent to a central location, which is why a computer application is necessary.

Designs and Sketches

Empty drinks, busy waiters, unhappy restauranters

Empty drinks, busy waiters/bartenders, unhappy restaurant/bar-goers

When below minimum pressure tolerance, signal is sent to central location

Central location gathers data for all drinks on all tables in restaurant/bar

SenseForm: Idea #2

Target User Group

This project is targeted towards anyone performing standard power lifting lifts in the gym. It will work best for lifts that have a static final position (i.e., the low-holding position in a squat). The system will tell them if they are performing the lift properly or not.

Problem Description and Context

Often, when people go to the gym to lift, they have a “gym buddy” that they can go with, one who can watch them and tell them if they are properly performing the lifts. For power lifting lifts in particular, the technique of the lift is of utmost importance. If a person goes to the gym alone, it becomes difficult to effectively determine whether one is correctly performing a lift or not. Mirrors are sometimes not available, and even when they are it is difficult to properly evaluate your form simply from a mirror view. This system could help alleviate that problem.

Justification of Technology Platform

A kinect, wekinator, and a laptop could be used to implement this system. The user could hold the final position and use these training examples to train the “finished” position on the wekinator, and then go through the motions before finishing and use these examples to train the “not finished” positions. The kinect’s skeletal tracking would be ideal for sensing the body positions. A simple footswitch could be used to turn example selection from the kinect on or off.

The Cereal Killers

Posted on February 22, 2013 by Bereket Abraham

Final Project Ideas

Bereket Abraham
Andrew Ferg
Ryan Soussan
Lauren Berdick

Part I
Final Project Ideas

1. Smart food bin:
Problem: forgetting / not knowing how much food is left in your fridge
Possible solution: special bins for certain common foods that get used a lot. Use a (pressure?) sensor to detect how much is left. A computer will then send you alerts if you are running low. Or you can look up recipes based on what you have left.

2. Automatic bartender:
Problem: the amount of liquid poured in each cup can vary a lot. We desire a system to pour standard, repeatable amounts into each cup.
Possible solution: A small reservoir tank and an arduino controlling a valve. Similar to a water cooler.
Possible solution: Same as above but now the cups travel on a convenient conveyer belt.

3. E-composer:
Problem: Writing music down on a score is annoying / difficult.
Possible Solution: create a computer program where you play an instrument and the computer transcribes the music.

4. E-composer II:
Problem: We want something to transcribe a music file, and break it into individual instrument parts.
Possible Solution: Some computer programs take in a music file and convert it into a score with notes. We just need to separate the individual instruments.

5. Smart light switch:
Problem: Hard to remember to turn off the light and leaving them on wastes electricity. It’s also annoying to turn off the lights from bed.
Possible Solution: Programmable light switch.
Possible solution: monitor a person’s light switch and then use machine learning to look for patterns. Similar to an implementation used for eco-friendly thermostats.

6. Indoor Directions:
Problem: It’s hard to make good directions for buildings and wall maps are annoying to use.
Possible Solution: Have a central screen where you input your destination. The lights in the floor will guide you to your destination. Your movements are tracked with (pressure/vibration?) sensors. Or just light up the entire path.

7. Shower Notes:
Problem: Can’t take notes during a shower.
Possible Solution: Create a waterproof iPad case. Create a way to use the touchscreen across the plastic casing. Indirect contact.

8. Networked Alarm Clock:
Problem: Several time keeping devices, including your online calendar, alarm clock, and cell phone.
Possible Solution: Program that interfaces mostly with google calendar. You can sleep until a set time or until your next class. Sets alarms on the clock and text/calls you on your phone.

9. Printed To Do List:
Problem: There are many good productivity tools online. How do they convert to the real world?
Possible Solution: Create printed post-it notes for each task or set of tasks.

10. Laundry folder
Problem: folding laundry is a pain.
Possible Solution: robot.

11. Breadboard Simulation / Avatar
Problem: Making circuits is hard.
Possible Solution: When using a breadboard, your parts are simulated on a computer. Computer can tell you stats on every wire/part.

12. Better oscilloscope
Problem: hard to use. Needs better interface.

13. Traveling Salesman
Problem: When you have a lot of destinations/errands, it is hard to plan the optimal route / order of events.
Possible Solution: event planner that takes into account opening/closing times, map directions, relative proximity, etc.

14. Digitizing notes
Problem: Lots of old notebooks are thrown away to save space.
Possible Solution: save your notes with some kind of automatic scanner. Enable searching.

15. Save energy
Problem: Several types of exercise can also produce energy.
Possible Solution: Attach a generator to a exercise bike, treadmill,

16. Dry shoes / socks
Problem: Canvas shoes and loafers get wet very easily. Uncomfortable.
Possible Solution: chemical treatment.
Possible Solution: Some kind of quick drying system.

17. Automatic tailor
Problem: Hard to find right size for online clothes shopping.
Possible Solution: special shirt / pants that finds your size.

18. Outlet finder
Problem: Hard to find a plug.
Possible Solution: detect radiation from AC current.

19. Bathroom finder.
Problem: Can never find the bathroom when you need it.
Possible Solution: App looks up the building plans and finds the nearest bathroom.
Possible Solution: tag bathroom with homing signal.

20. Lost Keys
Problem: Always losing your keys and other small objects.
Possible Solution: Close range tracker

21. Hologram
Problem: We don’t have holograms.
Possible Solution: use sound to set up standing waves. Trap gas in the nodes and shine a laser through it. Thus you get pixels.

22. Singing trainer
Problem: Hard to sing / embarrassed to practice in front of other people.
Possible Solution: program with feedback, scales, and other tools.

23. Music Updater
Problem: Hard to keep up to date with all of the musicians you like.
Possible Solution: Make an app that tracks whenever one of you artists releases a new song or album on Spotify or iTunes.

24. Outdoor refrigerator
Problem: During the winter, people heat their houses then cool them again to refrigerate food. Wastes energy.
Possible Solution: Pipe in outdoor air to cool the refrigerator. Somehow connect it to the outside world without cooling the rest of the house.

25. Robot Puppy
Problem: Usual robotic pets are stupid / unfeeling.
Possible Solution: Mod a robotic dog so that it recognizes faces and/or gestures.

26. Personalized ads
Problem: A lot of ads are not relevant.
Possible Solution: look at the clothes / other of a person to determine what types of ads to display.

27. Input automation
Problem: It’s annoying to do long sequences of clicks or enters on a computer, especially if you have to do it a lot of times.
Possible Solution: Have a program that monitors a sequences of clicks and turns them into an exe. Thus, you can simply a difficult or annoying interfaces into a series of chunks.

28. Commercial muter
Problem: TV commercials are annoying.
Possible Solution: Make a device/app that mutes ads and/or plays music. Maybe also include a short game during the commercial break.

29. Read while moving
Problem: It’s hard to read or watch movies while on the treadmill, bike, erg, etc.
Possible Solution: create a program that tracks your head and moves the display to match you.
Possible Solution: create a simple version of Google Glass to read books. Miniprojector / Mini screen?

30. Color Blind
Problem: People who are color blind might accidently mismatch clothes.
Possible Solution: Create an app that checks colors to make sure they are what you think they are. Or converts them into RGB numbers or sounds.

31. Artificial Synesthesia
Problem: It would be cool to experience colors / sounds in a new way.
Possible Solution: create a program that converts paintings into sounds or music into colors in an interesting way. Also leads to a new way to interface with music / painting.

32. Regular laptop into touchscreen
Problem: Regular laptop is not a touchscreen.
Possible Solution: use an attachable sensor to detect either finger touches or a special pen on the screen, turning any normal screen into a touchscreen.

33. Auto Tie Tyer
Problem: When tying a tie, it’s hard to judge the proper length of the end ahead of time.
Possible Solution: program measures the tie and determines how much slack you should give to get the right length.

34. Noise cancelling
Problem: Sometimes you want to play an instrument late at night.
Possible Solution: Use the same tech as noise cancelling headphones. A device at the end of your instrument will exactly cancel out its noise. Only someone in its immediate vicinity can hear it.

35. LateX Math Handwriting Recognition
Problem: It’s annoying to use LaTeX – why can’t we just write math and digitize it? but computer-typeset math is very readable and modifiable. let’s build a system that lets you interact with a LaTeX document with just a hand-edited document.

36. Gesture control TV remote
Problem: this does not exist.
Solution: Wear gesture detecting device, i.e. a camera or kinect thing. Then hack a TV remote and use gestures to control a TV.

36. Gesture control bluetooth
Problem: It’s hard to easily move files.
Solution: Wear gesture detecting device, i.e. a camera or kinect thing. Then use gestures to move files between a computer and your wearable device (possibly with cloud storage).

36. Gesture control magnets
Problem: I do not have superpowers.
Solution: Wear gesture detecting device, i.e. a camera or flex sensors. Then use gestures to control electromagnets in your gloves.

37. Surveillance Orb
Problem: It is difficult to monitor an entire room by looking at multiple camera feeds.
Solution: Stitch several cameras together and display them onto an orb (panoramic view). Then use control stick or gestures to move around in the room.

38. Surveillance Drone
Problem: Need to move to a different city, and need to apartment hunt.
Solution: Attach a camera to a robot (drone). Drive around the apartment.

39. Latex Checker
Problem: There’s no way to check the math in a latex document
Solution: Send the latex math to a math solver (like Mathematica) to make sure results are correct

40. Shortcut Recommender
Problem: Do lengthy or repetitive tasks inefficiently
Solution: Monitor computer use and automatically recommend shortcuts (keyboard, external buttons) for actions

41. Software/Hardware Recommender
Problem: We don’t know what efficient technology is available
Solution: Program to monitor computer use, recommend software and hardware devices that might help the user improve efficiency or generally improve their computing experience

42. Facial Hair Planner
Problem: Don’t know what facial hair would look awesome on face
Solution: Program to display different facial hair styles on image of user

43. Foot Pedal Interactor
Problem: We don’t use our feet when we use computers
Solution: Foot Pad/ Buttons to control actions on computer and games

44. Projectable Interactor
Problem: Some actions in software, games don’t map well to the keyboard
Solution: Have image projected on desktop, sense touches on the image and respond the the actions, change image/layout for each program

45. Guitar Ear Trainer
Problem: Hard to learn to play by ear, inefficient feedback or need a second person
Solution: Program to play notes on a guitar, listen to guitarist’s response and note mistakes and accuracy

46. Resistive Electronic Pen
Problem: Electronic pens don’t have any resistance / feedback.
Solution: Make a pen with a magnet in the tip. Underneath the surface will be an electromagnet. More strength = more attraction = more surface friction. Maybe different brushes are more resistive.

47. Repelling Electronic Pen
Problem: Electronic pens don’t have any resistance / feedback.
Solution: Same thing as above, but now the electromagnet is repelling. You would also have a way of forcing the electromagnet to be directly under the pen. Now, the pen will have a buffer directly over the surface that it cannot penetrate. But, you could push down with different strengths, which controls about an inch of z direction. Would make for a really interesting click or button.

48. Rotation Cubes
Problem: Its hard to rotate in 3D modeling or AutoCAD software.
Solution: A series of networked cubes. Each cube rotates and translates selected objects. A special cube can be assigned to rotate globally.

49. Thermal Beanie/other piece of clothing
Problem: Clothes don’t always keep you at the temperature you want
Solution: Have heat pads in clothing item that adjusts heat to keep you at desired temperature.

50. Magnetic Adjuster knob
Problem: Traditional slider knobs are boring / uninteresting.
Solution: Create a floating magnet suspended between two electromagnets. You can move the magnet up and down within the field, and the electromagnets will detect this and adjust accordingly.

51. Child Tracker
Problem: Nerve racking leaving a child alone in a room
Solution: Have the child wear a specific color and track them with cameras. Make sure they don’t go to dangerous areas.

Part II

Winner: Smart Food Bins
Why This Project: Out of our 51 ideas, this one had the most direct plan to carry it out, a very obvious utility, and easily fits within the budget. We liked the magnetic gloves and synesthesia idea as well, however, they gave difficult mechanical and creative concerns, respectively. The food pads had a clear cut implementation and utility that we felt we could create during the semester.

Part III

Smart Food Bins
Target user group: Very busy adults who have to cook for a household. This would most likely benefit families, where the parents often cook for three or more people, or roommates who cook for themselves, or co-ops who cook for a group. Their hectic lives mean that keeping track of food amounts is a very burdensome chore, and the fact that there are multiple people in the household means that it’s harder to keep track of what’s available since everyone is constantly eating and changing the amounts left. Their needs mainly center around adequately feeding their families. Their wants include saving time by reducing the number of trips to the grocery store and saving money by reducing waste.

The closest group we could find to model our target group is college students eating independently in large groups. This would include members of food cooperatives and residents of Spellman that cook together.

Problem Description: People are not always aware of the amount of groceries they have in their refrigerator or food pantry, especially when they are away from home. They might forget to buy certain items at the store and not be able to make certain recipes. Also, refrigerators and pantries can get very cluttered and disorganized. Thus, people may not be able to easily figure out they have left and what they are able to cook.

Possible Solution: We would make pads for food that would stream to a website the amount of each food that is available. We would make pads out of a durable, waterproof fabric with pressure sensors inside them to detect the food’s weight. The pads would communicate wirelessly over Bluetooth to an arduino, which would then stream data over wifi to a website. Users can login to the website to check the percent of each food that they still have. Users can label each pad on the website (i.e. eggs, milk, cereal) to indicate what it stores. The pads would have buttons to set (by measuring or by manual entry) their minimum and maximum weight, so that for example if a box of cereal is being used, the minimum weight would be the empty box, and the maximum weight would be the full box. The pads could come in different shapes and sizes.

An extension on the website could be a recipe checker, that allows the user to check what they still need in order to make a certain amount of servings for a dish. Additionally, the website could recommend dishes to make based on the ingredients that the user has.

P1 – Epple

Posted on February 22, 2013 by Andrew Boik

Group 21 – Epple

Andrew Boik (aboik@)
Brian Huang (bwhuang@)
Kevin Lee (kevinlee@)
Saswathi Natta (snatta@)

Brainstorming

Head tracking for use in dynamic crosstalk cancellation for 3D binaural audio
[Expansion on previous idea] Moving speakers and display to sweet spot you as you move around the room
Sign language to English translator
1. can translate to different languages, not just english
2. maybe as a design plan, put indicators on fingertips to make it easier to identify and track different finger positions ← can use this as a design alternate
3. May also do flag semaphores.
  1. Could actually be really useful for ships
[Expansion on previous idea] Sign language teacher ← similar to above but with error correction!
Chair that alerts you when you have bad posture
1. design would need lots of pressure sensors embedded in a chair – might be difficult mechanically for our group
Table on wheels that follows you
Take a picture from far away with a gesture instead of timer
Device to help you read in bed
1. Gestures or button to flip a page
2. One hand controller might be easier and more comfortable
3. Extensible to music (idea is basically for seeking through book without hands)
Alarm for if you look like you fell asleep while driving (or in office)
[Expansion on previous idea] Alarm clock that will ring if you stay or go back to bed
[Expansion on previous idea] Alarm that warns you if you are about to bump into something in the dark
[Expansion on previous idea] Alarm that sounds when a person falls, aimed at saving elderly
Cheap paper keyboard and use device to actually detect keystrokes
[Expansion on previous idea] Play air piano and actually generate sound…
1. a potential design for this would require a projector, maybe like a table top computer
Device to automatically turn off stove (or electronic device) when you are not present for a long time
1. default: maybe after 30 minutes, but have a button or something that would allow you to disable it if you decide to, say, simmer something for several hours
[Expansion on previous idea] Device to detect if you left the garage door/car door on/unlocked (or the toilet seat up!)
Use gestures as a universal remote control for your entertainment center
Use gestures to smoothly move through pdf or slides
Detector for raised hands used for fast polling (or alert professor that someone has a question)
Use gestures to conduct a virtual orchestra that will play related music
Create a robot servant that will identify different objects and bring them to you based on your command
1. Design: involves a camera, a claw to grab object, possibly an arm to move objects out of the way and a motor as well as movement vehicle that is stable enough to pick up simple objects
Scanning system to detect dirt on a surface
Object finding device
1. plays a sound to show location
2. shows on a screen where object is in the room
3. Design: would require a tracker to be placed on each object and involves wireless technology
Digital golf swing critiquer.
[Expansion on previous idea] Augmented tennis ball machine that can identify and drill your weakest strokes to improve practice.
1. Use similar idea to help you improve your baseball skills
[Expansion on previous idea] A digital personal trainer to watch your weight lifting form (and give encouragement! RAH!)
[Expansion on previous idea] A tracking digital coach that can critique swimming stroke form (would have to move with swimmer).
[Expansion on idea 26] Digital coach to watch you exercise, and then aggregate actions to determine what muscles might have experienced the most strain, and recommend a battery of stretches that target it.
1. Fancy treadmill or other exercise machine that, in addition to having a specialized routine that targets a certain group of muscles, also recommends stretches for it.
2. Design: would need complex systems of determining movement if it is kinect based. otherwise, would need complex systems of pressure sensors in exercise equipment to determine which muscles were used and need to be stretched
[Expansion on previous idea] An alternative to prescribing stretches is to use systems of pressure sensors to see if the user is doing the exercise correctly depending on where there is more or less pressure on the exercise equipment.
An instrument monitor that would provide feedback on when it has warmed to room temperature (helps a great deal for tuning, especially woodwind and brass instruments), or at least a stable temperature (if playing an instrument in a cold place, for example).
[Expansion on previous idea] An instrument tuner that would dynamically tune certain instruments (like a trumpet) using a tuner and robotic hand to manually move tuning slide.
Prevent cat from scratching sofa (more generally, prevent a pet from doing something bad) by detecting the pet scratching the furniture based on patterns of movement and sounding an alarm or sending a text to the owner.
[Expansion on previous idea] would detect if pet has a certain pattern of movement to signify that it needs/wants its owner back if they are outside the house and signals the owner of this. Similar to a baby monitor, or like a cellphone idea for pets to use
Child/pet/prisoner detector that detects how “lost” target is based on distance/obstacles between target and device.
1. Design: would have two modules. One is the base device with an alarm and a wireless receiver and the other would be the module that goes with the child/pet/prisoner with a transmitter and a circuit to calculate how far away it is from the base. If we want large distances, we can use GPS based technology, for small distance we can use signal strength from base or proximity to some preset border.
Detect how long the line outside a restaurant is.
Use a temperature sensor to make doors that are not double doors lock if temperature is below a certain level
1. Idea inspired by observing students being annoyed by the equad single doors being used in the wintertime, letting in cold wind instead of taking two steps to open the double doors
2. Design: detect temperature and wind level outside and enable automatic door locking mechanism
A group tracker and interface that will keep a tour group together – maybe point in direction of the tour guide if the person gets lost or play a noise.
A noise detector that plays an annoying alarm if a roommate plays music too loudly.
A student tracker for detecting whether students are bored in class – note nodding heads, facebook checking, etc. Mainly intended for use in large classes.
Pseudo-GPS system that uses Wi-fi and sensors like accelerometers to provide position tracking via dead-reckoning
1. Map new areas with sensors, and record relative positioning. Never get lost in a new place again!
Use level bar and adjustable legs to automatically level a pool table
Stalker detector that tells you if it looks like you’re being followed
Robot arms controller that looks at your arm motions and replicates it with a pair of robot arms
1. can be used for long distance control (like surgery from a different hospital)
2. or perhaps scale down movement so that large actions translate to small ones (for microscopic work?)
3. or scale up actions? Sounds like big robots now…
4. maybe try reversing actions to help, say, put on makeup
Use kinect to rotate virtual 3d objects
1. Use similar system to reorganize/redecorate a room
Device to check if a person limit is exceeded
Virtual tour where your leg movement is tracked and used to move through a displayed virtual 3d space
Head movement tracker that will allow for exploration of a remote environment according to your head movements and displays the image in that direction on a display.
Presentation trainer that will warn you if you are fidgeting or have other bad habits
Suit for providing remote tactile feedback
Smart window blinds controller that raises and lowers based on detected ambient light (may need an array to be accurate and to keep unscrupulous types from seeing inside by shining a flashlight at the window).
Motile scarecrow (or scaredog for those annoying Canada Geese?) that will move suddenly and/or make noise if pests get too close.
Recording device for creating beats from ambient noise.
Karaoke song selector (takes input from ambient noise/sound to select songs based on “mood”).
Automatic gardener: water plants when necessary (if it’s been very sunny and dry, or plants look dehydrated)
Virtual environment simulator that will map movements and show on screen a corresponding superpower. Such as a flick of the wrist will show your image on screen with a spiderman type web. (inspired by the movie BOLT) (this has practical use. We promise)
Virtual reality for providing virtual friends for lonely people (while pretending it’s a multiplayer game?).

Project Description

Choice 1:
Idea 47: Kinect based head movement tracker that will allow for exploration of a remote environment according to your head movements and pans around to display the image in that direction on a display.

We picked this amongst all the possibilities in the list because it has a clear application for us and what we are studying in terms of using HCI to break past the stagnant interface that is the mouse and keyboard. It involves a more natural way of computer interaction as it allows for humans to intuitively turn their heads and have an imaginary camera replicate this movement to show what one wants to see in a remote environment. This can be accomplished through panning around a panoramic image or actually controlling a rotating video camera through Kinect. It also involves a complex enough challenge in head tracking that it is not trivial.

Target users: Our project may be broadly aimed towards anyone who wants to remotely visualize an environment. However, for purposes of our project, we specifically identify people who wish to travel but do not have the time or funds to do so. These users seek a means for exploring a space as naturally and intuitively as possible (the stronger the sense of immersion, the better) and need a system that can bring the sights and sounds to the area to them with the least sense of artifice possible. Possible applications that we could apply this to include Google Streetview and virtual tours.

Problem description and context: Currently, exploring remote environments through applications such as Google Streetview rely on the clunky interface that is the mouse and keyboard. Unfortunately, this is an unintuitive tool that often causes people to awkwardly explore an environment through clicking arrows and pressing keys. This makes true immersion impossible. The goal of our project is to create an interface to make remote viewing of an environment much more intuitive. We aim to improve the situation by replacing the mouse and keyboard with Kinect-based head tracking used to pan around the environment. The image of the environment will then be displayed on a mobile display so as to always keep the display in view of the user. Aspects that may influence the problem solution include the demand for viewpoints; that is, each user will require a unique viewer of the environment that will respond only to his/her movements. If there is high demand, then an easily scaled solution may be necessary, such as virtualizing the experience into an exploration of a panoramic image. Panoramic images are easily supplied as anyone can take a panoramic image and upload it for multiple users to concurrently use. By having others upload panoramic images, this also allows us to solve a time and location problem, as we will not have to actually travel to the viewpoint to get the image. Our user group is characterized by little available time and money. We thus will want to aim to keep the cost of accessing our tool low. Related solutions to this problem include the Oculus Rift; however, the Oculus Rift is at this point an unproven technology and is far from being easily obtainable.

Technology platform choice: We choose to use the Kinect as it offers the necessary support for head tracking which we need to control the exploration of our remote environment with. We also choose to display the scenery on a mobile display such as a laptop screen, iPad, or smartphone held by the user as it allows us to change the view of the scenery while keeping the display in view of the user.

Sketches:

Choice 2 (non-Kinect backup):
Idea 10: Intelligent alarm clock controlled by pressure sensors on a bed, refusing to turn off until the owner gets up off the bed.

Although this is a simple idea, we think it would be very practical. Actually, we would really like to have this and wonder why no one has done this before. Our target user group is people who have trouble waking up in the morning. We’re specifically interested in users who frequently catch themselves sleeping through alarms and/or turning them off without remembering doing so. Conventional alarm clocks are not sufficient for these users because they are so easy to ignore or turn off. What they need is something more persistent, but intelligently so. An alarm that is excessively difficult to turn off will indeed force these users to get up in the desired situations, but in others (for example, if the user has been up all night, has already woken up, or is sleeping elsewhere), the system is an annoyance. These users want an alarm that is persistent only when it needs to be–that is, have sensitivity for context. We can always add features to enhance this device in case it is too simple (automatic prompt for alarm when body detected, context-aware alarm types, or daylight tracking).

Human-Computer Interface Technology

Princeton University

Monthly Archives: February 2013