Group 10 — Team X

-Junjun Chen (junjunc),

-Osman Khwaja (okhwaja),

-Igor Zabukovec (iz),

-(av)

Summary:

A “Kinect Jukebox” that lets you control music using gestures.

Introduction:

Our project aims to pro­vide a way for dancers to inter­act with recorded music through ges­ture recog­ni­tion. By using a Kinect, we can elim­i­nate any need for the dancers to press but­tons, speak com­mands, or gen­er­ally inter­rupt their move­ment when they want to mod­ify the music’s play­back in some way. Our moti­va­tion for devel­op­ing this sys­tem is twofold: first of all, it can be used to make prac­tice rou­tines for dancers more effi­cient; sec­ond of all, it will have the poten­tial to be inte­grated into impro­visatory dance per­for­mances, as the ges­tural con­trol can be seam­lessly included as part of the dancer’s movement. In this experiment, we want to use three tasks to determine how well our system accomplishes those goals, by measuring the general frustration level of users as they use our system, the level of difficulty they may have with picking up our system, and how well our system does in recognizing gestures and responding to them.

Implementation and Improvements

P5: https://blogs.princeton.edu/humancomputerinterface/2013/04/22/p5-group-10-team-x/

Changes:

• Implemented the connection between the Kinect gesture recognition and music processing components using OSC messages which we were not able to do for P5. This means that we did not need to use any wizard of oz techniques for P6 testing.

• Implemented the GUI for selecting music (this was just a mock up in P5).

Method:

Participants:

The participants were selected at random (from Brown Hall). We pulled people aside, and asked if they had any dancing experience. We tried to select users who had experience, but since we want our system to be intuitive and useful for dancers of all levels, we did not require them to have too much experience.

Apparatus:

The equipment used included a Kinect, a laptop, and speakers. The test was conducted in a room in Brown Hall, where there was privacy, as well as a large clear area for the dancers.

Tasks:

1 (Easy). The first task we have chosen to support is the ability to play and pause music with specific gestures. This is our easy task, and we’ve found through testing that it is a good way to introduce users to our system.

2 (Medium). We changed the second task from “setting breakpoints” to “choosing music” as we wanted to make sure to include our GUI in the testing. We thought that the first and third task was adequate for testing the gesture control (especially since the gesture control components are still being modified). We had participants go through the process of selecting a song using our graphic interface.

3 (Hard). The third task is to be able to change the speed of the music on the fly. We had originally wanted to have the music just follow the user’s moves, but we found that the gesture recognition for this would have to be incredibly accurate for that to be useful (and not just frustrating). So, instead, the speed of the music will just be controlled with gestures (for speeding up, slowing down, and returning to normal).

Procedure:

For test­ing, we had each user come into the room and do the tasks. We first read the general description script to the user, then gave them the informed consent form, and asked for verbal consent. We then showed them a short demo. We then asked if they had any ques­tions and clar­i­fied any issues. After that, we fol­lowed the task scripts in increas­ing order of dif­fi­culty. We used the eas­i­est task, set­ting and using pause and play ges­tures, to help the user get used to the idea. This way, the users were more com­fort­able with the harder tasks (which we wanted the most feed­back on). While the users were completing the tasks, we took notes on general observations, but also measured the statistics described below. At the end, we had them fill out the brief post-test survey.

Test Measures:

Besides making general observations as the users performed our tasks, we measured the following:

• Frustration Level (1 to 5, judged by body language, facial expressions, etc.), as the main purpose of our project is to make the processes of controlling music while practicing as easy as possible.

• Initial pickup – how many questions were asked at the beginning? Did the user figure out how to use the system? We wanted to see how intuitive and easy to use our system was, and whether we explained our tasks well.

• Number of times reviewing gestures – (how many times did they have to look back at screen?) We wanted to see how well preset gestures would work (is it easy to convey a gesture to a user?)

• Attempts per gesture  (APG) – The number of tries it took the user to get the gesture to work. We wanted to see how difficult it would be for users to copy gestures and for the system to recognize those gestures, since getting gestures to be recognized is the integral part of the system.

• Success – (0 or 1) Did the user complete the task?

Results and Discussion

Here are statistical averages for the parameters measured for each task.

From these statistics, we found that APG for each tasks was relatively high, and high APGs correlated with higher frustration levels. Also, unfortunately, the APG didn’t seem to go down with practice (from task 1 to task 3, and with the 3 different gestures in tasks 3). The tasks with the highest APG (10+ on normalize speed, and 8 on speed up) both happened towards the end of the sessions.

These issues seem to stem from misinterpretation of the details of the gesture on the screen (users tried to mirror the gestures, when in fact the gestures should have been performed in the opposite direction), as well as general issues in the consistency of the gesture recognition. All three users found the idea interesting, but as users 1 and 3 commented on, gesture recognition needs to be robust for the system to be useful.

Changes we plan to make include a better visualization of the gestures, so that users have less trouble following them. Based on the trouble some of our users had with certain gestures, we’ve decided that custom gestures are an important part of the system (users would be able to set gestures that they are more comfortable with, and will be able to reproduce more readily). This should alleviate some of the user errors we saw in testing. On the technical side, we also have to make our gesture recognition system more robust.

Appendices

i. Items Provided / Read to Participants:

• Consent Form

• Introductory Demo Script

• Task 1 Demo Script

• Task 2 Demo Script

• Task 3 Demo Script

• Post-Task Questionnaire

ii. Raw Data

Links to Participant Test Results:

• Participant 1 Results

• Participant 2 Results

• Participant 3 Results

Individual Statistics:

Participant 1:

Participant 2:

Participant 3:

Sam, Yared, Connie, Alejandro

i. Major problems that we found include:

• Inconsistencies in the left-hand menus (H4). We can fix this by keeping a left-hand menu present the whole time, rather than changing it constantly (this would also be consistent with the Finder).
• Difficulty finding the Help file in Windows, and Help doesn’t occur while not online (H10). Have a question mark icon in the upper left-hand bar.
• Inconsistency between top-bar menus (Music, Videos, etc.) (H4). Have “List” do the same thing in both, add an “Unplayed” option under Music, parallel to “Unwatched”.
• The status window is used for all statuses, and will block out other information (H1). This can be fixed by splitting the status window into  several windows with specific purposes.
• The user’s can’t stop the “Up Next” playlist from by default adding the next songs on a list to it. (H3). We could simply have this be something that the user can deactivate through the Preferences menu.

ii.

• We wouldn’t necessarily have looked for “help” if we hadn’t been told to.
• Most of the problems we found were issues that we encountered throughout extended use of the program in a normal setting. The challenge was then to see under which Nielsen heuristic these issues fit. In that sense, the Nielsen heuristics made us think more about what category these problems fit under and how we could change them.

iii.

• Many users found “problems” related to the system changing form generation to generation. These are more about getting used to the changes than real “problems”, and do not fit within the Nielsen heuristics.
• A heuristic we could use is “Generational Consistency” and “Cross-Platform Consistency”. 

iv.

• Name five Nielsen heuristics.
• How important do you think it is to keep programs consistent from one generation to the next? Should this be considered when judging the heuristics of a program?

Links to our PDFs:

Alejandro: https://www.dropbox.com/s/qobyn9i3zmoe59k/av_notes.pdf

Connie: https://docs.google.com/file/d/0B6ZddJLaXAx7N3duWUx5WVhRVTQ/edit?usp=sharing

Yared: https://www.dropbox.com/s/9g7frrclv2wc9hi/Heuristic%20Evaluation.pdf

Sam Payne: https://www.dropbox.com/s/9qgkodjdrnasy9u/SamPayneA3.pdf

Group 10:  “Team X”

Osman Khwaja (okhwaja)
Junjun Chen (junjunc)
Igor Zabukovec (iz)
(av)

Group Members:

We all observed dance rehearsals together, and afterwards met and discussed what features we’d want our project to have. Alejandro did some more in depth interviews of dancers, Junjun and Osman took care of the storyboarding part, Igor sketched out a user interface.

Problem and Solution Overview

We are addressing the problem of a dancer having to go back and forth to whatever device they are using to play music while practicing or choreographing, which results in inefficiency. Our system will recognized user-defined gestures to start or stop the music, to loop a section of the music, and to speed it up or down, so that they can control these parameters remotely, while retaining complete control over their movement.

Users Observed in Contextual Inquiry

Our target user group is dancers who practice or choreograph using previously recorded music. Obviously, if dancers do not use recorded music, then they do not need to control it. Furthermore, we did not feel the need to limit our system to specific styles of dance, because it can apply to all dance where recorded music is used. We conducted contextual inquiry on three different dancers. Dancers 1 and 2 are seniors in the Comparative Literature department and both are pursuing Certificates in Dance. They both do contemporary styles of dance and Dancer 2 also works as a choreographer. Dancer 1 is interested in the relationship between dance, theatre and music.  Dancer 3 is a graduate student in political economy who danced professionally for several years before going to college, primarily hip hop and some contemporary, and continues to dance for her own enjoyment. All three seem very interested in improvised dance and contemporary choreography, and as described they have different levels of commitment to dance.

Interview Descriptions

We observed Dancer 2 in rehearsals in New South Dance Studios for a thesis production, Dancer 3 practicing and improvising to music by herself, and Dancer 1 practicing for a dance sequence in a play. We watched these rehearsals taking place, asked the dancers how they felt that their movements connected to the music that they were using, watched how often they felt the need to change how the music was playing, and watched how they undertook that task. We interviewed them about their general practice routine, with a variety of questions that aimed to have them propose ways in which they thought it could be improved. We then described our design idea to them and asked them more specifically what they thought of it and how they thought it might fit into their practice and choreographing routines.

A lot of what we observed and were told by the dancers was fairly unexpected. Although it was clear that starting/stopping music was necessary, overall it seemed that they used those moments to rest or think more carefully about their moves, or to discuss what they were doing with someone else if they were not dancing alone. Furthermore, when asking them they said they felt that this was only somewhat annoying, and that it was a natural and important part of their practice routine. It seemed obvious to us that, although the system might improve efficiency, this wasn’t something crucial for dancers and therefore would not necessarily interesting them enough in order to have them adopt a new system.

When we explicitly discussed the system with them, all three of the dancers agreed that it would be a useful and interesting system to use. However, they seemed to be interested in using it in different ways. Dancer 1 thought that it would be most interesting as an actual performance tool, something with which a dancer would interact with in a live setting, rather than a practice setting. Dancer 2 thought it would a good tool for practicing but did not think it would be interesting for choreographing, as she prefers to choreograph without music, and then see how the music will fit with the dance (this is the same that we observed when watching her rehearsal: it was clear that the music was added afterward, and it didn’t seem like our system would be particularly applicable in that case). Like Dancer 1, she thought it would be most interesting when applied to a live setting, which would allow for dancers to interact with a computer playing recorded sound in a similar way that they could interact with live musicians. Dancer 3 suggested that it would be good both as a both a practice and choreographing tool, and thought that it could even be extended to be used to edit music on the fly when trying to develop both the dance and musical aspects of a performance.

In conclusion, these observations led us to see our project’s development in the following way: we will start from a practical perspective, where a dancer can use the system to practice, and the system will accomplish the very specific tasks that we will describe later. However, we hope that this will be the basis of what can then become an interactive system which can be introduced into improvised dance performances, allowing dancers to interact with the music on the fly, something that they seemed to think would be even more valuable than the improvement of the practice routine that we have in mind as a starting point.

Task Analysis Questions

1. Who is going to use system?
   Dancers and people who are trying to teach dancers who are trying to practice or choreograph their dance to a recorded song.
2. What tasks do they now perform?
   Currently, dancers use some sort of jukebox/stereo/computer to play their recording. They start the song from the beginning or some relevant point and then get into position. The music plays at its normal pace and they dance. When they mess up, they have to head over to the computer and start it again. This process repeats
3. What tasks are desired?
   Dancers want to be able to play a relevant part of music that they’re trying to learn and practice until they mess up. If they do, they want an easy way to go back and restart the music from that point. Additionally, if the music or moves are too fast to learn at the normal pace, they could use some way to slow down the music.
4. How are the tasks learned?
   The process of creating a methodology of learning how to dance basically comes down to one’s personal experience trying to learn dances. There isn’t a formalized approach that all dancers adhere to.
5. Where are the tasks performed?
   Normally performed in a dance studio
6. What’s the relationship between user & data?
   There isn’t necessarily user data created by the process of learning to dance. Optionally, one can create a video of the dance and watch it, but that’s not always necessary because dance studios usually have mirrors
7. What other tools does the user have?
   Normally the user has a stereo, iPod, computer, or something that can play music and allows them to start and restart it
8. How do users communicate with each other?
   There isn’t a relevant user to user interaction that needs to be addressed
9. How often are the tasks performed?
   Normally, Dancers practice dances with a show upcoming. Often, they also just do it for fun. Dancers practice as often as necessary to learn the dance.
10. What are the time constraints on the tasks?
    Dancers normally have a limited amount of time to learn a dance (before a show or something) so they want to have the most efficient practice.
11. What happens when things go wrong? (Pretty sure this question is not relevant)
    It is rarely the case that there is some sort of problem with the music playing that causes the dance practice to come to a halt. I guess if something happens, then they have to reboot, find a new way to play music, or just reschedule the dance practice. I don’t think this question is very relevant to our specific inquiry

 Description of three tasks

1. (Easy) The user would be able to stop and start the music. Currently, the user would have to either: have someone else control the music, walk over to a computer or other sound system and push start/stop (with a delay to get into back into position), or control the music with a remote (which they would have to carry with them, hindering their movements). With our system, there would be gesture and voice commands for pause and play. Ideally, the system would also know to pause the music when the user stops dancing, and start when the user starts. This would make this task incredibly easy with the proposed system, as user would not have to think about the task at all.
2. (Medium) The user would be able to repeat sections of the music. When practicing anything, repetition is key, and dancers often have to repeat sections of a dance to learn it. Currently, to go back to a section, the user would need to press buttons to skip to the proper section, or if the music isn’t divided into sections, they may have to fast forward or backward to seek to the proper place in the song. It is difficult to stop the music, seek to the right place, and then start it. With our proposed system, the dancer would be able to set “breakpoints” in the music with gestures while running through it, and then go back to that point later. This would make the process much easier.
3. (Hard) The speed of the music would change to follow the dancer.  Currently, there is no way to do this on the spot. Either the speed has to be changed beforehand, with some software, or the dancer has to follow the music (perhaps stopping and rewinding a section many times until they can follow that tempo). With our system, the speed of the music would changed based on the dancer. This would give dancers who are just learning the choreography an easier, and perhaps less frustrating, way of practicing with music, and it would give a dancer who knows the choreography more freedom of expression. It would be very easy to do on our system, as it could happen automatically (the user would indicate beforehand whether they want this to happen).

Interface Design

TEXT DESCRIPTION

Our system would provide a way for dancers to interact with music without having to think about it. The system would provide defined gesture and voice commands for the user to use to stop, start, and repeat a section of music, but additionally, system would also try to automatically follow the dancer’s intentions without additional gestures (the music will stop when the dancer stops moving, etc). The system would also allow the user to set the pace of the music, by following the dancer’s steps. All the user would have to do is tell the system to enter this “follow” mode, and the rest would be automatic. This allows the user to basically focus on their practice without having to think about the technology. There are existing systems for interacting with music, including keys on a computer, remotes, and even voice commands, but what makes our idea different it would be more “intelligent” and responsive, making the routine more fluid and efficent. Some others systems similar to ours are those which use the connect to interact with music. One example is a system that translates a user’s moves into music (http://www.mendeley.com/catalog/creating-musical-expression-using-kinect/). Our system would be different in that it focuses on practicing: the user group and aim is different.

STORYBOARDS

TASK 1:

TASK 2:

TASK 3:

SKETCHES

Group 10:

• Osman Khwaja (okhwaja)
• (av)
• Igor Zabukovec (iz)
• Junjun Chen (junjunc)

Brainstorming List

1. Fingerprint sensing bike lock opener.
2. Password gesture for opening a door.
   
3. Darkness detecting light system for dorm rooms.
4. Mood Llghting based on music.
5. Device that automatically scrolls Facebook based on your eye position.
6. Kindle that flips the page based on your eye position.
7. Switch computer windows with gestures.
8. Shades that close based on how bright it is outside.
   
9. Shades on a timer for taking naps.
10. Alarm clock that opens blinds.
11. Alarm clock that senses if you’re still in bed.
12. Drawers that open pants/shorts, t-shirts/long sleeve based on the weather.
13. Golf swing analyzer sunglasses.
14. Smart foot stool that softens when you put your foot on it but stays rigid when other things are on it.
15. Automatic coiling headphones that coil when you tug on them 3 times above a certain tension threshold.
16. A mouthguard type sensor that tells if, after brushing your teeth, your mouth has an acceptable amount of plaque.
17. A laptop monitor sensor that adjusts the angle of the screen based on your positioning in bed.
18. A smartphone keyboard that adjusts to how you’re holding it (i.e. if holding it with one hand, will make a viable keyboard that caters to your one-handedness).
19. When connecting external monitors to your laptop, you have to set your screen’s relative position. How about creating a system that that adjusts the external screen based on your position relative to the screen?
20. A running shoe step monitor that tells a runner if they’re pronating their foot properly and recommends what type of shoe to buy.
21. A gadget that lets you compare prices of a certain good while you’re at the store. Imagine you walk into Walmart and want to know if you’re getting a good deal on these headphones. Pull out smartphone/gadget, take picture, and a list of competitor prices shows up. (Kayak for real life).
22. A toothbrush that tells you if you’re brushing your teeth too hard.
23. Shoes that, as you’re jumping in the air, brace for impact by increasing cushion (I think laptops have something similar).
    
24. A phone, that when placed on your bed, senses that you’re asleep and automatically moves calls to voicemail.
25. Developing an alternative for knobs for music production programs (knobs are terrible to use with the mouse … we could just use physical knobs, but is there something more interesting?).
26. Turn a laptop into a touch screen.
27. Water dispenser that measure the amount of water in your cup (so it can stop pouring).
28. Refrigerator that tracks what you have (and how much) and tells you what you need to buy.
29. Sensor that reminds you to water your plants when the soil is dry
30. Clothing that tells you if you’re slouching
31. Clothing that indicates when it is stained / wrinkled / untucked, etc.
32. Glasses that indicate whether or not you are straining your eyes too much (for example if you spend a lot of time in front of your computer screen).
33. Headwear to help blind people navigate (senses when there is something in their path)
34. Sound visualizer: Turn sounds to colors (for deaf people, or just to visualize sounds)
35. System for dancers to learn moves that is synchronized to music. i.e. if they practice moves slowly, the music will play slowly, if they stop, it stops, etc.
    
36. Similarly, an interactive system that responds to a dancer’s movements by generating both sound and visuals, allowing the dancer to control an entire multimedia performance (not necessarily just for practice).
37. A similar system that compares a dancer’s moves against a previously recorded prototype (say by a teacher), so it can show mistakes in practice.
38. A similar system for musicians to practice with an accompaniment track (slows down when they do, etc.)
39. System that tells you if you have everything you need when you leave your room
40. System that helps you find stuff in your room
41. Live responsive software to music for a party : visuals are created according to the music played by the band/dj : sound, patterns, volumes. Possibility to use machine learning algorithms so that the software could recognize patterns in music
42. Live responsive software to people’s movement in space for party: create visuals according to their movements: combining micros and kinects and use effects.
43. Combine 40 and 41 : create a whole integrated systems for parties, where the experience keeps changing.
44. Tools on shopping carts that reads your shopping list on a USB key. Then minimize the distance in the shopping mall and tells you where to go exactly on a screen placed on the shopping cart.
    
45. Shopping cart as above, but instead of just telling you where to go, drives itself following the calculated path.
46. Bike padlocks that have heating function (using a battery) so that the lock does not get stuck when it freezes (this happened to me a few times and it is very annoying).
47. Electronic remote that can be used for all sorts of appliances: control your coffee machine, tv radio, etc.
48. Same idea, except using voice and gesture recognition:  by saying “coffee”, “TV”, etc. turn things on or off, and by raising or lowering your arm change the volume (for example).
49. Tool to plug and heat the coffee machine and make your coffee when your alarm clock goes off or when you go out of bed (in connection with idea 11) so that you do not have to wait in front of your coffee machine.
    
50. System to drive a car using only eye movements, for paraplegics.
51. Program that allows your to chance the source code file that you are working on without using the keyboard, so that you separate the functions of writing code and of choosing the file.
52. A new way of accessing different directories on the computer that simulates files organized in a 3d space.

Idea Chosen

We chose idea #35: System for dancers to learn moves that is synchronized to music (i.e. if they practice moves slowly, the music will play slowly, if they stop, it stops, etc). We chose this because this (more than many of our other ideas) applies to a specific target user group. We also think that learning moves is a real world problem that our system could realistically help solve. Our system wouldn’t interfere much with users’ normal practice, which means it wouldn’t be hard for them to use. Also, since there are many dancers on campus, we would have access to the target user group for testing. It seems feasible in terms of budget, as it wouldn’t require very many parts. It would be a Kinect-based application, but as a second idea, we could also use tethers and other sensors (large flex sensors, etc). It also seems feasible in terms of work, and it is also a good starting point from which we can build on if we have time to extend the project (see ideas #36 and #37).

Target User Group

Dancers are our target user group. Although this system would ideally be useful for all dancers, it will be particularly interesting to work with dancers who want to introduce non-traditional practices in their performance, where their craft consists of an interaction with the music, instead of being treated as a response to the music. A very important aspect of our system is to minimize any limitations to their movement, and to allow the system to work in a large space.

Project Description and Context

A system to allow dancers to practice moves without disturbing their practice. Currently, if  dancers want to change their music during practice, they must stop and go to a device that controls the music (like a computer or iPod). This would disrupt their practice, as it would require a lot of back and forth movement unrelated to the piece that they wish to perform. Additionally, since there is no easy way of controlling the tempo of the music on the spot, a dancer must adjust their moves to the predefined tempo. This may work for performances, but is not good for practice, when the dancer might want to practice some moves slower (but still to the music). This whole process might be improved with a technical solution that makes the music follow the person, instead of the other way around. Dancers must spend a lot of time practicing and they generally have a set practice location. This means that the problem solution does not have to be mobile, and that the user group would be motivated to use it. One related solution is using a remote control, but this means that the dancer must carry the remote with them. There are also other gesture based control systems, but they require the dancer to interrupt their practice in order to display these gestures to control the system. Our aim to develop a more “intelligent” and responsive system, to make the practice routine as fluid and efficient as possible.

Technology Platform

We chose to use several sensors :
– Kinect would probably be the best and most useful since it can easily capture movements. This would capture the whole body movement of the dancer, as well as allow the dancer to move around the room. Also, since we can use gesture controls with the Kinect, this also means that the dancer wouldn’t have to walk from their practice location to say a computer or other physical interface to control the music. A Kinect platform would be the least disruptive to a dancer’s normal practice.
– In case no kinect are available we could use tethers: thanks to them it is possible to determine the position of the dancer and  their movements. However, the dancers would have to be attached to physical objects on stage which is not as practical, but could create interesting performance possibilities
– Flex sensors with an Arduino can also be used as well, though seem more difficult.

Sketches

Group 10:

• Junjun Chen (junjunc),
• Igor Zabukovec (iz),
•  (av),
• Osman Khwaja (okhwaja).

Description: 

This DigiSketch is intend to do renew the famous Etch-a-Sketch, providing a new way to write on the computer. The principle is quite simple : use the two potentiometers to draw lines of circles and the photo-sensor to change the color of them. You can basically write and draw whatever you want. Start by writing letters, basic forms, and continue by increasing the complexity of your drawings and to test how skillful you are, try to draw a circle. If the user is adventurous, the Processing program can be modified in order to use a different color palette, a different sized canvas, or a different pen shape.

Our system overall achieved its goal. However, the calibration of the photocell could be improved, in order to allow the user more precise control over the color. Perhaps a different sensor could be used for this purpose. To make the system more sophisticated, we could have three different sensors controlling red/green/blue, rather than a single sensor controlling the shade of one color, as we have now. In general, our system proves how easy it is to replicate the Etch-a-Sketch concept; the possibilities for refinement are vast.

Pho­tos of sketches:

A system which allows you to draw in Processing using two rotary potentiometers to control  position, and using a photocell to control color.

A system which allows you to control the playback speed of a sound file by flexing your finger.

An LED night-light system: turns on the night light when it is dark.

Sto­ry­board: 

DigiSketch in Action:

The Arduino and the circuit:

An example of different colors that can be obtained by varying input to the photocell:

An attempt at drawing a circle:

List of parts

• Arduino, wires, USB cord, breadboard
• 10 k ohm resistor
• Photocell
• 2 rotary pots

Instruc­tions for Recreation

Connect a photocell to the bread board, with a 10kOhm pull down resistor from one leg of the photocell the ground. Connect the other leg to 5V. Connect the point between the photocell and the resistor to analog pin on the Arduino (we used A0). Connect a rotary potentiometer the the breadboard so that one other pin is connected to 5V and the other outer pin is connected to ground. Connect the middle pin to another analog pin (A1). Do the same for the other rotary pot to connect it to A2. Use the attached source code for Processing and Arduino – the Processing program receives messages from Arduino in order to draw on the screen.

Circuit diagram:

Arduino Source Code:

/* COS 436 - Lab 1 - Group 10 */
/* A Processing drawing system using two rotary potentiometers
   and a photocell. */
/* The Arduino program communicates with Processing with Serial port. */

int photoPin = 0; // the photocell is connected to A0
int pot1Pin = 1; // the 1st pot is connected to A1
int pot2Pin = 2; // the 2nd pot is connected to A2

int photoReading; // the analog reading from the photocell
int pot1Reading; // the analog reading from pot1
int pot2Reading; // the analog reading from pot2

// minimum and maximum values for calibration of photocell
int photoMin = 0;
int photoMax = 0;

void setup(void) {

  Serial.begin(9600);

  // Calibrate photocell during the first five seconds
  while (millis() < 5000) {     // Read in value from photocell     photoReading = analogRead(photoPin);     // Record the maximum sensor value     if (photoReading > photoMax) {
      photoMax = photoReading;
    }
    // Record the minimum sensor value
    if (photoReading < photoMin) {       photoMin = photoReading;     }   }      // Establish contact with Processing   establishContact(); } void loop(void) {   // Only send messages when there is a connection   if (Serial.available() > 0) {
    // Read input from Arduino's analog pins
    photoReading = analogRead(photoPin);
    pot1Reading = analogRead(pot1Pin);
    pot2Reading = analogRead(pot2Pin);

    // Map sensor value to desired range
    photoReading = map(photoReading, photoMin, photoMax, 0, 255);
    photoReading = constrain(photoReading, 0, 255);

    // Send sensor reading as comma separated values.
    Serial.print(photoReading);
    Serial.print(",");
    Serial.print(pot1Reading);
    Serial.print(",");
    Serial.println(pot2Reading);
  }
}

// Establish contact with processing.
void establishContact() {
  // Send an initial string until connection made
  while (Serial.available() <= 0) {
    Serial.println("0,0,0");  
    delay(300);
  }
}

/* COS 436 - Lab 1 - Group 10 */
/* The Processing program recieves input from sensors which the 
   Arduino sends by serial port. */

import processing.serial.* ;

/* Serial port for message communication. */
Serial port;

/* Drawing parameters */
float ypos;
float xpos;
float fgcolor;

void setup() {
  size(500, 500); /* Canvas dimensions */
  background(255); /* White background. */

  println("Available serial ports:");
  println(Serial.list());

  port = new Serial(this, Serial.list()[0], 9600);  

  port.bufferUntil('\n'); /* Read in one line at a time. */
}

void draw() {  
  fill(fgcolor, fgcolor, fgcolor); /* Set color. */
  ellipse(xpos, ypos, 20, 20); /* Draw circle with a center determined by pots. */
}

void serialEvent(Serial port) {
  String myString = port.readStringUntil('\n'); /* Read in a line. */
  myString = trim(myString); /* Remove leading / tailing whitespace. */

  int sensors[] = int(split(myString, ',')); /* Split CSV input into int array. */

  // Print out sensor values:
  for (int sensorNum = 0; sensorNum < sensors.length; sensorNum++) {     print("Sensor " + sensorNum + ": " + sensors[sensorNum] + "\t");   }   println();      if (sensors.length > 1) { /* else input is incorrectly formatted */
    fgcolor = 255 - sensors[0]; /* Set color of circles */
    xpos = map(sensors[1], 0, 1023, 0, width); /* Set x-position of circle center. */
    ypos = map(sensors[2], 0, 1023, 0, height); /* Set y-position of circle center. */
  }
  port.write("A"); // Send byte to get more data from Arduino program.
}