Author Archives: Elizabeth Qu

Final Project — AcaKinect

Posted on by

Group Name: Deep Thought

Group #25: Vivian Q., Harvest Z., Neil C., Alan T.

I. A 1-SENTENCE DESCRIPTION OF YOUR PROJECT

AcaKinect is voice record­ing soft­ware that uses a Kinect for gesture-based con­trol, which allows con­tent cre­ators to record mul­ti­ple loops and beat sequences and orga­nize them into four sec­tions on the screen, creating a more effi­cient and intu­itive way of pre­sent­ing a music record­ing inter­face for those less expe­ri­enced with the tech­ni­cal side of music production.

2. HYPERLINKS TO PREVIOUS BLOG POSTS

P1: http://blogs.princeton.edu/humancomputerinterface/2013/02/22/a-cappella-solo-live/
P2: http://blogs.princeton.edu/humancomputerinterface/2013/03/11/p2-2/
P3: http://blogs.princeton.edu/humancomputerinterface/2013/03/29/p3-vahn-group-25/
P4: http://blogs.princeton.edu/humancomputerinterface/2013/04/08/p4-acakinect/
P5: http://blogs.princeton.edu/humancomputerinterface/2013/04/22/p5-acakinect/
P6: http://blogs.princeton.edu/humancomputerinterface/2013/05/06/p6-acakinect/

3. VIDEOS & PHOTOS

Videos:




Description:

Gesture controls:

  • Record Loop (Raise either arm above head) — Records a single loop of 8 beats at a set tempo. Block representing loop appears in the column the user is standing in, when recording is finished. Blue loading circle appears at initial command.
  • Delete Loop (Put hands together in front of torso and swipe outward) — Deletes the last recorded loop in the column user is standing in. If there are no loops, does nothing. Green loading circle appears at initial command.
  • Delete Column Loop (Put hands together above hand and swipe straight down) — Deletes all recorded loops in the column user is standing in. If there are no loops, does nothing. Pink loading circle appears at initial command.

Photos:

4. BULLETED LIST OF CHANGES

  • Blocks added to visually indicate the number of loops in each column: this is easier to read than a number in each column and provides better visual indication of the structure and layering of loops across the sections.

  • More beautiful and colorful UI: since we couldn’t get the openGL to run on Processing without distorting our loop recording timing, we just changed the UI using bare Processing to have a more visually-engaging prototype. Each section is a different color (green, purple, blue, and red), all the loop blocks are different colors between columns, beat count is more visible (a complaint from user testing!), and messages displayed clearly and colorfully. We kept in mind conventional recording colors, so the “Begin recording in…” is green and the “Recording!” message is in red.

  • More prominent highlighting of the current section: Since some users needed additional guidance to figure out the fundamental structure of recording different parts in different sections so that each part can be edited or changed separately, we indicate the current section the user is in much more clearly by highlighting the entire column on screen instead of providing a small tab on the bottom.

  • More onscreen visual cues: message notifying user to calibrate, message notifying when user has reached maximum number of loops in a column (4 loops maximum) , count down to the next loop recording, etc. We pop up a prominent and visually consistent notification in the middle of the screen that is easy to read and provides immediate feedback.

  • Better gesture recognition with microphone in hand: the delete loop and begin recording gestures were both modified in order to be easier to do while holding a microphone; the delete loop gesture was entirely changed to be more intuitive and easy to perform with a mic while the begin recording gesture received a few code tweaks to prevent false positives.

  • A “delete column” gesture: during the demo, we noticed that people felt it was tedious when they were deleting repeatedly to clear a column. This gesture makes it faster and easier to delete all loops in the section rather than one by one.

  • Gesture loading indicators: one problem we noticed during user testing for P6 was that users often didn’t know if the gestures were being recognized. To make this clear, we based our UI fix off many Kinect games that exist. When a command gesture is recognized (record/delete/delete-column), a small circle starts filling up on the user’s hand which started the command. Once the circle fills up completely, then the command executes. This ensures that users aren’t accidentally causing gestures (since they have to hold the pose for 1-2 seconds) and notifies the user that the Kinect recognizes their command. Each loading circle is different colors for clarity and differentiation between commads.

5. HOW AND WHY YOUR GOALS/DESIGN EVOLVED

The original concept for this project was much more complex. Over the course of development and testing, the specification was drastically simplified — coincidentally making development simpler, but more importantly, reaching our target audience more effectively. The original idea was to create a gesture based system, that enables highly complex music creation and manipulation intended for both live performance and recording. With some testing, we discovered that what we really should create a tool for those with little or no music production experience. For those with experience, there are hugely complex systems that offer all the detailed functionality needed. However, there are a large number of musicians who don’t even want to worry about music production at the level of Garageband; these may be very talented musicians who want to use loops and effects, but they may not be interested enough in the technical side of music production to go learn any specific piece of software or hardware. We decided that AcaKinect would slot in at the very bottom of this technological chain: simple enough to pick up and use immediately, and yet fun and flexible enough to retain users and potentially allow them to develop an interest in learning more complex music production tools.

We also realized that the format does not particularly suit recording well; if a user has progressed to the point of wanting to record, edit, and polish these creations, there are a huge number of software loopers available that offer much more flexibility, as previously mentioned; in addition, more experienced musicians who are looking to produce a record will probably turn to a full-fledged Digital Audio Workstation that allows maximum control at the cost of a rather higher learning curve. Thus, we see this as an experimental tool. One tester, who is the music director for an a cappella group on campus, commonly uses a midi keyboard for just this purpose; when arranging a piece for a cappella, he writes down each part and then plays them together on a keyboard to see how they sound. In these cases, it may be easier and more flexible to just test out these parts using AcaKinect, encouraging more exploration and experimentation. To that end, we pared down the specification to the bare minimum needed to provide a flexible looping environment with spatial awareness (columns) to suggest to the user how a piece of music might be structured. There are only two primary functions – record and delete – so the user is not confronted with anything approaching “mixing board” levels of complexity. The division into four sections, and the physical requirement of moving left and right in order to record in each section, suggests a natural split in what to record in each section; users naturally choose to record all percussive sounds in one, all of the bassline in a second, and then maybe a more prominent harmony in a third. This sets them up well for moving on to more complex musical production tools, where structuring loops and tracks in a hierarchy is very important while organizing, say, twenty tracks all at once.

6. CRITICAL EVALUATION OF PROJECT 

We believe that with further work and development, this could be a viable and useful real-world system that fills a possible gap in the market that has never really been touched. We’ve already discussed how, in terms of simplicity and feature set, AcaKinect would be slotting in under all the music production software currently available; what we haven’t really covered is the opposite end of the music spectrum, which is currently occupied by games like Rock Band and Guitar Hero. These games do introduce a musical element in the form of rhythmic competence (and in the case of Rock Band, fairly lenient vocal pitch competence), but fundamentally the music is still generated by the game, and the user just kind of tags along for the ride. The goal for AcaKinect with further iteration is a product that is almost as game-like as Rock Band; a fun system for testing out loops and riffs in the living room, and a useful tool for playing with sound and prototyping arrangements. It’s important that AcaKinect is seen as more of an exploratory product; unlike working in a DAW where the user may have a very good idea what she wants to create, AcaKinect would be a live prototyping tool that enables a lot of exploration and iteration of various sounds and harmonies. The simplicity of the controls and the lack of any real learning curve only helps to make that a fairly easy task.

The application space, as far as music production (and even more loosely, music interaction) tools go, is giant, and spans all the way from Rock Band to Logic Pro. There is no real point going for new or extra functionality; whatever arcane feature you’re looking for, it probably already exists, or if it doesn’t, it would probably be best suited to a Logic plugin rather than a whole new product, since the pros are the ones looking for new features. Those who are intimidated by the sheer number of buttons and knobs and controls in a typical piece of music software or hardware seem to much prefer a great simplification of the whole music production process (as would make sense), and we found that there is a big opening in this space for software that can be picked up and used immediately by any musician, no matter whether she has technical background in music production or not.

7.  HOW WE MIGHT MOVE FORWARD WITH THE PROJECT 

There are still quite a few implementation challenges involved in making this product one that is truly easy to use for any musician. Firstly, given its intended use as a fun and exploratory product in the living room, it’s a little problematic that it only works for one person. If, say, a family is playing with it, it would be much better to allow several people to be recognized at once (even if only one loop is recorded at a time), so that several people may collaborate on a project. SimpleOpenNI is capable of tracking limbs for two people, which is generally what Kinects are used for in Xbox games as well; we could thus implement two people without too much extra trouble, but to do more may be difficult. Secondly, this prototype uses Processing and Minim for ease of coding, testing, and iteration; however, since Minim really wasn’t designed for low-latency audio looping, it has certain issues with timing and tempo for which we have implemented hacks to “get around.” However, in an actual polished product, latency would have to be much lower and the rhythm would have to be much more solid; to that end, a more robust audio framework (hooking directly into Apple’s CoreAudio libraries, for example) would allow us to achieve much better results.

Finally, there’s the issue of physical setup; the user needs to be able to hook up a microphone and play sound out of speakers such that the balance between the current vocal and the sound from the speakers is about right, but without triggering any feedback or recording the speaker output back in the microphone. There are advanced noise-cancellation techniques that can be implemented to signal-process away the feedback, but these will sometimes add artifacts to the recording; one way is just to require that the user use a highly directional microphone with a sharp signal strength falloff, so as to reduce the chance of feedback. An on-screen notification that informs the user of feedback and turns down the levels of the speakers when feedback occurs might also be convenient. Alternate setups may also be a good thing to test; a wireless clip-on mic of the sort used on stage during live performances, for example, may prove slightly more expensive, but it may let users feel less awkward and make gestures easier.

8. LINK TO SOURCE CODE 

AcaKinect Source Code (.zip file)

9. THIRD-PARTY CODE USED

10. LINKS TO PRINTED MATERIALS

AcaKinect poster (.pdf file)

 

P5: AcaKinect

Posted on by

I. Your group number and name

Group 25 – Deep Thought

 II. First names of everyone in your group

Neil Chatterjee Harvest Zhang Vivian Qu Alan Thorne

III. 1-sentence project summary 

AcaKinect is a voice recording software using the Kinect for gesture-based control, which allows the music users to record multiple loops and beat sequences and organize them into four sections on the screen, giving the users an more efficient and intuitive music recording interface.

IV. 1 paragraph describing 3 tasks you have chosen to support in this working prototype 

The first task (easy) is to create a simple one-track loops using gestures, by raising one hand above the head. This creates a loop in one of the four sections of the recorded voice. The second task (medium) is to create multiple loops in different sections of the screen, and even multiple loops in one section of the screen using the same gesture from the first task. The user does this by repeating the first task in the same and in different sections. The tracks will all loop over each other. The third task (hard) is to complete the second task with the addition of deleting tracks, by raising one arm and pointing the other arm to the elbow and swiping outward. This is the first part towards the creation of a fully editable sequencing and recording system.

 V. 1 short paragraph discussing how your choice of tasks has changed from P3 and P4, if at all, as well as the rationale for changing or not changing the set of tasks from your earlier work.

We have scaled back the set of tasks from P3 and P4. For our paper prototype we wanted to make sure that users could do everything the software had to offer. For this prototype we wanted to ensure that our core functionality worked properly. We put off implementing a master recording functionality in order to enable track deletion. The music recording process is more expressive and customizable if the users are able to delete tracks: we wanted our prototype to reflect that. We also have had some difficulty implementing a master track and so we could not include it in our tasks for now. In our paper prototype, we also had a “settings menu” where you could adjust the overall recording volume and EQ. Since this wasn’t essential to creating a minimum working prototype, we didn’t include it since it added additional complexities we would have to manage. We hope to be able to fully implement all the functionalities in our paper prototype by our final demo.

VI. 2–3 paragraphs discussing your revised interface design

We changed the display of the beat count (for a total of 8 beat loops) from a timing bar at the bottom of the screen, to simply print out a beat count. This was easier to implement and may actually be simpler for the user to understand. We will find out during the prototype testing how helpful this format of displaying the beat count is. We were also unable to get the section panels to successfully display transparency, so we changed the display to instead show a bar at the bottom of the section which the user are standing in.

From our own preliminary testing, we found that the Kinect is extremely sensitive in terms of gesture detection. So our “delete track” (raise one arm and swipe other arm) gesture often gets mistaken for a “start loop recording” (raise one arm) gesture since both involve raising a single arm. The beginning of the delete motion could start recording unintentionally. We plan on eliminating the gesture overlap, and make them clearly different. We have not yet implemented this change, but expect to before user testing on the hi-fi prototype.

Additionally, the GUI is incomplete. Currently, we are relying on the depth image from the Kinect to show the figures on the screen. In our paper prototype, we had a simple body outline which showed the person on the screen. Ideally, we would do something similar for our hi-fi prototype and outline the bodies of the users using OpenGL/Box2d to create a clean, simple interface. Adding color would be awesome too. See Section VIII for an attempt at creating a better UI — we were unable to get the timing for the beats exactly right, so we delayed implementing the better UI for the final demos. One important part of the UI which we have not yet implemented is displaying a rectangle in each panel corresponding to the loops which the user has recorded. This was due to some difficulties managing the visualization in Processing. We hope to add this in the upcoming weeks.

Storyboards:

Sketches for the still-unimplemented parts of the system:

VII. 3–4 paragraphs providing an overview and discussion of your new prototype

We have implemented the following functionality:

  • Visually track user using Kinect skeleton data
  • Built gesture-based functionalities for recording multiple loop, deleting a loop
  • Bring down the sound recording latency to a manageable level
  • Heads up beat/recording status display

The main thing we learned from the paper prototype testing was that the sound playback of the recorded loops was essential for the user to understand and fully utilize the system, which is why we focused on loop recording instead of the master recording for the hi-fi prototype. To have accurate and good sound playback, we needed to display to the user the number of beats per loop. We did this by printing out a beat count using text in the upper right hand corner (See Section VIII). In addition, we wanted to add visual indicators when the loop recording was about to start, during the recording, and when it finished. To do so, we displayed messages in the upper left hand corner — “get ready…” when the loop recording was about to start, “recording” when the loop is recording, and no message when the recording is finished.

We left out the following features: master track recording, and a settings menu. As a reminder, the master recording would be able to combine all the repeating loops that are playing back on screen plus any additional sound that the user adds during the recording time, and save to a single track. We weren’t able to implement this because it requires further investigation. It is not essential for a minimum working prototype. From paper prototype testing, we found that users got confused between “master track recordings” and “loop recordings,” so we decided to prioritize loop recordings in our working prototype and eliminate the confusion for the users. We aim to successfully add this master track functionality later (without confusing the user). For the settings menu, again the EQ and sound level adjustment is not part of the core functionality, so we are saving it until the more important parts work.

Wizard-of-oz techniques: We’re pretending that the master recording is working properly when in fact it is not yet. Our current plan of action is for users to use their imaginations to pretend it is recording. We may also use paper indicators (with text like “Master Track is recording…”) to help users understand, and test out the best ways to signal the users between loop and master track recordings. We implemented functionalities in order of importance to the main use of the software. We ranked the priorities: 1) get loop recording working and synchronized, with sound playback; 2) loop deletion; 3) show user position on screen; 4) show beat count and recording status. Todo: UI improvements (body outline and loop track blocks) and non-implemented functionalities (master recording and settings menu).

External Code and Libraries Used:

  • Minim Library (sound recording and processing)
  • SimpleOpenNI framework for reading in Kinect data (and Skeleton library)
  • For body outline and improved UI (not fully functional): followed the Kinect Physics Tutorial for the body outline, which utilizes the blobDetection, toxiclibs, PBox2D, and org.jbox2d libraries which are all freely available on the Internet.

VIII. Video & Images of Working Prototype

Videos:

A demo of our working prototype.

The non-functioning improved UI. Including in the openGL and polygon drawing libraries for drawing the figure caused additional latency (the beat count was not consistent, sped up during beats 2 and 3 and 5 and 6; this is really bad, since musicians need very constant tempo). We will try and work out the bug for this in the future so we can use this cool UI!

Photos:

P4 – AcaKinect

Posted on by

Group 25 — Deep Thought

(Vivian, Neil, Harvest, Alan)

I. PROJECT SUMMARY 

Aca-Kinect is a gesture based music recording software that interfaces with an Xbox Kinect, using gesture-based controls to allow on the fly track editing, recording, and overlaying.

II. DESCRIPTION OF TEST METHOD

i. Describe your procedure for obtaining informed consent, explaining why you feel this procedure is appropriate. 

We decided to use a modified adult consent form which includes an overview of the procedure, confidentiality, risks, and benefits. By signing, the user agreed to participate, and also optionally agreed to be filmed/photograph for our future reference. We felt the consent form worked well in covering all the concerns for the prototype testing process and was a good physical record of the testing. For the procedure itself, we first helped the participant understand what was on the consent form by providing a description of our project and giving a summary of each section on the consent form; then we gave the participant a hard copy of the consent form to read and sign. This process took around 3-4 minutes.

Here’s a link to our consent form.

ii. Describe the participants in the experiment and how they were selected. 

Participant 1 was selected for her experience with various singing groups. She has little recording experience but plenty of experience singing and arranging for live performance. She was selected partly for her expertise in singing and lack of recording experience.

Participant 2 has a lot of experience singing in several a cappella and choir groups on campus. She has been involved in music composing and arranging processes and uses existing recording software. She was selected for her knowledge of existing recording software and familiarity with a capella music-making.

Participant 3 was an electrical engineer that has had someone background with the XBox Kinect, so gesture based actions shouldn’t be a problem. He also has had no experience with singing, music, or recording. This allowed us to capture the other end of our user-group, while compensating that with experience with gesture-based software.

iii. Describe the testing environment, how the prototype was set up, and any other equipment

The testing environments were a bit less than ideal. Both locations (Frist 100 level, ELE lab) had the necessary space for gestures and adequate lighting. However, neither space was an actual stage or recording space, as it would have been difficult to secure a stage space, and a recording studio would be prohibitively small. Both locations had some background noise and the occasional passerby but ultimately suited our needs well. The paper prototype was set on a table a few feet from where the participant was standing. For any future testing, we would aim to find spaces that better simulate recording environments.

iv. Describe your testing procedure, including roles of each member of your team (order and choice of tasks, etc). Include at least one photo showing your test in progress. Provide links to your demo and task scripts.

Our testing procedure was as follows:

  • Team member describes the project, prototype, and testing procedure
  • Participant reads and signs consent form.
  • Gives the participant a demo (using the demo script) of how the system works. Asks if the participant if they have any further questions.
  • Read all of the first task instructions. Wait and observe the participant completing the task, prompting them if necessary.
  • Repeat for the second and third task. Since the tasks are increasing in order of difficulty, instructions for the last two tasks are read one-by-one for the participant to follow.
  • Ask for any further feedback/feelings from the participant.

Vivian introduced the project, walked the participant through the consent form, gave them time to sign the consent form, and ran through the demo script. Harvest wrote and read the tasks script with Vivian assisting (moving the prototype pieces). Alan took pictures and video during the testing process. Neil transcribed and took notes on the process, outcomes, interesting observations, etc.

Here’s a link to the demo script and the tasks script.

III. SUMMARY OF RESULTS

In general users performed well on each task. The main issue was whether the participants fully understood the concepts of master recordings and loop recordings. The third participant asked a question about this and we explained it to him in great detail, so he was able to easily complete the three tasks. On the other hand, the first two participants were slightly confused on the third task, when they were instructed to make a master recording and then record a loop in a specific panel while the master recording was going on. The concerns were twofold — the first participant didn’t understand the difference between (or remember the gestures for) the master recording and a loop recording. The second participant didn’t understand why someone wanted to do a loop recording during a master recording, when there was an option of doing it before. Therefore there was a minor usability problem in the first case, and a usage question in the second. With the second participant, we spent approximately 5 minutes discussing the reasons for doing the third task the way we framed it and the real-world applications. Besides the question of motivation, there was no problem with usability since she was quickly able to complete the task.

The participants all thought the prototype was simple and easy to use — they used words like “cool,” “easy and fun” and jokingly “a workout.” Some of the participants seemed a little embarrassed singing in front of a paper prototype or got tired by the end and just said words instead of simulating music-making — perhaps as we evolve our project into a higher-fidelity prototype the participants will be more engaged and interested because of the addition of audio feedback. Additionally, the second participant asked about volume control on each individual panel — this is a feature we will need to make more explicit in future iterations of our project! There was also a question about why we had four panels — this is a cosmetic problem and may require us to suggest to the user to organize their recordings into different sections (percussion, bass, tenor, alto, etc).

IV. DISCUSS RESULTS

A major result of the testing was confirmation that the workflow is fundamentally fairly accessible to musicians or otherwise who have no prior experience with recording music using tools ranging from Garageband to loop machines. The lack of the steep learning curve, expense, and complicated setup that comes with more advanced and more capable tools like a dedicated loop machine means that this Kinect application can introduce more people to at least basic music production with some interesting looping and multitrack features; those who outgrow this application will probably develop an interest in the more advanced tools that do offer much more functionality.

However, we also discovered that some parts of this product remain slightly confusing to users who had never encountered recording tools before; there was a little terminology confusion surrounding the meaning of “master recording” and “loops,” and it was very difficult to impress upon the users the fact that all the loops would be playing simultaneously on top of each other. This is a fundamental issue with paper prototyping an application like this, and a high-fidelity prototype that actually provides accurate audio feedback would go a long way to making the experience much more understandable. As such, the experiment also cannot reveal much about what kind of music users would actually create using this product, since we only simulated the loops. It would be instructive and useful to see the complexity of music that users would produce, whether they actually use various pieces of functionality or not, and so on, but we don’t know at this point since we are not fully simulating the generation of a piece of music with many parts.

V. PROPOSED SUBSEQUENT TESTING

We feel that our paper prototype demonstrated its purpose. We aimed to explore functionality and see how users interacted with the ability to control features via gestures. The biggest issue we discovered stemmed from the lack of audio feedback. It’s difficult for a user to have an idea of what he/she is doing in recording software without the ability to record/listen. The confusion our users had in the tests was not an interfacing problem, but rather an understanding that we feel will be evidently resolved when users interact with a higher-fidelity project.

L3 – Group 25 (Anteater)

Posted on by

Group 25 – Harvest, Alan, Vivian, Neil

I. PROJECT DESCRIPTION

Our robot is a vehicle which changes direction based on sensor input from a flex sensor. So essentially, it moves straight until the flex sensor on the head of the robot hits a wall/object, then it will always turn right. We named it “Anteater” because the long flex sensor makes the robot look similar to an anteater with its tongue stuck out. We thought that building such a robot would be really interesting since we’re using a flex sensor as a contact detector rather than the normal use of flex position. Unfortunately, the arduino was really heavy and our makeshift wheels (paper plates) could not hold the weight, so we built this scrappy box behind the robot which contained the arduino. We had to hold the box so the weight did not prevent the robot from moving. In the future, we’d like to use sturdier wheels and more powerful motors to be able to mount the arduino and everything on the vehicle itself. It would also be interesting to include more complex code which makes the robot turn in left or right with different probability, as well as different sensors to give the robot a richer source of information to navigate.

II. LIST OF BRAINSTORMED IDEAS

  1. Duck-foot robot: attach flippers to a DC motor
  2. One-wheeled motorcycle: use a single big wheel with all the controls mounted inside the wheel..
  3. Airboat: use a motor to rotate a fan propeller to make a robot slide across the floor.
  4. Cockroach: robot uses light sensor to run around aimlessly, remaining stationary if it detects its in the dark.
  5. Hovercraft: use an inverted styrofoam bowl as a hovercraft, mounting a fan pointed downwards to create hover effect.
  6. Tricycle car: Use 2 motors to drive rear wheels, servo to steer front wheel.
  7. Desktop Potentiometer Bot: small vehicle which can be steered based on input from sliding variable potentiometer.
  8. Egg car: attach a servo to one end of the egg, and the robot will rotate in circles.
  9. Spinning Top: stick axle of DC motor out of bottom of styrofoam bowl, to make the bowl spin.
  10. Propeller Plane
  11. 6-legged insect: servo motors move independently to simulate creepy looking insect
  12. Condiments car: wheels made out of mayonnaise/sugar packets
  13. Anteater: a robot which changes direction based on input from a flex sensor stuck in front.

III. SKETCHES

Sketch of our initial idea

However, we discovered that the weight of the arduino was too heavy to mount on the vehicle itself. So, we pivoted and ended up putting the arduino and associated wires in a separate box dragged behind the vehicle. So, effectively, we had to hold the box to assist the vehicle in moving (since the box was too heavy).

Here’s a sketch of our altered design: vehicle + box

Comparing our final system to an anteater — looks the same!

IV. VIDEO & PICTURES OF FINAL SYSTEM

Picture of final system, another point of view

Video of the Anteater: you can see a the anteater’s tongue pointing forwards, and when it hits the trash can, it rotates right by turning the left wheel. Success! Again, the box containing the arduino is kind of awkward to deal with.

V. LIST OF PARTS

  1. Arduino Uno x1
  2. Paper Plates x2
  3. Knives x2
  4. DC motors x2
  5. Breadboard x2
  6. Flex sensor x1
  7. Transistor x2
  8. 330 ohm resistor x2
  9. Wires

VI. ASSEMBLY INSTRUCTIONS

  1. Use one knife as the crossbeam, to which we attach the two motors.
  2. Stick a couple hubs on the motors and attach the plates to them to create wheels.
  3. Attach another knife perpendicular to the drive shaft knife, to which we attach a flex sensor on the end; this will act as the obstacle detector.
  4. Wire it all up to an Arduino/breadboard kit off chassis, since all that stuff is far too heavy to be located on the vehicle. Motors are connected via transistors for speed control, as in previous sections.
  5. Contain all Arduino/breadboard not mounted on the crossbeams in a box to keep all the wires from getting messed up.

VII. SOURCE CODE

// L3: Anteater
// Group 25 -- Harvest, Vivian, Alan, Neil

int impact = A0;
int left = 3;
int right = 5;

void setup() {
  Serial.begin(9600);
}

void loop() {
  // read the value from the sensor:
  int impactVal = analogRead(impact);  
  if (impactVal < 250)
  {
    analogWrite(left, 0);
    delay(50);
  }
  analogWrite(left, 100);
  analogWrite(right, 100);
  delay(15); 
}

P3 — VAHN (Group 25)

Posted on by

Group 25 — VAHN

Vivian (equ@), Alan (athorne@), Harvest (hlzhang@), Neil (neilc@)

1. MISSION STATEMENT

Our mission is to create a recording software solution by combining the true-to-life sound quality of live performance with the ease of gesture-control. The project will give a capella singers quick, fun, and easy-to-use interface to make music complete songs themselves.

 

  • Vivian: the artistic guru of the group. She directed most of the design efforts of the prototyping process, drawing and cutting the prototype.
  • Alan: made sure the group has all the necessary tools to get the job done, and wrote the majority of the blog post.
  • Harvest: the resident music expert and he formulated the interactions (gestures) with and helped build the prototype.
  • Neil: the hardware specialist and gave insight on how the kinect would interface with the user, as well as recorded the videos and took photos.

2. PROTOTYPE

We hope to uncover any inconsistencies in our ideas about how the software should behave and features we want to implement.  In the prototyping process we hope to refine the interface so that:

  • Our core functionality is immediately recognizable and intuitive.
  • We achieve a “minimalist” look and feel.
  • Secondary and advanced functionality is still accessible.
  • The learning curve is not so steep

Here’s a video of a girl using a complicated hardware sequencer to make an a capella song. We’d like to make this process easier — http://www.youtube.com/watch?v=syw1L7_JYf0

Our prototype consists of a paper screen (which would be the projector/TV). The user data (taken by the kinect) is shown as a paper puppet which can be moved around from panel to panel. Each panel represents a horizontal span of space. Users can move from between panels by moving horizontally in front of the kinect.

The following gestures manipulate the prototype:

  • Raise both hands: start/stop the master recording
  • Raise right hand: record a clip which will be saved in the panel that the user is standing in
  • Move side-to-side: switch between panels in the direction of movement
  • Both arms drag down: bring down settings screen
  • Both arms drag up: close the settings screen up
  • Touch elbow and drag horizontally: remove a sound clip on screen.

3. 3 TASKS

Task #1: Recording a simple master track.

This task is similar to hitting “record” on a traditional recording application — it just records one sound clip. The following video shows how the user would interact with the prototype:

The user raises both arms to start the recording. The user then sings the song into a microphone or directly to some sound-recording device. To finish recording, the user raises both arms again. Then a menu drops down which asks if the user wishes to save their recording. The user would indicate with their arm the choice to save or cancel the recording.

This task worked well with our paper prototype we built.

Task #2: Recording multiple clips and overlaying them.

This task involves the user moving between panels and recording clips of a specified length (ex. 16 beats). The bar on the bottom of the screen will indicate the user how much time they have left in the sound clip. After the clip is recorded in one panel, it will repeatedly play. The user can record multiple clips in each panel. All the clips will be playing at the same time.

The user raises their right arm to start the recording in one panel. The user then sings the song into a microphone or directly to some sound-recording device. The bar at the bottom of the screen shows how much time is left in the sound clip (ex. 16 beats total, 5 beats left). When time runs out, recording stops and a track appears in the panel. To switch between screens, the user moves horizontally into another panel. All the recorded clips are being played back repeatedly at the same time.

This task was hard to show because in our final project, once a sound clip is recorded it will continually loop and play back. When users add additional sound clips, they will simultaneously play back. This was hard to show because we had no sound playback in our paper prototype!

One issue we realized we had to consider after making this prototype is how to sync recording for each individual sound clip. We may have to add a countdown similar to the master recording, but for each sound clip.

Task #3: Recording multiple clips and combining them into one master recording.

The user may have no clips or some clips already recorded in each panel. The user starts recording the master track and all the clips on screen (currently repeating together). The user can also add more clips into the panel they are standing in. The user can remove clips from the panel they are standing in. Settings can also be adjusted by bringing down the menu and changing EQ and overall recording volume.

User raises both arms to start master recording. User can now move between panels and record individual clips (adding extra clips to the sound) like in Task #2. Point arm to elbow and drag horizontally outwards to remove the sound clip in current panel. User can also use both arms, dragging down, to pull down the settings menu. When users are finished they can drag both arms up to close the settings menu.

Similar difficulties as in task #2.

4. DISCUSSION

We made our prototype with paper, glue, cardboard, and markers. Since our system is gesture based, we also made little paper puppets to simulate usage in a more streamlined way than jumping around ourselves and panning a camera back and forth. The only real difficulty we encountered was determining precisely how to realize our conceptual ideas about the software, especially because paper does not express screen position (of the person) like we would like the kinect to do. To fix this, we created a little “puppet” which represented the user’s position on the screen. We think we were moderately successful at capturing the user-screen interaction, however in the future we would like to reflect gestures on screen to better teach users how to use our interface.

The main thing which the paper prototype could not show was the audio feedback, since the sound clips would be playing repeat after the user records it. In this way, paper prototyping was not good at giving an accurate representation of the overall feel of our system. However, it was paper prototyping was good at forcing us to simplify and streamline our interface and figure out the best gestures to interact with it. Paper prototyping forced us to answer the following questions precisely: what should and should not be on the screen? Should we prototype every single little scenario or just a representative cross-section? For which functions should we switch to mouse control?  We ended up prototyping representative actions and did not show some of the settings (such as beat count, represented by the bar on the bottom of the screen) which we assumed in the current prototype would already be set. Showing the separate screens for each standing position of the user worked really well. The status for recording the master track could be more visible (by having the screen turning a different color, for example), so we would improve on this in the future.

A3: Heuristic Evaluation

Posted on by

Names: Colleen, Vivian, Alan

1. What were the most severe prob­lems with the app/site? How do these prob­lems fit into Nielsen’s heuris­tic cat­e­gories? What are some sug­ges­tions to fix the UI, and how do your proposed changes relate to the heuris­tic categories?

  • Most severe problems:
    • No documentation (or tutorial) for how to use it
    • Not intuitive (have to switch between tabs)
    • No consistency across different platforms (Android/iPhone)
    • Can’t choose a stop and have to choose a route
    • Not useful for planning travel, can only pick one transit system
    • Really hard to figure out stops (on Android, can’t even see route intially, don’t know you can click on it)
    • Multiple routes were in the same color.
  • This application violates almost all Nielson’s categories except:
    • Doesn’t violate “Aesthetic design and Simplicity” (H8) but the simplicity comes at a sacrifice of functionality and instruction.
    • Also doesn’t violate “Match between system and real world” (H2) because its mostly based off of conventions from other technologies, and it relies on understood icons used already in Google maps and other technologies.
    • Did not violate “Help users recognize, diagnose, and recover from errors” (H9) because there were no error messages.
  • Suggestions to fix UI:
    • Make stops more visible (H6/H7)
    • Make specific stops searchable (H7)
    • Have all the routes initially selected so that the user doesn’t have to click and extra button when the map loads and nothing shows up (iPhone version) (H7)
    • Some sort of tutorial about how to get important information (H10)
    • Make sure routes are all different colors (H4) and that the routes are clearly distinguishable (H5)
  • Make changing the transit system easier, instead of needing to switch to the “Settings” tab (iPhone) (H3)
  • Doesn’t save previous states (which routes already selected) when switching between tabs or transit systems (H6/H7)
  • Doesn’t notify user when the times until the bus arrives at stop are updated (H1)
  • Announcements aren’t reflected in the route data or on the map (H5/H7)

2. Which problems, if any, were made easier to find (or potentially easier to correct) by the list of Nielsen’s heuristics?

  • Would not noticed the problems with loading data looking error (H5) and wouldn’t have been as bothered by lack of documentation (H10). Overall, the problems with the interface were pretty obvious and glaring. Without Nielson’s heuristics, probably would not have tried to access other transit systems (maybe would have just looked at Princeton ones) which helped identify user control and freedom (H3), recognition and recall (H6), and consistency and standards (H4).

3. Did anyone encounter usability problems that seemed to not be included under any of Nielsen’s heuristics? If so, what additional heuristics might you propose for this type of application or site?

  • Speed factor (each tab needs to load) is not included in the user heuristics but is kind of annoying for the user and definitely affects your experience (esp. on Android). For any time-relevant application, need to consider performance time.
  • For mobile, efficiency of interaction space (phone is considerably smaller than desktop) and should be considered. For example, choosing all routes clutters small screen with many colors/dots/icons (confusing). Can be considered part of Aesthetics (H8), but maybe should be a separate heuristic.

4. What might be some useful class discussion questions—or final exam questions— related to heuristic evaluation?

  • What would be most important heuristics to prioritize?
  • Would the important heuristics change depending platform (mobile vs. computer)?
  • Would the way you evaluate these heuristics differently depending on the time period (now vs. 1980)?
  • Should there be any heuristics added (like speed of interface)?

Individual Heuristic Evaluation:

View Vivian’s observations here.

View Colleen & Alan’s (worked together due to lack of computer) observations here.

 

L2 — Saxobeat

Posted on by

GROUP NAMES: Vivian, Harvest, Neil, Alan

GROUP NUMBER: 25

WHAT WE BUILT:

We played around with capacitive proximity sensing to detect when a user touches the instrument when we considered building 3 music instrument prototypes. We used the same circuit design for a 5-note musical instrument and varied the interface. Each uses aluminum foil in various configurations to play different notes. The first is a guitar tab style device, with aluminum covering stiff paper tabs. The second is a mini piano using pistachio shells to hold clumps of aluminum foil. The final design is a xylophone kit style device, using capacitive sensing with spoons hitting aluminum balls which act as the xylophone bars. We liked the xylophone the best because it had the most interesting visual setup and a comfortable interface, letting users play notes quickly and intuitively. It’s a fun, easy-to-use instrument with a creatively natural use of spoons 🙂

PROTOTYPE #1: GUITAR TABS

The user strums or just simple touches a tab in order to play a corresponding note.

Music: “When the Saints Go Marching In”

PROTOTYPE #2: PISTACHIO PIANO

Pistachio shells are used as the base to hold balls of aluminum. The user touches their finger to each aluminum-covered shell to play a note.

Music: “Turret Opera,” Portal 2

FINAL PROTOTYPE: SAXOBEAT

User holds the round end of the spoon, engaging the capacitive nature of the aluminum material. The round end is connected to the flat tip using a wire, and touching the aluminum balls generates a note.

Music: “Mr. Saxobeat,” Alexandra Stan

LIST OF PARTS:

  • Balls of aluminum (x5)
  • Plastic spoons (x2)
  • Aluminum sheet
  • 10 megaohm resistor (x5)
  • Arduino Uno (x1)
  • Piece of cardboard (x1)
  • Breadboard (x1)
  • Wires

SETUP INSTRUCTIONS: 

  1.  Wrap the round end of the spoon with aluminum. Wrap the thin, flat end of the spoon with aluminum as well. Tape a piece of wire connecting the round and flat end.
  2. Secure the five balls of aluminum to a stiff piece of cardboard using electric tape.
  3. Wire up the arduino. Place the five resistors on the breadboard and connect each end to a digital pin. Connect another end of each resistor to an aluminum ball.
  4. Upload the code and enjoy!

ARDUINO CODE:

// Team Deep Thought
// "Saxobeat" Musical Instrument

#include 

CapacitiveSensor   Eb = CapacitiveSensor(3,2);        // 10 megohm resistor between pins 4 & 2, pin 2 is sensor pin, add wire, foil
CapacitiveSensor   D = CapacitiveSensor(5,4);
CapacitiveSensor   C = CapacitiveSensor(7,6);
CapacitiveSensor   Bb = CapacitiveSensor(9,8);
CapacitiveSensor   A = CapacitiveSensor(12,11);
int spkr = 10;

// Turret Opera
int fA = 440;
int fBb = 466;
int fC = 523;
int fD = 587;
int fEb = 622;

// When the Saints Go Marching In
int wC = 262;
int wD = 294;
int wE = 330;
int wF = 349;
int wG = 392;

// Saxobeat
int sB = 494;
int sCs = 554;
int sD = 587;
int sE = 659;
int sFs = 740;

void setup()                    
{

   A.set_CS_AutocaL_Millis(0xFFFFFFFF);
   Bb.set_CS_AutocaL_Millis(0xFFFFFFFF);
   C.set_CS_AutocaL_Millis(0xFFFFFFFF);
   D.set_CS_AutocaL_Millis(0xFFFFFFFF);
   Eb.set_CS_AutocaL_Millis(0xFFFFFFFF);
   pinMode(spkr, OUTPUT);
   Serial.begin(9600);

}

void loop()
{
    long nA = A.capacitiveSensor(30);
    long nBb = Bb.capacitiveSensor(30);
    long nC = C.capacitiveSensor(30);
    long nD = D.capacitiveSensor(30);
    long nEb = Eb.capacitiveSensor(30);

    // Turret Opera
//    if (nA > 1000) tone(spkr, fA, 80);
//    else if (nBb > 1000) tone(spkr, fBb, 80);
//    else if (nC > 1000) tone(spkr, fC, 80);
//    else if (nD > 1000) tone(spkr, fD, 80);
//    else if (nEb > 1000) tone(spkr, fEb, 80);
//    
//    // Saints 
//    if (nA > 1000) tone(spkr, wC, 80);
//    else if (nBb > 1000) tone(spkr, wD, 80);
//    else if (nC > 1000) tone(spkr, wE, 80);
//    else if (nD > 1000) tone(spkr, wF, 80);
//    else if (nEb > 1000) tone(spkr, wG, 80);

    // Saxobeat
    if (nA > 1000) tone(spkr, sB, 80);
    else if (nBb > 1000) tone(spkr, sCs, 80);
    else if (nC > 1000) tone(spkr, sD, 80);
    else if (nD > 1000) tone(spkr, sE, 80);
    else if (nEb > 1000) tone(spkr, sFs, 80);

}

P2 — VAHN

Posted on by

GROUP NUMBER: 25
GROUP NAME: Deep Thought
GROUP MEMBERS: Vivian Qu, Neil Chatterjee, Harvest Zhang, Alan Thorne

All four group members conducted contextual interviews. All four worked on answering task analysis questions and interface design together, organizing a meeting and collaborating on a google docs.

Harvest, Alan, and Vivian drew sketches and final storyboards for the 3 defined tasks. Neil and Vivian compiled the blog post information to publish on the website.

PROBLEM SOLUTION AND OVERVIEW:

VAHN (pronounce vain) is Microsoft Kinect project that allows on-the-fly, gesture-controlled, audio editing for musical performers and recorders. Skeleton data allows the manipulation of recording software that includes features such as overlaying audio, playback, EQ modulation, and editing. Suppose you want to know what your a cappella performance is going to sound like, or you’re creating a one man a cappella song — stand in one location, use gestures, sing, record and edit. Physically move to the next location (simulating multiple singers). Use gestures, sing, record, and edit. Move to another location, realize you don’t want a middle of a snippet, and use gestures to remove that snippet. Finish the third section, and finally overlay them together. VAHN provides a cheap solution for gesture-controlled audio recording and editing.

DESCRIPTION OF USERS YOU OBSERVED IN THE CONTEXTUAL INQUIRY:

Our target users are amateur musicians with high level of skill and therefore need more music recording functionality, but who may not have much technical ability, and may lack familiarity with or access to complicated audio recording systems. This is a reasonable target user group because we imagine our system being a lightweight, simple tool which lowers the barrier to creating and enjoying music.

Interviewees were students who are deeply involved with playing and creating music. The majority consisted of students in a cappella groups and a member of Princeton University Orchestra ( PUO). Priorities included arranging music, improving their music skills, and the ability to experiment. In terms of technical skills, one was tech-savvy singer, two non-technical singer, one non-technical instrumentalists — all agreed they wanted a simpler way to record music and disliked the complicated interfaces of usual digital audio workstations where users are often limited by the fact that they don’t even know many functionalities exist.

CI Interview Descriptions

When conducting our interviews we followed this process:

  • We shared general idea of our project and asked why he would or wouldn’t use it.
  • Asked what the users would like to see in an interface (ideas without prompting).
  • Posed our ideas for interfaces to get their feedback, likes and dislikes.

We interviewed the users in their rooms or in public spaces such as coffee shops and dining halls. The process of recording music is generally isolated in a quiet spaces (usually with easy accessible software, such as those that come by default on computers), so the environment itself was not important for understanding the music-making and recording process.

Common suggestions included ease of operation, gesture control while recording, and a general expansion of features to incorporate as much functionality as possible as a recording studio. The “undo” functionality is very important for any music editing software, and needs to be intuitively integrated in the system. Editing should be able to happen in real time and be fine-tuned, so users can be precise how they edit the tracks and cut out segments on the fly. Overall, this should include all the features of high tech recording software, but maintains the simplicity of any kinect gesture software.

Suggestions from singers included the need for visual indication of loop position (like the start and finish of tracks), visual indications of beats/pulse, recommended auto-tune mode and various sound effects such as compressors and EQs. This product concept is extremely appealing to singers because other available software (Musescore, Sibelius, Finale, etc) all have electronically-generated MIDI files which doesn’t correspond to how parts will sound together in real life.

Feedback from a instrumentalist included the need to address handling of gestures — its weird to give gestures while playing an instrument. It would be useful to have different “modes” (switch from skeleton data input for kinect to depth perception data), allowing the musician to add notes. A timer countdown before recording starts would also be useful. Really like the concept of using spatial location to have different tracks.

Users also suggested that a video component would be valuable to the music-making process, especially if users could see multiple visual recordings at the same time.

ANSWERS TO 11 TASK ANALYSIS QUESTIONS:

1. What are unrealized design opportunities?

We would like people who can sing and play well (serious instrumentalists) to have an affordable pick-up-and-play recording and mixing system. Technology available is either too simplistic or too expensive and with complicated interfaces. Gesture-controls for the kinect are a new concept that can be integrated during performances or recordings. This allows people of all levels of technical skills to do a lot with the product without touching the computer.

2. What tasks do they now perform?

For music recording, our target user group uses freely available, bare-bones recording equipment instead of professional systems which are too expensive and complicated. For example, GarageBand is relatively easy to use, but still complicated for non-technical people and many often don’t recognize the power it has, and attribute undeserved limitations in sound quality because of lack of knowledge. Additionally, the editing and recording process takes many hours, at least 4 hours minimum to produce a do-it-yourself quality recording. Results are similarly for other digital audio workstations like ProTools, LogicPro, etc. Users may also use tools not intended for music recording (such as Photobooth on Mac computers) simply because they are easy and quick to use, though the quality is bad. There seems to exist a trade-off between time investment and quality with ease of use.

3. What tasks are desired?

Users would like to have a simple recording process that allows you to quickly and dynamically create music content alone or possibly collaboratively. This is allows them to improve their skills and quickly share the music, often useful in situations where others need to learn (such as new arrangements, where people would like to hear the balance of the overall blend). In particular, users want an easy interface allows musical people to take advantage of sound manipulation techniques and mixing/layering tracks without a technical tech background.

4. How are the tasks learned?

Trial-by-fire — trying to muddle through and figure out the functionalities. Our interviewees said they learned by asking people who already know how to do them, so they could never learn themselves how to make things work. There are no formal classes that they can take. Documentation online exists for these systems but they can sometimes be hard to understand — some say they don’t know where to start.

5. Where are the tasks performed?

Task performed in quiet environments (usually home, dorm room, practice room, recording spaces) and open areas. No influence of environment except for the impact on sound quality. It’s important to pay attention to how recording equipment is positioned. For example, a cappella groups sing in circular formations, which is hard to capture with a single one-directional mic, so a 360 degree mic is invaluable to capturing a recording that best matches a live performance. There is no effect due to bystanders, usually recording alone or with people who you are comfortable working with musically.

6. What’s the relationship between user & data?

Handling of data should be local, because recording music often results in catastrophic failure (the song sounds bad, the singer is off-key, etc) so you don’t want to broadcast bad recordings to others over the internet. The recordings should be private with the option to share with others. Not much of a privacy issue because the system is offline.

7. What other tools does the user have?

Laptop, home PC, cell phone, musical instruments, microphones, speakers. Microphones are extremely important to guarantee quality of sound, which might be useful to integrate into our system. There are mobile recording and mixing applications, but they are intended for casual interaction and have no ability to edit.

As mentioned before, currently available digital audio workspaces includes GarageBand, ProTools, LogicPro, and musicians even use PhotoBooth to record music.

8. How do users communicate with each other?

Not relevant for tasks related to music recording. Users often upload their recordings onto websites and youtube to share it with a larger audience.

9. How often are the tasks performed?

The tasks are performed whenever the urge to record and mix music occurs, which could be daily or once in long spans of time. Music recording is estimated to take a minimum of 4 hours, which become longer is significant editing time is taken into account. However, the time can be broken up into different chunks and worked on across an extended period of time. Therefore, it’s necessary to allow saving the workspace so users can later return to pick up where they left off.

10. What are the time constraints on the tasks?

No time constraints for the tasks, used whenever. If our device is used for live performance and collaborative music making, need minimal amounts of time to set up.

11. What happens when things go wrong?

Delete and start over again. Editing should be available on a very fine-grained scale and undos are very important.

DESCRIPTION OF 3 TASKS:

  1. Single recording — If you wanted simple gesture based recording and editing, one voice role could easily be performed this way. This would involve the performer recording with the kinect. Afterwards or during recording, the performer could edit different sections using gestures. The performer would spend minimal time interfacing with the computer this way and more time focusing on the actual performance/recording.
  2. A capella recording (“intro version”) — combining multiple tracks into one recording. This would be the “intro version” of recording because the performer would use default settings for everything (autotune on, equalizers set, beatbox helper).
  3. Real-time mixing and recording — The performer could manipulate sound properties while recording, including dubstep, EQs, and other sound processing features. This would mean that the performer has full control over every aspect of recording, much like the functionalities available in complex recording systems but with an easier, simpler interface.

INTERFACE DESIGN:

Text Description: 

The device focuses on real-time recording music with smooth playback and simple, intuitive interface allowing for many different functionalities for sound manipulation. It will allow easier and more intuitive forms of recording compared to digital audio workstations, and a more quality alternative to rock band. Users can separately record multiple tracks and combine them into one recording, adjusting sound levels on each track individually. Can add filters and change other sound properties. Body gestures would facilitate the recording and editing process. The main innovation is using spatial location of the person’s body to detect which “part” they are recording (for a capella, would correspond to soprano, alto, tenor, and bass voice parts). This would be easier for the user to understand the multiple track layering since they have to physically move to start a new track.

STORYBOARDS:

SKETCHES:

Lab 1 — Theremax

Posted on by

Team Deep Thought (Group 25):
Vivian Qu
Neil Chatterjee
Alan Thorne
Harvest Zhang

We built a device that mimics a theremin, controlled using a light sensor and touch sensitive slider (softpot). The goal of the device is to be a fun, interactive way to play music using the sensors. This device also could conceivably be used to teach young students about the physics of sound waves, since our visualizer shows the actual wave form changing in proportion to the user’s actions! One issue we had is that we needed to keep a finger on the softpot at all times or else there were spikes in the sound because the readings need to stabilize. This would be an aspect we would want to fix to streamline the experience. Future improvements could include adding buttons to change the tone of the sound and figure out how to control two different wave forms simultaneously. Otherwise, the device successfully works and allows the user to control the graphic sine wave to create different sounds.

I. Photos & Videos:

Pulse Monitor — Uses a light sensor to detect the changes in light filtered through the skin, detecting the heartbeat. Emits sound from the buzzer to match the heartbeat.

Precise Clicker — The device uses a flex sensor as a discreet clicker for presentations.

Bop-It — A fun interactive game based on the “Bop-It” games, where LED lights signal which sensor (Flex/FSR/Potentiometer) to manipulate.

Theremax — Uses the softpot and light sensor to control the frequency and amplitude of the sound wave.

We chose to implement the Theremax idea, coming up with the following storyboard for why a user would want to interact with the device.

 

 

 

 

 

Here’s a video of our device at work:

The touch sensitive slider (softpot) controls the frequency and the light sensor controls the amplitude. Sliding down on the slider increases the frequency, and covering the light sensor (less light input) reduces the frequency.

II. Parts Used

  • Arduino Uno x1
  • Photocell x1
  • Softpot x1
  • 330 ohm resistors x2
  • Breadboard
  • Wires

III. Setup

Connect the middle lead of the softpot to analog input (we used A0) and the leftmost lead to 5V and the third through a 330 ohm resistor to ground. Connect the left leg of the photocell to 5V and the other leg is connected to both analog input (we used A1) and through a resistor to ground. Download the source code, using Arduino and Processing. Upload the code to the Arduino and voila!

IV. Source Code

ARDUINO CODE

// Lab 1: Theremax
// Arduino code to read user input from softpot and thermoresistor.

// Frequency
int freq;
// Amplitude
int ampl;
// Receiving input from analog ports 0 and 1.
int thermo = 0;
int softpot = 1;

void setup(void) {
// Send input data to Processing via the Serial monitor.
Serial.begin(9600);
}

void loop(void) {
  // Read from input.
  ampl = analogRead(thermo);
  freq = analogRead(softpot);

  ampl = map(ampl, 0, 260, 0, 100); 
  freq = map(freq, 0, 500, 0, 100);

  // Send data to Processing.
  Serial.write(ampl);
  Serial.write(freq);

  delay(500);
}

ARDUINO CODE

// Lab 1: Theremax
// Processing code to output sound and graphically show the oscillator.

import ddf.minim.*;
import ddf.minim.signals.*;
import processing.serial.*; 
Serial port;
Minim minim;
AudioOutput out;
SineWave sine;
int volume;
int slider;

void setup()
{
  size(512, 200, P3D);
  port = new Serial(this, Serial.list()[0], 9600);
  minim = new Minim(this);
  // Get a line out from Minim, default bufferSize is 1024, default sample rate is 44100, bit depth is 16
  out = minim.getLineOut(Minim.STEREO);
  // Create a sine wave Oscillator, set to 440 Hz, at 0.5 amplitude, sample rate from line out
  sine = new SineWave(440, 0.9, out.sampleRate());
  // Set the portamento speed on the oscillator to 200 milliseconds
  sine.portamento(200);
  // Add the oscillator to the line out
  out.addSignal(sine);
  volume = 0;
  slider = 0;
}

void draw()
{
  // Compensate for variable readings at the extremes.
  if (port.available() > 0 )
  {
    volume = port.read();
    int temp = port.read();
    if (temp >= 0 )
    slider = temp;
  }
  float freq = map(slider, 0, 100, 60, 1500);
  // Change the frequency of the wave.
  sine.setFreq(freq);
  float volout = map(volume, 0, 100, 0, 1);
  // Change the amplitude of the wave.
  sine.setAmp(volout);
  background(0);
  stroke(255);
  // Draw the waveforms
  for(int i = 0; i < out.bufferSize() - 1; i++)
  {
    float x1 = map(i, 0, out.bufferSize(), 0, width);
    float x2 = map(i+1, 0, out.bufferSize(), 0, width);
    line(x1, 50 + out.left.get(i)*50, x2, 50 + out.left.get(i+1)*50);
    line(x1, 150 + out.right.get(i)*50, x2, 150 + out.right.get(i+1)*50);
  }
}

The Processing code relies on the Minim library and is based on the Sine Wave Signal code found here.

A2 — Seat Finder

Posted on by

1. OBSERVATIONS

I conducted both contextual interviews as well as personal observations of students and professors walking to and/or waiting for class. I asked the following prompting questions:

  • What kind of activities do you do if you get to class early?
  • What do you see your professors do?
  • Can you describe your actions if you are late lecture?
  • What’s your preferred method of walking to class? With friends, listening to music, etc. Are there any inconveniences related to that method of walking to class? (ex. can’t find friends)
  • What are some things you would like to do in the 10 minutes before class, but it’s hard to do?

The results from the contextual interviews are as follows:

Molecular Biology major (F), sophomore, on the way to class in Friend (Early Morning) —

  • If she gets to class early, plays with smartphone (checks email/facebook/reddit), may use laptop before class but depends on whether she will be taking notes on her laptop for the particular class.
  • She may check the following class-related material: lecture notes (download and open), reviewing notes from last lecture for the quiz at the beginning of lecture (cell & dev bioogy), syllabus and upcoming assignments.
  • Professors prepare lecture materials, don’t really do anything before lecture, erase the chalkboard.
  • If late to lecture, she will be frantically copying down things on the board and trying to follow from there.
  • Doesn’t ride a bike, mostly walks to class by herself and walks with a friend if she knows they are walking to the same class.

ORFE major (F), junior, waiting for class to start in Frist (Early Morning) —

  • If I get to class early, she usually talks with people that I’ve walked to class with or check email/facebook on phone/laptop.
  • Professors have played music and youtube videos (tangentially relevant to class), answer students’ questions, pass out papers before class, one professor has assigned seating so he places namecards out for every student.
  • It’s really awkward to walk into class if late to lecture, will feel bad and sit and listen for first 10 minutes without taking notes or getting laptop out of bag (feeling guilty). Early to seminar more often since participation matters more.
  • Difficult to find seats in large lecture halls, especially if it’s a large class. Don’t know where friends are sitting and sometimes need to climb over seats.
  • By the second week of class, basically know which friends are in similar classes and walk with them.
  • Would like to “preview” the lecture or read a little bit of what professor is going over. Other thing that would be nice is if we could go over what went on in previous class. Also, hard to find bathrooms in McCosh.

Woodrow Wilson school major (M), senior, eating lunch — 

  • Check email on iphone, depending on class review notes from last lecture, think about other work.
  • Professors speak with various students, fiddling with projectors, very often speaking with preceptors.
  • Late to lecture actions depend on size of lecture — large lecture, would just walk in and find first empty seat and sit down; small lecture/class, everyone sits in same seat (you have a seat), know exactly which seat to go to.
  • Like to walk to class quickly, optimal route.
  • Problems: don’t know where water fountains are, can’t sharpen pencil (no pencil sharpeners!).

Personal interviews — I also observed professors and students in my own classes.

Sociology Professor (morning class): early to class, uses projector (has some trouble setting up and interacting with the system), has folders open for students to hand in assignments on a table at the front of the lecture hall, checks lecture material on laptop, watches and waits for students to get seated, starts class a little after the beginning time for lecture after students stop trickling into the lecture room.

Students before lecture: checking email/read news on laptops, talking among themselves after walking into class with friend, eating snacks (continues into class), asking professor/TA specific questions about assignment or previously discussed topic, turning in an assignment before class, signing in on a paper sign-in sheet that the professor passes around.

In summary, I observed the following design opportunities:

  •  Make finding facilities simpler, ie. bathrooms, water fountains, pencil sharpeners (owned by other students?) seats in lecture halls…
  • Help students look up class information that can be accessed in 10 minutes — quick due dates, admin info, professor’s email.
  • Increase productivity of lecture learning with regards to reviewing material, previewing material, attendance…
  • Short fun, interactive games to pass the time before lecture.
  • Coordinating information with friends such as walking together to class.

2. BRAINSTORMING

The following are a list of ideas which Prakhar Agarwal (pagarwal@) and I came up with:

  1. Morning Announcements — broadcasting information about the campus activities of the day on the projector prior to the beginning of lectures.
  2. iSign-In — attendance mobile application which takes advantage of the smartphone’s “check-in” capability, letting students use phone GPS to record lecture attendance and replace sign-in sheets.
  3. Upvote Me — Mobile application forum where students can submit class-relevant questions, allowing other students to “upvote” the best questions so that the professor answers them first.
  4. ClickNAnswer — Mobile or web application which allows students to provide real-time feedback/answers to a professor’s questions, similar to the functionality of an i-clicker from physics classes.
  5. Flashcard Matching — Allows students to create flashcard games for other students to play, like Sporcle for language classes.
  6. Popcorn — Students log into a mobile app which allows the professor to randomly pick people to answer questions, facilitating class discussions, and may be a good way to record participation points.
  7. MyDeadline — A mobile app where you can check the upcoming deadlines for a class (assignments, projects, etc), available in both calendar or task list form and can integrate with Blackboard.
  8. SeatFinder — Mobile application that shows you which seats are full as a lecture starts to fill up and where your friends are sitting, hopefully motivating students to go to class.
  9. CampusLife — Mobile calendar tailored to daily Princeton campus events, integrating with Facebook events to also show how many of your friends are going so you can better coordinate, as well as how many TigerTickets you have left.
  10. WalkWithMe — Combines class schedule data from Ice and lecture room locations to show you a list of friends that have class nearby so you can walk the same direction together.
  11. MyStudyGroup — A discussion board viewable on your mobile phone where students can aggregate what they learned from the previous lectures, rather than the professor posting the content.
  12. Audit Course Recommender — For students with hour or more gaps in between classes, the application finds nearby lectures that are taught in that time so the student can audit the class.
  13. Grade Calculator — Takes the median/mean/grade data from Blackboard and instantly computes a score, as well as possibly projecting the scores needed on future assignments to achieve the desired grade.
  14. Map My Schedule — At the beginning of the semester, it’s annoying to keep looking up where your next class is so this app will plan out a route for you through the whole day!
  15. iHydrated — Shows the water fountains nearby based on the building you are in (using phone GPS or selecting the building in a map), plus it keeps track of how much water you drink everyday, making sure you are fully hydrated.
  16. ForgotMyCharger — Students can lend their electronics equipment for short-term use, like mac projector dongles/iphone charges/laptop chargers, could be useful for students who forgot chargers in class and can’t walk all the way back to their rooms.
  17. Lecture++ — For classes that taught many years in a row, students can rank specific days of specific classes that have been really rewarding so students outside of the class can go to that specific lecture.

3. SELECT 2 FAVORITE IDEAS

8. SeatFinder — This app would be for students trying to find where their friends are sitting in class, looking for seats in very full lecture rooms, or those who are coming in late and improves the seat finding process, which can be very awkward, wastes time, and often causes unnecessary distractions for students already seated.

2. iSign-in: For classes where attendance in lectures matter, it’s really low-tech and inefficient to have students initial a piece of paper — students have to wait a long time for the paper to pass around, it allows for students to sign their friends in who aren’t actually in lecture, as well as wasting a lot of paper making these sign-in sheets.

4. PROTOTYPES

SeatFinder prototype:

This slideshow requires JavaScript.

iSign-in prototype:

This slideshow requires JavaScript.

5. FEEDBACK

Here are the following students I consulted for user testing:

Some pictures from their tests:

This slideshow requires JavaScript.

  • Gary asked if the black seats were the ones that were taken. Lauren also asked that question, and suggested that the open seats should be in color to be more visible and intuitive.
  • Gary liked the idea that the star (“Faves”) option synced to phone contact. He would also use the Facebook friends option more than the other two.
  • All the users said that if the lecture is really full (as shown by the app), would still attend.
  • One question Gary asked about a potential problem: would we be trying to get this for every room on campus?
  • Gary and Amy asked how reserving/claiming seats would be enforced — brought up the good point that if there are other students not using the app, they could take your seat.
  • Amy thought this would be useful for more specific settings — like exams, could find seats quickly with time crunch before exam and enforce the honor code (after the face, see who was sitting next to whom if there was a honor code violation report).
  • Both Amy and Lauren said that they would use either star (“Faves”) contacts or Blackboard, thought one was redundant.
  • Amy didn’t think that either Option #1 or #2 is effective — suggested that the most fool-proof solution is to integrate it with the classroom (physically the seat is “marked” or show people in some way that it is taken).
  • Lauren brought up a potential usage problem with people who are biking/longboarding to class — may encourage dangerous use of phone + bike/skateboard.
  • All three users didn’t like the current way the Blackboard functionality is set up — hard to avoid people. Recommended solutions were a “permissions” system similar to Ice where you have to accept the other person’s  request. Another option suggested was an “allow people to find me” button which would make them findable through Bb.
  • All the users thought the ability to see where the outlets are in class was great.
  • Nobody picked the “Load more classes…” or “Looking for other classes” options.

6. INSIGHTS & FUTURE IMPROVEMENTS

  • Make the interface simpler — based on the user testing, there’s no need to have both the star (“Faves”) option and blackboard option, so one should be removed. The users seemed to prefer the “Faves” option. Also, none of the users tried the “Load more classes…” or “Looking for more classes?” option, which suggest they only want to see their immediately upcoming class. Can just show the upcoming class synced from Blackboard.
  • Improve the floor plan interaction — Making it clearer which seats are taken/available would be better for the users. One suggestion is to make the available seats colored in the default setting and not simply white.
  • Privacy — Users want to determine themselves what information is publicly available  (like who is sitting in what seat). This why the blackboard option was not popular
  • Feasibility — Need to consider how to effectively get the building plans for all the lecture halls. Do we personally go out to check where the outlets are? Need to find efficient way or if there are already available resources (online floor plans?).
  • Enforce usage — From the user input, it seems like the best way to ensure that seats aren’t unfairly taken is if the app integrates with the course as a way to take attendance for the professors. This would mean that the administration needs to be consulted about the possibility of this app being approved and used by the university.
  • Consider other applications — Amy brought up how this application would be very useful for taking exams and honor code enforcement. However, how often do honor code violations occur? Cost-benefit analysis for having this option need to be conducted and evaluated.
  • Safety — How would this application impact students who bike or skateboard? This may encourage unsafe cellphone usage. May consider a way to reserve a seat using voice (calling a specific phone number).