Group 25 — Deep Thought

(Vivian, Neil, Harvest, Alan)

I. PROJECT SUMMARY 

Aca-Kinect is a gesture based music recording software that interfaces with an Xbox Kinect, using gesture-based controls to allow on the fly track editing, recording, and overlaying.

II. DESCRIPTION OF TEST METHOD

i. Describe your procedure for obtaining informed consent, explaining why you feel this procedure is appropriate. 

We decided to use a modified adult consent form which includes an overview of the procedure, confidentiality, risks, and benefits. By signing, the user agreed to participate, and also optionally agreed to be filmed/photograph for our future reference. We felt the consent form worked well in covering all the concerns for the prototype testing process and was a good physical record of the testing. For the procedure itself, we first helped the participant understand what was on the consent form by providing a description of our project and giving a summary of each section on the consent form; then we gave the participant a hard copy of the consent form to read and sign. This process took around 3-4 minutes.

Here’s a link to our consent form.

ii. Describe the participants in the experiment and how they were selected. 

Participant 1 was selected for her experience with various singing groups. She has little recording experience but plenty of experience singing and arranging for live performance. She was selected partly for her expertise in singing and lack of recording experience.

Participant 2 has a lot of experience singing in several a cappella and choir groups on campus. She has been involved in music composing and arranging processes and uses existing recording software. She was selected for her knowledge of existing recording software and familiarity with a capella music-making.

Participant 3 was an electrical engineer that has had someone background with the XBox Kinect, so gesture based actions shouldn’t be a problem. He also has had no experience with singing, music, or recording. This allowed us to capture the other end of our user-group, while compensating that with experience with gesture-based software.

iii. Describe the testing environment, how the prototype was set up, and any other equipment

The testing environments were a bit less than ideal. Both locations (Frist 100 level, ELE lab) had the necessary space for gestures and adequate lighting. However, neither space was an actual stage or recording space, as it would have been difficult to secure a stage space, and a recording studio would be prohibitively small. Both locations had some background noise and the occasional passerby but ultimately suited our needs well. The paper prototype was set on a table a few feet from where the participant was standing. For any future testing, we would aim to find spaces that better simulate recording environments.

iv. Describe your testing procedure, including roles of each member of your team (order and choice of tasks, etc). Include at least one photo showing your test in progress. Provide links to your demo and task scripts.

Our testing procedure was as follows:

• Team member describes the project, prototype, and testing procedure
• Participant reads and signs consent form.
• Gives the participant a demo (using the demo script) of how the system works. Asks if the participant if they have any further questions.
• Read all of the first task instructions. Wait and observe the participant completing the task, prompting them if necessary.
• Repeat for the second and third task. Since the tasks are increasing in order of difficulty, instructions for the last two tasks are read one-by-one for the participant to follow.
• Ask for any further feedback/feelings from the participant.

Vivian introduced the project, walked the participant through the consent form, gave them time to sign the consent form, and ran through the demo script. Harvest wrote and read the tasks script with Vivian assisting (moving the prototype pieces). Alan took pictures and video during the testing process. Neil transcribed and took notes on the process, outcomes, interesting observations, etc.

Here’s a link to the demo script and the tasks script.

III. SUMMARY OF RESULTS

In general users performed well on each task. The main issue was whether the participants fully understood the concepts of master recordings and loop recordings. The third participant asked a question about this and we explained it to him in great detail, so he was able to easily complete the three tasks. On the other hand, the first two participants were slightly confused on the third task, when they were instructed to make a master recording and then record a loop in a specific panel while the master recording was going on. The concerns were twofold — the first participant didn’t understand the difference between (or remember the gestures for) the master recording and a loop recording. The second participant didn’t understand why someone wanted to do a loop recording during a master recording, when there was an option of doing it before. Therefore there was a minor usability problem in the first case, and a usage question in the second. With the second participant, we spent approximately 5 minutes discussing the reasons for doing the third task the way we framed it and the real-world applications. Besides the question of motivation, there was no problem with usability since she was quickly able to complete the task.

The participants all thought the prototype was simple and easy to use — they used words like “cool,” “easy and fun” and jokingly “a workout.” Some of the participants seemed a little embarrassed singing in front of a paper prototype or got tired by the end and just said words instead of simulating music-making — perhaps as we evolve our project into a higher-fidelity prototype the participants will be more engaged and interested because of the addition of audio feedback. Additionally, the second participant asked about volume control on each individual panel — this is a feature we will need to make more explicit in future iterations of our project! There was also a question about why we had four panels — this is a cosmetic problem and may require us to suggest to the user to organize their recordings into different sections (percussion, bass, tenor, alto, etc).

IV. DISCUSS RESULTS

A major result of the testing was confirmation that the workflow is fundamentally fairly accessible to musicians or otherwise who have no prior experience with recording music using tools ranging from Garageband to loop machines. The lack of the steep learning curve, expense, and complicated setup that comes with more advanced and more capable tools like a dedicated loop machine means that this Kinect application can introduce more people to at least basic music production with some interesting looping and multitrack features; those who outgrow this application will probably develop an interest in the more advanced tools that do offer much more functionality.

However, we also discovered that some parts of this product remain slightly confusing to users who had never encountered recording tools before; there was a little terminology confusion surrounding the meaning of “master recording” and “loops,” and it was very difficult to impress upon the users the fact that all the loops would be playing simultaneously on top of each other. This is a fundamental issue with paper prototyping an application like this, and a high-fidelity prototype that actually provides accurate audio feedback would go a long way to making the experience much more understandable. As such, the experiment also cannot reveal much about what kind of music users would actually create using this product, since we only simulated the loops. It would be instructive and useful to see the complexity of music that users would produce, whether they actually use various pieces of functionality or not, and so on, but we don’t know at this point since we are not fully simulating the generation of a piece of music with many parts.

V. PROPOSED SUBSEQUENT TESTING

We feel that our paper prototype demonstrated its purpose. We aimed to explore functionality and see how users interacted with the ability to control features via gestures. The biggest issue we discovered stemmed from the lack of audio feedback. It’s difficult for a user to have an idea of what he/she is doing in recording software without the ability to record/listen. The confusion our users had in the tests was not an interfacing problem, but rather an understanding that we feel will be evidently resolved when users interact with a higher-fidelity project.