Your group number and name.
Group 8 – Backend Cleaning Inspectors
First names of everyone in your group.
Dylan
Tae Jun
Green
Peter
A 1-sentence project summary.
Our project is to make a device to improve security and responsibility in the laundry room.
Introduction (5 points). 1 paragraph. Introduce the system being evaluated and state the purpose and rationale of the experiment.
We have built a laundry security system prototype that is intended to be used in Princeton University laundry rooms to help protect student’s personal clothing when it is left alone to be washed and/or dried. The purpose of this experiment is to test out three tasks that will be performed by users of our system. We wish to analyze and observe test participants attempting each task and to determine if there are any changes we need to make in the implementation of these three tasks.
Implementation and Improvements (5 points). Provide a link to your P5 submission. If you have changed your working prototype at all since submitting P5, supply a brief bulleted list of the changes made since P5. (It is not necessary to change your prototype from P5 before doing P6.) This section should be no more than 1 paragraph.
Method (10 points).
Participants: 1 paragraph. (Who they are, how they were selected.)
Participant 1) 21 year old Junior, Male, MAE major
Participant 2) 19 year old Freshman, Female, undecided
Participant 3) 19 year old Freshman, Male, undecided/Econ
See demographic questionnaire results below for more information.
We set up our study in a laundry room in Butler, and approached the first three people to enter the room. They all agreed to do the study and both genders were represented.
Apparatus: 1 paragraph. (What equipment did you use, where did you conduct the
test.)
We used our model prototype to test the three tasks. We do not need any wizard-of-oz techniques in our model prototype currently, so we were able to just use our prototype. We tested it out in a public laundry room at Princeton University.
Tasks. ~1/2 page. Describe the tasks you have chosen to support in this working
prototype (3 short descriptions, 2–3 sentences each; should be one easy, one
medium, one hard). If you have not changed the tasks from P5, you can re-use your text from P5 here. Otherwise, if you have changed the tasks, explain how and why. In any case, explain why you have chosen these tasks.
1. Current User: Locking the machine:
– The Current User inputs ID number into locking unit keypad. The product will then try and match his number to a netid in our system. If it finds a match, it will ask the user if this is their id. They can then answer yes or no and if yes, the machine locks. This netid is also the netid that will be sent emails if an alert is sent. This task would be medium in difficulty as the user has to ensure that he/she enters the right 9-digit number.
2. Next User: Sending message to current user that laundry is done and someone is waiting to use the machine:
– When there are no machines open, the next user can press a button to send an alert at any time during the cycle. When the button is pressed, and the id of the next user is verified, an alert will be sent to the current user saying someone is waiting to use the machine. The difficulty of this task would be easy/medium, as the next user has to input his/her 9-digit ID number.
3. Current User: Unlock the machine:
– If the machine is currently locked and the current user wishes to unlock the machine, the current user must input his princeton ID number. Once he has done this, the system then checks this id and tries to find a potential match to a net id. If it does, it will ask the user if this is his net id and on yes, it will unlock. This is a medium/hard task, as the user must input his number and confirm to unlock.
We chose these specific tasks, because these are the three main (and only) tasks our system is designed to execute (at least in this model…. in future models, additional features could be implemented).
Newly added to the machine was a grace period allowed after the first alert had been sent (if the alert button is pushed while the laundry is still running, aka during “laundry time”, then the alert is sent immediatly after “laundry time” is finished). This specific feature did not have any impact on the above tasks; however we did explain the concept to them and observed their reactions to it.
Procedure. 1 paragraph. Describe how you conducted the study.
Once we found a willing participant, we obtained his consent using our consent script, and then proceeded to describe the basics of what our system was and how it works, leaving out any important instructions on how to perform the three tasks. We also described the “think aloud” procedure following the protocol given at http://www.hu.mtu.edu/~njcarpen/hu3120/pdfs/thinkaloud.pdf. We then watched and listened to the participant attempt the three tasks and observed any and all things that they said and did. We then repeated this procedure for the other two participants.
Test Measures (5 points). Describe what you measured and why. Bullet points are
encouraged.
We measured the following:
-
task time – the time it took to complete each task, measured in seconds
-
self-reported satisfaction with each task
-
number of errors or mistakes
Results and Discussion (25 points). ~4 paragraphs. Provide results of your tests.
Describe what you learned from the user study. Document any changes that you plan to make in your prototype as a result of the study.
Times (seconds)
Task 1
1 – 20
2 – 54
3 – 27
Task 2
1 – 18
2 – 24
3 – 22
Task 3
1 – 16
2 – 14
3 – 32
Satisfaction (1-5)
Task 1
1 – 5
2 – 3
3 – 5
Task 2
1 – 5
2 – 5
3 – 5
Task 3
1 – 5
2 – 5
3 – 4
Number of errors (#)
Task 1
1 – 0
2 – 1
3 – 0
Task 2
1 – 0
2 – 0
3 – 0
Task 3
1 – 0
2 – 0
3 – 1
Overall, the results of our user study were encouraging. The users generally had a good feel for what was required of them to accomplish the tasks set before them. The first user was able to accomplish all tasks with no problems. The second user had a bit of trouble with locking the machine, and the third user had a small problem with locking it again.
Problems
The second user had a problem with locking the machine. She couldn’t figure out what the machine was asking for when it said “Enter ID: “. At Princeton, every student is assigned a Princeton ID number when they enroll at the school. However, this number is only used rarely, as most activities only require the student to swipe their prox. This is probably the main reason behind the problem. The third participant messed up when typing in his ID again to unlock the machine. We judged this is as a rather inconsequential error, but noted that there may be easier ways to authenticate a user rather than have them type in a 9-digit number the user most likely does not have memorized each time they lock or unlock the machine. Additionally, although not specifically part of any of the tasks, users one and three expressed a little bit of confusion when informed about the concept of the grace period. Although inconsequential to the tasks they were asked to perform, it was still noted and taken into consideration.
Cool! Comments
All three users, when prompted for the second task, made some kind of remark about how they thought the ability to send an email to the “current user” of the machine was really cool. We saw this as a very good sign for our product.
Although, we mostly received very positive reviews from each participant, there are some areas where the product could be improved for future usability tests. First, we could arrange an easier way for user authentication. Possibilities for this would include a prox swiper, a netid input (rather than Princeton ID), or other Princeton identification means. Second, we could attempt to inform the user more about what is going on with the machine currently. A couple of the users expressed confusion about the concepts of the grace period and alert sending when the laundry was not done. This could be easily improved in future versions.
Appendices (5 points).
Provide all things you read or handed to the participant: consent form,
demographic questionnaire, demo script, post-task questionnaire and/or interview
script.
consent form, demo script, demographic questionnaire
Also provide raw data (i.e., your merged critical incident logs, questionnaire
responses, etc.)
Demographic Questionnaire Responses:
Participant 1:
1.) Male
2.) 21
3.) single
4.) summer internship
5.) college junior
6.) MAE
7.) every 2 or 3 weeks
8.) the keypad on the princeton doors?
9.) Nope
Participant 2:
1.) Female
2.) 19
3.) single
4.) dining hall
5.) college freshman
6.) undecided
7.) every week
8.) No, I don’t think so?
9.) Yes, had my laundry taken out of dryer and put in washer, when some of the things shouldnt be washed. It made me very mad.
Participant 3:
1.) Male
2.) 19
3.) single
4.) unemployed
5.) high school
6.) college freshman
7.) every two weeks on Sundays
8.) Princeton keypads?
9.) Not really. Have had it taken out before, but I didn’t care too much.
***** Post-Demo Questionnaire
Based on the following scale from 1 to 5, please rate your agreement with the following questions:
0.) Strongly disagree
1.) Disagree
2.) Slightly Disagree
3.) Slightly Agree
4.) Agree
5.) Strongly Agree
1. I felt secure with my laundry using this device.
-Subject 1: 4
-Subject 2: 5
-Subject 3: 4
2. I felt less guilty pulling the laundry out of the machine after the grace period is expired.
-Subject 1: 5
-Subject 2: 5
-Subject 3: 5
3. I believe this device will increase efficiency in the laundry room.
-Subject 1: 4
-Subject 2: 3
-Subject 3: 5
4. I would be less irritated if my laundry was taken out after a warning and the grace period.
-Subject 1: 4
-Subject 2: 4
-Subject 3: 4
5. When mass-produced, we expect this device to take $30/device to manufacture. I believe this is a reasonable investment by the university.
-Subject 1: 5
-Subject 2: 4
-Subject 3: 5
6. I would definitely use this device in my daily laundry interactions.
-Subject 1: 5
-Subject 2: 5
-Subject 3: 4
7. It was easy to (secure the laundry machine) using the device (Task I/II/III)
-Subject 1: 5/5/5
-Subject 2: 5/5/5
-Subject 3: 5/4/5
8. The time it takes to perform the task I/II/III is reasonable
-Subject 1: 5/5/5
-Subject 2: 5/5/5
-Subject 3: 5/5/5
1. How long do you think the Grace Period should be?
-Subject 1: 10 minutes
-Subject 2: 5 minutes
-Subject 3: 7 minutes