Group 24 – the Cereal Killers
Andrew, Bereket, Ryan, and Lauren

The brisq bracelet is an easily-programmable, lightweight, wearable system that allows users to control their computers with gestures picked up by a motion-sensitive bracelet.

Introduction
Brisq is designed to be a simple, wearable interface that allows users to have customizable control over their computers without making use of a mouse or keyboard.

Implementation and Improvements
P5: P5 Blog Post Link

In addition to what we had in our previous prototype, we have finally integrated out GUI with the backend for the program, enabling users to actually record gestures without a “wizard of oz” approach to testing. However, since we do not yet have the accelerometer-based bracelet working, w will still be using one of our team members to simulate gesture execution.

Method
We chose 3 volunteers who had a diverse background of majors, in order to attempt to spread our test subjects across a wide spread of computer expertise. One of our test subjects Is a Woody Woo major, one is a Civil engineer, and one is a COS major. We chose this spread in order to give us a good spread of computer expertise in our test subjects.

For this test, we used our current, semi-functional brisq prototype. It consists of the program backend which can record and execute a series of mouse and keyboard shortcuts, our GUI which is now functionally connected to the program backend, and an arbitrary bracelet + “Wizard of Oz” acting in place of the accelerometer-based bracelet which is still under construction. The tests were all conducted in Quadrangle Club on members who volunteered for our experiment.

Task 1, Cooking: The test subject is asked to look up the recipe for a new dish they do not know how to prepare, and then prepare it while using brisq to navigate through the online recipe while cooking.

Task 2, Watching a movie: The test subject is sitting on a couch and is asked to map gestures to various controls for the movie, e.g. volume changes, pausing, fast forward, rewind, etc, and then use them to control the movie during playback.

Task 3, Disability test: The test subject is asked to perform a specific test while only using one arm to simulate a disability. The subject is given a cast to wear on their dominant arm, and then asked to type, and to open and close various programs.

Procedure
The user was able to record the computer action using the gui.

We used a remote desktop in order to replay the gestures to simulate the bracelet gesture recognition.

When the user did a gesture, the action was replayed using the remote desktop so the user did not have to press the replay button.

For each user, we changed up the order of the tasks

Test Measures
Qualitative:
Do users pick commands that are appropriate for the task.
– If their choice of commands if inefficient, do they need suggestions on tasks?
Do people easily remember which gestures they assigned to which actions?
– If they get confused, what confuses them? Do they mistake one gesture for another, or forget them entirely?
– Do they get flustered, confused, agitated and mess things up when they forget?
Do people actually use the gesture they create, or do they just forget after a while that they have the bracelet at their disposal?

Quantitative:
Tasks 1 and 2:
record the number of gestures performed during set cooking/movie time
Task 3:
record the time taken to complete the given task

Results and Discussion

User 1
Background: 21, male, sophomore, computer science, Dell Windows 8, computer level 5, no other input devices, no bracelets or watches, >10 hrs of active daily use, > 20 of passive daily use (confirmed), most common activity: running big stuff while I sleep

Cooking
-He was confused by the labels: record gesture or record computer action?
-maybe an explanatory paragraph would help to clarify
-we needed to explain that Esc ends the recording (the user didn’t want to minimise the new window to go back to the GUI to end the recording)
-he didn’t want to have the GUI window up after when he wanted to use the gesture
-page up/down doesn’t work

Gesture1 – swoosh up → open window with his favourite recipe site
Gesture2 – right/clockwise twist → page up
Gesture3 – left/counterclockwise twist → page down

TV
-he had trouble remembering functions and their associated gestures.
-he discovered he needed to record more gestures as he went

Gesture1 – big counterclockwise circle → open TV guide
Gesture2 – big clockwise circle → change the input activity
Gesture3 – twist clockwise → channel up
Gesture4 – twist counterclockwise → channel down
Gesture5 – hand palm up → volume up
Gesture6 – hand palm down → volume down
Gesture7 – punch → ok

Injured
-he had trouble thinking of good actions for computers, but using a computer involves complex tasks

Gesture1 – 90 degree right turn of the wrist → make a key work as its mirror key (to allow typing with one hand)
Gesture2 – a tap → backspace
Gesture3 – swipe to the left → close window
Gesture4 – swipe to right → open chrome
Gesture5 – lift hand up quickly → enter key

User 2
Background: 21, female, junior, civil and environmental engineering, laptop: OSX, level 4 computer user, no other input device, 9 hrs of active daily use, 11 hrs of passive daily use, most common task: doing homework, playing music, facebook

Injured
– she was confused about the order of pressing buttons
-she wanted multitouch gestures form IOS
-she used gestures in a limited way → need suggestions for gestures?
-used Esc button to not open window aagain
-play/pause doesn’t work
-page up/down and window key doesn’t get recorded
– she had trouble remembering actions
-she wanted to be able to hold and move icons, files and windows

TV Room
-wanted 2 handed gestures
-she wanted to have volume as level of arm height

Gesture1/2 – hand up/down → channel up/down
Gesture3/4 – hand side/side → volume up and down
Gesture5 – big swipe → mute
Gesture6 – circle hand clockwise/ counterclockwise→ forward / rewind
Gesture7 – punch → play/pause
Gesture8 – diagonal swipe → on/off

Cooking
-she thought that we could have a recipe sponsorship, so when the user does a gesture, it goes to the sponsored recipe site

Gesture1 – swipe hand down → scrolling
Gesture2 – hand to the right/ left → next page/ previous page
Gesture3 – hand in → magnify

User 3
Background: 20, female, sophomore, woodrow wilson school, computer level 2, input device: touchscreen, both bracelets and watches, 5-6 hrs of active daily use, 7-8 hrs of passive daily use, most common task: internet, playing music, writing papers

TV Room
– wanted to know how subtle the gestures are, i.e. how fine can the arm motions be?
– she thought arm motions are difficult, would rather do finger gestures. She wanted to replace the bracelet with a ring of glove.
– wanted more instructions. was confused about what gestures to pick and how to use the device.
– Some of the gestures she thought of were similar to each, and would be difficult for the classifying algorithm

Gesture1 – up/down → volume
Gesture2 – side/side → channel

Injured
Gesture1 – move arm in xy plane to control
Gesture 2 – punch in the z axis to click and double click

Cooking
Gesture1 – up/down → scroll
Gesture2 – diagonal arm swings → zoom in and out

User 4
Background: 21, male, junior, economics, lenovo laptop Windows 7, computer level 3, imput device: fingerprint scanner, neither bracelets nor watches, 6-7 hrs of active daily use, 6-7 hrs of passive daily use, most common task: facebook, buzzfeed, cracked articles, youtube videos

Injured
Gesture1 – hand side/side → switch between window
Gesture2 – diagonal hand swipe → close window
Gesture3 – hand up/down → minimise/maximise window

Tv Room
Gesture1 – wavy hand motion → play
Gesture2 – stop hand motion → pause

Test subject 1, cooking

Test subject 1, watching TV

Test subject 2, watching TV

Appendices

Consent form – Consent Form Link

Demographic Questionnaire –
Do you own a computer? If so, what kind (desktop or laptop? OS?)
Please describe your level of computer savviness, on a scale of 1 to 5 (1 – never used a computer/only uses email and facebook; 5 – can effectively navigate/control your computer through the terminal)
How many hours a day do you actively use your computer?
How many hours a day is your computer on performing a task for you (i.e. playing music, streaming a video)?
What activity do you perform the most on or with your computer?
Have you ever used something other than a keyboard or mouse to control your computer?
Do you wear bracelets or watches?
Have you ever needed crutches, a cast, a sling, or some other type of device to assist with a disability?

Demo Script –
Our project intends to simplify everyday computer tasks, and help make computer users of all levels more connected to their laptops. We aim to provide a simple way for users to add motion control to any of their computer applications. We think there are many applications that could benefit from the addition of gestures, such as pausing videos from a distance, scrolling through online cookbooks when the chef’s hands are dirty, and helping amputees use computers more effectively. The following tests will be aimed at seeing how users such as yourself interfaces with brisq. Brisq is meant to make tasks simpler, more intuitive, and most of all, more convenient; our tests will be aimed at learning how to better engineer brisq to accomplish these goals.

Our GUI has a simple interface with which users can record a series of computer actions to be replicated later with a motion-based command of the brisq bracelet.
[on laptop, demonstrate how users can record a mouse/keyboard command]

Now that a gesture has been recorded, you can map it to one of our predetermined gestures. shake brisq to turn it on or off, performing the gesture at any time when on to activate the series of actions on your computer.
[show how to set gesture to a command]

We think life should be simple. So simplify your life. Be brisq.

Interview script –
We have just finished showing you a quick demo of how to use brisq. Please use our program to enter the computer shortcuts you would like to use and the gestures to trigger them for the task of cooking with a new online recipe using your laptop. The bracelet we will give you is a replica of the real form, and one of our lab members will assist us in simulating the program.
We ask that you then carry on with the activity as you normally would. Do you have any questions?

[user performs task 1]

Now, please use the program to enter the computer shortcuts you would like to use and the gestures to trigger them for the task of watching a movie with your laptop hooked up to a tv, and you are lying on the couch. The bracelet we will give you is a replica of the real form, and one of our lab members will assist us in simulating the program.
We ask that you then carry on with the activity as you normally would. Do you have any questions?

[user performs task 2]

Lastly, please use the program to enter the computer shortcuts you would like to use and the gestures to trigger them for the task of using a computer with an injured arm/cast/etc. For this, we will ask that during the test, you refrain from using one of your arms for simulation purposes. The bracelet we will give you is a replica of the real form, and one of our lab members will assist us in simulating the program.
We ask that you then carry on with the activity as you normally would. Do you have any questions?

[user performs task 3]

Thank you! We will now ask you a short survey based on these tests.

Post-Task Questionnaire –
Did you find the device useful?
How well did the gestures help you perform the task? 1 to 5 where 1 is it hindered you, 3 is neutral/no effect, and 5 is it helped you a lot
How much would you be willing to pay for such a device?
What was the best thing about the product? What was the worst thing?
What other times do you think you would use this?
Any ideas for improvement/other comments?