PFinal – Name Redacted

Group 20 – Name Redacted

Brian, Ed, Matt, and Josh

Description: Our group is cre­at­ing an inter­ac­tive and fun way for students and others who do not know computer science to learn the fun­da­men­tals of com­puter sci­ence with­out the need for expen­sive soft­ware and/or hardware.

P1 –

P2 –

P3 –

P4 –

P5 –

P6 –


This video shows a compilation error that is detected by the TOY Program.  It tells the user where the error occurred and what the arguments of the command should be.

This video shows a TOY Program that prints out the numbers from 5 to 1 in descending order.The program takes 1.5 seconds to execute each step so that a user can see the program progress.  There is an arrow that is over the line that is currently being executed.

Number Bases

Bulleted List of Changes:

We made these changes after watching the testers use our code.

  • Debugged some of the assembly code.

  • There is application persistence – walking in front of a tag does not cause the application to restart.

  • Write on the back of the tags what it says on the front so that users can easily pick up tags that they need.

We made these changes based on the suggestions of the users.

  • We added debugging messages to the “Assembly” program to provide feedback when a user creates a compile or runtime error.  Before we just highlighted rows.

  • Clean up the numbers display so that decimal, binary, and hexadecimal have sections.

  • The “Numbers” program shows how the numbers are generated.

  • Initialization does not require users to cover up tags.

Evolution of the Goals/Design (1-2 Paragraphs):


Our idea began with the a problem: students in America are not learning enough about Computer Science.  The goal that we set during P1 was to create a technology that helped facilitate CS education to students of all ages.  The first step we took toward that goal was to refine it further.  At first we were thinking about how we could make learning CS more engaging.  We toyed with games and other teaching tools.  Ultimately, however, we decided that we wanted our goal to be to make a lightweight, relatively inexpensive teaching tool that could be used by teachers to engage students in the classroom and provide students with a tactile interaction with the concepts they are learning.


With this goal in mind, we set out on the process of design.  We immediately took up the technology of AR tags.  We really liked the fact that you could print them out for essentially pennies and make new lessons really cheaply.  We also liked the idea the teachers could throw together this tool from technology they might already have floating around a classroom (i.e. an old webcam or projector).  Again the process of design, like the process of goal making above was about specificity.  Once we have the basic design of AR Tags, we delved deeper into how this design will actually look.  We created metaphors such as “Tag Rows” where our system identifies horizontal rows of tags automatically.  We played with various types of user feedback.  Displaying lines and debug messages on top of the tags.

HCI as a Platform

Moving beyond the scope of the required sections of our projects, I think that one of the areas where we went above and beyond is not only creating a stellar interface but actually creating a usable platform for interface development.  We created sandboxes, metaphors, and ways that developers could safely create applications on our platform quickly.  We expose data structures (like the Tag Library) that allows our development users a myriad of options for accessing the data on the page.  Although clearly not as advanced or fleshed out, our platform is akin to the iOS platform which allows applications to take the complicated problems of capacitive touch and exposes them as simple events for developers.  In the process of making this platform we have learned a lot about the possibilities of our project.

Critical Evaluation of the Project (2-3 Paragraphs):

After working on this project for a few months, a future iteration of this project could be turned into a useful real-world system.  There is certainly a need for teaching computer science to people, particularly young students.  Very few middle schools teach computer science since they do not have the resources or the teacher training.  Many middle schools cannot afford to have computers for every students or even multiple computers per classrooms.  These schools do usually have one projector per classroom and one laptop for the teacher.  Our project combines the projector, the laptop, and a $30 webcam and would allow schools to teach computer science without expensive hardware and/or software.  There is certainly a need for computer science instruction, and a future iteration of our project could certainly help eliminate some of the obstacles that currently prevent the teaching of computer science.

We made our program modular so that future users could develop programs to teach what they wanted.  Just as the we created the “Numbers” program and the “Assembly” program, a teacher could create another program very easily.  All they would have to do is create a few tags corresponding to their lesson and some source code that would interpret the tags as needed.  Our program could provide them with a list of tags rows and a graphic.  They could change the graphic with their program and parse the tag rows to create a reasonable image.  If people ever used our program in the real world, they could easily customize the interfaces we provided to suit their needs.  Thus, although we only created a few lessons, other users could create many lessons to teach multiple topics.

Even outside of the classroom, there is a great need for computer scientists and people who know how to program.  Some of these people have taken some computer science courses in college but want to further their knowledge and understanding of the subject.  Many of our testers varied in their backgrounds in computer science.  However, they all were able to learn and practice by using our system.  By creating this project and interviewing and testing potential users, we have learned a lot about the difficulties of computer science and the topics that students find most difficult.  We geared our project towards teaching these topics better.  We did this by creating a binary program and by creating a TOY program which very easily shows memory management with the registers.  Thus, with our interviews and tests, we learned a great deal about the trouble with learning computer science and geared our project towards teaching those concepts.


Moving forward with the Project (1-2 Paragraphs):

Looking ahead, there are many ways we could improve our project design and develop it into a fully functional retail product. Currently our project is still a basic proof of concept, demonstrating its capabilities as a useful teaching tools. Keeping in mind our initial goal to develop an interface that would be used in a classroom environment by both teachers and students, our primary focus in developing our project further would be to improve the interface, make more built in lessons, and develop a platform for teachers to easily create new lessons.

In addition to making the overall presentation cleaner and more professional looking, the interface improvements would primarily be graphical additions such as animations and indicators that communicate to the user what is currently happening as they interact with the system. These improvements would not only make the system more intuitive, but also more entertaining for students. Additional useful features we could add may be things such as voice overs of instructions and demonstration videos on how to use various features of the system.

One of the most important things we would like to improve is the library of built in lessons available for users to chose from. Currently we only have two basic lesson: base-conversion and a TOY-programming environment. A more developed lesson library may contain tasks involving more advanced programming topics, teaching various programming languages, or even educational games. There would also be no need for lesson topics to be restricted to CS related topics. Our system could be effectively used to teach virtually any subject provided the right lessons are created.

Another key improvement we would like to make is developing a lesson creation environment where teachers can quickly and easily design their own lessons for their students. Any lesson library we make on are our will inevitably be lacking in some aspect. Giving users the power to create their own lessons and even share them online with each other will vastly improve the potential of our system as a teaching tool.

Our code base can be found at:

Our README can be found at:

Bulleted list of Third Party Code (including libraries):

  • Homography

  • Used to calculate transformation between camera coordinates and projector coordinates


Our Poster:

Additional Demo Print Outs: