Topic: Using Mechanical Turk and Qualtrics to Crowdsource Tasks and Surveys Speaker: Alfredo García
Time: Thursday, March 6, 12:00 pm – 1:00 pm Location: HRC Classroom, 012 East Pyne, Lower Level
The idea of “crowdsourcing” has increasingly been used in a variety of settings. With the help of social media, we can now crowdsource funds, software coding, information, and a variety of other tasks. But how can a scholar put crowdsourcing to use in his/her own work? This talk will focus on using Qualtrics, an online survey-generating software platform, and Amazon Mechanical Turk (AMT), a popular crowdsourcing website, to field surveys and collect hard-to-get information. Of particular interest in this session will be the use of AMT for running experimental survey designs. (Free Qualtrics accounts arevavailable for Princeton University members through the Princeton Survey Research Center.)
Princeton has a sophisticated survey creation and reporting tool, and if you are part of the PU community, it could be just the tool for your next survey.
On Wednesday, March 7th, 2012, Edward Freeland and Naila Rahman talked about Qualtrics, an online survey management solution used by the Princeton University Survey Research Center. The video of the session is below. The University’s Survey Research Center needed a survey solution that was web based, online, simple to use and learn easily, sophisticated enough to do experiments, secure, and respectful of survey participants personal information. They chose Qualtrics, and have used it since July of 2010. Since that time, they have amassed 300 survey creators, 1500 surveys, and 230,000 interviews.
Rahman explained that creating a survey with Qualtrics is as easy as clicking a button and creating or copying existing questions. Survey creators can build their own surveys, copy existing surveys, or browse and select from a few survey libraries in Qualtrics. With proper formatting, you can even import your survey using an uploaded Word document. Many question types exist for surveys in the system, including multiple choice, dropdown list, single answer, multiple answer, graphics based questions, ranking style questions in which you drag and drop to rank items, and sliders & matrices for likert styled survey questions. The editor for surveys has a clean and useful wysiwyg editor, where you can add page breaks, bold or italicize text, make specific color choices, custom headers, and more.
Rahman spent some time explaining the system’s skip and display logic, where you can set rules based on survey responses to display differing content or custom survey destinations. You can also pipe text, meaning you can use a previous question response to modify an upcoming question in a survey. For instance, if you ask a respondent what their favorite color is, and they say red, you can have Qualtrics ask how much the respondent likes red specifically with a scale based question.
Survey creators can add from pre-existing question libraries such as the Princeton survey question library in the system, which holds question pools on topics like demographics. Qualtrics libraries store your messages that are sent to your samples, graphics used in your surveys, questions and other items, so that they can be re-used repeatedly. Your results are stored online, and are graphed to give you a visual representation of your data. You can export your data to Excel, and you can even schedule an emailed report of updated data and survey results. The Survey Flow features allows you to randomize or re-order either portions or the entirety of the survey. Distribution can be done via email or direct link — The Panels feature allows you to upload a CSV file with first name, last name, and email for a sample. You can manage and schedule the announcement to your survey sample directly through Qualtrics.
Rahman went into some deeper detail about the sophisticated use of Survey Flow, question types, display logic, and skip logic in Princeton’s 2010 internal survey on Family Benefits, which gave the audience a sense of the power that Qualtrics has in delivering high-quality surveys.
If you are a part of the University community, please visit http://www.princeton.edu/~psrc to get started with Qualtrics.
To get overviews and tutorials on how to use Qualtrics in 5 steps, please visit: http://www.qualtrics.com/university/researchsuite/learn-qualtrics-in-5-steps
About Princeton’s Survey Research Center:
Our Mission: The SRC’s main purpose is to assist students and faculty in designing and implementing their own survey research projects. The SRC provides consultation and guidance on study design, sampling, instrument development, data collection and data processing. The Center has a 12-station computer-assisted telephone interviewing (CATI) facility, a library collection on survey research methods, and a network of external resources.
About Us: The Princeton University Survey Research Center (SRC) was founded in 1992 with a grant from the Andrew W. Mellon Foundation. The SRC is a resource for Princeton University students, faculty, and administration. The SRC has three principal activities: consultation, education and project management.