Jenna Rodrigues ’14, APPRISE

On Wednesday, January 11th, I took the long journey over to APPRISE, a center for public policy research that is located on Nassau Street. As soon as I arrived, I was welcomed into the office and introduced to everyone over a breakfast table of bagels. At the start of the day, I shadowed a Policy Analyst named Jen who has been intently working on the National Weatherization Assistance Program Evaluation for quite some time. Other projects that she spoke to me about included the CFL Saturization Market Characterization in New York, followed by the Cell Phone Landline Study through which they would compare landline phones to cell phones. She then introduced me to the process of survey research which proceeds as follows: pretest, survey revisions, contacting of the phone center to program the survey in CATI, testing of the CATI, correcting the mistakes, writing memos to train interviewers, and checking in on the phone service to monitor calls. After giving me some background on survey construct methods, Jen talked to me about the indoor air quality and the staff survey parts of the Weatherization evaluation. 

After receiving a wonderful background from Jen, I went over to shadow Brian. It was his last day at APPRISE, so he was attempting to finish up a report for the Weatherization Project. He told me about the goals of weatherization, in attempting to reduce energy costs and find ways to make the home more energy efficient based upon the auditor’s inspection. During my time with Brian, I read over the report from the social scientist, which discussed how the auditor visits went, and the final inspection reports. Brian’s current task was to write the reports for the WAP program for different states. He showed me how he transformed the report he obtained from the social scientist and from the weatherization expert into a summary report to provide to the agency.

Next, I shadowed Jeffrey who was working on the survey for the EEPS Workforce Development Program. In this survey, the main questions being asked were whether individuals had heard about the program or not, and if not, what would make them more interested. He introduced me to the idea of coding, and told me that if a question with verbatim responses had at least 5% of a selective response, they would create a new code for that response to clarify the analysis. He then showed me the disposition form, and told me that they use this to determine if they need to give the call center more sample in order to reach the desired number of survey completes. He explained that both APPRISE and the call center want to reach a certain number of completes in order to get the survey out of the field and begin the analysis process. The disposition form allows APPRISE to make sure the phone center is on track and performing correctly.

After the morning of shadowing, I went to lunch with Dan, Colleen, and Jeff at Theresa’s in town.

Next, I shadowed Colleen, who talked to me about her Puget Sound Energy Survey, which examines home energy reports, and examining how much energy saving is double counted. She also described to me the specific details in the way that she, as a Policy Analyst, must edit the surveys,.

Finally, I shadowed Daya who talked about the National WAP evaluation that she is currently working on. She described the WIPP initiative that involves a set of funds that are put aside for innovative methods, different uses of technology, or different financing techniques to enhance energy efficiency. Her role was to do pre and post-weatherization surveys, field visits, and billing analysis. Daya further taught me some of the details of the Microsoft ACCESS she uses to maintain a relationship between charts and enter data. I discovered that the main differences between ACCESS and Excel are that it is easier to set up data entry forms, it minimizes errors, and it automatically saves your work.

Once I had shadowed all of the Policy Analysts that were in for the day, I did a debriefing with Daya and Colleen. Some things that were surprising to me were the low target for complete samples on various projects and how high success rates were for people whose households had been weatherized. In the debriefing, we discussed further the major objectives and responsibilities in APPRISE, concluding that sometimes their role is from the beginning with the evaluation plan, and sometimes they come in at the middle stage of a project to do the data processing.

Overall, I had a great time at APPRISE and I gained a wonderful insight into data processing and survey research. I never knew how much programming and specificity was incorporated in the process of getting surveys to a phone center to conduct to various individuals. I found the research performed at APPRISE fascinating and I would love to learn more about the subject matter and research methods in the future. I want to thank everyone at APPRISE for spending the time to show me what they were working on and for being such a welcoming group of people.


Jenna McCarthy ’13, APPRISE

I made the quick journey up to 32 Nassau Street to the APPRISE office on January 11. Inside, I was greeted by staff members who graciously provided bagels for us. APPRISE is a nonprofit research institute that collects and analyzes data and information to assess and improve public programs. Throughout the day, I shadowed seven of the Policy Analysts, and each one discussed projects and showed me what they did on a daily basis.

Surveys are one of many research activities that APPRISE performs to obtain data and information used to assess public programs. Dan, one of the Policy Analysts, was working on a survey for the NYSERDA (New York State Energy Research and Development Authority). He explained that part of APPRISE’s job is to edit the surveys to make sure that the respondent understands the questions and can provide accurate responses. Another task is to test the computerized version of the survey instrument to ensure that it works correctly.  For example, often questions will be skipped based on the response to other questions, and this task ensures that these skips work correctly. After they ensure that the skip patterns function correctly, they send the surveys to a call center. APPRISE uses a number of call centers, and they often travel to their headquarters to train employees who will administer the surveys. In the beginning of each survey, they also listen in and take notes to see whether or not the call center employee is doing a good job. Once the surveys are complete, APPRISE will either analyze the data and assess what implications the survey results have for program improvement, or send the survey data to the client so that they can analyze it themselves. One of APPRISE’s biggest projects right now is the evaluation of the National Weatherization Assistance Program (WAP), which I heard a lot about from many of the Policy Analysts. There are many different research tasks that comprise this evaluation, including surveys, impact analysis, and on-site observation and data collection.

My day at APPRISE was a great learning experience, and I really got a feel for what it’s like to work in a nonprofit setting. The office was very open and informal, which was great – they even had a small party at the end with cake because one of the Policy Analysts was leaving for a new job. My experience at APPRISE definitely showed me that I would be interested in doing this kind of work someday, and I would recommend this Princeternship to anyone interested in nonprofit work, energy efficiency, or data analysis in general.