Jenna Rodrigues ’14, APPRISE

On Wednes­day, Jan­u­ary 11th, I took the long jour­ney over to APPRISE, a cen­ter for pub­lic pol­icy research that is located on Nas­sau Street. As soon as I arrived, I was wel­comed into the office and intro­duced to every­one over a break­fast table of bagels. At the start of the day, I shad­owed a Pol­icy Ana­lyst named Jen who has been intently work­ing on the National Weath­er­iza­tion Assis­tance Pro­gram Eval­u­a­tion for quite some time. Other projects that she spoke to me about included the CFL Sat­ur­iza­tion Mar­ket Char­ac­ter­i­za­tion in New York, fol­lowed by the Cell Phone Land­line Study through which they would com­pare land­line phones to cell phones. She then intro­duced me to the process of sur­vey research which pro­ceeds as fol­lows: pretest, sur­vey revi­sions, con­tact­ing of the phone cen­ter to pro­gram the sur­vey in CATI, test­ing of the CATI, cor­rect­ing the mis­takes, writ­ing memos to train inter­view­ers, and check­ing in on the phone ser­vice to mon­i­tor calls. After giv­ing me some back­ground on sur­vey con­struct meth­ods, Jen talked to me about the indoor air qual­ity and the staff sur­vey parts of the Weath­er­iza­tion evaluation. 

After receiv­ing a won­der­ful back­ground from Jen, I went over to shadow Brian. It was his last day at APPRISE, so he was attempt­ing to fin­ish up a report for the Weath­er­iza­tion Project. He told me about the goals of weath­er­iza­tion, in attempt­ing to reduce energy costs and find ways to make the home more energy effi­cient based upon the auditor’s inspection. During my time with Brian, I read over the report from the social sci­en­tist, which dis­cussed how the audi­tor vis­its went, and the final inspec­tion reports. Brian’s cur­rent task was to write the reports for the WAP pro­gram for dif­fer­ent states. He showed me how he trans­formed the report he obtained from the social sci­en­tist and from the weath­er­iza­tion expert into a sum­mary report to pro­vide to the agency.

Next, I shad­owed Jef­frey who was work­ing on the sur­vey for the EEPS Work­force Devel­op­ment Program. In this sur­vey, the main ques­tions being asked were whether indi­vid­u­als had heard about the pro­gram or not, and if not, what would make them more interested. He intro­duced me to the idea of cod­ing, and told me that if a ques­tion with ver­ba­tim responses had at least 5% of a selec­tive response, they would cre­ate a new code for that response to clar­ify the analysis. He then showed me the dis­po­si­tion form, and told me that they use this to deter­mine if they need to give the call cen­ter more sam­ple in order to reach the desired num­ber of sur­vey completes. He explained that both APPRISE and the call cen­ter want to reach a cer­tain num­ber of com­pletes in order to get the sur­vey out of the field and begin the analy­sis process. The dis­po­si­tion form allows APPRISE to make sure the phone cen­ter is on track and per­form­ing correctly.

After the morn­ing of shad­ow­ing, I went to lunch with Dan, Colleen, and Jeff at Theresa’s in town.

Next, I shad­owed Colleen, who talked to me about her Puget Sound Energy Sur­vey, which exam­ines home energy reports, and exam­in­ing how much energy sav­ing is dou­ble counted. She also described to me the spe­cific details in the way that she, as a Pol­icy Ana­lyst, must edit the surveys,.

Finally, I shad­owed Daya who talked about the National WAP eval­u­a­tion that she is cur­rently work­ing on. She described the WIPP ini­tia­tive that involves a set of funds that are put aside for inno­v­a­tive meth­ods, dif­fer­ent uses of tech­nol­ogy, or dif­fer­ent financ­ing tech­niques to enhance energy efficiency. Her role was to do pre and post-weatherization sur­veys, field vis­its, and billing analysis. Daya fur­ther taught me some of the details of the Microsoft ACCESS she uses to main­tain a rela­tion­ship between charts and enter data. I dis­cov­ered that the main dif­fer­ences between ACCESS and Excel are that it is eas­ier to set up data entry forms, it min­i­mizes errors, and it auto­mat­i­cally saves your work.

Once I had shad­owed all of the Pol­icy Ana­lysts that were in for the day, I did a debrief­ing with Daya and Colleen. Some things that were sur­pris­ing to me were the low tar­get for com­plete sam­ples on var­i­ous projects and how high suc­cess rates were for peo­ple whose house­holds had been weath­er­ized. In the debrief­ing, we dis­cussed fur­ther the major objec­tives and respon­si­bil­i­ties in APPRISE, con­clud­ing that some­times their role is from the begin­ning with the eval­u­a­tion plan, and some­times they come in at the mid­dle stage of a project to do the data processing.

Over­all, I had a great time at APPRISE and I gained a won­der­ful insight into data pro­cess­ing and sur­vey research. I never knew how much pro­gram­ming and speci­ficity was incor­po­rated in the process of get­ting sur­veys to a phone cen­ter to con­duct to var­i­ous individuals. I found the research per­formed at APPRISE fas­ci­nat­ing and I would love to learn more about the sub­ject mat­ter and research meth­ods in the future. I want to thank every­one at APPRISE for spend­ing the time to show me what they were work­ing on and for being such a wel­com­ing group of people.

 

Jenna McCarthy ’13, APPRISE

I made the quick jour­ney up to 32 Nas­sau Street to the APPRISE office on Jan­u­ary 11. Inside, I was greeted by staff mem­bers who gra­ciously pro­vided bagels for us. APPRISE is a non­profit research insti­tute that col­lects and ana­lyzes data and infor­ma­tion to assess and improve pub­lic pro­grams. Through­out the day, I shad­owed seven of the Pol­icy Ana­lysts, and each one dis­cussed projects and showed me what they did on a daily basis.

Sur­veys are one of many research activ­i­ties that APPRISE per­forms to obtain data and infor­ma­tion used to assess pub­lic pro­grams. Dan, one of the Pol­icy Ana­lysts, was work­ing on a sur­vey for the NYSERDA (New York State Energy Research and Devel­op­ment Author­ity). He explained that part of APPRISE’s job is to edit the sur­veys to make sure that the respon­dent under­stands the ques­tions and can pro­vide accu­rate responses. Another task is to test the com­put­er­ized ver­sion of the sur­vey instru­ment to ensure that it works cor­rectly.  For exam­ple, often ques­tions will be skipped based on the response to other ques­tions, and this task ensures that these skips work cor­rectly. After they ensure that the skip pat­terns func­tion cor­rectly, they send the sur­veys to a call cen­ter. APPRISE uses a num­ber of call cen­ters, and they often travel to their head­quar­ters to train employ­ees who will admin­is­ter the sur­veys. In the begin­ning of each sur­vey, they also lis­ten in and take notes to see whether or not the call cen­ter employee is doing a good job. Once the sur­veys are com­plete, APPRISE will either ana­lyze the data and assess what impli­ca­tions the sur­vey results have for pro­gram improve­ment, or send the sur­vey data to the client so that they can ana­lyze it them­selves. One of APPRISE’s biggest projects right now is the eval­u­a­tion of the National Weath­er­iza­tion Assis­tance Pro­gram (WAP), which I heard a lot about from many of the Pol­icy Ana­lysts. There are many dif­fer­ent research tasks that com­prise this eval­u­a­tion, includ­ing sur­veys, impact analy­sis, and on-site obser­va­tion and data collection.

My day at APPRISE was a great learn­ing expe­ri­ence, and I really got a feel for what it’s like to work in a non­profit set­ting. The office was very open and infor­mal, which was great – they even had a small party at the end with cake because one of the Pol­icy Ana­lysts was leav­ing for a new job. My expe­ri­ence at APPRISE def­i­nitely showed me that I would be inter­ested in doing this kind of work some­day, and I would rec­om­mend this Princetern­ship to any­one inter­ested in non­profit work, energy effi­ciency, or data analy­sis in general.