Effective training, publications evaluation process vital to our success

Consumer Action has been conducting evaluations of our trainings and publications for years. Initially, our evaluation program was limited to gathering immediate post-training feedback. In 2009, we decided that we wanted a broader and more rigorous evaluation process—one that would help us gauge not only the immediate impact of our efforts (such as the training participant’s confidence in being able to replicate the training in his or her community), but the intermediate- and long-term impact as well.

Consumer Action worked closely with Brandeis University’s Heller School for Social Policy and Management to develop a new evaluation process whereby training participants were given both pre- and post-training evaluations and also sent follow up Web-based surveys.

For some trainings, participants are asked to complete pre- and post-training written evaluations along with one-, three- and six-month online evaluations. For other trainings, participants complete a post-training evaluation and a 60-day online evaluation. To gauge our effectiveness we ask about the extent of intermediate- and long-term changes made by CBOs’ clients—things like paying down debt, developing a savings plan, improving credit scores, buying a home or starting a business.

The full participation of our CBO network is crucial for our evaluation program to be truly effective. The feedback we receive reveals what impact we are having in the community and what we can do to improve or enhance the training sessions, materials and support we provide to our network partners. To promote participation in the evaluation process, we offer a chance to win an iPod shuffle to those who completed the surveys.

In August, Consumer Action will launch a new evaluation program for our MoneyWi$e publications. Partners will receive a short letter with their MoneyWi$e orders explaining what we are looking for, and the actual evaluation surveys will be sent out approximately three months after the order is fulfilled. This new procedure grew out of a more informal publication evaluation process that we’ve used for the last couple of years. We hope to gain a better understanding of the value of the publications, learn about “best practices” for using the materials in both group and one-on-one counseling sessions, and discover ways we can improve our educational modules.