Measuring Impact
A disciplined approach to data collection and analysis.
As an organization, we hold ourselves to high standards of accountability in the pursuit of differential impact—evidence that our programs are producing achievement results that would not have been obtained in our absence. We do this for two reasons.
1) Demonstrating differential impact will help Blue Engine expand our operations and reach additional students in need of support, both in New York City and beyond. After two years working with entire grade levels (as opposed to individual students) in a school system where data collection procedures are implemented district-wide, Blue Engine will be in a position to state convincingly whether our programs appear to be having an impact on student achievement. Our analyses will compare results at Blue Engine schools with achievement data obtained in prior years (at our same school site(s)) as well as comparison schools across our “peer horizon”. We publish our Theory of Change to hold ourselves accountable for results and to signal to the education, policy, and philanthropic communities that we have a clear set of metrics that we are using to measure our impact.
2) Developing plans and procedures for data collection and analysis serves an important role in the optimization of program resources. By the summer of 2010, before our first class of Fellows sets foot inside a school building, we will have designed and implemented a data management system (incorporating surveys, program logs, and academic records) to track each of Blue Engine’s core activities, a trove of data that will be necessary to formally test whether the relationships outlined in our Theory of Change are working as hypothesized. These analyses will suggest, for example, why certain activities are accelerating achievement for some students and not others and allow us to adjust our program resources accordingly.
For more information on Blue Engine’s approach to data and metrics, please contact us at info@blueengine.org.
