Framework for assessments of impacts of international programme
How can a donor assess whether their project grants are having a real impact? And how can they track and report the overall impacts of a portfolio of projects? Is it wise to track energy savings or GHG emission reductions from capacity building projects where these type of impacts are indirect? When more than one donor or stakeholder supports a project, how does one attribute the impacts to the support? A consortium led by Eco, with IISD under contract to the REEEP tackled these complex questions.
In December 2008, REEEP engaged Eco Ltd to develop methodologies for the assessment of the energy, climate and development impacts of REEEP projects. The terms of reference for the assignment focused on the development of methodologies for measuring energy saved, green kilowatts generated, and greenhouse gas (GHG) emission reductions. This focus came out of an evaluation from “The Partnering Initiative” which emphasized the importance of evaluating REEEP’s overall impact.
Eco’s work started with a review of 84 projects financed by REEEP in 2007-2008. The review highlighted that the overwhelming majority of REEEP-funded projects (94%) focused on improving knowledge. Very few of the projects had direct outcomes for the generation of green kilowatts (6%) or reduction in GHG emissions (4%). Although REEEP-funded projects might ultimately contribute to saving energy, increasing generation of green kilowatts, or reducing GHG emissions, the findings highlighted that directly attributing such impacts to REEEP is problematic.
The review of the REEEP portfolio showed that the outputs and outcomes (those that projects are responsible for delivering during the course of the project) of the projects currently under implementation are predominantly of four kinds: the drafting of policies and regulation, creation of financial mechanisms, market development, and improving knowledge. The applicability of typical greenhouse gas or energy savings methods to this type of project is highly questionable. Methodologies that track impacts of this type of impact (the WBCSD measuring impact framework) or Outcome Mapping are therefore much more relevant to the REEEP portfolio than CDM or VER methodologies. Other approaches such as Most Significant Change, were also recommended.
Based on the analysis of REEEP projects, a review of literature on monitoring approaches was conducted. The literature review focused on approaches designed to accommodate initiatives focused on improving knowledge and building capacity. A total of 16 approaches were analysed in more detail, with special attention focused on the advantages and disadvantages of applying these approaches to REEEP. The review concluded with an overview of three key options.
Consultations with REEEP were held to discuss the preliminary findings and the way forward. It was determined that a synthesis monitoring approach combining key elements of Outcome Mapping with REEEP’s existing results based monitoring approach would be developed.
In 2011 REEEP incorporated Eco’s recommendations in their 8th Programme Cycle Call for Proposals.
The development of monitoring frameworks for complex projects, or, as demonstrated in this assignment, complex portfolios of projects requires specialist expertise such as that provided by Eco. Eco’s experts understand the impacts behind monitoring data.
Contact us
If you are an established organisation working to develop sustainable energy projects in developing countries, contact Dr Grant Ballard-Tremeer using our online form here.
08-110
