GEF Effectiveness
Chapter Nine GEF EffectivenessWhat's Working And What's Not Working
1. THE PILOT PHASE AND THE INDEPENDENT EVALUATION
Much criticism was leveled at the GEF Pilot Phase for its lack of a coordinated monitoring and evaluation (M&E) mechanism. Effective M&E helps to: avoid problems, correct problems before they become too serious and identify lessons for future application. Ultimately, M&E is critical to improving the efficiency and effectiveness of GEF resources.
In 1992, GEF participants requested the Independent Evaluation to assess progress and identify problems within the Pilot Phase. The evaluation was conducted by three separate teams, one from each GEF Implementing Agency (IA). Also, an independent experts panel was formed to provide guidance, guarantee an independent evaluation and channel input from the NGO community. The final report (published in 1994) of the Independent Evaluation provided a detailed assessment of the GEF's institutional structure, decision-making structure, project cycle and relationship with NGOs and local stakeholders. (GEFDOC #22)
Key recommendations raised in the evaluation include:
Mission and objectives:
Clearly articulate the GEF mission.
Develop programme objectives and strategies.
Programme management:
Reform leadership, management and organisational relationships.
Clarify and establish clear lines of accountability.
Establish a permanent mechanism for identifying lessons and promoting their application.
Relations with Implementing Agencies:
Establish common guidelines for management of GEF operations by IAs.
Undertake an independent review of IAs' capacity.
Improve participation at the country and community levels.
NGO involvement:
Establish mutually beneficial collaboration with NGOs.
Project review and approval:
Ensure that strategies and program guidelines are in place before program initiatives are undertaken (with funds from the replenishment).
Many of the key issues identified by the evaluation were addressed during the negotiations to restructure the GEF (and finalise the GEF Instrument; GEFDOC #27), which were conducted in parallel with the evaluation. For example, the GEF Instrument adopted by governments in 1994 provides a clearer governance structure and the basis for developing an operational strategy, containing a clear mission statement, programme objectives, strategies, etc. Finally, a key recommendation of the Independent Evaluation was the establishment of a permanent M&E system within the GEF Secretariat, which was approved by the GEF Council in May 1995.
2. ONGOING MONITORING AND EVALUATION (M&E) UNDER THE GEF
In October 1996, the GEF Council requested a major evaluation of accomplishments and overall performance of the GEF to be available at the GEF Participants Assembly in April 1998. This evaluation includes three studies:
the Project Implementation Review,
the Study of Overall GEF Performance and
the Study of Project Lessons.
The studies are being supervised by the GEF's M&E coordinator.
Project Implementation Review (PIR). IAs and the GEF Secretariat annually undertake PIRs to (i) examine the status of implementation and goal-attainment for GEF projects; (ii) identify portfolio strengths and weaknesses; and (iii) distill and disseminate the lessons learned from GEF programs. The PIR was conducted using the reporting formats of the IAs with some agreed upon reporting guidelines. Generally, IAs prepare a brief report for projects one year or older, identifying implementation progress, prospects for attaining objectives and recommended actions to respond to any problems. Many NGOs see the self-monitoring activities by IAs as welcome and necessary but not as a substitute for rigorous independent evaluations.
Overall GEF Performance. Scheduled for completion in early 1998, the Overall GEF Performance study will focus on the following four areas:
provision of resources (e.g., are GEF funds leveraging other resources? What is the efficacy of financing incremental costs?);
country level coordination issues (e.g., are GEF resources helping governments to fulfill their obligations under the Biodiversity and Climate Conventions?);
institutional issues and procedures (e.g., how well are the institutions involved in the GEF coordinating and communicating their activities? Is the GEF encouraging mainstreaming of environmental concerns into the programs of the GEF IAs); and
programming efforts (e.g., are Council decisions and guidance from the Biodiversity and Climate Conventions being operationalised effectively? How effective is the project selection process, especially in regards to identifying systems for learning and replicating promising approaches?).
The study will also include an annex to document and assess the GEF's follow-up to the recommendations from the 1994 Independent Evaluation of the Pilot Phase. (GEFDOC #27) Information on GEF projects collected in 16 countries will provide the basic inputs for analysis. An Expert Advisory Panel has been formed to review the work of this study (as well as the study on Evaluation of Lessons Learned described below) and to assure that the final report is comprehensive and independent.
Evaluation of Lessons Learned. In conjunction with the Overall GEF Performance Study, the Evaluation of Lessons Learned will assess the experience of projects approved during the Pilot Phase to determine what factors have been most important to their success or failure. Building on the PIRs and information from documents and interviews with GEF and IA staff, as well as NGOs, the study team has already identified three priority lessons during the first phase of the project:
Considerable effort/time must be devoted to building partnerships and understanding among project implementors and communities.
Coordination of local activities by individual projects with national policies and priorities is necessary to ensure that linkages between local efforts and global environmental benfits can be made and sustained.
Involvement of private businesses and other organisations engaged in economic activities in the design and implementation of projects can help ensure the project efforts are sustained, which often requires innovative approaches.
The second phase of the study will focus on these three lessons, examining the experience of selected Pilot Phase projects to better understand their dynamics.
3. NGO PERSPECTIVES
NGOs have advocated the need for independent evaluations of the GEF's overall performance, as well as evaluations of specific project lessons. For many NGOs, systematic monitoring and evaluation of GEF projects and dissemination of results has been too slow and has not contributed enough to the international learning process on how to address global environmental problems. As such, many feel that the current studies should start with the results of the 1994 Independent Evaluation as a departure point or baseline from which to assess the ability of the GEF and IAs to respond to initial criticisms. (GEFDOC #27) However, the studies' inception report and terms of reference only included one line regarding use of the Independent Evaluation. Additionally, many NGOs have stressed that to maintain their independence the study teams should report directly to the GEF Council, as opposed to the GEF's senior M&E officer. NGOs have stressed a number of areas for the focus of the three current M&E studies:
GEF structure:
Restructuring and problems with the governance system.
Barriers within IA policies (for example, the World Bank will not develop GEF projects that are not in line with its Country Assistance Strategy).
IAs' (in)ability to mainstream GEF goals into their overall portfolios.
IA monopoly:
Necessity of developing projects through one of the IAs.
Alternatives to the existing IAs within the project cycle for project development and implementation.
Stakeholder input:
Role of public involvement and information disclosure throughout the project cycle.
Procurement reform:
Restrictions on NGO involvement.
Incremental costs:
Utility as a formula for project eligibility.
Efficiency and cost-effectiveness:
Efficiency throughout the GEF organisational structure and within projects.
Project size:
Appropriate project size for national structures and local organisations.
Exploration of national environmental funds.
Portfolio balance:
Role of IA priorities and geopolitics.
Alleged caps on the number of Medium-Size Grants to be accepted by IAs.
Underrepresentation of sustainable use in the biodiversity portfolio.
Strategic leverage:
Utility of large investments vs. smaller grants to catalyse innovative approaches building on local momentum and capacities.
Quality assurance:
Selection of baseline data and information for M&E.
Need for systematic quality control.
Responsibility of IAs for quality data and monitoring.
Replenishment:
Need for rigorous independent evaluations as a critical component for discussion of replenishing GEF funds.
Information access and disclosure:
Access to information at the country and project levels that is translated into local languages to enable effective grassroots participation.