Terms of Reference for Mid-Term Evaluation
Consultants’ Terms of Reference
For the Mid-term Independent Evaluation (MTE) of IW:LEARN
Capacity to Sustain Transboundary Waters:
The International Waters Learning
Exchange and Resource Network, Operational Phase
1. Introduction & Background
The Global Environment Facility (GEF) International Waters: Learning Exchange and Resources Network (IW:LEARN), operational phase project commenced in October 2004. The project is based on the outcomes of a successful 3-year pilot and recommendations from the three GEF Implementing Agencies (IAs) – United Nations Development Programme (UNDP) United Nations Environment Programme, (UNEP), and the World Bank (WB) -- and their partners engaged in International Waters projects. This new phase scales up successful IW:LEARN initiatives and provides additional services to the GEF’s IW projects.
IW:LEARN’s overall objective is to improve the management of transboundary water systems by increasing the capacity to identify, disseminate and replicate best practices and lessons learned across the GEF IW projects. This objective is being achieved through a suite of learning exchanges, information sharing activities, joint demonstration activities, and related capacity building activities among GEF supported IW projects and their partners.
In pursuit of its global objective, IW:LEARN seeks to improve GEF IW projects’ information base, replication efficiency, transparency, stakeholder ownership and sustainability of benefits through the following 5 project components and outcomes:
Facilitating Access to Information on Transboundary Water Resources Among GEF IW Projects
TWM** improved across GEF IW project areas through projects’ and stakeholders’ access to TWM data and information from across the GEF IW portfolio and its partners.
Structured learning among IW Projects and co-operating partners.
Enhanced TWM capacity at project- and basin-levels through sharing of experiences among subsets of the GEF IW portfolio, including projects, their partners and counterparts.
Biennial International Waters Conferences
GEF IW portfolio-wide increase in awareness and application of effective TWM approaches, strategies and best practices; numerous new and enhanced linkages and exchanges between GEF IW and other TWM projects with shared TWM challenges
Testing innovative approaches to strengthen implementation of the IW portfolio
A widely available suite of tested and replicated ICT and other tools and approaches for strengthening TWM.
Fostering partnerships to sustain benefits of IW:LEARN and associated technical support
TWM learning and information sharing mechanisms mainstreamed and institutionalized into GEF IA and ongoing projects, as well as institutional frameworks of completed projects (e.g., Regional Seas and freshwater basin secretariats)
*Source: Logical framework. http://www.iwlearn.net/abt_iwlearn/docs/iwl2_execsumm.pdf
**TWM: Transboundary Water Management
To increase the ownership of this global project among GEF IAs and those served by IW:LEARN, this GEF Full-Sized Project (FSP) is being co-implemented by all three IAs. Working with these and other partners, the intercontinental IW:LEARN Project Coordinating Unit (PCU) directs IW:LEARN’s Project Coordination Team (PCT) from Washington, Nairobi and [starting in 2007] Bratislava. Each IA oversees a subset of activities. The management structure is diagramed below in Annex I. (See details in IW:LEARN Operational Phase FSP Project Document (ProDoc) or its abridged Executive Summary, both on-line via http://www.iwlearn.net/abt_iwlearn/pubs.)
This TOR is designed to support a consultant conducting an independent mid-term evaluation (MTE) for the Operational Phase IW:LEARN project, including project performance vis Project Document outcomes, institutional arrangements, financing (disbursements and co-financing), impacts and opportunities for learning and mid-term course corrections to improve upon expected project delivery of its products and services. Both UNDP- (UNOPS-) and UNEP-implemented (executed) sub-projects will be considered .
2. Objectives and scope of the mid-term evaluation
The objective of the MTE is to enable IW:LEARN, the GEF International Waters Task Force (IWTF) members and UNOPS to assess the progress in delivery of Project outcomes and based on this assessment, to take decisions on the future orientation and emphasis of the project during its remaining time.
The evaluation is an activity in the project cycle which attempts to determine, as systematically and objectively as possible, the relevance, efficiency, effectiveness, impact and sustainability of the project. The evaluation will assess the achievements of the project against its stated outcomes, including a re-examination of the relevance of the outcomes and of the project design. It will also identify significant factors that are facilitating or impeding the delivery of outcomes. Whilst a review of the past is in itself very important, the evaluation is expected to lead to recommendations and lessons learned for the future.
The mid- term evaluation will, using the methodology described below, and bearing in mind the following points, review the status of each project outcome:
- · Relevance of project design within the framework of GEF guidelines and global concern regarding improving transboundary waters management (TWM)
- · Appropriateness of the project’s concept and design to the current economic, institutional and environmental situations across the GEF IW portfolio;
- · Contribution of the project to the overall development objective (i.e., the top-level outcome) as declared in the Project logical framework; and
- · The likely sustainability of project interventions;
- · General implementation and management of both UNDP/GEF and UNEP/DGEF sub-projects by the PCU in terms of quality of inputs and activities, adherence to workplans and budgets, major factors which have facilitated or impeded the progress of project implementation;
- · Adequacy of management arrangements as well as monitoring and backstopping support given to the project by all parties concerned;
- · Institutional set-up throughout the Project Coordination Team (PCT; see Annex I), PCU and various partners, and the degree to which it has encouraged full involvement of GEF target projects;
- · Responsiveness of project management to changes in the environment in which the project operates;
- · UNOPS and UNEP execution, including coordination between them;
- · Co-operation among PCT partners (IWTF, GEF Secretariat, IAs, PCU, Partnership Activity Leads (PALS), specifically with regard to the integration and support of IW:LEARN
Project progress and impact (outcomes)
- · Achievement, to date, of the project outcomes as detailed in the project document and the Project Implementation plan;
- · Awareness of the participating projects regarding IW:LEARN outcomes;
- · Level of ownership of the project by the participating projects and IWTF;
- · Commitment of IW projects and their partners to support the ongoing IW:LEARN project, including learning and information sharing across the GEF IW portfolio;
- · Likely degree of support from the projects and their national partners in integrating IW:LEARN objectives into their regional institutional frameworks and other related projects, and how well the project fits into their own project plans;
- · Impacts on GEF IW projects’ policies, strategies and work plans;
- · Project impact on improving GEF IW projects’ capacity to prepare and implement collaborative, targeted and effective TWM efforts
- · Project impact on enhancing inter-agency and inter-project co-operation within and between regions;
- · Cooperation among international organisations, NGOs and other stakeholders;
- · Cooperation with sister projects in the GEF IW portfolio especially OP10 capacity-building projects;
- · Catalytic impacts arising from the performance of the project;
- · Sustainability of the project’s impact.
Learning from First Half of Project and Recommendations for its Remainder
- · Learn from efforts to date, as basis for exploring ways to adapt or restructure the project design or institutional arrangements, if needed.
- · Challenges or difficulties faced, e.g.,
- o Where, why and how do we need knowledge sharing among IW projects?
- o How participatory vs. “top-down” has IW:LEARN been? Is this properly balanced? If not, how so (and how to adjust)?
- o What went wrong with e-forums? Where have the e-forums worked well and where have they not worked well and why? Potential for other approaches to engage projects in peer-to-peer learning (e.g., blogs, wikis, face-to-face workshops)?
- o How to reconcile implementation plans with asynchronous start of UNDP and UNEP sub-projects?
- · Identify implementation challenges and recommendations for improvement.
The mid-term evaluation mission will also briefly review the current proposed activities for the remainder of the Operational Phase IW:LEARN project and provide perspectives and recommendations to improve their feasibility and impacts.
The evaluation will consist of three
- Document review
- Field visits and
- Interviews with individuals who are either affiliated with the project in some way or who have or might be expected to be impacted by the project.
IW:LEARN PCU staff will be available upon request to administratively facilitate the MTE as required.
(a) Document Review
The evaluator shall familiarise himself with the project through a review of relevant documents prior to the field visits. These documents include inter alia:
- · Final evaluation from IW:LEARN Pilot Phase (provided by CTA)
- · Pertinent information available at the project web site: www.iwlearn.net as well as [limited access] Steering Committee Web page, e.g.,
o UNDP and UNEP IW:LEARN Project Documents (on Web site)
o IW Experience Notes series (www.iwlearn.net/experience)
o UNDP-GEF Project Implementation Review (PIR/APR) and GEF Annual Project Performance Results (APPR), October 2006 (in SC area)
o Minutes of meetings of IW:LEARN Steering Committee from 2004 through end of 2006 (Ordinary Meetings) (in SC area)
o M&E Plan/reports (see CTA/SC Web space, respectively)
- · UNDP Handbook for Programme Managers: Results-Oriented Monitoring and Evaluation and UNDP Handbook on Monitoring and Evaluation for Results; Guidelines for Outcome Evaluators: Electronic forum archives
Selected documents which are not available through the Web page shall be provided by email to the evaluator in advance of the mission and/or during the mission upon request from the evaluator.
(b) Field visits
The evaluator will visit at 1-2 IW:LEARN-sponsored workshops or events, as well as PCU offices in Nairobi (for IW:LEARN workshop and UNEP “NBO team” consultations) and Washington, DC (UNOPS “DCO team” consultations and November 28, Steering Committee meeting). Local desk, telephone and internet access will be extended by PCU to the evaluator during DC and Nairobi missions for IW:LEARN-related communications.
The evaluator will be available to carry out 12-15 interviews. Interviews will use a simple questionnaire designed to solicit feedback on opportunities and constraints to the delivery of project outcomes. The questionnaire will be developed by the Evaluator in consultation with the PCU and tested with an interviewee nominated by the PCU before use. Interviews will be face-to-face or by telephone. The PCU will arrange the interviews and where necessary provide a venue and communications facilities:
· Project Staff (Chief Technical Adviser (CTA), USA- and Nairobi-based PCU teams).
· Selected members of the IW:LEARN Steering Committee and IW Task Force (IWTF)
· Selected members of the Project Coordination Team (PCT), Partnership Activity Leads (PALs) and other partners;
· Representatives of the relevant beneficiaries: GEF IW project and national partners
· Other constituencies and stakeholders not directly involved in the project who may have experienced, or may be expected to experience, its impacts.
Suggested interview categories are noted in Annex III.
Although the independent evaluator should feel free to discuss with authorities concerned all matters relevant to their assignment, they are not authorised to make any commitment on behalf of UNOPS, UNDP, UNEP or GEF.
4. Conclusions and Recommendations
Based on the above objectives and methodology, the evaluation mission should provide brief, clear, conclusions and recommendations, including:
· The degree to which the project outcomes are likely to be delivered;
· Significant lessons that can be drawn from the experience of the project and its results, to date, particularly those elements that have worked well and those that have not
· General recommendations on improving implementation for the remainder of the Operational Phase project
· Recommendations on further action upon completion of the current project
5. MTE Mission Report
The evaluation mission will produce a concise report according to the structure outlined in the UNDP Guidelines for Outcome Evaluators:
· Executive Summary
· The Development Context
· Lessons and Recommendations
In addition, the final report should contain at least the following annexes:
· Terms of Reference for final evaluation
· List of meetings attended
· List of persons interviewed
· Summary of field visits
· List of documents reviewed
· Any other relevant material
A template following this format is provided as Annex IV below. In consultation with the UNDP representative or his designee, this report format may be revised insofar as no significant elements are omitted.
As the report is the product of an independent evaluation, it is up to the evaluator to make use of the information provided during the mission. However, the evaluator is responsible for reflecting any factual corrections brought to his/her attention prior to the finalisation of the report. Therefore, in order to ensure that the report considers the view of all parties concerned, is properly understood, and is factually accurate, it is necessary for the evaluator to submit draft reports to the project, UNDP/GEF and UNOPS, not later than contract day 47, i.e., 2 weeks prior to delivery deadline for the finalized MTE mission report (day 60). In the intervening period, UNOPS will solicit and revert promptly collective feedback from project partners in order that the evaluator may finalise the report.
The final version of the evaluation mission report should be submitted in electronic format (MS Word) and hard copy to UNOPS no later than 3 months following activation of this contract.
6. Composition of the mid-term evaluation mission
The evaluation will be performed by one internationally recruited consultant. The consultant will have considerable knowledge and experience regarding (&/or assessing) GEF IW operational programme, peer-to-peer facilitated learning and knowledge management. A good knowledge of TWM issues and relevant scientific understanding and in-depth experience of project evaluation techniques, particularly of those projects which are funded by GEF, is preferred.
· Advanced degree in knowledge management, international relations, natural resource management, development studies or related fields, or equivalent demonstrated experience.
· Expertise in Knowledge Management (KM) at multi-institutional scale, with professional-level understanding of Information and Communications Technology (ICT) to support KM.
· Notable experience with transboundary waters management in GEF IW project regions, particularly where pertinent to Monitoring and Evaluation and/or documenting TWM lessons.
· Demonstrated ability to reliably contribute to output- and outcome-based evaluations, both assessment and learning aspects.
· Familiarity with MTE process, UN and/or World Bank M&E procedures preferred.
· Excellent ability to work in English, effective oral and written communication skills;
· Experience with researching and writing on international environment and/or international development issues;
· Facility and access to use of Internet email and Microsoft Word (2000 or later).
The consultant shall not have been directly involved in the design or implementation of the project. To ensure unbiased impartiality, consultant shall also not have had prior involvement with any GEF IW:LEARN Operational Phase project activities.
7. Indicative mission timetable, deliverables and itinerary
The duration of the consultancy working days, including travel time, are presented as Annex II. Electronic submission of all deliverables is required unless otherwise pre-arranged in writing with the UNDP-GEF Steering Committee member or his representative. Any hard copy reports should be delivered to UNOPS per contract to which this TOR is appended
This schedule may be updated through written (or emailed) agreement between the UNDP representative and the Evaluator, as needed, during the course of the contract.
The Consultant will conduct the evaluation for up to 24 work days over a period of 2 calendar months from the inception of this TOR. If there is delay in any of the inputs from IW:LEARN required for evaluator to full the role presented above, the evaluator will bring this to the attention of the Steering Committee.
Under no circumstances should consultant invoice for more than the maximum number of days above without formal advance approval from UNDP Representative, along with associated amendment to this TOR.
Candidates for Evaluator will provide their proposed daily rate during the selection process (along with any expected adjustment to number of days required). The selected Evaluator will then be compensated at rate approved by UNOPS Personnel Division, considering candidate proposal, and commensurate with experience and responsibilities of the Consultant (as provided in the Contract accompanying this TOR).
Full payment will be contingent on receipt of all deliverables per Annex II below. UNOPS will provide an advance for subsistence on contract signature and 25% of the fee on acceptance of the questionnaire. Partial payment of no more than 85% of total will be provided with evaluator’s invoice accompanying delivery of Draft MTE Report (Task 8).
Upon submission of F-10 form, UNOPS will also reimburse travel costs associated with Evaluator missions to Nairobi (4-5 days travel) and Washington DC (4-5 days travel) according to standard UN and UNOPS rates.
9 . Contact information
Contact information for IW:LEARN PCU, UNEP, UNOPS and UNDP/GEF:
IW:LEARN DC Office
Dann Sklarew, Ph.D.
Chief Technical Advisor
IW:LEARN Nairobi Office
Andrew Menz, Ph.D
Senior Portfolio Manager
UNOPS: Global and Interregional Division
Mr. Andrew Hudson, Ph.D.
Principal Technical Advisor – International Waters UNDP-GEF
 Taking note of the Guidelines for developing TOR for Final Evaluations in UNDP M&E Resources kit.