MONITORING AND EVALUATION
Monitoring and evaluation (M&E) serves two essential purposes. By looking forward it provides a key to learning and improving performance, and when necessary revising approaches and designs with a view toward ensuring the intended outcomes and impacts are achieved. By looking backwards is provides a basis for accountability and is a foundation for good governance and effective management. Rapid change makes learning and adaptability increasingly important. Performance measurement and accountability are cornerstones of governance and a core part of sector approaches and cooperation modalities that aim to put development processes in the control of the countries and people that represent the beneficiaries. M&E is relevant across development sectors and themes and across the Sustainable Development Goals.
PEMconsult has a considerable expertise carrying out independent monitoring and evaluation including reviews using results based management approaches. PEMconsult’s services include:
Evaluations of specific projects, programmes, sectors and themes
Reviews of projects and programmes, as well as institutions and their work
Capacity development and design of systems of information management
Different techniques and approaches have evolved to suit different situations. It can be that a highly independent, expert driven approach is called for, or a participatory bottom-up learning approach is more appropriate. Objective and verifiable indicators of results are the key to successful monitoring (SMART indicators are specific, measurable, achievable, realistic and time-bound). In other cases, it is a guided process to reflect over a change process that is most useful. Development interventions are increasingly being guided by a theory of change approach that is flexible and dynamic, and M&E systems must similarly be dynamic and enable change as implementation is underway. In many development interventions, inception reviews and mid-term reviews are useful. PEMconsult’s approach is to draw on our experience of a variety of quantitative and qualitative approaches and techniques and to tailor them to the particular assignment. Our team has a strong reputation among clients for carrying out high quality evaluations and reviews at the strategic, programme, and project levels. Moreover, PEMconsult is experienced with assisting clients with ensuring that monitoring moves beyond measuring activities and outputs and towards capturing results, outcomes and impact.
It is important in most if not all situations that the monitoring and evaluation is:
– Set in a contextual understanding
– Representative and systematic
– Evidence based
– Independent and free from bias.
OUR WORK - EXAMPLES:
AN EVALUATION OF EIB SUPPORT FOR WATER AND SANITATION OUTSIDE THE EU
For the period from 2010 to 2021, water and sanitation has been EIB’s fourth largest sector of engagement outside EU. The EIB water lending sector orientation is due for review in 2022/23. It is with this upcoming revision in mind it was decided to launch this evaluation. The evaluation covers all regions outside EU from 2014-2021. This evaluation examines the combined effects of EIB lending, blending and technical assistance, including on SDG 6 and on the climate ambitions of the EIB, the role of technical support in addressing capacity issues, including to unlock signed but undisbursed amounts, the coordination of EIB support with other IFIs, and synergies between the EIB’s project-based approach and other IFIs’ and donors’ interventions. The thematic content covers water resources development and management including flood control, water supply, sanitation and wastewater, and irrigation. The evaluation team is undertaking the entire evaluation process, from structuring the evaluation (including developing EQs) to the drafting and the dissemination of the final evaluation report. The analysis of projects involve a review of project documentation (including appraisal reports, project progress and completion reports, results measurement framework-REM reports) and interviews with loan officers, clients (promoters or financial intermediaries), and final beneficiaries, as well as field visits.
- European Investment Bank (EIB) (2022 – 2023), contract value EUR 393,910
TECHNICAL ASSISTANCE TO MONITORING OF THE IMPLEMENTATION, VERIFICATIONS AND O&M ACTIVITIES OF THE “ASSELA WIND FARM PROJECT”
Danida Sustainable Infrastructure Finance (DSIF) is financing the construction of a 100 MNø windfarm around 150 km south of -Àddis Âbaba, Ethiopia, with a soft loan. The owner of the windfarm to be built is Ethiopian Electric Power (EPP), the State-owned electricity company which manages the electric transmission network and generates most of the electricity in the country. The purpose of the present assignment is the Technical Assistance to the monitoring of the construction, commissioning and technical assistance for operation and management. This assignment includes a variety of tasks like monthly monitoring and progress reports, the final verification report, several reports on output and outcome indicators, and a final project completion report on output and outcome indicators 12 months after the end of the operation and monitoring period.
- Danida Sustainable Infrastructure Finance (DSIF) (2022 – 2024), contract value EUR 268,000
PROJECT EVALUATION: SUPPORT TO PUBLIC FINANCIAL AND ECONOMIC MANAGEMENT, MALAWI
Subject to this evaluation is the project “Support to Public Financial and Economic Management”. The focus of the project is on strengthening the budget implementation (at the national level). The module objective is “to support the Malawian Government’s efforts to improve public financial management” in terms of efficiency, regularity and propriety of budget management, domestic resource mobilization, and internal and external control mechanisms. Central project evaluations are divided into two phases. In the beginning is an inception phase with the preparation of the inception report on the methodological approach. Secondly, there is a data collection and evaluation phase with a field mission and the preparation of the evaluation report. Based on preparations during the inception phase, the field mission intends to collect the necessary data for the evaluation by conducting interviews with the involved stakeholders and by other methods defined in the inception phase. Data are then analysed, triangulated, assessed, and validated systematically. Based on the findings, the evaluation report is elaborated.
- Deutsche Gesellschaft für Technische Zusammenarbeit (GTZ) (2021 – 2022), contract value EUR 63,547
INDEPENDENT EVALUATION OF SDC’S PERFORMANCE IN NATIONAL POLICY DIALOGUE 2013 - 2020
PEM is excited to have been contracted by the Swiss Agency for Development and Cooperation (SDC) to undertake an evaluation on SDC’s performance in national policy dialogue. Policy dialogue processes can deliver a transformational impact on development aid that is bigger than the sum of individual projects and programmes. The aim of the evaluation is to provide a better understanding of what constitutes successful national policy dialogue and to contribute to the improvement of national policy dialogue as an instrument for SDC. PEM will be responsible for developing the overall methodological approach for the evaluation, in addition to defining the Evaluation Questions, constructing the Intervention Logic, a Theory of Change and an Inception Report. Field missions will be undertaken to conduct Interviews and Focus Group Discussions with SDC staff, government partners, implementing partners, other donors and development partners. The final Evaluation Report presenting the overall conclusions and recommendations will be submitted at the end of the year. Policy change is a highly complex process influenced by numerous interacting forces and stakeholders. In this context, monitoring and evaluation is a considerable challenge, but also presents exciting opportunities to understand how policy dialogue can be successfully implemented in future development cooperation.
- Swiss Agency for Development and Cooperation (SDC) (2021), contract value EUR 150,220
EVALUATION OF THE EU COOPERATION WITH GUATEMALA 2007 – 2020
PEM has been commissioned by the European Commission to undertake an Evaluation of the EU’s cooperation with Guatemala from 2007-2020. Thematically, the evaluation is centred around Food security; Conflict resolution, peace and security; and Competitiveness. The evaluation aims to provide an overall independent assessment of the EU’s past and current cooperation with Guatemala and key lessons and recommendations on how to improve current and future strategies, programmes, and actions. PEM will provide the evaluation team and carry out the development and refinement of the overall evaluation methodology. Field visits will be carried out at selected project sites. The evaluation aims to deliver strategic recommendations based on the findings to assist the European Commission’s goal to contribute towards a more democratic and inclusive society, eradicating poverty and moving towards environmentally sustainable development.
- European Commission (2021) contract value EUR 349,712
INDEPENDENT EVALUATION OF SDC’S PERFORMANCE IN AGRICULTURAL MARKET SYSTEMS DEVELOPMENT 2013-2019
PEM has been commissioned to evaluate the performance of the Swiss Agency for Development and Cooperation (SDC) in Agricultural Market Systems Development (MSD) 2013-2019. The evaluation purpose is, in particular, to provide evidence on how the SDC MSD development programmes in Agriculture contribute to increasing income, supporting food security, reducing poverty, and improving resilience and livelihoods of smallholder farmers. The evaluation seeks to identify successes, difficulties and failures, as well as good practices of how MSD in Agriculture is implemented and assess to which extent and how such practices could be applied systematically within SDC. The evaluation team will look at the relevance, coherence, effectiveness, efficiency, impact and sustainability of SDC agricultural MSD programmes and projects in all four domains: South Cooperation, Cooperation with Eastern Europe, Humanitarian Aid, and Global Cooperation. Looking backwards and forward, the evaluators will assess accountability and performance, document evidence of results and focus on making strategic use of the findings.
- Swiss Agency for Development and Cooperation (SDC) (2021) contract value EUR 190,702
EVALUATION OF THE EU COOPERATION WITH THE UNITED NATIONS
PEM is part of an evaluation of the EU’s cooperation with the United Nations in 2014-2020. The total budget under the evaluation is approximately EUR 21.4 billion. The evaluation team is looking at the following EU priority areas: a) the Green Deal (environment, climate change, agriculture, green economy); b) transition to a new digital era; c) sustainable economic, human, and social development; d) peace, security, and governance (incl. democracy, rule of law, human rights); and e) migration and forced displacement. PEM provided the Team Leader and the senior expert for the Green Deal (Deputy Team Leader) who are leading the methodology development, the definition of the intervention logic (theory of change), the elaboration of evaluation questions. PEM will also lead review and analysis of the results framework for the EU-UN cooperation, data collection, the synthesis of findings and elaboration of conclusions and recommendations, as well as the overall technical supervision of the evaluation team. Remote interviews and focus group discussions will be carried out with key stakeholders. Depending on the COVID-19 situation, field visits may also be carried out. A public dissemination event will be held in Brussels in January 2022.
- ADE for the European Commission (2020) contract value EUR 415,496
EVALUATION OF THE DANISH SUPPORT TO CLIMATE CHANGE ADAPTATION IN DEVELOPING COUNTRIES
Yesterday PEM in association with ODI officially started the evaluation of the Danish support to Climate Change Adaptation in developing countries. Climate change is high on the political agenda with the current Climate Action Summit and increased attention in the Danish government. Countries with climate-vulnerable and poor people need to find ways to achieve climate-resilient societies and economies while addressing both current increased climatic variability and future climate change. Hence, a key focus area of the evaluation is to assess the mainstreaming of climate considerations in bilateral interventions that do not have climate adaptation as its main objective. The evaluation team is focusing on the climate resilience of marginalised and vulnerable groups, resilient rural livelihoods and interventions that contribute to transformation and long-term resilience of the societies. In addressing the above the team will examine interventions supported under the Climate Envelope as well as broader Danish Official Development Assistance to assess the joint Danish support to climate change adaptation and identify mechanisms that have been successful (or less successful) in this respect. Case studies will be conducted in Bangladesh, Burkina Faso, Kenya, Ethiopia.
- Ministry of Foreign Affairs Denmark (Danida) (2020) contract value EUR 536,603
EVALUATION OF THE SWEDISH WATER HOUSE 2010 - 2018
PEM is carrying out an evaluation of the Swedish Water House (SWH). Being an independent platform committed to mobilize and support coordination of Swedish actors interested in international water issues, SWH creates an interface for dialogue, collaboration and knowledge sharing. PEMconsult has been commissioned to reflect upon the success and lessons between 2010 – 2018 through a systemic review of the implementation of SWHs work and associated learnings. PEM will assess the relevance, effectiveness and sustainability of the implementation of SWHs activities and formulate recommendations regarding avenues to further strengthen, replicate and/or scale-up activities that will accelerate the engagement of Swedish actors in international water issues.
- Stockholm International Water Institute (2019) contract value SEK 327,340
EVALUATION OF THE SWEDFUND/SWEDPARTNERSHIP PROGRAMME
PEM provided two international experts to conduct an evaluation of Swedpartnership’s activities and achievements, including an assessment of the results of Swedpartnership’s operations, working methods and methodology as well as to come up with proposals as to how Swedpartnerhip’s activities should continue in the future. Swedpartnership offers financial support aimed at Swedish SMEs, which establishes their businesses in one of Swedfund’s operating countries. The overall goal is to promote the growth of sustainable, profitable and productive SMEs in the countries where Swedfund operates.
- NIRAS Sweden AB (2018-2019) contract value SEK 272,000
INDEPENDENT EVALUATION OF THE SWISS AGENCY FOR DEVELOPMENT AND COOPERATION’S PERFORMANCE IN DISASTER RISK REDUCTION (DRR) 2010-2017
PEM will carry out an evaluation of the Swiss Agency for Development and Cooperation’s (SDC) performance in Disaster Risk Reduction (DRR) including difficulties and solutions as well as good practices of how DRR is being implemented within the SDC. PEM will assess to which extent and how best practices could be systematically applied within the SDC. The Evaluation and Corporate Controlling (E+C) division of SDC conducts such strategic evaluations to gain evidence of its contribution to international cooperation at institutional level. It is foreseen that the evaluation will support strategic decision-making for future engagements on policy and operational level and will therefore include a benchmarking of the SDC’s performance compared to evidences from international practices and research is an important element of the evaluation.
- Particip (2018-2019) contract value CHF 29,230
MID-TERM REVIEW OF THE BANGLADESH COUNTRY PROGRAMME (2016-2021)
PEM will carry out a mid-term review of the Danish Bangladesh Country Programme 2016 – 2022 in accordance with the Danida Aid Management Guideline. The objective of the mid-term review is to assess the performance of the Bangladesh Country Programme focusing on the progress against the results framework and the theory of change, development in the risk scenario, sustainability/exit strategy, and provide recommendation for further implementation of the country programme. The Bangladesh Country Programme consists of three thematic areas, namely: Agricultural Growth and Employment; Climate Resilience and Sustainable Energy; and Governance and Rights and contributes to poverty reduction, growth and sustainable development and the promotion of democracy, respect for human rights, rule of law and good governance.
- Ministry of Foreign Affairs of Denmark (2019), contract value DKK 897,921
MID-TERM REVIEW OF UKRAINE DENMARK ENERGY CENTRE (UDEC) II (2018 - 2020)
PEM has been tasked to review the UDEC II with regards to relevance, efficiency, effectiveness and sustainability of the programme. Danish support to UDEC II is part of the Danish Neighbourhood Programme (DANEP) 2017 – 2021 and the DANEP Country Programme for Ukraine, under the thematic pillar on Sustainable and Inclusive Economic Growth. The mid-term review will include an assessment of progress against the results framework, an assessment of theories of change including assumptions and risks for each development output, and an assessment of relevance of the thematic focus. The mid-term review will also include an assessment of the level of technical quality and relevance of technical assistance provided, level of documentation and level of country ownership and sustainability of activities.
- Ministry of Foreign Affairs of Denmark (2019), contract value DKK 209,788
EVALUATION OF EU’S COOPERATION WITH MYANMAR
PEM led (in association with ADE) the evaluation of EU’s development cooperation with Myanmar in 2012-17. The Myanmar country programme is EU’s third largest country programme with an investment in 2012-17 of approximately EUR 420 million. The evaluation covered the four focal sectors: a) Education; b) Peacebuilding; c) Governance, rule of law, state capacity building; and d) Rural development, agriculture, food and nutrition security. Three international experts and three national consultants carried out the assignment, which entailed field work in Myanmar and a comprehensive desk review of available documentation and data.
- EU (2018/2019) contract value Euro 249,892.
MID-TERM EVALUATION OF THE LAO COUNTRY PROGRAMME OF THE GRAND DUCHY OF LUXEMBOURG
PEM has recently embarked on the mid-term evaluation of the current phase of the development cooperation programme between PDR Lao and Luxembourg with ongoing interventions for app. 75 mill. Euros. The team is commissioned to conduct the review of the four sectors which include the health sector, local and rural development, the tourism sector with focus on skills development and governance.
- Ministère des Affaires étrangères et européennes, Luxembourg (2018), contract value app USD 115,000.
EVALUATION OF EU’S COOPERATION WITH AFGHANISTAN
PEM provided the team leader/rural development expert for the evaluation of EU’s development cooperation with Afghanistan in 2007-2016. The Afghanistan country programme is EU’s largest country programme with an investment in 2007-2016 of EUR 1.8 billion. The evaluation covered the four focal sectors: a) Agriculture and rural Development; b) Health; c) Police and rule of law; and d) Democratisation & Accountability. The assignment entailed stakeholder consultations in Afghanistan and a comprehensive desk review of available documentation and data.
- EU (2016/2018) contract value Euro 79,748.
EVALUATION OF EU’S REGIONAL-LEVEL SUPPORT TO CENTRAL ASIA
PEM provided the team leader/environment expert for the evaluation of EU’s regional-level development cooperation with Central Asia in 2007-2014. The evaluation covered the four focal sectors: a) Environment; b) SME development; c) Border management; and d) Higher education. The assignment entailed field visits to Kazakhstan, Kyrgyzstan and Tajikistan, and a comprehensive desk review of available documentation and data.
- EU (2015/2017) contract value Euro 56,527.
OTHER ASSIGNMENTS:
- Mid-term evaluation of Smallholder Agriculture Reform through Enterprise Development (SHARED) Project, iDE Zambia, Sida, 2018.
- Evaluation of EU support for regional integration in EASAIO, 2016-2018.
- Global Evaluation of EU Sustainable Energy Cooperation, EU, 2017-2018.
- Evaluation of SIDAs International Training Programme on Private Sector Development, 2017.
- Mid-term review of UNDP-GEF project establishing a protected areas system in Afghanistan. UNDP, 2017.
- Mid-term review of WWF project on forest protection and livelihoods in Eastern Cambodia, WWF, 2016.
- Review of the UNEP DTU Partnership, Ministry of Foreign Affairs of Denmark/Danida, 2016-2017.
- Evaluation of the European Instrument for Democracy and Human Rights (EIDHR), EU, 2016-2017.
- Inception Review, Support in the Field of Energy Efficiency and Sustainable Energy, Georgia, 2015-2019
- Evaluation of EU, Sida and Danida cooperation with Bangladesh – climate change adaptation, EU, 2015.
- Mid-Term Evaluation of the UNDP Low Emission Capacity Building (LECB) Programme, UNDP, 2015.
- Review of Danish Institute of Human Rights, Danida, 2013-2014.
- Mid-Term Review of the Democratic Governance Facility (DGF), Uganda, Danida, 2013-2014.
- Evaluation of the European Union’s co-operation with the Region of Eastern Africa, Southern Africa and the Indian Ocean, 2008-2015.
- Evaluation of the Danish Neighbourhood Programme 2008-2017.
- Global Midterm Review of UNDP-UNEP Poverty-Environment Initiative (PEI).
- Development of an outcome-impact results monitoring framework for UNEP-DHI, Phase 5.
- Development of Theory of Change, Results Framework and M&E System for the Swedish International Agricultural Network Initiative (SIANI).