Given the frequently unstable nature of post-conflict settings, where conditions may change rapidly for better or worse, rigorous monitoring and evaluation of peacebuilding interventions, together with ongoing conflict analysis updates, is critical for effective programme management and achievement of results. One of the long-standing challenges to successful peacebuilding, however, has been the difficulty of measuring results and generating evidence to identify what types of interventions work best. Often, peacebuilding interventions seek change that is more abstract than concrete: to positively affect citizens’ confidence in governing institutions, transform behavior of security sector personnel, or build social cohesion through national dialogue – none of which is easily measurable. The conditions that contributed to the conflict, moreover, also frequently result in weaker state institutions, and absent or unreliable administrative data. Also, peacebuilding interventions may take a long time to show clear peace-relevant results. These challenges highlight the importance of robust and regular assessment. In addition, the changeable nature of post-conflict societies means that planning assumptions must be regularly reviewed during the implementation cycle, and intervention strategies evaluated to determine if they remain valid, or require reassessment. As such, PBSO requires a clear, well-structured and appropriately costed Monitoring and Evaluation plan, including Results Framework(s), to plan for and capture the results of all of its investments. The following chapter provides guidance to project designers, implementers and M&E experts on how to establish M&E systems that will enable them, their senior managers and PBSO management to make informed decisions about the effectiveness of their planned interventions, and to generate knowledge for the broader international community about what works and does not work to support peacebuilding in post-conflict settings.
At the stage of conceiving and drafting the Priority Plan/projects, the drafting team should refer to the PBF’s Performance Management Plan (PMP), which is PBF’s Global Results Framework. The PMP can act as a guide to RUNOs on the kind of strategic results that PBF intends to support and how to measure PBF’s added value to country portfolios. The PMP is organized around the PBF’s four priority areas, which are given by the Fund’s 2009 Terms of Reference. The four priority areas include: i) support for the implementation of peace agreements and political dialogue, ii) promotion of the coexistence and peaceful resolution of conflict, iii) revitalization of the economy and generation of immediate peace dividends, and iv) re-establishment of essential administrative services. Within these four given priority areas, as noted in section 3.3 of these Guidelines, the PBF has identified nine Focus Areas that further define PBF’s strategic focus. RUNOs are required to situate their project within one of these Focus Areas at the time of submission. Starting in June 2014, the PMP will provide a menu of standard indicators for each of the nine Focus Areas. RUNOs will be required to select at least one of the standard indicators that pertain to the Focus Area of their project and include this indicator in the project’s Results Framework. PBF recognizes, however, that some projects may have outcomes that can be attributed to more than one Focus Area. For example, a DDR effort may be identified as an (Outcome 1) Focus Area 3 project. Through this project, however, ex-combatants may be provided with livelihoods support, which would fall under (Outcome 3) Focus Area 8. In cases such as this, the implementing UN Agency would be strongly encouraged to include a standard indicator from both Focus Area 3 and Focus Area 6 in the project’s Results Framework.
Whether submitting a Peacebuilding Priority Plan for PRF funding, or an IRF project proposal, all submissions need to be accompanied by a results framework that helps applicants and PBF to ensure the coherence, logic and complementarity of their interventions and to measure the results that will contribute to peacebuilding. The results framework establishes the basis for the M&E plan, which – during the implementation phase – will measure fund users’ performance in contributing to peacebuilding results. The Results Framework contains Outcomes, Outputs, Indicators, Baselines, Targets and Milestones.
Developing indicators for peacebuilding interventions is not an easy task. As noted above, peacebuilding interventions often are not easily quantifiable, and so qualitative measures must also be employed. PBSO encourages an appropriate mix of both qualitative and quantitative indicators to best capture the range of likely interventions, although in certain settings a heavier emphasis on qualitative indicators and process level indicators will be most realistic/ useful in order to capture behavior change. In many cases, indicators based on surveys of the population’s perceptions of peace, security, justice and government services will be useful.
Measuring Outcomes: Change of behavior (individual or institutional) or changed perceptions (e.g. increased confidence in government capacity to maintain public order during the election process) that impact conflict factors (e.g. distrust of the population towards government authorities).
Measuring Outputs: Short-term deliverables that fall within the direct responsibility of the Fund user (e.g. skills of police officers in crowd control and de-escalation tactics improved).
Regardless of whether using qualitative or quantitative indicators, when formulating indicators for PBF results frameworks, bear in mind the SMART criteria: Specific, Measurable, Attainable, Relevant, and Time-bound. Within this framework, applicants should ensure that their indicators measure the specific peacebuilding outcomes of their work, as opposed to more traditional sector-focused development indicators. They should also ensure that they measure both the effectiveness of the project/programme (i.e. achievement of intended results) and its efficiency (i.e. value for money). Finally, the results framework and the indicators should be aligned with the theory of change/ logic of the project/ programme and measure what the project/ programme intended to achieve.
In order for the identified indicators to serve their function, baselines and appropriate targets need to be established. Without baselines, project managers and JSCs will be unable to determine whether their interventions have actually contributed to the desired change. To establish baselines, RUNOs need to consider what data is already being collected by the Government, the civil society and the UN to see if this could be used as a baseline. If baseline data needs to be collected from scratch, this will come at a cost and so provisions for this will need to be included in the M&E budget.
Without appropriate targets, RUNOs will not be able to articulate the scale and scope of the change they are expecting to make, and PBSO will be unable to determine if this scale and scope is appropriate for the context and level of likely funding. Targets, then, are essential to help all stakeholders understand what specific change is expected. Targets, however, apply to the entire life of a given intervention. As such, RUNOs and JSCs may find it useful to establish incremental milestones to determine if the given intervention is progressing at an expected pace, or if adjustments are required. For example, a project may hope to achieve a 50% increase in the level of confidence in local governing institutions of a given population over the course of a 2 year project. Within that two year span, project managers will be required to report on the project’s progress three times. By breaking the project’s target into achievable milestones that are connected to the reporting cycles, project managers and PBF will be better able to assess if the project is on track to meet its 50% goal. For example, in the first 6 months of implementation the project may be expected to achieve only a 5% increase because the project has just begun and not all project activities have been launched. By the first year, however, the project manager may expect that she will see a 20% increase, as the project begins to mature, and a further increase to 35% by the 18 month reporting period. By setting project milestones such as this, period progress assessments can more meaningfully determine if implementation is on track or if various barriers need to be addressed and rectified.
In addition to ensuring the specific peacebuilding focus on results, indicators should also be examined for how they capture differences in gender vis-à-vis the outcomes. As noted in Section 3.4, gender responsiveness may enter into programming in one of two typical ways: either as an explicit aim of an outcome area (for example, to increase women’s empowerment within the socioeconomic and political arenas in support of a peace process), or as an indirect measure through sex and age disaggregated data of an outcome that is not explicitly addressing gender inequality. Without disaggregation, PRF-funded initiatives will be unable to state with certainty what impact they have had on gender equality and women’s empowerment within peacebuilding processes.
A final, general consideration when devising results frameworks is to account for differences in timing implied by certain outcomes and the projects that fit under them. For the case of PRFs, in many Priority Plans, the achievement of key results from one strategic outcome may be critically important for the launch of another outcome. For example, an outcome on social cohesion through improved ethnic minority rights at the community level may depend on the prior achievement of an outcome on revision of key national legal and policy frameworks ensuring minority protection. These kinds of sequencing considerations should be accounted for within the results framework.
While PRF Priority Plans and projects (both projects approved within the Priority Plans as well as IRFs) require results frameworks, the requirements for each are slightly different, given the different length of time, level of funding and complexity of the two. The below sections first outline the results framework components for the Priority Plan before shifting to results framework components for individual projects.
Priority Plan Results Framework
The Priority Plan’s results framework consists of two phases: phase one to accompany the submission of the Priority Plan for approval by PBSO, and phase two to be submitted to PBSO within 30 days of the Joint Steering Committee’s approval of projects against the Priority Plan.
Phase One (in blue shade within the PPP results framework template)
Given the strategic nature of PBF interventions, the results framework that accompanies the submission of the Priority Plan to PBSO should focus exclusively on identifying appropriate indicators for each Priority Plan outcome, together with baselines, appropriate targets, and means of verification. When formulating indicators, success or failure at measuring peacebuilding achievements will depend on how high or low you pitch the indicators. Indicators (and targets) pitched at too high a level – for example, achieving peace within a country – are not appropriate, given that PBF funding is relatively modest in comparison with the overall needs and that implementation timeframes last no more than 36 months, making deeply systemic, longer-term change an unrealistic goal. Conversely, pitching the outcome indicator at too low a level – for example, training demobilized former-combatants in life skills – is more appropriate for a project-level indicator. The Priority Plan outcome indicators should seek a middle ground, and measure results at a level above likely project-level results.
Phase Two (in yellow shade within the PPP results framework template).
Once the Priority Plan has been approved by PBSO, and the Joint Steering Committee has approved projects against the PPP outcomes, the M&E focal point for the Secretariat should work with RUNOs on elaborating the second phase of the PPP results framework. The second phase expands the results framework from the higher-level PPP strategic outcomes to include a focused selection of key project-level outcomes and their indicators, arrayed across the Priority Plan strategic outcomes. All project-level indicators should meaningfully relate to and flow into the higher-level PPP strategic outcome indicators. This second phase results framework should be completed within 30 days of the JSC project approval date. The second phase PPP results framework includes indicators, targets, baselines and means of verification for each outcome. It also includes shading boxes for Year 1, Year 2 and Year 3 to allow project managers to think when each outcome will be achieved and to visualize if some outcomes will depend on and flow from others. It also has a column to list any key milestones and dates which will allow the project managers to track the achievement of the targets over time. The second phase of the PPP results framework needs to be accompanied by an M&E Plan (Template 4.1). Where only a part of the Priority Plan has been projectized at one time by the Joint Steering Committee, the update of the Priority Plan Results Framework and the M&E Plan will be a rolling task.
Project-level Results Framework
As noted above, every project – whether a stand-alone IRF or a project within the Priority Plan – requires a results framework to provide evidence of how the individual project is achieving its intended peacebuilding results (Template 3.2 and 3.3). The project results framework makes clear the linkages between the immediate results (outputs) of a project and their intended contribution to the peacebuilding objectives and outcomes.
For projects associated with a Priority Plan, individual project outcome statements and indicators should be reflected in the yellow-shaded boxes in the Phase Two results framework. For IRF projects, the Results Framework should be included in the project document (the template for the Framework is included in the project document template). It includes outcomes as well as outputs and their indicators, baselines, targets, means of verification. For each outcome and output, there should be a shading of quarters and years to indicate when they are expected to be achieved and whether certain outputs or outcomes lead to others or are dependent on them.
High quality results frameworks are the starting point for measuring progress on desired outcomes. However, a results framework – even with SMART indicators – only provides an answer to the question “what do we measure”, it does not address the question of how or when to measure, nor does it set aside time to reflect on what has been achieved. For that, PBSO requires a sound and adequately budgeted Monitoring and Evaluation Plan.
An M&E Plan is a tool used to coordinate the collection and analysis of data for informed decision-making and substantive reporting and for evaluating achievements. It also helps to ensure the consistent and timely flow of information from data collection to reporting, using different data sources. The M&E Plan builds on the Results Framework by determining how the indicators will be tracked and stating clearly who is responsible for what and when, and what budget is allocated for this purpose. Similar to results frameworks, PBSO requires separate M&E Plans for Priority Plans as well as for individual projects.
Priority Plan Level
Given that project selection usually occurs after the submission and approval of the Priority Plan, the M&E Plan that is elaborated for Priority Plan submission will be an indicative one. Elements of an adequate indicative M&E Plan should include: 1) surveys or other data collection needed to obtain baseline data, 2) the scheduling of annual and mid-term reviews at the PPP level, 3) any required data collection activity a mid-term review may require, and 3) data collection toward the end of the project to enable both project staff and evaluators to assess the project’s outcomes. M&E Plans may also include any evaluative exercises the Secretariat or individual RUNOs may wish to undertake in the programme cycle.
While an indicative M&E Plan will need to be included in the Priority Plan proposal, finalization of the PPP M&E Plan will occur upon finalization of the second phase results framework. Upon acceptance of the Priority Plan by PBSO and project selection against the Plan by the JSC, the PBF M&E focal point should convene a working group of all project managers receiving PBF funding. Drawing upon phase two of the results framework, the working group should list all of the means of verification for all indicators within the results framework, including those at the strategic outcome level as well as project outcome level. Once the list is drawn up, the working group should identify which means of verification are routinely collected at no additional cost, and which require special data collection efforts (for example, the implementation of a perception survey). For those requiring special efforts, a budget should also be provided.
Regardless of whether data collection is routine or via a special, dedicated mechanism, the working group should indicate the relative frequency of data collection. Considerations for setting the frequency should include: what is the implementation timeframe and, if quite short, how many opportunities will project managers have to adjust their approach; how much time is likely needed in order to see meaningful change in the indicator; and what are programme management’s opportunities for analyzing and adjusting programme implementation through mid-year or mid-term reviews and evaluative exercises.
Once a list of means of verification and their frequency and budget implications are clear, all information should be included in template 4.1, and submitted at the same time as the Phase Two results framework submission to PBSO.
Priority Plan evaluations do not need to be included in the M&E Plan of the Priority Plan because they will be planned and managed by PBSO/PBF in New York.
Funding for these evaluations will be drawn from the Priority Plan allocation, under the Secretariat support project, and retained by MPTF-O for use by the PBF M&E Unit at the close of the Priority Plan. Individual project-level evaluations for projects funded through the Priority Plan are not required by PBF but could serve as additional data sources for reporting should the country teams decide to undertake them.
All projects must have an M&E Framework, however not all projects require an individual M&E Plan. For PRF projects, it is sufficient that there is an M&E Plan at the Priority Plan level, which takes into account all the projects contributing to the implementation of the Priority Plan.
IRF projects will require an M&E Plan only if they require an independent evaluation. PBSO requires evaluations for IRF projects when: funding amount is $1.5 million or more, or if they last for more than 12 months, or if they are implemented in new PBF country contexts or using interventions that are particularly risky or innovative. Project managers should contact PBF M&E staff when drafting their proposals to discuss whether their projects fall within our outside any of these criteria. If an evaluation/M&E Plan is required, it needs to be part of the project proposal, with funding set aside to conduct the evaluation, which will be managed at the country level.
The main purpose of reporting is to offer evidence – based on the sound analysis of M&E data – of which PBF interventions performed well and achieved expected results, what challenges were encountered, what were the lessons learned, and which of these could be used to build knowledge of how PBF interventions have successfully addressed conflict drivers. Such reports are a crucial means for programme managers to reflect on implementation and make adjustments where necessary, as well as for PBSO to enhance accountability on results and sharing of knowledge on successes, as well as failures. For PBSO, such accountability extends to the UN General Assembly and Secretary General through the Security General’s Annual Report on the Peacebuilding Fund, as well as to Fund donors and PBF recipient member states.
Reports are required at the level of the Priority Plan as well as for each project, regardless of whether the project is an IRF or funded under the PRF. Reports are required if the project or the Priority Plan commenced at least 6 months before the due date of the report. All RUNOs and Secretariats are required to utilize the PBF reporting templates for reporting on PBF-funded initiatives. Narrative descriptions of project or PPP implementation should be based on evidence and be grounded in the achievement of the results framework milestones and targets.
Reporting on programmatic results of the Priority Plan (Template 4.2)
As the Joint Steering Committee is responsible for ensuring the achievement of the Priority Plan outcomes, it is responsible for submitting an annual report summarizing progress against the milestones and targets established in the Priority Plan’s results framework. The main purpose of this report is to assess PBF’s overall added value in contributing significantly to peacebuilding processes at the national and local level, to review the quality of programmatic results, and to monitor carefully risks and changes within conflict dynamics. The deadline for submission of this report is the first week of December, as it is used as input into PBSO’s Annual Report to the Secretary General, Security Council, General Assembly, donors and UN. The report will be reviewed by PBSO according to established criteria elaborated in guidance available to JSCs to assist with the preparation of the report.
The JSC, the Office of the Senior Resident UN Representative and the PBF Secretariat are also responsible for quality control of the half year, annual and end of project narrative reports submitted by the RUNOs (see below).
Reporting on project specific results (Templates 4.3 – 4.5)
PBF’s reporting procedures are governed by the standard MoU signed between the MPTF Office and RUNOs. RUNOs are responsible for reporting on progress against anticipated outcomes twice a year, using PBF reporting templates: a six-month report due on 15 June and an end-of-year annual report due on 15 November. Quarterly reporting to the MPTF Office is no longer mandatory. In addition, RUNOs are required to submit annual financial reports by 30 April to the MPTF Office. Please note that financial reports are submitted by the RUNO’s headquarters. Joint projects should submit a single consolidated project report.
For projects that have concluded operational activities, End of Project reports must be submitted (Template 4.5) within three months of operational closure. If the project is finishing at the end of the year, the End of Project report can be submitted instead of the Annual Report for that year.
Summary of reporting responsibilities
PBF/MPTF-Office Reporting Cycle
|Who reports?||To whom?||When?||What and How?|
|- Joint Steering Committee (PRF)||PBSO||- Once a year (deadline 1 December)||Reporting on the status of programmatic results against the Priority Plan (Template 4.2)|
|- RUNOs (jointly)||JSC / RCO
|- Half year (deadline 15 June);
- Annual report (deadline: 15 November)
- End of Project Report (3 months after operational closure or instead of Annual report)
|Reporting on the status of project results (Templates 4.3, 4.4 and 4.5); financial statements to MPTF-O (30 April)|
|- Implementation Partners (project manager and team)||RUNOs||- Quarterly or more frequent, depending on context||RBM / early warning systems at project level: Review of target achievements / milestones of Annual Workplan|
Independent evaluations provide an impartial assessment of the project/ programme. They also help determine the strategic positioning of PBF based on its success, added value, and lessons learned on how to achieve strategic impact on addressing conflict factors. As noted above, PBSO requires a portfolio-wide, independent evaluation for all PRF allocations. These evaluations are designed and managed by the PBF M&E Unit in New York, in close cooperation with the JSC and PBF Secretariat. Final external project evaluations are mandatory for certain IRF projects (those that are above $1.5 million, those that last more than 12 months and those that are particularly innovative/risky) – PBSO should be consulted when the project is being developed to determine whether a given project will need to conduct an evaluation. End of programme/ project evaluations and should take place 6 months prior to the financial closure of the programme, and at or around operational closure. Mid-term evaluations or other evaluative exercises remain optional. An independent evaluation is required prior to any consideration for a renewal allocation.
PBSO has developed standard ToRs for evaluations (Guidance Note 5.3) that follow the OECD/DAC (and UNEG) quality norms and standards for evaluating programmes in fragile countries1. In most cases, the evaluations are jointly conducted by an international and a national consultant. The final decision on the Team Leader is the responsibility of a PBSO selection panel (comprised of three members) in compliance with PBF rules and regulations. National consultants will be selected by the UNCT according to their procurement procedures and in consultation with PBSO. The M&E Advisor at PBSO is in charge of oversight during the whole evaluation process and is accountable for quality assurance of the final evaluation report. The cost for external evaluations of IRF projects is fully covered by project funds and should be included in the budget for every project proposal.
Evaluation findings are shared with JSC, PBSO and UN interagency working groups and their comments on the evaluation findings are included in the final report as an Annex. A management response is due into PBSO within 30 days of receipt of the final report. For PRF evaluations, PBF M&E Unit will coordinate the drafting of the response in close coordination with the JSC. For IRF project evaluations, RUNOs are responsible for finalizing the management response and submitting it to PBF M&E Unit. Once finalized, all evaluations will be posted on the PBF website, together with the management responses.
- M&E plan (Template 4.1) Download M&E Plan- Template 4.1 – JSC annual reporting (Template 4.2) Download Annual Reporting of the Joint Steering Committee- Template 4.2 – RUNO half year reporting (Template 4.3) Download Project Half Yearly Progress Report- Template 4.3 – RUNO annual reporting (Template 4.4) Download Project Annual Progress Report- Template 4.4 – RUNO final project reporting (Template 4.5) Download End of Project Report- Template 4.5
Related Guidance Notes
- How to use PBF Global Results framework (PMP) (Guidance Note 5.1) Download PBF Global Results Framework- Guidance Note 5.1 – How to develop TORs for evaluating PBF country portfolio (Guidance Note 5.3) Download TORs for PBF Country Evaluations- Guidance Note 5.3
- 2.Guidance on Evaluating Conflict Prevention and Peacebuilding Activities, OECD/DAC, June 2012 ↩