Skip the NI Direct Bar
Skip navigation

11.2 Conducting an Evaluation

11.2.1PERs should be conducted by the Project Manager at project closure. However, all other evaluations should be led by individuals who have not been involved in the management or implementation of the proposal under consideration. This is so that they are in a position to take an independent and unbiased view. It is desirable and should be possible to maintain this principle even for the evaluation of lower value projects, although it may not be practical for all de minimis expenditures (i.e. those below £500k). Basic principles for planning and managing evaluations are outlined at 1.6 above.
11.2.2

DFP will generally expect Departments to conduct project evaluations in accordance with the Post Review section of the Successful Delivery (NI) website and according to PRINCE2 procedures. This requires evaluation to be conducted in two stages:

  1. A Project Evaluation Review (PER):
    • This reviews the effectiveness of the project management up to the point of project closure. Led by the Project Manager, it should result in an End Project Report and a Lessons Learned Report as described in the PRINCE2 guidance.
  2. A Post Project Review (PPR), often referred to as a Post Project Evaluation (PPE):
    • A PPR or PPE should be planned before project closure and is the main substance of the ex post evaluation. It compares outturn against estimated costs and benefits, and generally reviews success in achieving objectives. It should be conducted 6 to 12 months after project closure, led by an individual independent of the Project Board and Project Team.
11.2.3Departments should apply the guidance provided at Successful Delivery (NI) website and in PRINCE2 on the conduct of PERs and PPRs. That guidance is not repeated here. However, the following paragraphs provide some general principles for ex post evaluation that are relevant to PPRs/PPEs and apply generally to ex-post evaluation of policies, programmes and projects.
11.2.4Planning for evaluation must begin at the appraisal stage and should ensure that the appraisal reports contain the information needed for evaluation. This should include an outline plan, setting out the general boundaries of the proposed evaluation, as indicated at 2.9.12-17 above.
11.2.5

An evaluation should normally follow this sequence:

  • Establish exactly what is to be evaluated and how the outturns can be measured.
  • Define the counterfactual i.e. estimate what would have happened if the intervention (e.g. the project, programme policy or financial assistance) had not occurred.
  • Compare the outturn with the target outturn, and with the effects of the chosen counterfactual(s).
  • Present the results and recommendations.
  • Disseminate and use the results and recommendations.
11.2.6The status quo or other baseline option used in the original appraisal should normally inform the counterfactual. However, viewing events from a post hoc position, evaluators may judge that the counterfactual would actually have been quite different from what was envisaged at the time of the appraisal, due to, for example, alternative states of the world and/or alternative management decisions. In such circumstances it may be helpful to consider other counterfactuals in addition to the original baseline option. The streams of costs and benefits that would have occurred in the counterfactual(s) should be estimated and set out so that the actual outturn costs and benefits can be compared with them.
11.2.7The above sequence applies broadly as much to projects as to policies and programmes. However, proportionate effort should be applied e.g. when dealing with individual projects, particularly smaller projects, there tends to be less emphasis upon detailed consideration of alternative counterfactual states of the world. In most cases effort should be concentrated upon evaluating the extent to which objectives have been achieved, whether assumptions have proved accurate (e.g. by comparing outturns with target outturns), and what lessons can be learned.
11.2.8

In general, evaluation reports should summarise

  • whether, and if so, why the outturn differed from that foreseen in the appraisal;
  • how effective the activity was in achieving its objectives, and why;
  • the cost-effectiveness of the activity; and
  • what the results imply for future management or policy decisions.
11.2.9The results obtained should generally lead to recommendations for the future. These might include, for example, changes in procurement practice, improvements to methods for estimating costs or benefits, changes to management procedures, or the continuation, modification or replacement of a project, programme or policy.
11.2.10The results and recommendations should feed into future decision making. The methods used to achieve this may require senior management endorsement. Efforts should be made to disseminate the results widely within the organisation, and for this purpose it may be useful to employ summaries of the main points, and synthesis reports incorporating the results from a number of evaluations with common features.
11.2.11A PPR template is provided at the Successful Delivery (NI) website. This indicates the minimum content of a PPE and is recommended for general use to document PPRs/PPEs.

Read on to DFP Monitoring of Post Project Evaluations (PPEs)

Back to Topic Index