Technical Supplement

A.

Rationale for Government Intervention

Government generally implements expenditure programmes in order to overcome problems caused by 'market failure' or to address social/equity considerations. The following describes a number of 'market failures' and looks at the social/equity rationale.

Market Failure

The main economic justification for government intervention in an economy rests on the concept of market failure. As a result of market failure the government may choose to restructure, complement or supplement the unrestricted workings of the market economy. Several types of market failure exist:

Public goods - certain goods or services which if they provide benefits to anyone provide benefits to everyone. These are known as public goods. An example of a public good is national defence. An adequate defence system protects everyone even if they would otherwise choose not to pay for it. If the free market were left to provide for national defence some people would buy the service but others would not, knowing that they will have equal protection, (the "free rider" problem). In a free market system there is no mechanism to compel payment for a public good since there is no way of preventing a person from receiving the benefits of the good even if they choose not to pay. Only government through their power to tax can compel payment.

Externalities - an externality exists where all the costs and/or benefits of production of a good or service are not reflected in the price of that good or service at its transaction. For example, firms which discharge trade effluent into a lake may impose a negative external cost in the form of reduced catch on those who depend on fishing from the lake for their livelihood.

Merit goods - Merit goods are those which it is socially desirable for people to consume. However, without government intervention, individuals might not consume the optimum amount in terms of benefiting society as a whole. Education and vaccination are examples of merit goods. Government usually provides education and vaccination services because it recognises that individuals, left to their own devices, might not choose to consume these.

Natural Monopolies - these services were originally provided by the public sector because the start-up capital costs were so high that the private sector was unwilling to undertake the risk involved, for example, the network infrastructure of water supply, electricity and telecommunications industries. Current economic debate focuses on whether these services should be provided by the public sector or by a commercial concern which is controlled by a regulatory authority e.g. OFREG, OFWAT, OFTEL.

Incomplete Information and Uncertainty - this form of market failure could lead to insufficient investment for the future if people or firms are unwilling to undertake the efficient amount of risky activities because, for example, there was not an adequate set of contingent or insurance markets for dealing with risk. A complete set of insurance markets would allow risk to be transferred from those who dislike risk to those who are prepared to bear risk at a price and so build risk into the market system to achieve an efficient outcome. There is also the problem that information is incomplete because gathering information is costly and this may lead to socially inefficient allocations, for example, a worker who is unaware that exposure to high levels of benzene, as happens in some chemical plants, might cause cancer, will be willing to work for a lower wage than if he knew the risks associated with benzene. The firm's production costs will understate the true social cost and the good will be over-produced. Government can intervene, in this case, to regulate health and safety.

Social Priorities - Here the Government's concern is not about the efficiency of the market but rather about the equity or the social justice of the distribution of resources and with the quality of life of members of society. It may, for example, be considered important that income is distributed so that no individual's income falls below a level adequate for basic subsistence. The Government could therefore intervene to bring the distribution of incomes into line with that considered to be just and fair.

B.

The Base Case and Evaluation Analysis

Evaluation of a programme necessarily involves consideration of what would have happened if the programme had not been implemented, i.e. the Base Case (or sometimes referred to as the Counterfactual position). This can be viewed as an attempt to predict an alternative outcome and is essential in assessing whether the activity has changed things for the better or would they have improved anyway. This is a crucial aspect of evaluation, bringing out its comparative nature and it also provides the basis for analysis.

Description of Designs - True Control Group Designs

Two truly random groups are selected from the client population. Only one of these actually receives the programme, the other - the control group - does not. Comparisons of the selected output measure(s) can then be made between the two groups in order to determine if the policy or programme has made any significant difference. A comparison of the groups at the pre-policy period either as a time series (design 1) or at single points in time (design 2) can act as a check that the two groups initially do have identical characteristics. However, if there are statistically significant differences in the pre-programme period then it may be necessary to have, for example, a stratified sample based on the characteristics of the programme affected group. If pre-programme implementation measures for the two groups are not available, then the groups will have to be compared in the post-programme implementation period only (design 3).

Non-equivalent control group designs

In the situation where a time series is available for the non-equivalent group, it may be possible to use trend extrapolation (see paragraph B7) of the group. Here the trend of a non-equivalent control group is applied to the baseline position of the policy affected group to determine the base case situation. This design could be used in the absence of pre-programme time-series data for the policy affected group but selection of an appropriate comparison group is crucial. An example of this is where the number of small businesses in NI was not monitored prior to the programme's implementation, then the trend of small business growth in an area in Britain (the non-equivalent control group) could be applied to the baseline to determine what would have happened in the programme's absence. The use of this design assumes that the non-equivalent control group has not been affected by GB policies.

Single Group Designs

With the single group post-policy only design (design 9), information is necessary on what would have happened without the policy for the design to be feasible. Usually this type of information can be obtained through a survey of the policy affected group and it may even be possible to derive retrospective data on the pre-policy position and so have a 'before and after' or more appropriately, a 'with and without policy' comparison.

A further type of design -'what if' - is also possible but it is a design of the last resort where there is little or no information for the pre and post phases of the programme to enable proper comparisons to be made. In this case the key question is 'what would happen if the programme was terminated now?' The design then takes the form of comparing the consequences of stopping the programme with allowing it to continue. The evaluation then becomes like an appraisal (a look forward) rather than an evaluation (a look back).

Selecting and Using designs

Comparison of the policy affected and a non equivalent control group in a single period (i.e. post programme) is less reliable as the two groups may not have had identical characteristics in the period immediately prior to the programme's implementation. If, in practice, it proves problematical to find a suitable comparison group then reliance has to be placed on the single policy affected group and constructing a base case using trend projection and/or surveys. However, evaluations in which only the policy affected group is measured makes interpretation of the results more difficult. Moreover, with respect to trend extrapolation, it should be recognised that, for example, external shocks can complicate the construction of a counterfactual position in the 'policy on' period.

The type of evaluation design chosen and data availability will influence whether the quantitative or qualitative approach is more appropriate, although most evaluations would benefit from a combination of the two. Conclusions which are supported by a range of methods and data sources should be the most reliable. Quantitative and qualitative approaches should therefore be seen as complementary rather than alternatives and can be used together in addressing different questions within each evaluation. The choice of methods and designs must however be made in the context of the questions that need answered and the timescale and resources for the evaluation. The selection of the most appropriate research methods can therefore be a difficult task but the starting point is a recognition that there are options.

Analysing Net Additionality

Different types of net additionality can be measured:

• Full additionality is where the programme's benefits are wholly attributable to the programme, i.e. deadweight and displacement are zero.
• Partial additionality is where the activity would have been carried out earlier, or on a larger scale or to a higher specification or has displaced existing activity.

The comparison between the base case and the programme's actual outcome, in terms of the output indicator(s) chosen, encapsulates activity that would have occurred in the programme's absence, (deadweight) and also activity which has been displaced by the programme's existence, ( displacement). It also takes account of supplier multiplier and local multiplier effects. These impacts are described in more detail below.

Displacement of activity within a local area can occur:

• - through product markets, where the output of a supported programme takes market share from other local firms producing the same or similar goods or services;
• - through factor markets, where a supported programme uses locally scarce factors of production (e.g. certain skills or land) or by bidding up their prices.

Additional local activity is likely to raise local income and this will generate additional expenditure in the area adding to local employment. The wider an area is defined, (provided it remains small relative to the total national economy), the higher will be the income multiplier. For small areas it is important to estimate how many of the additional employees are resident within the policy area as those who came in from outside (e.g. for construction work) may spend little of the additional income in the area covered by the policy or programme. For most activities local income multiplier effects are fairly small: estimates are generally around 1.1. Estimates significantly above this will need strong analytical support. Regional multipliers, where relevant, may be larger: estimates have ranged from 1.2 - 1.5. If such an estimate is proposed, supporting analysis will be required for the particular region or wider area.

Use of Control Groups to measure net additionality

Using this approach will not allow separate identification of deadweight, displacement or the multiplier effects for the indicator chosen. Moreover, it should be recognised that the programme may impact on other factors not covered by the indicator(s) chosen, for example, it may impose external costs or benefits on third parties.

Use of Trend Extrapolation to Measure Net Additionality

Figure B3: Single group trend extrapolation

As with the control group analysis, this approach will not allow separate identification of deadweight, displacement or the multiplier effects for the indicator chosen. Moreover, it should be recognised that the programme may impact on other factors not covered by the indicator(s) chosen, for example, small business growth might be obtained at the expense of a reduction in larger businesses. This will not be picked up by the estimate of the base case but it should be considered in the analysis.

The framework for an ex-post CBA will normally follow a similar sequence as that outlined for evaluation:

• Construct the base case;
• Identify, quantify and, where possible, value the costs and benefits of the programme;
• Weigh up the uncertainties;
• Compare the outturn with the base case;
• Present the results in terms of whether the programme has represented value for money and the extent to which the market failure which was the original justification for the programme has been resolved.

Costs and benefits covered by an evaluation will often include:

1. initial capital costs: including buildings, equipment and land;
2. capital cost of any buildings or equipment which need to be replaced over the evaluation period;
3. residual values of capital assets at the end of the evaluation period;
4. operating costs over the whole term of the evaluation;
5. other costs or benefits which can be valued in money terms, in the form of revenues, cost savings or non-marketed outputs;
6. measures or descriptions of those costs or benefits which cannot be valued in money terms.