Evaluation Plan Guidance (Page 37 of 68) in pdf

EVALUATION PLAN GUIDANCE

SOCIAL INNOVATION FUND

VII. Evaluator Qualifications and Independence

Overview

Evaluator qualifications are critical to a successful evaluation that will strengthen the level of evidence. A

single evaluator or a team of evaluators is fine, so long as all the necessary skills are covered. When selecting

an evaluator, it helps if the evaluator has worked with similar programs and has demonstrated experience in

conducting the specific type of evaluation described in the evaluation plan.

Specific Guidance: Evaluator Qualifications

This section should focus on the program evaluator and the evaluator’s background and qualifications.

Explain why the evaluator was selected, including the extent of the evaluator’s experience with both the

content area and the type of evaluation. This will likely include listing the evaluator’s experience with similar

interventions and with the type of RCT or

QED

that the evaluation is using (e.g., an RCT in which schools,

rather than students, are randomly assigned to treatment or control). List the key people designing and

overseeing the evaluation and ensuring its quality along with their education/training and type and years of

experience.

Verify that the evaluator can handle the scale and size of the proposed evaluation. Provide at least one

example of an evaluation that is similar in size, complexity, and number of sites. Discuss the experience the

evaluator has in managing similar evaluation protocols (e.g., this type of sampling, data collection, analysis). If

relevant, does the evaluator have the capacity to conduct an evaluation with multiple sites across a broad

geographic area?

Specific Guidance: Evaluator Independence

Include information that describes the independence of the evaluator from the program. To achieve a high

quality evaluation, it is important that the evaluator have enough independence to render an honest and

unbiased opinion of the program’s outcomes and impacts. However, it is also important that the project staff

be able to provide the oversight needed to ensure that the evaluation meets expected standards. Finally, it is

important that the evaluator not have conflicts of interest regarding the evaluation.

To address these issues, provide a description of the relationship between the program and the evaluator, and

what steps are being taken to ensure independence and alleviate any apparent or real conflicts of interest. In

doing this, it is important to explain the structure of the relationship between the intervention and the

evaluation, including both the role of the program staff with respect to the evaluator. This includes

determining whether the evaluator is hired by the program organization, and if that evaluator is an employee,

consultant, or outside firm. In addition, what are the reporting and financial arrangements between the

program and the evaluator? Who is responsible for giving direction and providing oversight?

Address whether or not there are conflicts of interest related to the evaluation. Conflicts of interest could be

related to a part of the program, the evaluator, or the relationship between the two. For example, has the

evaluator played a role in designing the program, or is the person supervising the evaluator also responsible

for program implementation and success? If there are conflicts of interest, they should be disclosed and

measures taken to mitigate them discussed.

nationalservice.gov/SIF

Evaluation Plan Guidance Page 37