Evaluation Plan Guidance (Page 64 of 68) in pdf

EVALUATION PLAN GUIDANCE

SOCIAL INNOVATION FUND

results obtained from a sample to a population is when the sample is randomly selected from that population.

Otherwise, this generalization must be made on extra-statistical ground – that is, on a non- statistical basis.

Idiographic: Research that focuses on the outcomes for individuals rather than for groups. This is in contrast

to research that is nomothetic, which is research that focuses outcomes at the group level. For between-group

designs, the strength of the causal attribution depends on how the control or comparison group was formed

(random assignment, matching, non-random assignment).

Impact Evaluation: An evaluation designed to determine if the outcomes observed among program

participants are due to having received program services or the intervention.

Implementation Fidelity (Fidelity of Intervention Implementation): The extent to which the program or

intervention was implemented as intended. The intention usually is expressed prior to intervention delivery

and, when available, in intervention developer documents such as the program theory and logic model.

Informed Consent: A dimension of Human Subjects Protection that requires researchers to make sure that

potential study participants (both program participants and control or comparison group members) are fully

informed of the potential risks or benefits, if any, and conditions of study participation.

Intent-to-Treat (ITT): An approach for analyzing data from between-group designs in which study

participants are analyzed, (1) in the group they were assigned to at the start of the study, regardless of the

group they end up in at the end of the study, and (2) for individuals in the intervention group, whether they

participate in the intervention or not. In intent-to-treat analysis, the aim is to estimate the impact of the “offer”

of the intervention regardless of whether it is received, as opposed to focusing on how participants’ experience

of program participation affects an outcome.

Internal Validity: For a given design, the extent to which the observed difference in the average group

outcomes (usually program participants versus control or comparison group members) can be causally

attributed to the intervention or program. Randomized controlled trials allow for high causal attribution

because of their ability to rule out alternative explanations (usually unobserved characteristics) other than the

intervention as the reason for the observed affect.

Intervention: A term used to describe the services or activities a program does to achieve its stated outcomes,

goals, or desired results.

Intervention Level: The level (e.g., at the individual, group, community, or structural level of society) at which

a specific program offers treatment or services to address a particular problem or issue.

Level of Evidence: The quality of findings, based on empirical data, from an evaluation or research

study. Although there is no consensus within the evaluation field concerning what constitutes a particular

level of evidence, the SIF program divides evidence into three categories: preliminary, moderate, and

strong. These divisions are based on how well a particular evaluation is able to address concerns about

D.3

Evaluation Plan Guidance Page 64