Evaluation Plan Guidance (Page 26 of 68) in pdf

EVALUATION PLAN GUIDANCE

SOCIAL INNOVATION FUND

most respondents will interpret it in similar ways. Appropriateness refers to

Additional Resources

matching measures to the population or scale of a study. To illustrate,

measures that require literacy should not be used with young children or

More details on generating

under-educated populations. Multiple ways exist to test validity and

reliable, valid, and

reliability of measures; see May et al. (2009) or the University of Leeds’ (2011)

appropriate measures, see

May et al. (2009)

“Using state

guide to designing questionnaires for more information.

tests in education

experiments: A discussion of

Specific Guidance: Measures

the

issues” at

The evaluation plan should provide a clear indication of how each measure

aligns with the outcomes in the logic model. As such, each outcome should be

9013.pdf.

clearly defined as a confirmatory measure, or an exploratory measure (see

For information specifically on

above for definitions of ‘exploratory’ and

‘confirmatory’ research

questions).

questionnaire design, see the

In-depth detail regarding each variable to be measured should be provided. If

University of Leeds

“Guide to

the variable is to be measured by a survey, test, interview, or structured

the design of questionnaires”

observation, describe, to the extent feasible, the following:



The intended respondents;

ds/file/125/guide_to_the_desig



The proposed administration method;

n_of_questionnaires



The number of questions included;



The anticipated administration time;



How the questions are organized and worded;



The response categories used (if appropriate);



Potential score/response ranges; and



Typical measure distributions in a general sample (if available – such information allows evaluators to

detect any potential mismatch between analysis technique and the proposed measures).

If the measure has already been developed and tested, then its origin should be described, including relevant

citations. Of particular importance is whether the measure was developed by the researchers themselves, or if

it is a commercially available measure.

If the measure is part of an existing data set (e.g., program records of service provided, medical test results,

financial data), describe the following:



The proposed data source (e.g., program records, public data sets, patient records);



When the data were collected and by whom;



Who funded the original data collection; and,



The type of data provided by the data source (e.g., Are financial/medical data provided for each

participant or are they aggregated? Are data on program services provided available as counts,

percentages, for individual services or for groups of individuals?).

If different measures of the same constructs are to be used for different portions of the sample, this should be

clearly indicated and the implications this has for the analysis plan should be described.

If data are to be provided by a third party, such as a school, it is highly recommended that a letter of

agreement from that third party be included in the evaluation plan, if available.

nationalservice.gov/SIF

Evaluation Plan Guidance Page 26

Related Articles

Related forms

Related Categories