Basic Impact Assessment At Project Level Page 20

ADVERTISEMENT

Beneficiaries are the easiest group to approach as generally they accept ‘answering
questions’ as one of the unavoidable transaction costs of being in a program, particularly
one supported by a foreign donor agency.
Motivation can be enhanced by having
interviewers introduced by program officers: but, this has the danger of linking the
assessor with field level staff and encouraging the recounting of ‘the right answers’. For
both data quality and ethical reasons the personal introductions that interviewers make
prior to interview need to be carefully worked out so that respondents understand why they
are being interviewed and have an opportunity to ask their own questions before the
interview begins.
Motivation is a more difficult issue with control groups as, having by definition no
connection with a program, they have no incentive to cooperate. In many cases, however,
the novelty value of being interviewed is sufficient encouragement (though expatriates
should note that when they are working at a field site the willingness of people to be
interviewed may be higher than is the norm because of the rarity value of foreigners). The
problems of response increase significantly if longitudinal data is collected, as second and
third interviews have much less perceived value. In such cases rewarding interviewees
should be considered to promote data quality and for ethical reasons (what right have
impact assessors to assume that the opportunity costs of an interview, particularly for poor
people, are zero?). This can take the form of a social reward, such as bringing soda
waters and snacks to share with respondents (this works well in East and Southern Africa),
or ‘bribery’ where the interviewee is paid cash for surrendering her/his time.
Program drop-outs represent a particular problem, and a failure to pursue drop-outs may
have led to some IAs underestimating the negative impacts of projects. When the drop-
out is traceable then significant effort is merited to obtain an interview/re-interview. Where
drop-outs cannot be traced, or death has occurred, then a replacement respondent
sampled at random from the original population, and preferably from the same stratum,
should be interviewed.
Participatory and rapid appraisal methods that work with groups generally manage to
muster respondents because of the social interaction they create. However, care needs to
be taken to observe who has turned up and, perhaps more significantly, who has not come
to the meeting, and why. It is not necessarily the case that participants in a PLA exercise
represent ‘the community’. Additional interviews or focus groups may be necessary to
collect information from people who do not turn up for communal PLA or RRA sessions.
• The problem of ‘low impact’ impact assessments
A final problem of IA concerns the impact of IAs on policy and practice. This depends in
part on the original objectives of a study. It applies to both ‘proving’ and ‘improving’ IAs.
The evaluation literature of the 1980s bemoans the limited influence of evaluation on
subsequent decision-making.
There are a number of ways this problem can be ameliorated:
(i)
Impact assessors need to devote more time to the ‘use’ of their studies (and
perhaps a little less time to the product itself!). Their focus must go beyond ‘the
report’ into a dissemination strategy aimed at decision-makers.
Bullet point
20

ADVERTISEMENT

00 votes

Related Articles

Related forms

Related Categories

Parent category: Education