Survey Methodology #2006-02 - Use Of Dependent Interviewing Procedures To Improve Data Quality In The Measurement Of Change - U.s. Census Bureau Page 10
ADVERTISEMENT
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25addition, is also highly sensitive to the definition of the analysis universe. To permit a
comparison between the two SIPP panels in the likelihood of observing “yes-to-no” and “no-
to-yes” changes at the seam, we also show, in the 2004 row, the change in the statistic from
2001 to 2004 as a percent of the 2001 estimate.
Despite the large amount of information in Table 1, and the wide variations among the different
characteristics in the levels of the estimates presented, we find the essential features of the results
to be remarkably consistent. We see those essential features as follows:
1. Seam bias has declined substantially in the 2004 SIPP panel. The non-dependent 2001
procedures were significantly less effective at controlling seam bias than are the dependent
procedures introduced in 2004. This conclusion is readily apparent in the first data column,
which shows the proportion of all month-to-month changes observed at the seam. Scanning down
that column in Part 1 reveals a completely consistent trend across all 15 analyzed characteristics
subject to the new DI procedures: in every case the 2004 panel estimate is lower than the 2001
estimate. As noted earlier, the 15 pairs of estimates in Part 1 represent 45 separate comparisons,
of which 42 showed a statistically significant difference (see Table 1 notes) according to a simple
t-test of the difference between two proportions; 36 of the 42 significant differences were
significant at the p<.001 level or beyond. Not surprisingly, the “Change Rate Ratio” column
presents a similar consistency: across all of the Part 1 characteristics, the ratio of the likelihood
of a seam change to the likelihood of an off-seam change is always lower in 2004 than it was in
2001. In many cases the 2004 estimate is less than half its 2001 counterpart.
2. The decline is attributable to the new DI procedures. Just as clearly as seam bias declined
in 2004 where SIPP implemented new DI procedures, it did not decline where the interview
procedures were the same in both panels, as shown in Part 2. For both Medicare coverage (the
US government health insurance program for people over 65), and employment at a particular
job, the questionnaire procedures in use in the 2004 panel are very similar to the procedures used
in 2001. Not surprisingly, the use of very comparable interviewing procedures yields effectively
identical seam bias results. This finding offers strong support for the essential argument of this
chapter, which is that the seam bias differences in Part 1 are due to the new DI procedures
introduced in 2004, and not to different samples, different interviewing staffs, the different times
that the measurements were collected, or other artifacts.
3. DI shows positive seam bias effects for all types of characteristics and respondents. Seam
bias is not a problem that affects one class of respondent, or one type of characteristic, and the
improvements observed with the implementation of DI seem similarly general. This is evident in
the consistent, positive results, already noted, across all of the diverse characteristics included in
Part 1. We divided Part 1 into two general subcategories of characteristics – “need-based”
programs to assist the low income population, and other programs and characteristics which apply
to the general population (or which are in fact skewed toward the upper end of the economic
spectrum) – primarily to make it easy to see that there was really no need to do so. The impact of
DI appears to have been the largely the same in both categories. On some dimensions, in fact, it
was nearly identical. For example, the average 2001-to-2004 decline in the percent of changes
-9-
ADVERTISEMENT
0 votes
Related Articles
Related forms
Survey Methodology #2012-02 - New Relationship And Marital Questions: A Reflection Of Changes To The Social And Legal Recognition Of Same-sex Couples In The U.s. - U.s. Census Bureau
Legal
Early Childhood Program Participation, From The National Household Education Surveys Program Of 2012 - U.s. Department Of Education
Legal
Form Hud-9887 And 9887-a - Document Pachagefor Applicant's/tenant's Concent To The Release Of Information - U.s. Department Of Housing And Urban Development
Legal
Related Categories
Parent category: Legal