2017 YRBS Mischievous Responders
Principal Investigator(s): View help for Principal Investigator(s) Joseph Cimpian, New York University
Version: View help for Version V1
| Name | File Type | Size | Last Modified |
|---|---|---|---|
|
|
text/x-stata-syntax | 493.6 KB | 10/22/2019 05:17:PM |
|
|
application/pdf | 30.9 KB | 10/22/2019 05:16:PM |
|
|
application/pdf | 128.1 KB | 10/22/2019 05:17:PM |
|
|
text/csv | 11.2 MB | 10/22/2019 05:35:PM |
Project Citation:
Cimpian, Joseph. 2017 YRBS Mischievous Responders. Ann Arbor, MI: Inter-university Consortium for Political and Social Research [distributor], 2019-10-22. https://doi.org/10.3886/E115086V1
Project Description
Summary:
View help for Summary
The data for this project correspond to Cimpian and Timmer's AERA Open paper on mischievous responders in estimates of LGBQ–heterosexual disparities using the 2017 YRBS data. Their abstract is:
Although numerous survey-based studies have found that students who identify as lesbian, gay, bisexual, or questioning (LGBQ) have elevated risk for many negative academic, disciplinary, psychological, and health outcomes, the validity of the types of data on which these results rest have come under increased scrutiny. Over the past several years, a variety of data-validity screening techniques have been used in attempts to scrub datasets of “mischievous responders,” youth who systematically provide extreme and untrue responses to outcome items and who tend to falsely report being LGBQ. We conducted a pre-registered replication of Cimpian et al. (2018) with the 2017 Youth Risk Behavior Survey (YRBS) to (1) estimate new LGBQ–heterosexual disparities on 20 outcomes; (2) test a broader, mechanistic theory relating mischievousness effects with a feature of items (i.e., item response-option extremity); and (3) compare four techniques used to address mischievous responders. Our results are consistent with Cimpian et al.’s findings that potentially mischievous responders inflate LGBQ–heterosexual disparities, do so more among boys than girls, and affect outcomes differentially. For example, we find that removing students suspected of being mischievous responders can cut male LGBQ–heterosexual disparities in half overall, and can completely or mostly eliminate disparities in outcomes including fighting at school, driving drunk, and using cocaine, heroin, and ecstasy. Methodologically, we find that some methods are better than others at addressing the issue of data integrity, with boosted regressions coupled with data removal leading to potentially very large decreases in the estimated of LGBQ–heterosexual disparities, but regression adjustment having almost no effect. While the empirical focus of this paper is on LGBQ youth, the issues discussed are relevant to research on other minority groups and youth generally, and speak to survey development, methodology, and the robustness and transparency of research.
Although numerous survey-based studies have found that students who identify as lesbian, gay, bisexual, or questioning (LGBQ) have elevated risk for many negative academic, disciplinary, psychological, and health outcomes, the validity of the types of data on which these results rest have come under increased scrutiny. Over the past several years, a variety of data-validity screening techniques have been used in attempts to scrub datasets of “mischievous responders,” youth who systematically provide extreme and untrue responses to outcome items and who tend to falsely report being LGBQ. We conducted a pre-registered replication of Cimpian et al. (2018) with the 2017 Youth Risk Behavior Survey (YRBS) to (1) estimate new LGBQ–heterosexual disparities on 20 outcomes; (2) test a broader, mechanistic theory relating mischievousness effects with a feature of items (i.e., item response-option extremity); and (3) compare four techniques used to address mischievous responders. Our results are consistent with Cimpian et al.’s findings that potentially mischievous responders inflate LGBQ–heterosexual disparities, do so more among boys than girls, and affect outcomes differentially. For example, we find that removing students suspected of being mischievous responders can cut male LGBQ–heterosexual disparities in half overall, and can completely or mostly eliminate disparities in outcomes including fighting at school, driving drunk, and using cocaine, heroin, and ecstasy. Methodologically, we find that some methods are better than others at addressing the issue of data integrity, with boosted regressions coupled with data removal leading to potentially very large decreases in the estimated of LGBQ–heterosexual disparities, but regression adjustment having almost no effect. While the empirical focus of this paper is on LGBQ youth, the issues discussed are relevant to research on other minority groups and youth generally, and speak to survey development, methodology, and the robustness and transparency of research.
Funding Sources:
View help for Funding Sources
United States Department of Education. Institute of Education Sciences (R305B170009)
Related Publications
Published Versions
Report a Problem
Found a serious problem with the data, such as disclosure risk or copyrighted content? Let us know.
This material is distributed exactly as it arrived from the data depositor. ICPSR has not checked or processed this material. Users should consult the investigator(s) if further information is desired.