
Despite the growing popularity of online opt-in samples in criminology, recent work shows that resultant findings often do not generalize. Not all opt-in samples are alike, however, and matching may improve data quality. Replicating and extending prior work, we compare the generalizability of relational inferences from unmatched and matched opt-in samples. Estimating identical models for four criminal justice outcomes, we compare multivariate regression results from national matched (YouGov) and unmatched (MTurk) opt-in samples to those from the General Social Survey (GSS). YouGov coefficients are almost always in the same direction as GSS coefficients, especially when statistically significant, and are mostly of a similar magnitude; less than 10% of the YouGov and GSS coefficients differ significantly. By contrast, MTurk coefficients are more likely to be in the wrong direction, more likely to be much larger or smaller, and are about three times as likely to differ significantly from GSS coefficients. Matched opt-in samples provide a relatively inexpensive data source for criminal justice researchers, compared to probability samples, and also appear to carry a smaller generalizability penalty than unmatched samples. Our study suggests relational inferences from matched opt-in samples are more likely to generalize than those from unmatched samples.
SocArXiv|Social and Behavioral Sciences|Sociology|Methodology, sample matching, Criminology, Social and Behavioral Sciences, bepress|Social and Behavioral Sciences|Sociology, SocArXiv|Social and Behavioral Sciences|Sociology, web survey, Criminology and Criminal Justice, bepress|Social and Behavioral Sciences|Sociology|Quantitative, Qualitative, Comparative, and Historical Methodologies, bepress|Social and Behavioral Sciences|Sociology|Criminology, public opinion, Amazon Mechanical Turk, bepress|Social and Behavioral Sciences, Legal Studies, SocArXiv|Social and Behavioral Sciences, SocArXiv|Social and Behavioral Sciences|Sociology|Crime, Law, and Deviance
SocArXiv|Social and Behavioral Sciences|Sociology|Methodology, sample matching, Criminology, Social and Behavioral Sciences, bepress|Social and Behavioral Sciences|Sociology, SocArXiv|Social and Behavioral Sciences|Sociology, web survey, Criminology and Criminal Justice, bepress|Social and Behavioral Sciences|Sociology|Quantitative, Qualitative, Comparative, and Historical Methodologies, bepress|Social and Behavioral Sciences|Sociology|Criminology, public opinion, Amazon Mechanical Turk, bepress|Social and Behavioral Sciences, Legal Studies, SocArXiv|Social and Behavioral Sciences, SocArXiv|Social and Behavioral Sciences|Sociology|Crime, Law, and Deviance
| selected citations These citations are derived from selected sources. This is an alternative to the "Influence" indicator, which also reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically). | 88 | |
| popularity This indicator reflects the "current" impact/attention (the "hype") of an article in the research community at large, based on the underlying citation network. | Top 1% | |
| influence This indicator reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically). | Top 10% | |
| impulse This indicator reflects the initial momentum of an article directly after its publication, based on the underlying citation network. | Top 1% |
