Isaac Scientific Publishing

Psychology Research and Applications

Can the Null Hypothesis Be “Proven” by Using Large Numbers of Predictor Variables?

Download PDF (228.1 KB) PP. 35 - 40 Pub. Date: June 28, 2019

DOI: 10.22606/pra.2019.12001

Author(s)

  • Michelle Perez
    School of Family Studies and Human Services, College of Health and Human Sciences, Kansas State University, USA
  • Walter R. Schumm*
    School of Family Studies and Human Services, College of Health and Human Sciences, Kansas State University, USA
  • Abdullah AlRashed
    School of Family Studies and Human Services, College of Health and Human Sciences, Kansas State University, USA
  • Duane W. Crawford
    School of Family Studies and Human Services, College of Health and Human Sciences, Kansas State University, USA

Abstract

Multivariate techniques have become commonplace in the social sciences. Some scholars have attempted to “prove” the null hypothesis by predicting one variable from a host of others. At least three risks or dangers accompany such attempts. First, the addition of multiple predictors may eventually render the apparent significance of any one independent variable to non-significance even if the underlying effect size remains medium to large. Second, the use of multiple predictors implies the potential for hundreds to millions of alternative models that could have been tested along with the models reported in an article. Third, ruling out the direct effects of a variable does not necessarily prove that the variable has no indirect effects on the outcome(s). Using a small data set of survey research in which truck drivers participated as respondents, we show how a theoretically and empirically strong relationship between income satisfaction and job satisfaction could be rendered non-significant (despite a medium or larger effect size) by adding enough other predictor variables, even if those additions did not make much theoretical sense. Thus, it is possible that other misinterpretations of the null hypothesis have occurred in the scientific literature when models with dozens of predictor variables have been used by authors who have had a goal of “proving” the null hypothesis.

Keywords

Truck drivers, income satisfaction, job satisfaction, methodology, indirect effects

References

[1] Aczel, B., Palfi, B., Szollosi, A., Kovacs, M., Szaszi, B.,,,,,,Wagenmakers, E-J. (2018). Quantifying support for the null hypothesis in psychology: an empirical investigation. Advances in Methods and Practices in Psychological Science, 1(3), 357-366.

[2] Allen, D. W., & Price, J. (2015). Same-sex marriage and negative externalities: A critique, replication, correction of Langbein and Yost. Econ Journal Watch, 12, 142-160.

[3] Bakker, M., & Wicherts, J. M. (2011). The (mis)reporting of statistical results in psychology journals. Behavioral Research, 43, 666-678.

[4] Calzo, J. P., Mays, V. M., Bjorkenstam, C., Bjorkenstam, E., Kosidou, K., & Cochran, S. D. (2017). Parental sexual orientation and children’s psychological well-being: 2013-2015 National Health Interview Survey. Child Development, online advance.

[5] Cohen, J. (1988). Statistical power analysis for the behavioral sciences. San Diego, CA: Academic Press.

[6] Cohen, J. (1992). A power primer. Psychological Bulletin, 112, 155-159.

[7] Herek, G. M. (2006). Legal recognition of same-sex relationships in the United States: a social science perspective. American Psychologist, 61, 607-621.

[8] Langbein, L., & Yost, M. A. (2009). Same-sex marriage and negative externalities. Social Science Quarterly, 90, 292-308.

[9] Langbein, L., & Yost, M. A., Jr. (2015). Still no evidence of negative outcomes from same-sex marriage. Econ Journal Watch, 12, 161-163.

[10] Perrone, K. M., Webb, L. K., Wright, S. L., Jackson, Z. V., & Ksiazak, T. M. (2006). Relationship of spirituality to work and family roles and life satisfaction among gifted adults. Journal of Mental Health Counseling, 28, 253-268.

[11] Rosenfeld, M. J. (2010). Nontraditional families and childhood progress through school. Demography, 47, 755- 775.

[12] Schumm, W. R. (1982). Integrating theory, measurement, and data analysis in family studies survey research. Journal of Marriage and the Family, 44, 983-998.

[13] Schumm, W. R. (2010). Statistical requirements for properly investigating a null hypothesis. Psychological Reports, 107, 953-971.

[14] Schumm, W. R. (2018). Same-sex parenting research: a critical assessment. London: Wilberforce Press.

[15] Schumm, W. R., Southerly, W. T., & Figley, C. R. (1980). Stumbling block or stepping stone: path analysis in family studies. Journal of Marriage and the Family, 42, 251-262.