New Polystochastic Statistical Inference in Social Sciences - Defining new Rules and Thresholds

Sinisa Opic

doi:10.18690/rei.4907

Sinisa Opic University of Zagreb

Keywords: Bayesian, effect size, NHST, p-value, polystochastic, social science, statistical inference.

Abstract

The Null Hypothesis Significance Testing (NHST) framework has sparked considerable debate within the scientific community, leading to numerous studies advocating for a re-evaluation of the current system. New polystochastic statistical inference defines methods of statistical inference that integrate rules and thresholds for both rejecting the null hypothesis and confirming the alternative hypothesis. This approach unifies the control of respondents' influence on statistical significance and introduces criteria such as effect size and Bayesian inference for confirming the alternative hypothesis. Unlike NHST, polystochastic statistical inference controls Type I error (p-value) and aims to optimize the confirmation of evidence without increasing the risk of Type II errors.

Downloads

References

Ali, S., Waheed, M., Shah, I., and Raza, S. M. M. (2023). Bayesian sample size determination for coefficient of variation of normal distribution. Journal of Applied Statistics, 51(7), 1271–1286. https://doi.o–rg/10.1080/02664763.2023.2197571

Andraszewicz, S., Scheibehenne, B., Rieskamp, J., Grasman, R., Verhagen, J., and Wagenmakers, E. J. (2015). An Introduction to Bayesian Hypothesis Testing for Management Research. Journal of Management, 41(2), 521-54. https://doi.org/10.1177/0149206314560412

Assaf, G. A., and Tsionas, M. (2018). Bayes factors vs. P-values. Tourism Management, 67,17-31.

Balow, C. (2017). The “effect size” in educational research: What is it and how to use it? [sic] Illuminate Education. Retrieved from

www.illuminateed.com/blog/2017/06/effect-size-educational-research-use/ on July 14, 2019.

Bartlett, M. S. (1957). A comment on D. V. Lindley’s statistical paradox. Biometrika, 44, 533–553.

Benjamin, D. J., Berger, J. O., Johannesson, M. et al. (2017). Redefine statistical significance. Nature Human Behaviour, 2(1), 6-10.doi: 10.1038/s41562-017-0189-z.

Bollen, K. A., Harden, J. J., Ray, S., and Zavisca, J. (2014). BIC and Alternative Bayesian Information Criteria in the Selection of Structural Equation Models. Structural Equation Modeling: A Multidisciplinary Journal, 21,1–19.

Cohen, J. (1969). Statistical Power Analysis for the Behavioral Sciences (1st ed.). New York, NY: Academic Press.

Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Erlbaum.

Cohen, J. (1992). Power Primer. Psychological Bulletin [PsycARTICLES]; 112, 1; PsycARTICLES, pp. 155-159.

De Ruiter, J. (2019). Redefine or justify? Comments on the alpha debate. Psychonomic Bulletin & Review, 26, 430–433, https://doi.org/10.3758/s13423-018-1523-9.

Fisher, R. A. (1925). Statistical Methods for Research Workers. Edinburgh, UK: Oliver and Boyd.

Goodman, S. (2005). Introduction to Bayesian methods I: Measuring the strength of evidence. Clinical Trials, 2 (4), 282-290.

Goodman, S. (2008). A Dirty Dozen: Twelve P-Value Misconceptions. Seminars in Hematology, 45 (3), 135-140.

Hoijtink, H., van Kooten, P., and Hulsker, K. (2016). Why Bayesian psychologists should change the way they use the Bayes Factor. Multivariate Behavioral Research, 51, 2-10. doi: 10.1080/0027317–1.2014.969364

Holmberg, A. (2024). Toward a Better Understanding of Statistical Significance and p Values in Nursing. Nursing Forum, Article ID 7263781, 1-11 pageshttps://doi.org/10.1155/2024/7263781

Huisman, L. (2023). Are P‑values and Bayes factors valid measures of evidential strength? Psychonomic Bulletin & Review, 30, 932–941. https://doi.org/10.3758/s13423-022-02205-x

Jarosz, A., and Wiley, J. (2014). What are the odds? A practical guide to computing and reporting Bayes factors. Journal of Problem Solving, 7, 2-9. doi: 10.7771/1932-6246.1167

Jeffreys H. (1961). Theory of Probability (3rd ed.). New York: Oxford University Press.

Lakens, D., Adolfi, F., Albers, C., Anvari, F., Apps, M., Argamon, S., ... Bradford, D. (2018). Justify your alpha. Nature Human Behavior, 2, 168-171.

Lavine, M., and Schervish, M. J. (1999). Bayes factors: What they are and what they are not. The American Statistician, 53 (2), 119-122.

Lindley, D. V. (1957). A statistical paradox. Biometrika, 44, 187–192.

McGrath, R. E., and Meyer, G. J. (2006). When effect sizes disagree: the case of r and d. Psychological Methods, 11(4), 386-401. doi: 10.1037/1082-989X.11.4.386.

McShane, B. B., Gal, D., Gelman, A., Robert, C., and Tackett, J. L. (2019). Abandon Statistical Significance. The American Statistician, 73 (1), 235-245.

Morey, R. D., Romeijn, J.W., and Rouder, J. N. (2016). The philosophy of Bayes factors and the quantification of statistical evidence. Journal of Mathematical Psychology, 72, 6–18.

Opić, S, and Rijavec, M. (2022). Misconceptions of the p-value - let us use new approaches and procedures // 2. Međunarodna znanstvena i umjetnička konferencija Suvremene teme u odgoju i obrazovanju - STOO 2 In memoriam Prof. Emer. Dr sc. Milan Matijević u suradnji sa Zavodom za znanstvenoistraživački i umjetnički rad Hrvatske akademije znanosti i umjetnost / D. Velički and M. Dumančić, (eds.). Zagreb: Sveučilište u Zagrebu Učiteljski fakultet, 2022. pp. 1-21.

Pearson, K. X. (1900). On the Criterion that a Given System of Deviations from the Probable in the Case of a Correlated System of Variables Is Such That It Can Be Reasonably Supposed to Have Arisen from Random Sampling. London, Edinburgh and Dublin Philosophical Magazine and Journal of Science, 50 (302), 157–175, https://doi.org/10.1080/14786440009463897

Perezgonzalez, J. D. (2015). Fisher, Neyman-Pearson or NHST? A Tutorial for Teaching Data Testing, Frontiers in Psychology 6. https://doi.org/10.3389/fpsyg.2015.00223-

Rovetta, A. (2024). Abandon Statistical Significance: A Gentle Introduction to S-values and S-intervals.

https://doi.org/10.31219/osf.io/ywhu9 (https://osf.io/preprints/osf/ywhu9)

Ruscio, J. (2008). A Probability-Based Measure of Effect Size: Robustness to Base Rates and Other Factors. Psychological Methods, 13(1), 19 –30.

Sullivan, G. M., and Feinn, R. (2012). Using effect size—Or why the p-value is not enough. Journal of Graduate Medical Education, 4(3), 279–282. https://doi.org/10.4300/JGME-D-12-00156.1

Stern, H. S. (2016). A test by any other name: P-values, Bayes Factors, and statistical inference. Multivariate Behaviour Research, 51(1), 23-39. doi:10.1080/00273171.2015.1099032

Vacha-Haase, T., and Thomson, B. (2004). How to Estimate and Interpret Various Effect Sizes. Journal of Counseling Psychology, 51(4), 473–481. https://doi.org/10.1037/0022-0167.51.4.473

Wasserstein, R. L., and Lazar, N. A. (2016). ASA Statement on Statistical Signiﬁcance and P-Values - Context, Process, and Purpose. The American Statistician, 70 (2), 129-133.

Yuan, Y., Johnson, V. E. (2008). Bayesian Hypothesis Tests using Nonparametric Statistics. Statistica Sinica, 18 (3), 1185-1200.