“THEY DON’T KNOW THAT THEY DON’T KNOW”: REVEALING AND QUANTIFYING THE SOCRATES BIAS
Keywords:
Socrates bias, medical education, self-assessment, questionnaire, biasAbstract
The objective of the present study was to determine whether bias (over- or under-estimation of self-competence) affects pre-training ratings and hence distorts the actual participation effect of experiential workshops.
Assessments were held during “empathy in doctor-patient relationship” elective courses held during winter 2014, spring 2015 and winter 2016 at Ioannina's Medical School, University of Ioannina, Greece.
Twenty-eight women and 19 men aged 21-28 years (mean = 22.8, SD = 1.52), in 4th (n = 18), 5th (n = 19) and 6th (n=10) year of medical studies took part, voluntarily, in the empathy training.
The Jefferson Scale of Physician Empathy was used on a total of 47 medical undergraduates to measure empathic performance both before (B) and a-posteriori-before (P) training. Overestimation of empathic ability was calculated as the difference B-P, and its significance was checked through paired t-test, while effect size (Cohen’s d) was used to reveal any practical importance.
Participants’ mean B score (+SD) was 110.6 (10.5) whereas P was 88.6 (13.8; p(B-P) < 0.001). Assuming total P as the basis (100), total B was 124.8, i.e 24.8% overestimation. A very large effect size was found (d = 1.81) for B-P indicating a highly practical importance. There were no significant differences between the 3 cohorts nor between men &women.
This study revealed the existence of the “didn't know that they didn’t know” (DNKDNK) bias, offered a simple and easy method to measure it, and estimated it to be 24.8%.
References
Bloom, A. (1968). The Republic, Basic Books, New York.
Blanch-Hartigan, D. (2011). Medical Students’ Self-Assessment of Performance: Results from Three Meta-Analyses. Patient Education and Counseling, 84(1), p.3-9.
Cohen, J. (1988). Statistical Power Analysis for the Behavioral Sciences, 2nd ed. Lawrence Earlbaum Associates, Hillsdale NJ.
Davis, A. D., Mazmanian, P. E., Fordis, M., Van Harrison, R., Thorpe, K. E., and Perrier, L. (2006). Accuracy of Physician Self-assessment Compared With Observed Measures of Competence: A Systematic Review. Journal of American Medical Association, 269(9), p. 1094-1102.
Ehrlinger, J., Johnson, K., Banner, M., Dunning, D. and Kruger, J. (2008). Why the unskilled are unaware: Further explorations of (absent) self-insight among the incompetent. Organizational Behavior and Human Decision Processes, 105(1), p.98-121.
Gilovich, T. and Griffin, D. W. (2002). Heuristics And Biases: Then And Now, in Heuristics and Biases: The Psychology of Intuitive Judgment, Gilovich, T., Kahneman, D. (ed), Cambridge University Press. Cambridge, p. 695-698.
Hazelton, M. G., Nettle, D., and Andrews, P. W. (2005). The Evolution Of Cognitive Bias, in: The Handbook of Evolutionary Psychology, Buss, D. M. (ed), John Wiley & Sons Inc. Hodoken NJ, p. 968-987.
Kim, Y. H., Chiu, C. Y., and Bregant, J. (2015). Unskilled and Don’t Want to Be Aware of It: The Effect of Self-Relevance on the Unskilled and Unaware Phenomenon. PLoS ONE, 10(6).
Kiosses, V., Tatsioni, A., Dimoliatis, I., and Hyphantis, T. (2017). “Empathize With Me, Doctor!” Medical Undergraduates Training Project: Development, Application, Six-Month Follow-Up. Journal of Education and Training Studies, 5(7), p.20-27.
Kruger, J., and Dunning, D. (1999).Unskilled and Unaware of It: How Difficulties in Recognizing One's Own Incompetence Lead to Inflated Self-Assessments. Journal of Personality and Social Psychology, 77(6), p.1121-34.
Miller, R.G. (1981). Simultaneous Statistical Inference 2nd Ed. Springer Verlag, New York.
Ouzouni, C., and Nakakis, K. (2012). An Exploratory Study Of Student Nurses’ Empathy. Health Science Journal, 6(3), p.534-552.
Rogers, C. R. (1951). Client-Centered Therapy: Its Current Practice, Implications, And Theory, Houghton Mifflin Co, Boston.
Schierkirka, S., Anders, S., and Raupach, T. (2014). Assessment Of Two Different Types Of Bias Affecting The Results Of Outcome-Based Evaluation In Undergraduate Medical Education. BMC Medical Education, 14, 149.
Additional Files
Published
How to Cite
Issue
Section
License
Copyright (c) 2021 International Education and Research Journal (IERJ)
This work is licensed under a Creative Commons Attribution 4.0 International License.
