“THEY DO NOT KNOW THAT THEY DON’T KNOW”: REVEALING AND QUANTIFYING THE SOCRATES BIAS

Vasileios Kiosses, Claire de Burbure, Ioannis D K Dimoliatis

Abstract


The objective of the present study was to determine whether bias (over- or under-estimation of self-competence) affects pre-training ratings and hence distorts the actual participation effect of experiential workshops.

Assessments were held during “empathy in doctor-patient relationship” elective coursesheld during winter 2014, spring 2015 and winter 2016 at Ioannina's Medical School, University of Ioannina, Greece.

Twenty-eightwomen and 19men aged 21-28 years (mean=22.8, SD =1.52),in 4th (n=18), 5th (n=19) and 6th (n=10) year of medical studies took part, voluntarily, in the empathy training.

The Jefferson Scale of Physician Empathy was used on a total of 47 medical undergraduates to measure empathic performance both before (B) and a-posteriori-before (P) training. Overestimation of empathic ability was calculated as the difference B-P, and its significance was checked through paired t-test, while effect size (Cohen’s d) was used to reveal any practical importance.

Participants’ mean B score (+SD) was 110.6 (10.5) whereasPwas 88.6 (13.8; p(B-P)<0.001).Assuming total P as the basis (100), total B was 124.8, i.e 24.8% overestimation. A very large effect size was found (d=1.81) for B-P indicating a highly practical importance.There were no significant differences between the 3 cohorts nor between men &women.

This study revealed the existence of the “do not know that they do not know” bias, offered a simple and easy method to measure it, and estimated it to be 24.8%.


Keywords


The Socrates bias, medical education, self-assessment, questionnaire, bias.

Full Text:

PDF

References


Bloom, A. (1968). The Republic,Basic Books, New York.

Blanch-Hartigan, D. (2011). Medical Students’ Self-Assessment of Performance: Results from Three Meta-Analyses. Patient Education and Counseling, 84(1), p.3-9.

Cohen, J. (1988). Statistical Power Analysis for the Behavioral Sciences, 2nd ed. Lawrence Earlbaum Associates, Hillsdale NJ.

Davis, A. D., Mazmanian, P. E., Fordis, M., Van Harrison, R., Thorpe, K. E., and Perrier, L. (2006). Accuracy of Physician Self-assessment Compared With Observed Measures of Competence: A Systematic Review. Journal of American Medical Association, 269(9), p. 1094-1102.

Ehrlinger, J., Johnson, K., Banner, M., Dunning, D. and Kruger, J. (2008). Why the unskilled are unaware: Further explorations of (absent) self-insight among the incompetent. Organizational Behavior and Human Decision Processes, 105(1), p.98-121.

Gilovich, T. and Griffin, D. W. (2002). Heuristics And Biases: Then And Now, in Heuristics and Biases: The Psychology of Intuitive Judgment, Gilovich, T., Kahneman, D. (ed), Cambridge University Press. Cambridge, p. 695-698.

Hazelton, M. G., Nettle, D., and Andrews, P. W. (2005).The Evolution Of Cognitive Bias,in: The Handbook of Evolutionary Psychology, Buss, D. M. (ed), John Wiley & Sons Inc. Hodoken NJ, p. 968-987.

Kim, Y. H., Chiu, C. Y., and Bregant, J. (2015). Unskilled and Don’t Want to Be Aware of It: The Effect of Self-Relevance on the Unskilled and Unaware Phenomenon. PLoS ONE, 10(6).

Kiosses, V., Tatsioni, A., Dimoliatis, I., and Hyphantis, T. (2017). “Empathize With Me, Doctor!” Medical Undergraduates Training Project: Development, Application, Six-Month Follow-Up. Journal of Education andTraining Studies, 5(7), p.20-27.

Kruger, J., and Dunning, D. (1999).Unskilled and Unaware of It: How Difficulties in Recognizing One's Own Incompetence Lead to Inflated Self-Assessments. Journal of Personality and Social Psychology, 77(6), p.1121-34.

Miller, R.G. (1981). Simultaneous Statistical Inference 2nd Ed. Springer Verlag, New York.

Ouzouni, C., and Nakakis, K. (2012). An Exploratory Study Of Student Nurses’ Empathy. Health Science Journal, 6(3), p.534-552.

Rogers, C. R. (1951). Client-Centered Therapy: Its Current Practice, Implications, And Theory, Houghton Mifflin Co, Boston.

Schierkirka, S., Anders, S., and Raupach, T. (2014). Assessment Of Two Different Types Of Bias Affecting The Results Of Outcome-Based Evaluation In Undergraduate Medical Education. BMC Medical Education,14, 149.


Refbacks

  • There are currently no refbacks.

Comments on this article

View all comments




Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 International License.

Copyright © 2020 INTERNATIONAL EDUCATION AND RESEARCH JOURNAL