Article Information

Bindu Menon¹, Coral D. Matus^*,1,2, Jeremy J. Laukka^{*, 1, 3}

¹Department of Medical Education, University of Toledo College of Medicine and Life Sciences, Toledo, OH

²Department of Family Medicine, University of Toledo College of Medicine and Life Sciences, Toledo, OH

³Department of Neurology, University of Toledo College of Medicine and Life Sciences, Toledo, OH

*Corresponding author: Jeremy J. Laukka, Senior Associate Dean for Undergraduate Medical Education, Interim Associate Dean for Foundational Sciences, Associate Professor, Department of Medical Education, 318 Mulford, Library, Mail Stop 1050, 3000 Arlington Ave., Toledo, Ohio, USA.

Coral D. Matus, Associate Dean for Clinical Curriculum, Associate Professor, Department of Medical Education, 330 Mulford Library, Mail Stop 1050, 3000 Arlington Ave., Toledo, Ohio, USA.

Received: 31 October 2023; Accepted: 07 November 2023; Published: 05 December 2023

Citation: Bindu Menon, Coral D. Matus, Jeremy J. Laukka. Two Novel Approaches for the Implementation and Assessment of Self-Directed Learning in the Pre-Clinical Medical School Curriculum. Fortune Journal of Health Sciences. 6 (2023): 473-482.

Abstract

Keywords

Undergraduate Medical Education (UME); Self Directed Learning; Lifelong Learning; Master Adaptive Learner

Article Details

1. Introduction

Spiro and colleagues introduced a Cognitive Flexibility Theory that articulates how the human mind can acquire, manage, and restructure our existing knowledge based on new information learned [1]. Their seminal work inspired a deeper process of learning that highlighted the importance of being adaptive, flexible, and self-reflective in thinking during learning activities to develop greater efficacy in higher-order thinking and problem-solving. A fundamental tenet of advancing knowledge acquisition is self-directed learning (SDL) which has been shown to refine and develop presentation, communication, collaborative learning, and information-handling skills [2, 3]. Medical educators acknowledge the importance of medical students, as future physicians, acquiring the skills necessary for identifying learning needs and developing lifelong learning habits due to the constantly changing landscape of knowledge and practice of medicine. The Liaison Committee on Medical Education for Accreditation (LCME) requires medical schools to provide opportunities for students to participate in SDL activities as described in Standard 6.3 (The faculty of a medical school ensures that the medical curriculum includes self-directed learning experiences and time for independent study to allow medical students to develop the skills of lifelong learning). Skills that embody a lifelong learner include the ability to recognize one’s knowledge gaps, to know where and how to find credible sources with relevant information, and then synthesize this information and apply it to one’s clinical practice (http://lcme.org/publications). There are similar requirements from the Accreditation Council for Graduate Medical Education (ACGME) that residents should have the ability to identify their knowledge deficiencies, and set learning goals and be able to locate, appraise, and assimilate evidence from scientific studies [4].

However, recent studies show that despite the proven advantages and the concurrent emphasis on SDL in the undergraduate and graduate medical curriculum, institutions are found to face several challenges in implementation [5, 6]. Studies show that these challenges could either be of the method/methods applied for SDL or based on the readiness scale, taking the perception of students, teachers' viewpoints, and the challenges faced by self-regulated learning studies [5]. We have designed our models based on the SDL in Medical Education, (SDL-ME) construct which builds on the foundations of self-directed and self-regulated learning theories and is specifically contextualized in the context of medical education with the purpose of health professional identity formation [7]. SDL, as defined by LCME, involves ‘ self-assessment of learning needs; the independent identification, analysis, and synthesis of relevant information; and the appraisal of the credibility of information sources’ is a necessary part of a medical education program (www.lcme.org/publications/2015-16-functions-and-structure-with-appendix.pdf). We have tried to incorporate all these aspects into both our models. In addition, we are assessing the success of the process by measuring student satisfaction through a survey. Survey items measured students' approach towards SDL, thoughts on the role of SDL in their learning process, and factors influencing their engagement and satisfaction. We also evaluated the role of SDL in enhancing their content mastery by analyzing the internal assessments. For this purpose, we used a novel method of “cognitive diagnostic assessment” namely the DINA method which has been used and tested by us previously [7]. This study is the first of its kind to use this multidimensional, implementation and assessment, approach.

2. Methods

2.1 Curriculum

The MD program at University of Toledo College of Medicine begins with the 18-month systems-based foundational sciences curriculum (curriculum scheme is presented in supplementary figure 1). Across the continuum of the preclinical phase, 4 courses are designated as curricular “threads.” Each thread constitutes a summation of integrated systems (i.e. Thread 3 includes Building Foundations, Cardio-Renal, and Cardio-Pulmonary). Both the SDLs were introduced in the academic year (AY) 2021-2022 in thread 3 and again repeated the following year with small modifications. The survey was done in the second year (AY 2022-2023).

Figure 1: The rubric for evaluation of SDL model 1. The faculty evaluated the SDL reports submitted by the students based on this rubric. A group of faculty, who were content experts, took charge of grading the reports and started with a clear consensus to avoid variations in the grading pattern and narrative. The students (n=175) were divided into separate groups so that each faculty had about 15-16 students to grade.

2.2 Research design

The target population was the second-year medical students at our institution. The SDLs were “required” sessions in their curriculum. Hence, all the (n=175) students participated in the SDL process. The survey was distributed to the entire population and 90% of the eligible students participated in it. The content mastery in “Covid-19” was measured for the entire cohort after obtaining IRB approval. The rubrics for assessment were developed by the faculty.

2.3 SDL Model 1:

The first SDL model was implemented in the Cardiovascular-Renal System in the second year of medical school. In the Hematology Oncology system of the first year, students learned about anticoagulants and the coagulation cascade. The goal of SDL in this context was to reactivate and expand previous knowledge about anticoagulants through their application to the cardiovascular system. Through the SDL process, students self-assess their own learning needs and identify deficits in knowledge and understanding. For this purpose, the faculty assigned a detailed clinical vignette based on the topic (anticoagulants) to the students through our learning management system (LMS), Blackboard. Each student was asked to identify a specific learning objective based on the case, identify appropriate resources to answer their clinical questions and submit their findings on Blackboard. The faculty also provided modules and suggested other sources of information including a sample worksheet (Supplementary Figure 2) to help the students. A group of faculty experts analyzed and scored the reports based on a well-defined rubric (Figure 1). They also provided detailed narrative feedback to each student with further specific suggestions for reflection, and revision, if needed. In addition to this, each student was required to provide feedback on at least 5 of their peer’s assignments on Blackboard. The completion of this was also part of their SDL grade which in turn comprised 10% of their total system grade. The concept of SDL was introduced on the first day of the system and the students were given approximately 4 weeks to complete the assignment. The rubric and the assessment criteria were provided to the students on the learning management software.

2.4 SDL Model 2:

This model was introduced in the Cardio-pulmonary system. The intent of this SDL was to introduce the concept of “how to read medical literature and use it appropriately in clinical practice.” We chose to explore the topic of COVID-19 as it demonstrates well the importance of physicians as master adaptive/ self-directed learners during times of uncertainty within healthcare. The faculty designed a series of questions/learning objectives based on the chosen topic (COVID-19; questions shown in Table 1). The students were divided into small groups of 5-6 students, and each group was asked to select one question to investigate. Groups were given specific instructions such as how to find a credible original research paper that addressed their clinical question.

Table 1: The topics which were chosen for the COVID-19 SDL. The students (n=175) were divided into 18 separate groups for their TBL sessions from day 1 of their medical school. Each TBL group, which had 5-6 students, were asked to select a topic from this given list to prepare for their SDL experience.

They were redirected to resources from the MD program research curriculum to help them identify an appropriate and credible research article. The group members then presented their findings to the entire class. The faculty graded the presentations based on a well-defined rubric (Figure 2) and provided detailed feedback with helpful suggestions for reflection and revision. In the second year of its iteration, the student presentations were changed to the Pecha Kucha forma to consolidate the time required for presentations and make it more efficient. The concept of SDL was introduced in the first day of the system and the students were given approximately 3 weeks to complete the assignment and prepare the presentation. The rubric and the assessment criteria were provided to the students on the learning management software.

Figure 2: The rubric for evaluation of SDL model 2. The faculty, who were content experts, evaluated the SDL presentations by the students based on this rubric. The students (n=175) were divided into separate groups (4-5 students) and each presentation involved participation of all the members of the team.

2.5 Survey

A descriptive survey was used to understand student discernment about SDL. The questions focused on whether one or both approaches attained the goals and helped the students in fulfilling their learning needs as well as their development of lifelong learning skills. The survey, conducted anonymously, was administered via a Qualtrics survey link sent to the entire second-year MD class (175 students). Responses were collected over a period of one month. The survey instrument (Supplementary Figure 3) had five closed-ended questions and one open-ended question for each model. The quantitative data was analyzed using Qualtrics and Microsoft Excel. The qualitative responses were coded manually. The common terms were grouped and ranked from most to least cited. We employed a frequency threshold of 20% for identifying themes in the qualitative responses.

2.6 DINA model analysis:

A cognitive diagnostic assessment method was selected to assess skill mastery comparatively [8]. For this study, we analyzed content mastery in the major content area (COVID-19) that came under the SDL curriculum, by using the deterministic input, noisy “and” gate (DINA) model [9-11]. The DINA model is a type of cognitive diagnostic model that predicts the probability of mastery of latent variables such as skills or attributes. Any instances of a student answering a question without mastery of all required skills were considered to be correct by chance (i.e., guessing). To identify which skills were required for each question, a Q-matrix had to first be developed. A Q-matrix [[12] is a confirmatory matrix that identifies the skills required to answer each item in an assessment in a binary format, where “1” indicates the requirement of skill to answer the item, and “0” indicates that skill is not required by an item. We developed an Q-matrix, where j different skills or attributes were required to correctly answer a question from the internal assessments. For example, to correctly answer the question is on the given topic “Covid-19,” we determined that students needed to possess knowledge or “skill” in 2 content areas: j₁, respiratory system; j₂, infectious diseases; and j₃, COVID-19 infection. Similarly, each question had 3-5 skills matched to them. This method has been described in detail in our previous publication [8].

3. Results

The details of the survey results are presented in Figures 3 and 4. There was a solid engagement in the survey with a robust 90% participation by the entire cohort who was part of the study. We have only included the strongly agree/agree and strongly disagree/disagree percentages in the graphs. The neutral category responses are not represented in the figures to keep it simple. Figure 3 shows the results for SDL model 1. We found that 66% percent of respondents agreed/or strongly agreed that the SDL model “anticoagulants” helped them to identify critical knowledge gaps and that it helped their development as lifelong learners. 69% of the respondents found that the SDL experience helped them to reinforce their learning of the specific content matter. Interestingly, almost half of the students found the experience to help identify their weaknesses and help to improve their SDL style.

Figure 3: The results of the Qualtrics survey for anticoagulants SDL. The survey was distributed to the entire student body (second year medical students) at University of Toledo College of Medicine with a response rate of 90% (n=158/175). Some stems have been condensed for presentation here. The full survey is provided in supplementary Figure 3.

The survey results for COVID-19 SDL, depicted in Figure 4, show that this experience had comparatively less student satisfaction in all elements than the first model. Despite that, 61 percent of respondents agreed/strongly agreed that this SDL model helped their development as lifelong learners and was helpful to reinforce their learning of the specific content matter. Furthermore, 59% of the students who took the survey found that it helped them to identify critical knowledge gaps while 46% believed that it helped their development as lifelong learners. The qualitative part of the survey showed that students found many aspects of both exercises extremely useful. For example, there was a comment on how the most valuable part of anticoagulant SDL was “reading over how other students approached the project and seeing the conflicting evidence and opinions.” Another comment read “Covid-19 SDL showed how important it is to have good research skills as a physician so you can correctly interpret new evidence”. There were comments on the peer feedback component such as “I had to learn about the medications well enough to ask a meaningful question and respond to other student's posts.” Some of the challenges described by the students included difficulty in “identifying and classifying sources according to the levels of evidence criteria.” A representation of the major comments are summarized in Table 2.

Table 2: The results of the qualitative segment of the Qualtrics survey for anticoagulants (a) and Covid-19 (b) SDL. Selected comments are presented without any changes except for formatting.

Figure 4: The results of the Qualtrics survey for Covid-19 SDL. The survey was distributed to the entire student body (second-year medical students) at the University of Toledo College of Medicine with a response rate of 90% (n=158/175). Some stems have been condensed for presentation here. The full survey is provided in supplementary figure 3.

DINA model analysis was conducted [8] for the content “COVID-19” in the internal assessment done at the end of the system for both academic years 2021-2022 and 2022-2023. We found that 91% of the students from both years attained mastery of this content. The average grade of the students in the respective end-of-the-system internal assessments were 78.49±8.57% (AY 2022-2023) and 86±6.3% (2021-2022).

4. Discussion

Principles of “cognitive flexibility theory of knowledge acquisition,” show that learners are better able to acquire and retain knowledge if they develop their own representation of information. Self-directed learning (SDL) is an "approach where learners are motivated to assume personal responsibility and collaborative control of the cognitive and contextual processes in constructing and confirming meaningful and worthwhile learning outcomes". It is important to note that SDL is different from self-regulated learning (SRL) [13]. While SRL involves “The process of a learner controlling their cognition, motivation, emotion, and behavior to achieve specific learning tasks or goals in a given context” [13], SDL is considered “a general approach to learning in which the learner takes responsibility for self-identifying gaps in knowledge and skills and then applies SRL strategies to remedy those gaps.” In SDL, learning is linked to assessment of learning, because learners must monitor their progress and obtain external feedback. This helps them make necessary adjustments, which allows them to progress to higher levels of learning and performance. SDL has been shown to help students improve their presentation skills, communication skills, collaboration, learning skills, and information handling skills [2, 3]. However, recent studies show that despite the proven advantages and the concurrent emphasis on SDL in the undergraduate and graduate medical curriculum, institutions are found to face several challenges in implementation [5, 6]. Faculty guidance and lack of organizational skills were some of the shared challenges faced by institutions [5, 6].

Here, we present two different methods of SDL, both of which address these challenges and meet the requirements defined by the Liaison Committee on Medical Education (LCME). They both focus on two distinct aspects of student learning. The first is focused on encouraging the students to identify learning needs and find resources to contribute to their learning while the second addresses a major challenge faced by future physicians, i.e., how to effectively use scientific literature to reinforce medical knowledge acquisition. Both models are designed to help students develop lifelong learning skills. The survey results showed that the majority of the students are satisfied with the process and that we have been able to achieve the desired consequence. Furthermore, our data show that the majority of students successfully attained content mastery in the topic related to the second SDL. Ricotta et al, in a study conducted by faculty teams from eight major US medical schools in partnership with the Association of American Medical Colleges (AAMC) and published in 2021, emphasize that SDLs introduced in medical curricula should capture the unique environment of medical training and clinical education [7]. Their imperative that ‘self-directed learning requires placing the construct in the context of patient care and has an obligation to society at large’ was instrumental in our conception of both models. It comes from the understanding that learning is a lifelong process for physicians and our learners have to be cognizant of some of the major healthcare challenges and be prepared to adapt their learning approach accordingly. For example, COVID-19 SDL aimed to prepare our students with the best approach to practice evidence-based medicine in unprecedented situations such as those witnessed with the arrival of the pandemic in 2020.

We have based both our SDL models on the SDL in Medical Education, (SDL-ME) framework, which is a construct of learning and pedagogy specific to medical students and physicians in training as described by Ricotta et. al. in their publication [7]. In addition to the most popular elements in SDL, our “anticoagulants” model also focused on the fourth aspect of self-regulated learning described in the SDL-ME framework, viz. applying new knowledge and skills to real-world situations [7]. We had a clinical scenario as the basis of SDL, which required students to apply their knowledge and bridge the gap while keeping in mind the various important aspects of clinical decision-making. SDL-ME emphasizes a “safe, supportive, and rigorous learning environment within the context of the professional obligation and responsibilities of medicine” and we strongly believe that our model captured this essence.

Motivation is a major draw of the SDL-ME model and is described to involve both internal (for example: intrinsically wanting to improve patient care) and external drivers (for example: driven by evaluations or assessments). While the grade component of both of our SDL models specifically targeted the external driver, a faculty grading and feedback based on a rubric that emphasized the quality of the assignment, as well as the peer feedback component attempted to capture the internal driver in the case of the first model (anticoagulants). SDL-ME framework necessitates that the feedback component should be not only meaningful, and timely, but also centered on the process of learning (effort). In this model, each student is expected to think like a physician while assessing the patient described in the clinical case. Therefore, the clinical faculty who evaluated their assignment and provided feedback addresses adaptive expertise, which is the ability to reason from foundational principles to create solutions to problems not previously encountered. By including peer feedback as part of their assignment and grade in this model, we ensured that the students, while giving feedback to their peers on their strengths and weaknesses, have an opportunity to encounter various other approaches to learning, thereby giving themselves an added advantage and opportunity to re-evaluate and adapt their own methods. While our SDL model “anticoagulants” focused more on fostering professional identity formation, teamwork was the major facet of the “COVID-19” model. Teamwork was an added important advantage of the COVID-19 SDL, which created opportunities to interact with peers and teachers and this has shown to be a major advantage in prior studies [2, 14-17]. In the medical curriculum, learning has to be enjoyable and motivating to help better retention and improve professional acumen. The teamwork and presentation in the classroom along with your teammates helped to create these qualities [3, 16, 17]. SDL also helped students improve their time management [3, 5, 18]. Thus, SDL will have several advantages if applied with the appropriate methodology.

5. Strengths and Limitations

By evaluating content mastery using DINA model, we found that students benefited from expanding their content mastery through the SDL exercises. The content mastery in Covid-19 related content was very high compared to the class averages in internal assessments. Survey responses show that out of the 90% of students who responded to the survey, the majority (> 60%) agreed that the goals of implementation were successful. However, there are additional constructive comments in the narrative section that can be useful in making necessary changes for the upcoming iterations (see sections marked* in Table 2). An example is taking into consideration comments that some of the students did not find the assignment challenging enough, and therefore we may consider carefully balancing the process so that all the students benefit from the exercise. Another limitation was that the authors were not able to assess the topic of “anticoagulants” using DINA model analysis. We intend to rectify this in the upcoming iteration.

6. Conclusion and Next Steps

Our ultimate goal is to introduce the concept of Master Adaptive Learners (MAL) to our medical students from the very beginning of their education in the medical school. We believe the principles of SDL will allow students to take ownership of their learning and support their professional growth. This, in turn, will help them develop the skills necessary for a successful career in the field of medicine. Some of the proposed future plans include taking the ideas presented in this study and expanding them into other organ systems. It is our goal to develop rubrics that course directors can easily incorporate into organ systems and establish an operational system to help faculty construct clinical vignettes using the PICO (Patient/Population, Intervention, Comparison, Outcomes) model [19].

List of abbreviations

Self Directed Learning (SDL); Liaison Committee on Medical Education for Accreditation (LCME); Accreditation Council for Graduate Medical Education (ACGME); Learning Management System (LMS); Deterministic Input, Noisy “and” Gate (DINA); Academic Year (AY); Self-Regulated Learning (SRL); SDL in Medical Education, (SDL-ME); Master Adaptive Learners (MAL); Patient/Population, Intervention, Comparison, Outcomes (PICO).

Ethics approval and concent to participate

The authors declare that all methods were conducted in accordance with relevant guidelines and regulations. All experimental protocols were approved by The University of Toledo College of Medicine and Life Sciences “Institutional Review Board (IRB)” which has determined that this study meets the “exempt” category. Informed consent was obtained from all subjects as per IRB guidelines.

Consent for Publication

Not applicable

Availability of data and Materials

The datasets generated and analyzed are available from the corresponding author on reasonable request.

Competing Interests

The authors report no declarations of interest.

Acknowledgement

We thank all patients and healthy subjects who participated in this study and all the individuals who helped us to complete the study.

Funding:

None

Authors' Contributions

BM supervised the SDL implementation and was responsible for data analysis and manuscript writing. CDM conceived the study and contributed to manuscript writing. JJL was responsible for the overall supervision of the study, manuscript writing and critical revision.

Acknowledgments

The authors wish to thank Dr. Danae Hamouda, (Associate Professor, Assoc Chief of Clinical Hematology/Oncology, & Program Director Hematology/Oncology Fellowship, University of Toledo College of Medicine and Life Sciences) and Dr. James Willey (Professor, Director, Cancer Research Center, & George Isaac Endowed Chair in Cancer Research, Department of Medicine, University of Toledo College of Medicine and Life Sciences) for their assistance in implementing SDL 1 and 2 respectively.

References

1. Spiro RJ, Coulson RL, Feltovich PJ, Anderson D: Cognitive flexibility theory: Advanced knowledge acquisition in ill-structured domains. Hillsdale, NJ: Erlbaum (1988).
2. Abraham RR, Torke S, Gonsalves J, Narayanan SN, Kamath MG, Prakash J, et al. Modified directed self-learning sessions in physiology with prereading assignments and Pecha Kucha talks: perceptions of students. Adv Physiol Educ 42 (2018): 26-31.
3. Premkumar K, Vinod E, Sathishkumar S, Pulimood AB, Umaefulam V, Prasanna Samuel P, et al. Self-directed learning readiness of Indian medical students: a mixed method study. BMC Med Educ 18 (2018): 134.
4. Murad MH, Coto-Yglesias F, Varkey P, Prokop LJ, Murad AL: The effectiveness of self-directed learning in health professions education: a systematic review. Med Educ 44 (2010): 1057-1068.
5. Buch A, Rathod H, Naik M. Scope and Challenges of Self-Directed Learning in Undergraduate Medical Education: A Systematic Review. J Med Edu 20 (2021).
6. Jeong D, Presseau J, ElChamaa R, Naumann DN, Mascaro C, Luconi F, et al. Barriers and Facilitators to Self-Directed Learning in Continuing Professional Development for Physicians in Canada: A Scoping Review. Acad Med 93 (2018): 1245-1254.
7. Ricotta DN, Richards JB, Atkins KM, Hayes MM, McOwen K, Soffler MI, et al. Self-Directed Learning in Medical Education: Training for a Lifetime of Discovery. Teach Learn Med 34 (2022): 530-540.
8. Matus AR, Matus LN, Hiltz A, Chen T, Kaur B, Brewster P, et al. Development of an assessment technique for basic science retention using the NBME subject exam data. BMC Med Educ 22 (2022): 771.
9. Bangeranye C, Lim YS. How to Use Cognitively Diagnostic Assessments of Student Performance as a Method for Monitoring and Managing the Instructional Quality in Undergraduate Medical Education. Acad Med 95 (2020): 145-150.
10. de la Torre J. The Generalized DINA Model Framework. Psychometrika 76 (2011): 179-199.
11. Junker BW, Sijtsma K. Cognitive assessment models with few assumptions, and connections with nonparametric item response theory. Applied Psychological Measurement 25 (2001): 258-272.
12. Tatsuoka KK. Rule Space - an Approach for Dealing with Misconceptions Based on Item Response Theory. J Educ Meas 20 (1983): 345-354.
13. Papanagnou D, Corliss S, Richards JJ, Artino AR, Schwartzstein R. Progression of Self-Directed Learning in Health Professions Education: Clarifying Terms and Processes. Acad Med (2023).
14. Peine A, Kabino K, Spreckelsen C: Self-directed learning can outperform direct instruction in the course of a modern German medical curriculum - results of a mixed methods trial. BMC Med Educ 16 (2016): 158.
15. Kershaw G, Grivna M, Elbarazi I, AliHassan S, Aziz F, Al Dhaheri AI: Integrating Public Health and Health Promotion Practice in the Medical Curriculum: A Self-Directed Team-Based Project Approach. Front Public Health 5 (2017): 193.
16. Patra S, Khan AM, Upadhyay MK, Sharma R, Rajoura OP, Bhasin SK. Module to facilitate self-directed learning among medical undergraduates: Development and implementation. J Educ Health Promot 9 (2020): 231.
17. Zheng B, Zhang Y: Self-regulated learning: the effect on medical student learning outcomes in a flipped classroom environment. BMC Med Educ 20 (2020): 100.
18. Cleary TJ, Sandars J. Assessing self-regulatory processes during clinical skill performance: a pilot study. Med Teach 33 (2011): e368-374.
19. Richardson WS, Wilson MC, Nishikawa J, Hayward RS: The well-built clinical question: a key to evidence-based decisions. ACP J Club 123 (1995): A12-13.