Designing Educational Activities based on Learning Goals

So, I received this request from a friend and ex-colleague: 

Dear Red,

I hope everything is good at your end.

My department wants to improve the effectiveness of their educational activities, but our teachers cannot agree on anything. Some claim that learning from examples is most effective and propose more modelling and demonstrations. Others claim learning-by-doing is most effective and want more problem-solving and discovery-based learning. Yet others claim that simulation-based learning is the way to go and want to buy simulators and include roleplays. We have endless discussions.

I know you are doing your MHPE and was hoping you could shed some light on these issues. Could you help us settle our disputes and explain which instructional methods are most effective?

Looking forward to your response.

Thanks,

George

Dear George,

I reflected on this concern that you sent me. This is quite exciting as your faculty is actively getting involved in the decision-making on teaching methods and that they are not shying away from the typical lecture. That is credit to the hard work that your team has been putting in since a few years back, when you started the curriculum redesign.

The issues that you presented me are actually quite complex but focused on instructional methods. Allow me to disentangle them.

My response to you is below – and I hope it is useful. Do contact me if you have any queries and concerns.

Best regards,

Red

What seems to be the problem?

First, what do your colleagues mean when they said that that they want to "improve the effectiveness of their educational activities"? This question does not have a generic answer, instead it needs to be contextualized, depending on the “educational activities” in question. Which educational activities are most effective? The answer is, it depends.

Second, we need to determine how to teach depending on what we need to teach. Do we need to teach knowledge? Do we need to teach a skill set? Or do we need to hone their attitudes and instill certain qualities on our students. Every one of these will necessitate different activities mentioned in the previous paragraph.

This response to George will begin with the general underpinnings of educational activities and its relations to learning goals. Then I will narrow it down to my experience in my own setting, focusing on one particular learning activity.

How do we improve effectiveness of our educational activities?

Weston and Cranton (1986) defined teaching method or educational activities as the vehicle or technique for instructor-student communication. Reigeluth (2013) described the “Level of effectiveness is a matter of how well the instruction works, as indicated by how well (to what degree of proficiency) the learning goals are attained. “(p9) When we talk of effectiveness we talk of the mastery that the learners achieve using a particular method of instruction. This is usually measured by adding a criterion in the learning outcomes, such as “getting the correct answer 80% of the time.”

But to effectively improve the educational activities that we use, we must match those educational activities with the educational goals and the kind of learning that the student will derive from an activity. Kern (2016) ascribed activities that can best teach the educational objectives. For example, knowledge acquisition can be achieved through lectures and even online, self-paced modules. Whereas, for clinical reasoning skills, problem-based learning may be optimal. Attitudes such as professionalism may be developed through metacognition using reflection, and by working in interprofessional teams. He termed this Congruent Educational Methods (Kern, 2016).

While we must be aware that in choosing Instructional methods, there is no perfect method even for those that are aligned to the educational objectives. Effectiveness is one of 3 criteria that makes up the instructional design iron triangle concept described by Reigeluth and Carr-Chellmann (2013), the 2 others being efficiency, and appeal. In the iron triangle, only 2 of the 3 criteria are reasonably achievable, one has to be traded off. Honebein and Honebein (2016) found that designers had to decide on the trade-offs or “sacrifices” that they will accept in different teaching scenarios. Clearly, to achieve good instruction, instructional design must suit different situations (Reigeluth & Carr-Chellman, 2013).

How do we teach what we need to teach?

Kern (2016) mentioned in his chapter on A Six-Step Approach to Curriculum Development that “Once objectives have been clarified, curriculum content is chosen and educational methods are selected that will most likely achieve the educational objectives.” This goes back to Weston and Cranton (1986) who categorized instruction into four: (1) instructor-centered, (2) interactive, (3) individualized, and (4) experiential, depending on what is needed to be taught, and on the ID practitioner’s or the teacher’s personal preferences.

Notwithstanding, Christensen and Osguthorpe (2004) determined that the ID practitioners’ decision on what instructional design to use is often made upon discussion with other practitioners. They also found that only half use theories as basis for instructional strategies, but rather use instructional designs, implying that IDs are more practical than theoretical in their educational activity decisions.

While this may seem straightforward, Honebein and Reigeluth (2021) remind would-be instructional designers that their work happens in real-time, in a complex, living situation that may be unpredictable. Oftentimes, teaching the same course twice may yield different results.

My context

              Our medical school, St Luke’s Medical Center College of Medicine (SLMCCM) is a small but growing private medical school that was incepted by the St Luke’s Medical Center, the largest single-entity private medical center in the Philippines, that integrates 2 medical center sites/campuses, and a medical school. As such, the one-system approach unites governance of administrative and fiscal resources of the college within the same processes as the medical center. In short, there is a deeper resource pool, but to access that is more circuitous.

In our medical school, we are using a few different activities, most of which I have discussed above. The setting is onsite, unless specifically mentioned online, wherein the students will typically be at home.

A quick survey of the educational activities that we use in our institution, along with literature (Table 1) showing the effectiveness of our teaching methods (how to teach) depending on the educational goals (what needs to be taught). Didactics, and one-to-many approach maybe the most efficient in delivering knowledge, with online didactics being available now in a manner that is inexhaustibly repeatable (Branzetti, et. al., 2011); and learning how to communicate to colleagues and patients by doing is a good way to learn communications skills (Chi & Verghese, 2013); Sawyer, et al (2015) reiterated the 6-step procedural skills training that involved the end-to-end process from knowledge acquisition (learn), to observation (see), demonstration (practice), simulation (prove), performance (do) and all the way to retaining that skill (maintain). Other individual educational goals are included in the survey as well, such as collaboration, critical thinking and character formation.

Table 1. Survey of educational activities (how) that we use in our institution and our learning goals (what) and literature citing their effectiveness.

What we need to teach	How do we teach	Sources
Knowledge acquisition	Online didactics	Branzetti, et al., 2011
Communications	Learning by doing	Chi & Verghese, 2013
Motor skills/ Procedural skills	Demonstration and practice *Learn, See, Practice, Prove,  Do, and Maintain	Weeks & Anderson, 2000 Sawyer, et al., 2015
Collaboration and Knowledge application	**Team-based learning	Michelsen & Sweet, 2008
Character formation and values	Role-modelling	Cruess, et al, 2008
Critical thinking and knowledge application Clinical skills	Simulation	Swanson, et al., 2011   Al-Elq, 2010

* being considered;  **still in demonstration phase

For knowledge acquisition, a fundamental activity is lecture (both synchronous onsite, and synchronous online), and online didactics that use pre-recorded lecture or voice-annotated PowerPoint and quizzes. Lecture is very efficient as it utilizes one teacher to teach one whole class of almost 200 students. It is an effective form of educational activity as the students generally pass with high marks on examinations that are generally knowledge-recall MCQs. But in terms of appeal, the students are not very engaged. They often miss live lectures and opt to watch recordings of these lectures.

              We also have skills training and we use models and task trainers. This ensures that the students can practice as many times as they need, for those critical skills that they need to perform when they reach the clinical years. It makes for safer doctors as the students will be skillful by the time they encounter their first patients. It is effective, as OSCE shows most of the students can do the skills. The students find this appealing as they are engaged most of the time. But, this is not very efficient, as it requires a lot of resources.

              For clinical reasoning, we use standardized patients to perform patient centered interview, physical examinations and form their diagnosis, request for diagnostics and manage the patients. This is learning-by-doing. The students are very much engaged, and the students are able to arrive at correct diagnosis and management through clinically relevant critical thinking. But like the use of models, the use of standardized patients is resource intensive, difficult to design, and not very efficient.  

Spotlight on Team Based Learning

              I would like to focus on an educational activity that we had long planned in doing but was disrupted by the pandemic, and that is Team Based Learning (TBL). It was initially demonstrated in one session in the Academic Year (AY)2019-2020. It shall be brought back in the coming AY2023-2024.  The course that we employed it in is called Foundations of Medicine (FOM). This is an integrative course for year level 1 medical students, who have an introductory knowledge of medicine and clinical reasoning. But over the first year, they have didactics and laboratory sessions across 4 main disciplines (Anatomy, Biochemistry, Physiology, and Preventive Medicine). In FOM, they teach clinical reasoning, and integration of knowledge learned from the abovementioned disciplines. It meets a total of 18 Fridays in an academic year to integrate the learnings from days and weeks prior. This is an onsite course and delivered partly by TBL. Out of a total of about 18 sessions, 6 sessions are planned to be delivered in TBL format. This happens onsite, in class (face-to-face). When using TBL, the knowledge from the previous sessions of the 4 disciplines becoming the pre-learning materials, with the individual and team readiness assurance tests (iRAT/tRAT) items coming from the disciplines (for example on the topic of the Respiratory System), and then they explore the clinical application using case vignettes in the Application Exercises.

              During the first and only demonstration session of the TBL, the students initially preferred the small group discussion (SGD) format over TBL. In these SGDs, a faculty guides them as they discuss the case vignettes and then answer the quizzes immediately after. The students did not need to prepare when they come to class and they only learn while in class, and they immediately get tested on the content. Whereas in TBL, the students had to come prepared, get assessed for their readiness (iRAT/tRAT) before they even discuss the cases (Michaelsen, et al., 2011). During the demonstration, the faculty acted like a facilitator and did not provide the answer. It was only during the actual Application Exercises that the answers were revealed as the case slowly rolls out.

Though surprisingly, the students surveyed at the end of the session liked the TBL session, with the comments for TBL coming in as: “Fun, Interactive, Promotes teamwork, Promotes communications among peers, Structured questions, Teaches time management, Maximizes use of technology” (SLMCCM, 2020). In terms of effectivity, Table 2 summarizes the general score of the iRAT/tRAT comparisons to show that the Team scores are higher than the individual scores (SLMCCM, 2020).  

Table 2. Summary of iRAT/tRAT scores

	iRAT	tRAT	Difference
Median	6	9	3
Average	6	9	3
Students with improved grades			149
Unchanged			17
Students with worse grade			5
Sample size			171

The first of two main constraints is human resource – the faculty need to be trained to create TBL learning materials, to facilitate, and to form and manage teams. This constraint is being addressed. In the months leading to AY 2019-2020, there were at least 30 trained faculty over 2 training bootcamps. But as they were ready to put in their learnings into practice, the face-to-face sessions were canceled because of the pandemic. Currently, training has been restarted, and in the past 2 months, refresher courses have been offered and new faculty were recruited for training. It is envisioned that the training will continue. This is resource intensive but can be managed. The second main constraint is the physical space. The students commented on being cramped in a small space, and that the ambience and AV technology not quite optimal. And because of the increasing class size, space constraint has taken even more prominence. The current venue is the Discussion Rooms, which consists of 6 small group discussion rooms with collapsible walls, which can then be combined to hold 20 tables with 6 students each, and therefore big enough for 120 students. The current intake has grown to 192 students. The planned work-around is to hold 2 sessions of the same TBL content for the Year Level 1 class, once in the morning (0830 hrs to 1230 hrs) and another in the afternoon (1330 hours to 1730 hours).

The medical center has already committed in-principle to TBL Theatre that is capable of 300 students. This will also be the TBL training center for the whole medical center. This will only be available when the new hospital building that is yet to begin construction finishes and various movements of the wards have been concluded. The main decision-makers here are the Board of Trustees of the medical center, the CEO, and the Dean/Chief Academic Officer (CAO). They have bought in to the idea of employing innovative teaching to ensure competency based medical education among our medical students, who will form the future medical staff of the medical center, ensuring that we reach the organizational goal of becoming global by 2030. Because this TBL Theatre will be the home of the TBL Training Center which aspires to becoming a national center of excellence in TBL, it is necessary that it is backed by adequate amount of resources. This can only be provided by the Board and the CEO, with the advocacy and guidance of the Dean/CAO.

What is our final advice to George and his team?

Well, besides knowing the suitability of each learning activity vis-à-vis their learning goals, open-mindedness is also key. The instructional designer must be able to recognize personal biases when choosing educational activities, and must be open to other activities that might be considered suitable for the instructional objective. In Honebein’s study (2017), he was able to demonstrate the instructional designers’ bias towards certain instructional methods, and that this poses a risk when one method is rejected despite it being effective for the objective at hand.

 

References

Al-Elq, A. H. (2010). Simulation-based medical teaching and learning. Journal of family and Community Medicine, 17(1), 35.

Branzetti, J. B., Aldeen, A. Z., Foster, A. W., & Mark Courtney, D. (2011). A novel online didactic curriculum helps improve knowledge acquisition among non–emergency medicine rotating residents. Academic Emergency Medicine, 18(1), 53-59.

Chi, J., & Verghese, A. (2013). Improving communication with patients: learning by doing. Jama, 310(21), 2257-2258.

Christensen, T. K., & Osguthorpe, R. T. (2004). How do instructional‐design practitioners make instructional‐strategy decisions?. Performance improvement quarterly, 17(3), 45-65.

Cruess, S. R., Cruess, R. L., & Steinert, Y. (2008). Role modelling—making the most of a powerful teaching strategy. Bmj, 336(7646), 718-721.

Kern, D. E. (2016). A six-step approach to curriculum development. Curriculum development for medical education, 3, 5-9.

Honebein, P. C. (2017). The influence of values and rich conditions on designers’ judgments about useful instructional methods. Educational Technology Research and Development, 65(2), 341-357.

Honebein, P. C., & Honebein, C. H. (2015). Effectiveness, efficiency, and appeal: pick any two? The influence of learning domains and learning outcomes on designer judgments of useful instructional methods. Educational Technology Research and Development, 63(6), 937-955. 

Honebein, P. & Reigeluth, C. M. (2021). Making Good Design Judgments via the Instructional Theory Framework. In J. K. McDonald & R. E. West (Eds.), Design for Learning: Principles, Processes, and Praxis. EdTech Books. 

Michaelsen, L. K., & Sweet, M. (2008). The essential elements of team‐based learning. New directions for teaching and learning, 2008(116), 7-27.

Michaelsen, L., Sweet, M., & Parmelee, D. (2011). Team-Based Learning: Small–Group Learning’s Next Big Step: New Directions for Teaching and Learning.

Reigeluth, C. M. (2013). Instructional-design theories and models: A new paradigm of instructional theory, Volume II. Routledge.

Reigeluth, C. M., & Carr-Chellman, A. A. (Eds.). (2009). Instructional-design theories and models, volume III: Building a common knowledge base (Vol. 3). Routledge.

Sawyer, T., White, M., Zaveri, P., Chang, T., Ades, A., French, H., ... & Kessler, D. (2015). Learn, see, practice, prove, do, maintain: an evidence-based pedagogical framework for procedural skill training in medicine. Academic Medicine, 90(8), 1025-1033.

St. Luke’s Medical Center College of Medicine. (2020) Executive Summary: Team Based Learning. Unpublished internal company document.

Swanson, E. A., Nicholson, A. C., Boese, T. A., Cram, E., Stineman, A. M., & Tew, K. (2011). Comparison of selected teaching strategies incorporating simulation and student outcomes. Clinical Simulation in Nursing, 7(3), e81-e90.

Weeks, D. L., & Anderson, L. P. (2000). The interaction of observational learning with overt practice: effects on motor skill learning. Acta psychologica, 104(2), 259-271.

Weston, C., & Cranton, P. A. (1986). Selecting instructional strategies. The Journal of Higher Education, 57(3), 259-288.