As a member of the Curriculum Management Committee I am often involved in assessing proposed changes in the Palmer instructional program. This may involve evaluating a newly proposed course, or considering changes in existing course objectives, among other decisions that must be made. And when we design such changes, there are lessons we might apply to our effort. The excellent text “Designing and assessing courses and curricula: a practical guide” (1) provides a number of characteristics that can help make your efforts succeed. These include:
Have a plan and follow it: The text offers a model for curriculum design that has been shown to be successful, and it recommends you follow it carefully.
Do not do it alone: Involve others in your design efforts. Include your chair, quite obviously, but also include others who may be affected by what you are doing. And include those whose expertise you may need, such as those with expertise in, for example, instructional technology.
Strive for the ideal: Focus not on what exists but rather on what ideally could be. This can also help reduce potential turf war problems and can lead to new and novel curriculum approaches.
Collect information before you begin. In my estimation, this often does not take place. People focus on the end result more than the tasks involved in producing the end result and often do not do sufficient preparatory work. The information you should collect should address the need for your innovation, it should allow you to test your assumptions and it can provide base data to measure change.
Create ownership and keep key individuals informed. You need to make sure you have support before you begin a change process. All affected need to be kept informed and you need to consider who it is that may be affected. Involve key administrators.
Be sensitive to human problems. Curriculum innovation and change can significantly affect others. Change can be an emotionally laden process and we need to keep mindful of that fact. Those who are affected may see this in a negative light, forgetting that this is about institutional need, not personal need. Use administrative help. Talk to people. Answer questions respectfully.
Do not reinvent the wheel. Keep abreast of what is new, what has been tried and what has succeeded. Model your efforts after that, and don’t waste time and resources uing approaches that have failed in the past.
Pay attention to support systems and logistics. This really means that you need to work carefully with the registrar’s office to see how what you are planning fits into what can actually occur and be scheduled.
This is just a short overview of some of the issues you need to consider when you decide to innovate. This is something we all wish to do. We just need to smooth the path for innovation to occur.
References
1. Diamond R. Designing and assessing courses and curriculum: a practical guide, 2nd edition. San Francisco, CA; Jossey-Bass, 1998:237-238
Monday, January 31, 2011
Monday, January 24, 2011
Selecting Treatments: Healthcare Education
While not all of us are going to be involved in training students in the use of therapeutic interventions and treatment, the work we do here is geared toward making that happen. In the evidence-based world, how does one go about selecting treatment? Sackett and colleagues (1) offer some thoughts.
They proceed from the decision that you have reached a point where you believe the patient’s condition does warrant treatment (and there are many reasons why you may not). And further, you have selected the goal of treatment; in the case of a chiropractic physician, it may be to decrease pain, to lessen disability, or something else. But now you need to select the appropriate and proper therapy (whether adjustment, exercise, counseling, or so on). According to Sackett, there are three ways to choose the intervention:
1. You consider your own uncontrolled clinical experience and of your colleagues and friends, or perhaps you extend certain concepts of mechanisms of disease so that you arrive at a therapy that “seems to work or ought to work.” This is an inductive approach to therapy.
2. You obtain clinical research papers, mainly comprised of clinical trials which are designed to expose worthless or dangerous treatments, and you select treatments that “are able to successfully withstand formal attempts to demonstrate their worthlessness.” This is a deductive approach to therapy.
3. You obtain recommendations from your teachers or colleagues, or from advertisements and you then accept the treatment “on faith.” Sackett refers to this as the method of abdication or of seduction.
Now, it will not come as a surprise that Sackett and colleagues prefers the method of deduction. They argue that using your own clinical experience carries risks that you may not be aware of, notably that what you are really doing is using “historical controls.” By this, they mean that when you compare your latest patient to those you have seen before and for whom you were able to treat successfully, you may be comparing your current and potentially new treatment to an older form of therapy. And if you are able to help the current patient, you judge this new approach as efficacious and superior. In this they see risk.
However current formulations of evidence-based care recognizes that clinical experience is one plank of a triple-planked construct: best available evidence from the literature, combined with clinical expertise and patient values. All are important. But as I see evidence-based care, we need to stay mindful that one of the most common reasons for using it is to determine what to do for a patient where we have questions about what to do; that is, for situations where we might not know what to do. In such cases, our past clinical experience may not be much of a guide. I like to use the example of Brucellosis (in fact, I will hear from a certain academic administrator cough cough Dr. Weinert cough cough for using this example). This is not typically seen in chiropractic settings, so when you are confronted with a patient with this condition, how would you proceed? You would not have clinical expertise in managing the condition. Thus, you would likely need to turn to the literature for best information. When you do have experience, well, that experience will tell you what to do.
Conversations such as this are important in understanding the basis for modern practice, and the literature is replete with them. I heartily urge readers to look into the philosophical basis for evidence-based practice.
References
1. Sackett DL, Haynes RB, Guyatt G, Tugwell P. Clinical epidemiology: a basic science for clinical medicine, 2nd edition. Boston, MA; Little, Brown, 1991
They proceed from the decision that you have reached a point where you believe the patient’s condition does warrant treatment (and there are many reasons why you may not). And further, you have selected the goal of treatment; in the case of a chiropractic physician, it may be to decrease pain, to lessen disability, or something else. But now you need to select the appropriate and proper therapy (whether adjustment, exercise, counseling, or so on). According to Sackett, there are three ways to choose the intervention:
1. You consider your own uncontrolled clinical experience and of your colleagues and friends, or perhaps you extend certain concepts of mechanisms of disease so that you arrive at a therapy that “seems to work or ought to work.” This is an inductive approach to therapy.
2. You obtain clinical research papers, mainly comprised of clinical trials which are designed to expose worthless or dangerous treatments, and you select treatments that “are able to successfully withstand formal attempts to demonstrate their worthlessness.” This is a deductive approach to therapy.
3. You obtain recommendations from your teachers or colleagues, or from advertisements and you then accept the treatment “on faith.” Sackett refers to this as the method of abdication or of seduction.
Now, it will not come as a surprise that Sackett and colleagues prefers the method of deduction. They argue that using your own clinical experience carries risks that you may not be aware of, notably that what you are really doing is using “historical controls.” By this, they mean that when you compare your latest patient to those you have seen before and for whom you were able to treat successfully, you may be comparing your current and potentially new treatment to an older form of therapy. And if you are able to help the current patient, you judge this new approach as efficacious and superior. In this they see risk.
However current formulations of evidence-based care recognizes that clinical experience is one plank of a triple-planked construct: best available evidence from the literature, combined with clinical expertise and patient values. All are important. But as I see evidence-based care, we need to stay mindful that one of the most common reasons for using it is to determine what to do for a patient where we have questions about what to do; that is, for situations where we might not know what to do. In such cases, our past clinical experience may not be much of a guide. I like to use the example of Brucellosis (in fact, I will hear from a certain academic administrator cough cough Dr. Weinert cough cough for using this example). This is not typically seen in chiropractic settings, so when you are confronted with a patient with this condition, how would you proceed? You would not have clinical expertise in managing the condition. Thus, you would likely need to turn to the literature for best information. When you do have experience, well, that experience will tell you what to do.
Conversations such as this are important in understanding the basis for modern practice, and the literature is replete with them. I heartily urge readers to look into the philosophical basis for evidence-based practice.
References
1. Sackett DL, Haynes RB, Guyatt G, Tugwell P. Clinical epidemiology: a basic science for clinical medicine, 2nd edition. Boston, MA; Little, Brown, 1991
Tuesday, January 18, 2011
Stages of Survey Development
Over the course of the past week or so, I have had several contacts from faculty related to surveys they are in process of developing and for which they wish guidance on IRB application. As a result of looking over some of the proposed surveys I thought I would offer a few comments about the nature of developing them. Czaja and Blair, authors of a fine text on survey development (1), suggest a 5-step process for developing and completing a survey and I thought I would use their approach in offering these comments to you.
Stage 1: Survey design and preliminary planning. This stage specifically looks at the research problem and the research questions that the survey is designed to address. What is your goal in doing this survey? Is it to test a hypothesis, to test a causal model or to estimate the proportion of people who hold certain beliefs or attitudes? In asking this question, there are important issues embedded in it. For example, who is our population of interest? Is it all students at PCC, for example, or a particular trimester, or a specific subgroup from one of these? What is the sampling frame, since it is unlikely we will be able to capture every student in our target population? How will we contact them? In our initial planning, we also need to consider the kinds of questions we will ask, how much time we will need, and how we will analyze the results.
Stage 2: Pretesting. This is where we begin testing our initial design decisions. We need to think about how to reach our sampling frame, what kinds of records we will use for collecting information, how to word questions and so on. We have to draft an initial version of our questionnaire, and it is okay to borrow them from other past research, ensuring that we contact the originator for his or her permission. But keep in mind that past use in a different population is not a guarantee of success in using it in ours. Ru our draft past others for their comments, and incorporate them. Then we need to pretest the questionnaire in a small population of individuals. For example, in the PCCR we might use our research fellows to pretest a questionnaire. Once done, we gather information from that group, either via written comments or via personal interviews and/or focus group meetings, and from their comments we revise our questionnaire accordingly. We need to feel that we are asking what we think we are asking; pretesting helps to accomplish that goal.
Stage 3: Final survey design and planning. This is revision of our initial draft based on the input we have collected. It may also lead us to revise who our sampling frame is or how we analyze the data (we may find that a question we felt was close-ended is actually better asked as an open-ended question, shifting us from a quantitative analysis to a qualitative one.
Stage 4: Data collection. In this stage, you must monitor the results of the sampling and data collection activities. You should also begin coding your answers and preparing your data files ( for example, if you have a “yes- no” question, are you coding the yes response as “1” and the no as “2” or something else? Are you using Excel or SPSS or something else? What do you do with non-answers or double marks? Are you using double data entry, having a second person also code the data for reliability purposes?
Stage 5: Data coding, date-file construction, analysis and final report. This is now where you ensure all your data was properly entered, that you have coded it properly for analysis, that it makes sense when you do if you are looking at anything more than a simple descriptive analysis (how many answered each question).
All of these steps ensure that in the end you collect data that answers questions you are interested in clearly and without miscomprehension of your questions. These are necessary steps; please consider using them as you develop your own surveys.
References
1. Czaja R, Blair J. Designing surveys: a guide to decisions and procedures. Thousand Oaks, CA: Pine Forge Press, 1996:11-30
Stage 1: Survey design and preliminary planning. This stage specifically looks at the research problem and the research questions that the survey is designed to address. What is your goal in doing this survey? Is it to test a hypothesis, to test a causal model or to estimate the proportion of people who hold certain beliefs or attitudes? In asking this question, there are important issues embedded in it. For example, who is our population of interest? Is it all students at PCC, for example, or a particular trimester, or a specific subgroup from one of these? What is the sampling frame, since it is unlikely we will be able to capture every student in our target population? How will we contact them? In our initial planning, we also need to consider the kinds of questions we will ask, how much time we will need, and how we will analyze the results.
Stage 2: Pretesting. This is where we begin testing our initial design decisions. We need to think about how to reach our sampling frame, what kinds of records we will use for collecting information, how to word questions and so on. We have to draft an initial version of our questionnaire, and it is okay to borrow them from other past research, ensuring that we contact the originator for his or her permission. But keep in mind that past use in a different population is not a guarantee of success in using it in ours. Ru our draft past others for their comments, and incorporate them. Then we need to pretest the questionnaire in a small population of individuals. For example, in the PCCR we might use our research fellows to pretest a questionnaire. Once done, we gather information from that group, either via written comments or via personal interviews and/or focus group meetings, and from their comments we revise our questionnaire accordingly. We need to feel that we are asking what we think we are asking; pretesting helps to accomplish that goal.
Stage 3: Final survey design and planning. This is revision of our initial draft based on the input we have collected. It may also lead us to revise who our sampling frame is or how we analyze the data (we may find that a question we felt was close-ended is actually better asked as an open-ended question, shifting us from a quantitative analysis to a qualitative one.
Stage 4: Data collection. In this stage, you must monitor the results of the sampling and data collection activities. You should also begin coding your answers and preparing your data files ( for example, if you have a “yes- no” question, are you coding the yes response as “1” and the no as “2” or something else? Are you using Excel or SPSS or something else? What do you do with non-answers or double marks? Are you using double data entry, having a second person also code the data for reliability purposes?
Stage 5: Data coding, date-file construction, analysis and final report. This is now where you ensure all your data was properly entered, that you have coded it properly for analysis, that it makes sense when you do if you are looking at anything more than a simple descriptive analysis (how many answered each question).
All of these steps ensure that in the end you collect data that answers questions you are interested in clearly and without miscomprehension of your questions. These are necessary steps; please consider using them as you develop your own surveys.
References
1. Czaja R, Blair J. Designing surveys: a guide to decisions and procedures. Thousand Oaks, CA: Pine Forge Press, 1996:11-30
Monday, January 10, 2011
The Leadership Challenge
I’ve been thinking a lot lately about leadership: what makes a good leader, how are leaders trained, and so on. And in considering this, I have come across what appears to be one of the leading, if you will, texts on this subject, The Leadership Challenge, by Kouzes and Posner (1). Their text has been a best seller and it proposes 10 commitments to leadership that good leaders follow. These comprise the main part of the book, and they discuss each commitment and offer tactics to help you learn and assignments to help you understand them. But these 10 commitments are built around 5 practices which Kouzes and Posner feel are essential for good leadership. I thought I would briefly note each one.
Model the Way: We can easily translate into vernacular: walk the walk and talk the talk. It is your behavior which earns you respect. We need to act in a way we would like to see others act. The authors note that to effectively model behavior, you have to be clear about guiding principles, and you must clarify values. Leaders do not speak for themselves; they speak for their organization and for its values. But words only go so far; deeds matter even more. What you do is important; I can’t ask you to write papers, for example, if I do not do so myself.
Inspire a Shared Vision: Leaders have a vision of what can be, not just what is, and they believe in what they see and what they think can happen. They are confident they help people get there, and they see opportunity and possibility. Some of us do this every day; we see where we want our students to be, and we help them get there; we see their future. We need to enlist others in common vision, speaking in a common language, and have others’ interests at heart. We excite others, and they then will follow us as we go forward.
Challenge the Process: we all face challenges. Perhaps it is declining enrollment; perhaps it is challenges from a competitor, or something else. We cannot keep everything the same forever; we need to change, and we need to look at what we do in order to change. Leaders venture out, says Kouzes and Posner. They do not wait for fate to smile on them, but they actively work to address challenge; they search for opportunity to innovate and grow. Our students, for example, come to this college with excellent skills in using technology; we need to answer that new challenge by learning how we can yoke the technology to better educate them. No longer can we simply stand still. Leaders create ac climate where we can feel free to try innovation, and where we can even feel safe if we fail. Leaders understand risk but accept it.
Enable Others to Act: In order to get things done, we have to act, and our leaders allow us to do so. No, they encourage us to do so. And to do so, leaders foster collaboration and help build trust. Teamwork results and we all engage. We are empowered. You will hear leaders not use “I” very much, but you will hear them use “we.”
Encourage the Heart: We are all in this together and we have all been at this a long time. I have spent, for example, 31 years in academic chiropractic, and it is always possible we can become exhausted or disenchanted (I have not, let me be clear!). Leaders encourage the heart, they care about those who work with them, and they help draw people forward. A simple note of thanks can do wonders to the person thanked. Recognizing the contributions people make is important, and celebrating them collectively is critical.
In the end, leadership is a relationship that can be developed. We are leaders in the classroom, in our professional lives, and in many other ways. There are skills we can use to become better at this, if we but take the time to do so.
References
1. Kouzes JM, Posner BZ. The leadership challenge, 4th edition. San Francisco, CA: Jossey-Bass 2007
Model the Way: We can easily translate into vernacular: walk the walk and talk the talk. It is your behavior which earns you respect. We need to act in a way we would like to see others act. The authors note that to effectively model behavior, you have to be clear about guiding principles, and you must clarify values. Leaders do not speak for themselves; they speak for their organization and for its values. But words only go so far; deeds matter even more. What you do is important; I can’t ask you to write papers, for example, if I do not do so myself.
Inspire a Shared Vision: Leaders have a vision of what can be, not just what is, and they believe in what they see and what they think can happen. They are confident they help people get there, and they see opportunity and possibility. Some of us do this every day; we see where we want our students to be, and we help them get there; we see their future. We need to enlist others in common vision, speaking in a common language, and have others’ interests at heart. We excite others, and they then will follow us as we go forward.
Challenge the Process: we all face challenges. Perhaps it is declining enrollment; perhaps it is challenges from a competitor, or something else. We cannot keep everything the same forever; we need to change, and we need to look at what we do in order to change. Leaders venture out, says Kouzes and Posner. They do not wait for fate to smile on them, but they actively work to address challenge; they search for opportunity to innovate and grow. Our students, for example, come to this college with excellent skills in using technology; we need to answer that new challenge by learning how we can yoke the technology to better educate them. No longer can we simply stand still. Leaders create ac climate where we can feel free to try innovation, and where we can even feel safe if we fail. Leaders understand risk but accept it.
Enable Others to Act: In order to get things done, we have to act, and our leaders allow us to do so. No, they encourage us to do so. And to do so, leaders foster collaboration and help build trust. Teamwork results and we all engage. We are empowered. You will hear leaders not use “I” very much, but you will hear them use “we.”
Encourage the Heart: We are all in this together and we have all been at this a long time. I have spent, for example, 31 years in academic chiropractic, and it is always possible we can become exhausted or disenchanted (I have not, let me be clear!). Leaders encourage the heart, they care about those who work with them, and they help draw people forward. A simple note of thanks can do wonders to the person thanked. Recognizing the contributions people make is important, and celebrating them collectively is critical.
In the end, leadership is a relationship that can be developed. We are leaders in the classroom, in our professional lives, and in many other ways. There are skills we can use to become better at this, if we but take the time to do so.
References
1. Kouzes JM, Posner BZ. The leadership challenge, 4th edition. San Francisco, CA: Jossey-Bass 2007
Tuesday, January 4, 2011
Innovative Teaching
Let me first offer you my best wishes for the new year. I hope that during the past couple of weeks you were able to catch your breath, enjoy your time off and spend time with friends and family during the holidays. I know I very much appreciated the time off; we have all been working hard and this will help clear out the cobwebs and set us on the road for a great 2011.
While I was off, I was sent links to a site which featured the well-known statistician Hans Rosling. In viewing one of the clips I got to thinking about innovation in teaching methods and I have some thoughts to offer on that, in the context of commenting of a few of the clips I watched over the break. Let me begin by offering this link:
200 Countries, 200 years, 4 minutes: http://www.flixxy.com/200-countries-200-years-4-minutes.htm
Now, I know that we do not have access the kind of technology seen in this clip. But stop a minute and think about what you see here. Note that what Rosling does is intriguing. First, he is obviously passionate and animated about what he is discussing, and he has obviously planned out what he is going to say and the movements he makes in order to synch them properly with something he cannot actually see, since that has to be edited in later. But also note that his use of this technology helps him pass on an important learning message that would otherwise take far longer to present if he were to simply lecture or even use bullet points in PowerPoint. He has been able to show you the relationship between life expectancy and income (wealth) for 200 countries over a period of 2 centuries. And he has done so in just 4 minutes- and I think everyone who sees this could tell me what that relationship is. The key point he makes is this: “Having the data is not enough; I have to show it in ways people both enjoy and understand.” He has done so masterfully.
New Insights on Poverty: http://www.ted.com/talks/lang/eng/hans_rosling_reveals_new_insights_on_poverty.html
Another clip with Rosling. Note again how he combines passion, humor and data to present an engaging and informative lecture. Not only does he use his animated projections, but note how he integrates video clips and panoramic photography to illustrate points. And note his subtle humor about PowerPoint at around the 14.50 mark here! And I absolutely guarantee that youw ill not foget the end to this presentation. Guarantee it!
Animating the Cell: http://www.ted.com/talks/david_bolinsky_animates_a_cell.html
Another use of graphics to illustrate complex points. Imagine accessing this for use in class, or even learning how to do something like this. (PS. Ted.com is a great source of informative and educational videos).
The general idea, of course, is to use innovation in our teaching. Other ideas exist as well; Drunk History, which can be found in youtube, presents clips from American history through the lens of a purportedly drunk person discussing historical episodes and then has them acted out by famous actors. I will not post links due to the profanity on them, but I list it here as yet another novel method for presenting information. I hope to encourage us all to look for new ways to present information; new technologies do exist and we need to remain up to date with them and then use them to help our students learn more effectively.
Welcome back!
While I was off, I was sent links to a site which featured the well-known statistician Hans Rosling. In viewing one of the clips I got to thinking about innovation in teaching methods and I have some thoughts to offer on that, in the context of commenting of a few of the clips I watched over the break. Let me begin by offering this link:
200 Countries, 200 years, 4 minutes: http://www.flixxy.com/200-countries-200-years-4-minutes.htm
Now, I know that we do not have access the kind of technology seen in this clip. But stop a minute and think about what you see here. Note that what Rosling does is intriguing. First, he is obviously passionate and animated about what he is discussing, and he has obviously planned out what he is going to say and the movements he makes in order to synch them properly with something he cannot actually see, since that has to be edited in later. But also note that his use of this technology helps him pass on an important learning message that would otherwise take far longer to present if he were to simply lecture or even use bullet points in PowerPoint. He has been able to show you the relationship between life expectancy and income (wealth) for 200 countries over a period of 2 centuries. And he has done so in just 4 minutes- and I think everyone who sees this could tell me what that relationship is. The key point he makes is this: “Having the data is not enough; I have to show it in ways people both enjoy and understand.” He has done so masterfully.
New Insights on Poverty: http://www.ted.com/talks/lang/eng/hans_rosling_reveals_new_insights_on_poverty.html
Another clip with Rosling. Note again how he combines passion, humor and data to present an engaging and informative lecture. Not only does he use his animated projections, but note how he integrates video clips and panoramic photography to illustrate points. And note his subtle humor about PowerPoint at around the 14.50 mark here! And I absolutely guarantee that youw ill not foget the end to this presentation. Guarantee it!
Animating the Cell: http://www.ted.com/talks/david_bolinsky_animates_a_cell.html
Another use of graphics to illustrate complex points. Imagine accessing this for use in class, or even learning how to do something like this. (PS. Ted.com is a great source of informative and educational videos).
The general idea, of course, is to use innovation in our teaching. Other ideas exist as well; Drunk History, which can be found in youtube, presents clips from American history through the lens of a purportedly drunk person discussing historical episodes and then has them acted out by famous actors. I will not post links due to the profanity on them, but I list it here as yet another novel method for presenting information. I hope to encourage us all to look for new ways to present information; new technologies do exist and we need to remain up to date with them and then use them to help our students learn more effectively.
Welcome back!
Monday, December 13, 2010
New Papers on Healthcare Education
These new papers touch on areas that we at Palmer College are involved with. I thought I would post these as a result.
1. Clark ML, Hutchison CR, Lockyer JM. Musculoskeletal Education: A Curriculum Evaluation at one University. BMC Medical Education 2010, 10:93doi:10.1186/1472-6920-10-93
ABSTRACT
Background: The increasing burden of illness related to musculoskeletal diseases makes it essential that attention be paid to musculoskeletal education in medical schools. This case study examines the undergraduate musculoskeletal curriculum at one medical school.
Methods: A case study research methodology used quantitative and qualitative approaches to systematically examine the undergraduate musculoskeletal course at the University of Calgary (Alberta, Canada) Faculty of Medicine. The aim of the study was to understand the strengths and weaknesses of the curriculum guided by four questions: (1) Was the course structured according to standard principles for curriculum design as described in the Kern framework? (2) How did students and faculty perceive the course? (3) Was the assessment of the students valid and reliable? (4) Were the course evaluations completed by student and faculty valid and reliable?
Results: The analysis showed that the structure of the musculoskeletal course mapped to many components of Kern's framework in course design. The course was subject to a high level of commitment to teaching, included a valid and reliable final examination, and valid evaluation questionnaires that provided relevant information to assess curriculum function. Analysis also identified several weaknesses in the course: the apparent absence of a formalized needs assessment, course objectives that were not specific or measurable, poor development of clinical presentations, small group sessions that exceeded normal 'small group' sizes, and poor alignment between the course objectives, examination blueprint and the examination. Both students and faculty members perceived the same strengths and weaknesses in the curriculum. Course evaluation data provided information that was consistent with the findings from the interviews with the key stakeholders.
Conclusions: The case study approach using the Kern framework and selected questions provided a robust way to assess a curriculum, identify its strengths and weaknesses and guide improvements.
2. Botezatu M, Hult H, Fors UG. Virtual Patient Simulation: what do students make of it? A focus group study. BMC Medical Education 2010, 10:91doi:10.1186/1472-6920-10-91
ABSTRACT
Background: The learners' perspectives on Virtual Patient Simulation systems (VPS) are quintessential to their successful development and implementation. Focus group interviews were conducted in order to explore the opinions of medical students on the educational use of a VPS, the Web-based Simulation of Patients application (Web-SP).
Methods. Two focus group interviews - each with 8 undergraduate students who had used Web-SP cases for learning and/or assessment as part of their Internal Medicine curriculum in 2007 - were performed at the Faculty of Medicine of Universidad el Bosque (Bogota), in January 2008. The interviews were conducted in Spanish, transcribed by the main researcher and translated into English. The resulting transcripts were independently coded by two authors, who also performed the content analysis. Each coder analyzed the data separately, arriving to categories and themes, whose final form was reached after a consensus discussion.
Results. Eighteen categories were identified and clustered into five main themes: learning, teaching, assessment, authenticity and implementation. In agreement with the literature, clinical reasoning development is envisaged by students to be the main scope of VPS use; transferable skills, retention enhancement and the importance of making mistakes are other categories circumscribed to this theme. VPS should enjoy a broad use across clinical specialties and support learning of topics not seen during clinical rotations; they are thought to have a regulatory effect at individual level, helping the students to plan their learning. The participants believe that assessment with VPS should be relevant for their future clinical practice; it is deemed to be qualitatively different from regular exams and to increase student motivation. The VPS design and content, the localization of the socio-cultural context, the realism of the cases, as well as the presence and quality of feedback are intrinsic features contributing to VPS authenticity.
Conclusions. Five main themes were found to be associated with successful VPS use in medical curriculum: Learning, Teaching, Assessment, Authenticity and Implementation. Medical students perceive Virtual Patients as important learning and assessment tools, fostering clinical reasoning, in preparation for the future clinical practice as young doctors. However, a number of issues regarding VPS design, authenticity and implementation need to be fulfilled, in order to reach the potential educational goals of such applications.
3. Deom M, Agoritsas T, Bovier PA, Perneger TV. What doctors think about the impact of managed care tools on quality of care, costs, autonomy, and relations with patients. BMC Health Services Research 2010, 10:331doi:10.1186/1472-6963-10-331
ABSTRACT
Background: How doctors perceive managed care tools and incentives is not well known. We assessed doctors' opinions about the expected impact of eight managed care tools on quality of care, control of health care costs, professional autonomy and relations with patients.
Methods: Mail survey of doctors (N=1546) in Geneva, Switzerland. Respondents were asked to rate the impact of 8 managed care tools on 4 aspects of care on a 5-level scale (1 very negative, 2 rather negative, 3 neutral, 4 rather positive, 5 very positive). For each tool, we obtained a mean score from the 4 separate impacts.
Results: Doctors had predominantly negative opinions of the impact of managed care tools: use of guidelines (mean score 3.18), gate-keeping (2.76), managed care networks (2.77), second opinion requirement (2.65), pay for performance (1.90), pay by salary (2.24), selective contracting (1.56), and pre-approval of expensive treatments (1.77). Estimated impacts on cost control were positive or neutral for most tools, but impacts on professional autonomy were predominantly negative. Primary care doctors held more positive opinions than doctors in other specialties, and psychiatrists were in general the most critical. Older doctors had more negative opinions, as well as those in private practice.
Conclusions: Doctors perceived most managed care tools to have a positive impact on the control of health care costs but a negative impact on medical practice. Tools that are controlled by the profession were better accepted than those that are imposed by payers.
This will be the last blog post of 2010. We are all soon heading into our all-too-short vacation break. I wish you all the very best for the upcoming holiday season and for the new year.
1. Clark ML, Hutchison CR, Lockyer JM. Musculoskeletal Education: A Curriculum Evaluation at one University. BMC Medical Education 2010, 10:93doi:10.1186/1472-6920-10-93
ABSTRACT
Background: The increasing burden of illness related to musculoskeletal diseases makes it essential that attention be paid to musculoskeletal education in medical schools. This case study examines the undergraduate musculoskeletal curriculum at one medical school.
Methods: A case study research methodology used quantitative and qualitative approaches to systematically examine the undergraduate musculoskeletal course at the University of Calgary (Alberta, Canada) Faculty of Medicine. The aim of the study was to understand the strengths and weaknesses of the curriculum guided by four questions: (1) Was the course structured according to standard principles for curriculum design as described in the Kern framework? (2) How did students and faculty perceive the course? (3) Was the assessment of the students valid and reliable? (4) Were the course evaluations completed by student and faculty valid and reliable?
Results: The analysis showed that the structure of the musculoskeletal course mapped to many components of Kern's framework in course design. The course was subject to a high level of commitment to teaching, included a valid and reliable final examination, and valid evaluation questionnaires that provided relevant information to assess curriculum function. Analysis also identified several weaknesses in the course: the apparent absence of a formalized needs assessment, course objectives that were not specific or measurable, poor development of clinical presentations, small group sessions that exceeded normal 'small group' sizes, and poor alignment between the course objectives, examination blueprint and the examination. Both students and faculty members perceived the same strengths and weaknesses in the curriculum. Course evaluation data provided information that was consistent with the findings from the interviews with the key stakeholders.
Conclusions: The case study approach using the Kern framework and selected questions provided a robust way to assess a curriculum, identify its strengths and weaknesses and guide improvements.
2. Botezatu M, Hult H, Fors UG. Virtual Patient Simulation: what do students make of it? A focus group study. BMC Medical Education 2010, 10:91doi:10.1186/1472-6920-10-91
ABSTRACT
Background: The learners' perspectives on Virtual Patient Simulation systems (VPS) are quintessential to their successful development and implementation. Focus group interviews were conducted in order to explore the opinions of medical students on the educational use of a VPS, the Web-based Simulation of Patients application (Web-SP).
Methods. Two focus group interviews - each with 8 undergraduate students who had used Web-SP cases for learning and/or assessment as part of their Internal Medicine curriculum in 2007 - were performed at the Faculty of Medicine of Universidad el Bosque (Bogota), in January 2008. The interviews were conducted in Spanish, transcribed by the main researcher and translated into English. The resulting transcripts were independently coded by two authors, who also performed the content analysis. Each coder analyzed the data separately, arriving to categories and themes, whose final form was reached after a consensus discussion.
Results. Eighteen categories were identified and clustered into five main themes: learning, teaching, assessment, authenticity and implementation. In agreement with the literature, clinical reasoning development is envisaged by students to be the main scope of VPS use; transferable skills, retention enhancement and the importance of making mistakes are other categories circumscribed to this theme. VPS should enjoy a broad use across clinical specialties and support learning of topics not seen during clinical rotations; they are thought to have a regulatory effect at individual level, helping the students to plan their learning. The participants believe that assessment with VPS should be relevant for their future clinical practice; it is deemed to be qualitatively different from regular exams and to increase student motivation. The VPS design and content, the localization of the socio-cultural context, the realism of the cases, as well as the presence and quality of feedback are intrinsic features contributing to VPS authenticity.
Conclusions. Five main themes were found to be associated with successful VPS use in medical curriculum: Learning, Teaching, Assessment, Authenticity and Implementation. Medical students perceive Virtual Patients as important learning and assessment tools, fostering clinical reasoning, in preparation for the future clinical practice as young doctors. However, a number of issues regarding VPS design, authenticity and implementation need to be fulfilled, in order to reach the potential educational goals of such applications.
3. Deom M, Agoritsas T, Bovier PA, Perneger TV. What doctors think about the impact of managed care tools on quality of care, costs, autonomy, and relations with patients. BMC Health Services Research 2010, 10:331doi:10.1186/1472-6963-10-331
ABSTRACT
Background: How doctors perceive managed care tools and incentives is not well known. We assessed doctors' opinions about the expected impact of eight managed care tools on quality of care, control of health care costs, professional autonomy and relations with patients.
Methods: Mail survey of doctors (N=1546) in Geneva, Switzerland. Respondents were asked to rate the impact of 8 managed care tools on 4 aspects of care on a 5-level scale (1 very negative, 2 rather negative, 3 neutral, 4 rather positive, 5 very positive). For each tool, we obtained a mean score from the 4 separate impacts.
Results: Doctors had predominantly negative opinions of the impact of managed care tools: use of guidelines (mean score 3.18), gate-keeping (2.76), managed care networks (2.77), second opinion requirement (2.65), pay for performance (1.90), pay by salary (2.24), selective contracting (1.56), and pre-approval of expensive treatments (1.77). Estimated impacts on cost control were positive or neutral for most tools, but impacts on professional autonomy were predominantly negative. Primary care doctors held more positive opinions than doctors in other specialties, and psychiatrists were in general the most critical. Older doctors had more negative opinions, as well as those in private practice.
Conclusions: Doctors perceived most managed care tools to have a positive impact on the control of health care costs but a negative impact on medical practice. Tools that are controlled by the profession were better accepted than those that are imposed by payers.
This will be the last blog post of 2010. We are all soon heading into our all-too-short vacation break. I wish you all the very best for the upcoming holiday season and for the new year.
Monday, December 6, 2010
Rubrics Continued
Rubrics are comprised of 4 general parts: a task description, characteristics to be rated (usually placed in rows), levels of mastery (usually placed in columns) and a description of each mastery level; that is, of each cell.
The task description is the outcome being assessed or the instructions a student is provided for an assignment. The characteristics to be rated are the skills, knowledge or behaviors to be demonstrate by the student. The levels of mastery should be written clear language. An example might be something along the lines of: exemplary, proficient, marginal, unacceptable. Finally, each cell would contain a description of the what is required for each mastery level.
The University of Hawaii at Manoa (1) suggests that there are 6 steps to developing a rubric:
Step 1: Identify what it is you wish to assess.
Step 2: Identify the characteristics you wish to rate. Here, you would detail the skills or knowledge you plan on evaluating, limiting them to those you feel are most critical or important.
Step 3: Identify the levels of mastery: The authors recommend that you use an even number of categories to avoid the middle category being a sort of “catch all” for scoring.
Step 4: Describe each level of mastery for each characteristic (cell). Start by describing the best work you could reasonably expect to receive for that characteristic, and set that as your top category. Determine what would comprise unacceptable work and set that as your bottom category. Finally develop your mid-categories, ensuring that there is no overlap between any of them.
Step 5: Test the rubric. Apply it to an assignment, and share it with colleagues for their input. You also need to determine the minimal work that you would find acceptable for passing. This could be based on an average, a total score, or achieving a score of, say, marginal on every cell. Or, of course, you could set the standard higher than that.
Step 6: review and revise. It takes work to set these up and ensure they measure what you wish to measure. Rubrics also allow us to share grading expectations, which may be of help; for example, consider how a rubric might be used to assess a technique practical examination.
References
1. http://manoa.hawaii.edu/assessment/howto/rubrics.htm, accessed Dec 3, 2010
The task description is the outcome being assessed or the instructions a student is provided for an assignment. The characteristics to be rated are the skills, knowledge or behaviors to be demonstrate by the student. The levels of mastery should be written clear language. An example might be something along the lines of: exemplary, proficient, marginal, unacceptable. Finally, each cell would contain a description of the what is required for each mastery level.
The University of Hawaii at Manoa (1) suggests that there are 6 steps to developing a rubric:
Step 1: Identify what it is you wish to assess.
Step 2: Identify the characteristics you wish to rate. Here, you would detail the skills or knowledge you plan on evaluating, limiting them to those you feel are most critical or important.
Step 3: Identify the levels of mastery: The authors recommend that you use an even number of categories to avoid the middle category being a sort of “catch all” for scoring.
Step 4: Describe each level of mastery for each characteristic (cell). Start by describing the best work you could reasonably expect to receive for that characteristic, and set that as your top category. Determine what would comprise unacceptable work and set that as your bottom category. Finally develop your mid-categories, ensuring that there is no overlap between any of them.
Step 5: Test the rubric. Apply it to an assignment, and share it with colleagues for their input. You also need to determine the minimal work that you would find acceptable for passing. This could be based on an average, a total score, or achieving a score of, say, marginal on every cell. Or, of course, you could set the standard higher than that.
Step 6: review and revise. It takes work to set these up and ensure they measure what you wish to measure. Rubrics also allow us to share grading expectations, which may be of help; for example, consider how a rubric might be used to assess a technique practical examination.
References
1. http://manoa.hawaii.edu/assessment/howto/rubrics.htm, accessed Dec 3, 2010
Subscribe to:
Posts (Atom)
Posts
