House, MD is a great program for discussion about medical ethics, or bioethics. And the scenario I presented in last week’s post is rife with interesting twists and turns. We can see the following: Foreman convinced Thirteen to enter a trial in which he was one of the co-investigators because he is falling in love with her and wants to give her a chance to live a longer life. To do so, he has to manipulate her emotionally, because she was initially not interested in being in the trial. To do so, he arranges to bring Thirteen into contact with a woman suffering a far more advanced case of Huntington’s, meaning he had to manipulate that patient’s schedule. Thirteen catches on to this, and because of bleed-over from a patient she is involved in treating, whose condition has taken its toll on his family, she decides to end her relationship with Foreman. When she returns for her next treatment, she finds the same woman patient she had seen before, but very much improved. She accuses Foreman of bringing her to manipulate her, but Foreman lies by telling her he did not know what drug the other woman was getting, was not in charge of scheduling, and did not do it- we later learn he did do all of this. House knows, but House also does nothing about it. And the nurse unblinds the treatment regimen that Thirteen is getting. So we see violations of medical ethics by Foreman, the nurse, and House. Foreman is now aware that Thirteen is not getting the active treatment, but is getting the placebo. This is not going to end well.
All research projects have protocols that dictate how they are to be conducted. And all trials are based on something called clinical equipoise, which is a foundational setting that says that we do not know which of the treatments will have better improvement. If we do not start with this at baseline, ethically we cannot do the trial, because if we know that one treatment is better than another we cannot knowingly enter people into the trial knowing that some will be given a less effective treatment. Part of this also involves ensuring that research participants have an equal chance to be in either groups, and in our informed consent documents, we will tell them what we believe to be the risks and benefits of participation. What we will not do is tell them that they will categorically receive personal benefit; we can’t. Often, the only benefit will be the knowledge that they contributed to scientific information that may benefit others.
When patients believes that their involvement in a trial will personally benefit them, and that the doctors involved are doing everything they can on their behalf, they are said to be suffering from the therapeutic misconception. That is because in a research project, the investigators cannot do everything that would be done were we seeing this person as a patient in our clinic; we are constrained by the project protocols. In Thirteen’s case, her intervention seems to be nothing but the drug, and I do not know what outcomes they are measuring since Thirteen has yet to have any choreaform movements. But certainly, she believes she is receiving help, which is what her smile indicated at the end of the scene.
We have seen coercion used in this scenario. Forman is using his knowledge of his new girlfriend to manipulate her, and to do so he is using a second patient to show her how effective the experimental drug is. He has knowingly broken the scheduling log, and was caught by House, who did nothing as yet about it. The nurse should be taken to task as well for breaking the blinding. And these are only some of the issues involved. When we conduct research here at our PCCR, these issues are discussed and resolved long before we ever begin entering patients into our trials. Princeton Plainsboro (House’s hospital) certainly should know better.
Monday, January 26, 2009
Tuesday, January 20, 2009
Bioethics on Television: The Case of House, M.D.
I am about half-way through a master’s degree in biomedical ethics and public health, and I have an abiding interest in how ethics issues are portrayed in media. One of television’s most popular programs is House, M.D. It features the misanthrope Dr. Gregory House, who is brilliant but who pays utterly no attention to social conventions or rules of human behavior in his zeal to solve the difficult medical cases that he sees each week. This includes the rules of medical ethics that all institutions have to live by. So each week House does what he thinks necessary in order to diagnose the condition; this may include breaking into a patient’s house, treating them against their will, bullying them into compliance or forcing his co-workers to do his dirty deeds.
Last night’s episode contained a number of issues that are illustrative, but in this case they do not actually involve House himself, but the people who work for him and who have been influenced by him whether they wish to admit it or not. To set the stage, let me note that House’s colleague Dr. Foreman has begun a new relation with House’s latest hire, Dr. Remy Hadley, better known by her nickname “Thirteen.” Recently, Thirteen was positively diagnosed with Huntington’s Chorea, which has not yet begun to manifest; however, she understands it is a death sentence and has begun acting badly as a result. With Foreman’s coaching and after facing death from a deranged patient, she has decided to live better. Thus, she accepts Foreman’s suggestion that she enter a trial of a new drug for her disease, which is being held at their hospital, and which Dr. Foreman is one of the doctors overseeing the trial. In the course of showing up for treatment, she has befriended a woman whose Huntington’s disease is much more advanced, though the constant reminder of what is to come troubles her. As a result, Foremen rescheduled the patient so that they did not have coincident visits.
Which sets the stage for yesterday’s developments. In the episode, Thirteen tells Foreman she cannot see him again. She does not wish, in her words, to bring him down with her as she worsens (the case they are seeing outside of the trial involves a man in constant pain and the effect it has on his family). When she returns for her own treatment in the trial, the patient who had been rescheduled is there again, but she is greatly improved. She is obviously getting the experimental drug and it is working. Thirteen therefore accuses Foreman of setting this up in attempt to get her to remain in the relation. Foreman points out that it is a double blind trial, he does not do the scheduling, and he could not know whether or not this patient is getting the active drug or placebo. Thirteen is mollified and even happy to hear this.
Finally, to the point. Thirteen is in treatment again, and Foreman is there helping to fix a small leak in her IV bottle. They agree to go out that night. When Foreman leaves the room, the nurse outside asks him how he can stand the smell. He notes that he does not smell anything at all. The nurse then states that, in that case the patient (Thirteen) must be getting the placebo because the active drug smells very bad. We see Thirteen sitting inside, alone, but smiling in the thought of a successful treatment and a date to come. And then we see House pull Foreman aside, and tell him that he looked over the treatment logs and saw that Foreman had indeed switched the scheduling and manipulated the other patient in order to maintain his new and growing relation with Thirteen. And now Foreman knows that the woman he is growing to love is getting the placebo. And he is involved with the trial.
So, what violations have occurred here? What is wrong in the picture I have painted? Please feel free to send comments in here if you would like. Next post I will provide some answers, but for now, have at it, please.
Last night’s episode contained a number of issues that are illustrative, but in this case they do not actually involve House himself, but the people who work for him and who have been influenced by him whether they wish to admit it or not. To set the stage, let me note that House’s colleague Dr. Foreman has begun a new relation with House’s latest hire, Dr. Remy Hadley, better known by her nickname “Thirteen.” Recently, Thirteen was positively diagnosed with Huntington’s Chorea, which has not yet begun to manifest; however, she understands it is a death sentence and has begun acting badly as a result. With Foreman’s coaching and after facing death from a deranged patient, she has decided to live better. Thus, she accepts Foreman’s suggestion that she enter a trial of a new drug for her disease, which is being held at their hospital, and which Dr. Foreman is one of the doctors overseeing the trial. In the course of showing up for treatment, she has befriended a woman whose Huntington’s disease is much more advanced, though the constant reminder of what is to come troubles her. As a result, Foremen rescheduled the patient so that they did not have coincident visits.
Which sets the stage for yesterday’s developments. In the episode, Thirteen tells Foreman she cannot see him again. She does not wish, in her words, to bring him down with her as she worsens (the case they are seeing outside of the trial involves a man in constant pain and the effect it has on his family). When she returns for her own treatment in the trial, the patient who had been rescheduled is there again, but she is greatly improved. She is obviously getting the experimental drug and it is working. Thirteen therefore accuses Foreman of setting this up in attempt to get her to remain in the relation. Foreman points out that it is a double blind trial, he does not do the scheduling, and he could not know whether or not this patient is getting the active drug or placebo. Thirteen is mollified and even happy to hear this.
Finally, to the point. Thirteen is in treatment again, and Foreman is there helping to fix a small leak in her IV bottle. They agree to go out that night. When Foreman leaves the room, the nurse outside asks him how he can stand the smell. He notes that he does not smell anything at all. The nurse then states that, in that case the patient (Thirteen) must be getting the placebo because the active drug smells very bad. We see Thirteen sitting inside, alone, but smiling in the thought of a successful treatment and a date to come. And then we see House pull Foreman aside, and tell him that he looked over the treatment logs and saw that Foreman had indeed switched the scheduling and manipulated the other patient in order to maintain his new and growing relation with Thirteen. And now Foreman knows that the woman he is growing to love is getting the placebo. And he is involved with the trial.
So, what violations have occurred here? What is wrong in the picture I have painted? Please feel free to send comments in here if you would like. Next post I will provide some answers, but for now, have at it, please.
Monday, January 12, 2009
Evidence-Based Chiropractic
Over the past few weeks, I’ve engaged in several discussion and meeting where the concept of evidence-based chiropractic (EBC) has been discussed. The administration at Palmer College has committed to the inclusion of EBC in our DC curriculum, and the ninth trimester research course is largely devoted to a thorough presentation of its concepts. Yet still some confusion remains, and I thought I would take time here to discuss some general concepts involved with EBC.
One the challenges members of the chiropractic profession face is how to respond to a patient that stands outside of the normal day-to-day patient management protocols. What do we do when a patient walks in, say, and says that they have a condition such as brucellosis? Certainly, we’ve likely received little training on the management of brucellosis in our clinical education, yet our patient hopes there is something we can do for him. Likely, we will attempt in some fashion to locate information about the chiropractic management of brucellosis, and using that information attempt to move forward with the treatment. But there is a great deal embodied in what I just said. First, we need to recognize that we are essentially trying to construct a clinical question. That is the first skill we need to develop. Once the question is developed, we need to locate information. That is a second skill. Once we have located good information, we need to appraise it to determine what literature, out of all that we find, is germane to answering our question, and that is yet another skill. We need to understand the data in the reports we read, which is still another skill. And then we need to apply the information and monitor what happens, potentially leading to a new question and a new cycle of information gathering, etc. In a nutshell, EBC is the means to do just that.
All too often, EBC is seen as nothing more than using the results of clinical trials to inform clinical practice. As a result, many people fear that doing so will lead to managed care corporations limiting what chiropractors can do in practice. There is, of course, some merit to this fear, but I would remind readers that chiropractic is not simply whatever 65,000 chiropractors say it is (some use crystal waving, for example, but I ask, is that really chiropractic?). And we are all aware that clinical guidelines are often developed using clinical trial results; that is, after all, the entire reason for the Cochrane Collaboration. But this is not what EBC is.
David Sackett described evidence-based medicine as “the conscientious, explicit, and judicious use of current best evidence in making decisions about the care of individual patients. The practice of evidence-based medicine means integrating individual clinical expertise with the best available external clinical evidence form systematic research.” (1) To this has been added the need to take into account patient values as well.
There are a number of points to note here: (1) Clinical expertise is an important part of EBC. In chiropractic, often what we have is significant clinical expertise, but little evidence from clinical trials. (2) Best evidence. This does not necessarily mean “clinical trials.” Often, such information does not exist, but other forms of information, perhaps from lower on the evidence hierarchy, exists. We can use that information in EBC. (3) Patient values come to the fore, and are respected. Why do patients seek chiropractic care and what do they expect from that care? This is taken into account in EBC.
All EBC really is, is a set of tools. These tools help a clinician develop a clinical question, perhaps using a PICO format (patient, intervention, comparison, outcome), develop a literature search, locate and appraise literature, and then apply the information. These tools can be sophisticated, but in the end, they can be easily be learned. As we move forward with implementing our R25 grant, we are working to help provide the skills necessary to use these tools.
References
1. Sackett DL Rosenburg WMC, Muir Gray JM, Haynes RB, Richardson WS. Evidence-based medicine: what it is and what it isn’t. Br Med J 1996;312:71-72
One the challenges members of the chiropractic profession face is how to respond to a patient that stands outside of the normal day-to-day patient management protocols. What do we do when a patient walks in, say, and says that they have a condition such as brucellosis? Certainly, we’ve likely received little training on the management of brucellosis in our clinical education, yet our patient hopes there is something we can do for him. Likely, we will attempt in some fashion to locate information about the chiropractic management of brucellosis, and using that information attempt to move forward with the treatment. But there is a great deal embodied in what I just said. First, we need to recognize that we are essentially trying to construct a clinical question. That is the first skill we need to develop. Once the question is developed, we need to locate information. That is a second skill. Once we have located good information, we need to appraise it to determine what literature, out of all that we find, is germane to answering our question, and that is yet another skill. We need to understand the data in the reports we read, which is still another skill. And then we need to apply the information and monitor what happens, potentially leading to a new question and a new cycle of information gathering, etc. In a nutshell, EBC is the means to do just that.
All too often, EBC is seen as nothing more than using the results of clinical trials to inform clinical practice. As a result, many people fear that doing so will lead to managed care corporations limiting what chiropractors can do in practice. There is, of course, some merit to this fear, but I would remind readers that chiropractic is not simply whatever 65,000 chiropractors say it is (some use crystal waving, for example, but I ask, is that really chiropractic?). And we are all aware that clinical guidelines are often developed using clinical trial results; that is, after all, the entire reason for the Cochrane Collaboration. But this is not what EBC is.
David Sackett described evidence-based medicine as “the conscientious, explicit, and judicious use of current best evidence in making decisions about the care of individual patients. The practice of evidence-based medicine means integrating individual clinical expertise with the best available external clinical evidence form systematic research.” (1) To this has been added the need to take into account patient values as well.
There are a number of points to note here: (1) Clinical expertise is an important part of EBC. In chiropractic, often what we have is significant clinical expertise, but little evidence from clinical trials. (2) Best evidence. This does not necessarily mean “clinical trials.” Often, such information does not exist, but other forms of information, perhaps from lower on the evidence hierarchy, exists. We can use that information in EBC. (3) Patient values come to the fore, and are respected. Why do patients seek chiropractic care and what do they expect from that care? This is taken into account in EBC.
All EBC really is, is a set of tools. These tools help a clinician develop a clinical question, perhaps using a PICO format (patient, intervention, comparison, outcome), develop a literature search, locate and appraise literature, and then apply the information. These tools can be sophisticated, but in the end, they can be easily be learned. As we move forward with implementing our R25 grant, we are working to help provide the skills necessary to use these tools.
References
1. Sackett DL Rosenburg WMC, Muir Gray JM, Haynes RB, Richardson WS. Evidence-based medicine: what it is and what it isn’t. Br Med J 1996;312:71-72
Monday, January 5, 2009
An Overview of Research Methodologies
While I think we are all familiar with the general concepts of quantitative and qualitative research, we may be less familiar with key methodologies that are used in each of these forms of scientific research. One of the more heartening things I have seen here at Palmer is a growth in research conducted by our faculty; we have seen papers from our faculty presented at conferences such as the Association of Chiropractic Colleges/Research Agenda Conference, the annual conference of the American Public Health Association, and the World Federation of Chiropractic. In this blog entry, I’d like to provide a brief overview of several forms of research methodology.
Historical Research: This involves the investigation and study of past events. Generally this is done in order to bring better understanding to current events or in anticipation of future events. For those who do historical research, they will look for either primary sources (such as eyewitness reports or review of original documents) or through secondary sources (second-hand information, from, say, a friend of an eyewitness). In addition, it becomes necessary to assess the faithfulness of the information and information source. At present, for example, we see some concern raised about the inability of historians to review documents from the current administration as it leaves office. Historical research can be used to look at current issues, investigate individuals (such as D. D. Palmer), institutions or even movements (straight chiropractic, for example). Our profession does have an organization dedicated to our history and to historical researh within chiropractic, the Association for the History of Chiropractic.
Descriptive Research: Here, we gather data in order to test a hypothesis or answer a question. We might ask Palmer faculty about their understanding of, say, the IRB process. Typically, the information is gathered via survey questionnaires or through interviews, and it is sometimes best to use more than one method, in a process called triangulation. Though survey research sounds simple enough- write questions, administer, analyze- it is actually a complicated process involving question development and testing, identifying populations to survey and methodologies to reach that population, and then tracking response rates and finally doing the analysis. However, this is a form of research that faculty may find valuable in their own disciplines.
Correlational Research: Here, our interest is in determining relationships between 2 variables in our data. And then we try to determine its direction and its strength. For example, we might develop a study where we look to see if the presence of a short leg relates to the presence of low back pain. To do so, we’d first need to find a method for determining whether a short leg is present, and once we do that, seeing if the population of those with short legs has a higher rate of low back pain than the population of those without short leg. Typically, the statistical test used here is the Pearson product moment correlation coefficient. This test ranges from -1.0 (perfect negative correlation) to +1.0 (perfect positive correlation), so the closer to +1.0 we get, the more we can feel comfortable that the two variables are linked.
Experimental Research: Here, we control the conditions for the events we are interested in. A randomized trial is a clear example of experimental research; one group is a control group which does not get the experimental intervention (whether therapeutic for biomedical research, or educational for educational research) and one group gets the intervention. As a reminder, the variable we control is the independent variable and the one that changes as a result of this is the dependent variable. Issues we need to consider in experimental research are various forms of bias, and threats to internal and external validity.
Causal Comparative Research: This form of research is done to establish cause-effect relations. However, here, the independent variable cannot be manipulated, because it has already happened (for example, past drug abuse), and so we look only to the dependent variable (say, your grade in school). The challenge here is that confounding is always potentially present.
Action Research: This is a form of research quite positively suited for educational research. In education, action research has been defined as “systematic inquiry by practitioners to improve teaching and learning.” While hard to explain, in action research we gather data and then act upon it while monitoring its effects. The researcher does this while acting, for example, in the workplace, (i.e., the classroom).
As always, we need the best tool for our “problem.” We need to determine what our research question is, and in doing so, the proper research methodol will typically reveal itself. Each has its own challenges, weaknesses and strengths, but proper choice goes a long way toward improving rigor and answering the questions in which we are interested.
I welcome you all back from the holiday break and wish you all a very happy new year.
Historical Research: This involves the investigation and study of past events. Generally this is done in order to bring better understanding to current events or in anticipation of future events. For those who do historical research, they will look for either primary sources (such as eyewitness reports or review of original documents) or through secondary sources (second-hand information, from, say, a friend of an eyewitness). In addition, it becomes necessary to assess the faithfulness of the information and information source. At present, for example, we see some concern raised about the inability of historians to review documents from the current administration as it leaves office. Historical research can be used to look at current issues, investigate individuals (such as D. D. Palmer), institutions or even movements (straight chiropractic, for example). Our profession does have an organization dedicated to our history and to historical researh within chiropractic, the Association for the History of Chiropractic.
Descriptive Research: Here, we gather data in order to test a hypothesis or answer a question. We might ask Palmer faculty about their understanding of, say, the IRB process. Typically, the information is gathered via survey questionnaires or through interviews, and it is sometimes best to use more than one method, in a process called triangulation. Though survey research sounds simple enough- write questions, administer, analyze- it is actually a complicated process involving question development and testing, identifying populations to survey and methodologies to reach that population, and then tracking response rates and finally doing the analysis. However, this is a form of research that faculty may find valuable in their own disciplines.
Correlational Research: Here, our interest is in determining relationships between 2 variables in our data. And then we try to determine its direction and its strength. For example, we might develop a study where we look to see if the presence of a short leg relates to the presence of low back pain. To do so, we’d first need to find a method for determining whether a short leg is present, and once we do that, seeing if the population of those with short legs has a higher rate of low back pain than the population of those without short leg. Typically, the statistical test used here is the Pearson product moment correlation coefficient. This test ranges from -1.0 (perfect negative correlation) to +1.0 (perfect positive correlation), so the closer to +1.0 we get, the more we can feel comfortable that the two variables are linked.
Experimental Research: Here, we control the conditions for the events we are interested in. A randomized trial is a clear example of experimental research; one group is a control group which does not get the experimental intervention (whether therapeutic for biomedical research, or educational for educational research) and one group gets the intervention. As a reminder, the variable we control is the independent variable and the one that changes as a result of this is the dependent variable. Issues we need to consider in experimental research are various forms of bias, and threats to internal and external validity.
Causal Comparative Research: This form of research is done to establish cause-effect relations. However, here, the independent variable cannot be manipulated, because it has already happened (for example, past drug abuse), and so we look only to the dependent variable (say, your grade in school). The challenge here is that confounding is always potentially present.
Action Research: This is a form of research quite positively suited for educational research. In education, action research has been defined as “systematic inquiry by practitioners to improve teaching and learning.” While hard to explain, in action research we gather data and then act upon it while monitoring its effects. The researcher does this while acting, for example, in the workplace, (i.e., the classroom).
As always, we need the best tool for our “problem.” We need to determine what our research question is, and in doing so, the proper research methodol will typically reveal itself. Each has its own challenges, weaknesses and strengths, but proper choice goes a long way toward improving rigor and answering the questions in which we are interested.
I welcome you all back from the holiday break and wish you all a very happy new year.
Monday, December 15, 2008
Portfolios in Chiropractic Education
Part of our strategy as teachers in chiropractic educational settings is to encourage our students to take greater responsibility for their own learning, and where possible to personalize their learning experiences. For example, in the clinical bioethics course I teach to graduate students in clinical research, students use portfolios in part to collect articles and other ephemera that they themselves are specifically interested in. There is no caveat on what those articles might be; they simply have to reflect the student’s interests. Of course, as a means of assessment, any tool we use should enhance and support learning as well as be able to measure performance (1). One method that is gaining greater use is the student portfolio.
A portfolio is “a collection of papers and other forms of evidence that learning has taken place” (2) or “a collection of student work that exhibits the student’s efforts, progress and achievements in one or more areas” (3). Portfolios have been introduced into public school education as early as elementary school, and have found homes within professional nursing and medical education.
Friedman Ben David et al (4) note that portfolios contain student work over time demonstrating evidence for learning and progress toward educational outcomes or learning objectives. A portfolio may contain any of the following: best essays, written reports or research projects, samples of performance evaluations, videotapes of interactions with patients, records of practical procedures, letters of recommendation, annotated patient records, written reflections, journal articles, etc.
They further note that portfolios can contribute quite well to assessment. They do allow for assessment of learning outcomes, even to the point of being an effective method for examining outcomes not easily addressed by other means, such as personal growth and reflective ability. A portfolio can provide evidence of performance, which can come from numerous sources. That evidence is collected over a period of time, thus demonstrating development over time. It can track progress toward the learning outcomes as well. Finally, it has use for both summative and formative assessment.
Other benefits include the fact that portfolios may enhance interactions between students and teachers, by allowing dialogue between students and educators, by reminding students that learning is a two-way process, by stimulating teachers to reassess their teaching strategies, and by raising teacher expectation in relation to thinking ability and problem solving. Further, portfolios can enhance the use of reflective strategies on the part of the student.
As I noted above, I require my clinical bioiethics students to keep a portfolio. In it, they place their responses to a series of clinical ethics scenarios which they must address and analyze. They keep journal articles that they use for preparing a short classroom presentation, and they also keep other journal articles that they collect regarding their own personal interests. They can place newspaper articles which they find over the course of the term. They will add their final paper to the portfolio. They are encouraged to ask me to review it at any time, but they are not required to have me do so until the very end of the term. No assignment has a specific due date, except that all my students know the date at which the portfolio is to be turned in with all work completed. This allows them to allocate their time as they see fit. It has been a great success for me to see the quality of the work they do. I recommend that those interested review the references below and consider the use of the portfolio in class.
This will be my final post of the year. We are all getting ready to go on winter break, and I am doing so as well. I want to wish you all a great holiday break, good time with family and friends, and I will be back when 2009 begins.
References
1. Koretz D, Broadfoot P, Wolf A. Editorial. Assessment in Education, 1998;5(3)
2. Davis MH, Friedman Ben David M, Harden RM, et al. Portfolio assessment in medical students’ final examinations. Med Teacher 2001;23:357-366
3. Martin-Kneipp GO. Becoming a better teacher. Alexandria, VA; ASCP, 2000
4. Friedman Ben David M, Davis MH, Harden RM, Howie PW, Ker J, Pippard MJ. AMEE Medical Education Guide No. 24: portfolios as a method of student assessment. Med Teacher 2001;23:535-552
A portfolio is “a collection of papers and other forms of evidence that learning has taken place” (2) or “a collection of student work that exhibits the student’s efforts, progress and achievements in one or more areas” (3). Portfolios have been introduced into public school education as early as elementary school, and have found homes within professional nursing and medical education.
Friedman Ben David et al (4) note that portfolios contain student work over time demonstrating evidence for learning and progress toward educational outcomes or learning objectives. A portfolio may contain any of the following: best essays, written reports or research projects, samples of performance evaluations, videotapes of interactions with patients, records of practical procedures, letters of recommendation, annotated patient records, written reflections, journal articles, etc.
They further note that portfolios can contribute quite well to assessment. They do allow for assessment of learning outcomes, even to the point of being an effective method for examining outcomes not easily addressed by other means, such as personal growth and reflective ability. A portfolio can provide evidence of performance, which can come from numerous sources. That evidence is collected over a period of time, thus demonstrating development over time. It can track progress toward the learning outcomes as well. Finally, it has use for both summative and formative assessment.
Other benefits include the fact that portfolios may enhance interactions between students and teachers, by allowing dialogue between students and educators, by reminding students that learning is a two-way process, by stimulating teachers to reassess their teaching strategies, and by raising teacher expectation in relation to thinking ability and problem solving. Further, portfolios can enhance the use of reflective strategies on the part of the student.
As I noted above, I require my clinical bioiethics students to keep a portfolio. In it, they place their responses to a series of clinical ethics scenarios which they must address and analyze. They keep journal articles that they use for preparing a short classroom presentation, and they also keep other journal articles that they collect regarding their own personal interests. They can place newspaper articles which they find over the course of the term. They will add their final paper to the portfolio. They are encouraged to ask me to review it at any time, but they are not required to have me do so until the very end of the term. No assignment has a specific due date, except that all my students know the date at which the portfolio is to be turned in with all work completed. This allows them to allocate their time as they see fit. It has been a great success for me to see the quality of the work they do. I recommend that those interested review the references below and consider the use of the portfolio in class.
This will be my final post of the year. We are all getting ready to go on winter break, and I am doing so as well. I want to wish you all a great holiday break, good time with family and friends, and I will be back when 2009 begins.
References
1. Koretz D, Broadfoot P, Wolf A. Editorial. Assessment in Education, 1998;5(3)
2. Davis MH, Friedman Ben David M, Harden RM, et al. Portfolio assessment in medical students’ final examinations. Med Teacher 2001;23:357-366
3. Martin-Kneipp GO. Becoming a better teacher. Alexandria, VA; ASCP, 2000
4. Friedman Ben David M, Davis MH, Harden RM, Howie PW, Ker J, Pippard MJ. AMEE Medical Education Guide No. 24: portfolios as a method of student assessment. Med Teacher 2001;23:535-552
Monday, December 8, 2008
Relative Risk vs. Risk Difference
It is necessary to distinguish between relative risk (RR) and the risk difference (RD), because the relative risk is typically much larger than then the risk difference so that when you read results in the form of a RR it can be rather misleading. Think about reading a statement that says something has reduced the patient’s risk by 50%. This sounds quite good. However, if you think about it, it might mean nothing more than you reduced the risk from, say, 2% to 1%.
More clinically, imagine that there are three subpopulations of patients, each getting the same treatment, and for each group we see a decrease in risk of 1/3 (that is, RRR= 0.33; RR= 0.67). If this was administered to a subpopulation that has a 30% risk of dying, getting the treatment reduces that risk to 20%. When it is given to a subpopulation that has a 10% risk of dying, the treatment reduces the risk to 6.7%. And when given to a subpopulation that was just a 1% risk of dying, the treatment reduces the risk to 0.67%.
In each group, the treatment reduces the risk of dying by a third, but it really fails to adequately express the full impact of treatment. What if this treatment had a high rate of serious adverse events? Would it make much sense to offer it to the patients in the lowest risk group (where the RD is only 0.3%)? For the middle group, we’d certainly want to let them know about the risks and benefits of treatment, inasmuch as the as the absolute reduction in risk is about 3%. In the high risk group, it makes complete sense to offer this treatment, because there is an absolute benefit of 10%.
The authors of the Users’ Guide suggest that the relative risk reduction be examined in light of your assessment of your patient’s baseline risk. Obviously, you need to have information on the risk reduction of a therapy, which is not always available for chiropractic interventions but could be calculated as per our last blog post. If your know your therapy offers a relative risk reduction of 30% in a general group of patients, you might view it differently if one patient is 40 years old and in generally good health compared to one who is 70 years old and has significant co-morbidity.
This is not the easiest concept to grasp. However, the Users’ Guide offers a great resource in understanding this (1). Further information can be found at http://www.annalsofian.org/temp/AnnIndianAcadNeurol104225-401063_110826.pdf
NOTE: As I was home over the weekend, I came across this article in the Washington Post: Howell D. Making sense of science reporting (2). In it, the Washington Post Ombudsman, Deborah Howell, defends the nature of science reporting in her newspaper, but offers the following comment regarding one such article: “FIAR Director George M. Carter's chief complaint was that stories emphasized a change in "relative risk" -- a 44 percent fall in the number of heart attacks, strokes and surgical procedures among people taking the statin, compared with those in the placebo group. He said the fact that everyone in the study had an extremely low "absolute risk" for heart problems should have been emphasized more. About 1.36 percent of people taking the placebo suffered a heart attack or stroke; that fell to 0.8 percent among those taking the statin. That means that nearly 97 percent of the people using the drug would not see any benefit, he said.”
The article goes on to note the author of the news report “quoted a skeptic in the ninth paragraph and noted near the story's end that ‘the actual risk reduction for an individual would be very small, given the relatively low risk for most middle-age people, so that the benefits easily could be outweighed by the costs of thousands more people taking tests, drugs and being monitored by doctors.’”
Finally, the author said "While I would have liked to have explored many of the nuances of this study more fully, I feel confident we struck a responsible balance. I think it's crucial to provide readers with both the evidence supporting new claims and enough context and interpretation to help them gauge its significance." Independent experts, he said, concluded the study was "a very well done, very convincing piece of research."
Which, of course, was not the point. It is only that the average reader cannot draw this distinction between the absolute and relative risk reductions. That much remains clear, and points to why we need to better understand such concepts.
References
1. Guyatt G, Rennie D, Meade MO, Cook DJ. Users’ guides to the medical literature: a manual for evidence-based practice. New York, NY; McGraw Hill;208:90-91
2. Howell. D. Making sense of science reporting. http://www.washingtonpost.com/wp-dyn/content/article/2008/12/05/AR2008120502959.html?hpid=opinionsbox1
More clinically, imagine that there are three subpopulations of patients, each getting the same treatment, and for each group we see a decrease in risk of 1/3 (that is, RRR= 0.33; RR= 0.67). If this was administered to a subpopulation that has a 30% risk of dying, getting the treatment reduces that risk to 20%. When it is given to a subpopulation that has a 10% risk of dying, the treatment reduces the risk to 6.7%. And when given to a subpopulation that was just a 1% risk of dying, the treatment reduces the risk to 0.67%.
In each group, the treatment reduces the risk of dying by a third, but it really fails to adequately express the full impact of treatment. What if this treatment had a high rate of serious adverse events? Would it make much sense to offer it to the patients in the lowest risk group (where the RD is only 0.3%)? For the middle group, we’d certainly want to let them know about the risks and benefits of treatment, inasmuch as the as the absolute reduction in risk is about 3%. In the high risk group, it makes complete sense to offer this treatment, because there is an absolute benefit of 10%.
The authors of the Users’ Guide suggest that the relative risk reduction be examined in light of your assessment of your patient’s baseline risk. Obviously, you need to have information on the risk reduction of a therapy, which is not always available for chiropractic interventions but could be calculated as per our last blog post. If your know your therapy offers a relative risk reduction of 30% in a general group of patients, you might view it differently if one patient is 40 years old and in generally good health compared to one who is 70 years old and has significant co-morbidity.
This is not the easiest concept to grasp. However, the Users’ Guide offers a great resource in understanding this (1). Further information can be found at http://www.annalsofian.org/temp/AnnIndianAcadNeurol104225-401063_110826.pdf
NOTE: As I was home over the weekend, I came across this article in the Washington Post: Howell D. Making sense of science reporting (2). In it, the Washington Post Ombudsman, Deborah Howell, defends the nature of science reporting in her newspaper, but offers the following comment regarding one such article: “FIAR Director George M. Carter's chief complaint was that stories emphasized a change in "relative risk" -- a 44 percent fall in the number of heart attacks, strokes and surgical procedures among people taking the statin, compared with those in the placebo group. He said the fact that everyone in the study had an extremely low "absolute risk" for heart problems should have been emphasized more. About 1.36 percent of people taking the placebo suffered a heart attack or stroke; that fell to 0.8 percent among those taking the statin. That means that nearly 97 percent of the people using the drug would not see any benefit, he said.”
The article goes on to note the author of the news report “quoted a skeptic in the ninth paragraph and noted near the story's end that ‘the actual risk reduction for an individual would be very small, given the relatively low risk for most middle-age people, so that the benefits easily could be outweighed by the costs of thousands more people taking tests, drugs and being monitored by doctors.’”
Finally, the author said "While I would have liked to have explored many of the nuances of this study more fully, I feel confident we struck a responsible balance. I think it's crucial to provide readers with both the evidence supporting new claims and enough context and interpretation to help them gauge its significance." Independent experts, he said, concluded the study was "a very well done, very convincing piece of research."
Which, of course, was not the point. It is only that the average reader cannot draw this distinction between the absolute and relative risk reductions. That much remains clear, and points to why we need to better understand such concepts.
References
1. Guyatt G, Rennie D, Meade MO, Cook DJ. Users’ guides to the medical literature: a manual for evidence-based practice. New York, NY; McGraw Hill;208:90-91
2. Howell. D. Making sense of science reporting. http://www.washingtonpost.com/wp-dyn/content/article/2008/12/05/AR2008120502959.html?hpid=opinionsbox1
Monday, December 1, 2008
Risk, Risk Reducation and Relative Risk
One of the more common reasons we read clinical trial literature is look at the association between a given treatment, such as spinal adjustment, and the outcome of that treatment (for example, reduced pain or disability). But often we do not delve with any depth into the results, and the concept of relative risk is one that we should be looking at as we assess the outcomes for a research paper.
Let us say we have a paper in which in the course of a randomized trial we are comparing a dichotomous outcome (for example, mortality rates) in patients undergoing either Treatment A or Treatment B. And further, let us say that 36 of 128 patients in Treatment group A died, while 58 of 130 in Treatment group B died. From this, we can construct what is called a 2x2 contingency table (here referring to the columns “Death” and Survival” and the rows “Treatment A” and “Treatment B,” in which we put exposure on the left and the outcome across the top. When we do, we will see that in Treatment Group A, 36 people died and 92 survived, and we will call these "a" and "b"; in Treatment group B, 58 died and 72 survived, and we will label these "c" and "d."
This now allows us to be able to calculate risk. First, we can look at the absolute risk for mortality in both groups. This is defined as the risk of the adverse event in each group, sometimes called baseline risk. So, the risk of mortality in Treatment group A is the number of people who died divided by the total number of people in the group: a/(a+b) = 36/(36+92) = 36/128 = .28 or 28%, and in Treatment group B = 58/130 = .45, or 45%.
The difference between these rates is known as the risk difference, or absolute risk reduction. That is, if we compare these risks, we can see that a patient is less likely to die if they have treatment A. The absolute risk reduction is simply the difference between groups (difference meaning subtraction is involved) = c/(c+d) – a/(a+b). Here, that is .45-.28= .17, or 17%.
Finally, rather than look at the difference between groups, we can also look at the ratio; ratio implies that we will be dividing the rates, rather than subtracting them. When we do this, we calculate the risk ratio, or relative risk. Mathematically, this would be the risk in the Treatment group A divided by the risk in Treatment group B (which is typically a control group), or [a/(a+b) / c/(c+d)]. Here, we can see that the math is .28/.45 = .63, meaning that the risk of death is Treatment group A is about 63% (or about two-thirds) that of Treatment group B.
There are complexities to this assessment, which our next entry will take up. For those who wish to read a bit further on this, let me recommend the “Users’ Guide” as a good resource (1).
References
1. Guyatt G, Rennie D, Meade MO, Cook DJ. Users’ guides to the medical literature: a manual for evidence-based practice. New York, NY; McGraw Hill;208:87-90
Let us say we have a paper in which in the course of a randomized trial we are comparing a dichotomous outcome (for example, mortality rates) in patients undergoing either Treatment A or Treatment B. And further, let us say that 36 of 128 patients in Treatment group A died, while 58 of 130 in Treatment group B died. From this, we can construct what is called a 2x2 contingency table (here referring to the columns “Death” and Survival” and the rows “Treatment A” and “Treatment B,” in which we put exposure on the left and the outcome across the top. When we do, we will see that in Treatment Group A, 36 people died and 92 survived, and we will call these "a" and "b"; in Treatment group B, 58 died and 72 survived, and we will label these "c" and "d."
This now allows us to be able to calculate risk. First, we can look at the absolute risk for mortality in both groups. This is defined as the risk of the adverse event in each group, sometimes called baseline risk. So, the risk of mortality in Treatment group A is the number of people who died divided by the total number of people in the group: a/(a+b) = 36/(36+92) = 36/128 = .28 or 28%, and in Treatment group B = 58/130 = .45, or 45%.
The difference between these rates is known as the risk difference, or absolute risk reduction. That is, if we compare these risks, we can see that a patient is less likely to die if they have treatment A. The absolute risk reduction is simply the difference between groups (difference meaning subtraction is involved) = c/(c+d) – a/(a+b). Here, that is .45-.28= .17, or 17%.
Finally, rather than look at the difference between groups, we can also look at the ratio; ratio implies that we will be dividing the rates, rather than subtracting them. When we do this, we calculate the risk ratio, or relative risk. Mathematically, this would be the risk in the Treatment group A divided by the risk in Treatment group B (which is typically a control group), or [a/(a+b) / c/(c+d)]. Here, we can see that the math is .28/.45 = .63, meaning that the risk of death is Treatment group A is about 63% (or about two-thirds) that of Treatment group B.
There are complexities to this assessment, which our next entry will take up. For those who wish to read a bit further on this, let me recommend the “Users’ Guide” as a good resource (1).
References
1. Guyatt G, Rennie D, Meade MO, Cook DJ. Users’ guides to the medical literature: a manual for evidence-based practice. New York, NY; McGraw Hill;208:87-90
Subscribe to:
Posts (Atom)
Posts
