Monday, January 23, 2012

Developing Survey Questions

A good many of our faculty have used surveys in their classroom. Each time they do, time must be spent constructing the questions that are asked, and there is, as might be expected, a great deal to take into account when writing those questions. Here are issues to consider when writing survey questions.

1. Wording items: Sometimes the answers to your question depend on whose opinions are under consideration. Nardi notes that “if you ask respondents to agree or disagree with ‘Merit raises should be eliminated for all workers” you will get a different answer that if you asked “I feel that merit raises should be eliminated for all workers.” His point is that the first construction is more about a general belief while the second is much more personal and asks the person to consider for him or herself the answer. This suggests that as you write questions, you would mist in an occasional item asked in a different way and compare its results to other similar questions. You should, however, generally stick to writing either with “I” or “You” but not switch later to more general questions. A second suggestion is to avoid negatives in sentences, because for some people they will not know whether agreeing with a negative sentence means they are disagreeing with it.

2. Statement directions; Mix the direction to your statements so that not all the answers for a specific set of opinions lead to “agree or all lead to “disagree.” You should word some questions sot that people must disagree with some and agree with others. An example might be to ask “Staying up late the night before a test helps a lot,” later followed by “Getting a good night’s rest before a test is helpful.” You could not really agree with both of these. Mixing the direction removes what is known as response bias, which occurs when people simply answer most questions the same way by checking, say, “disagree” for all.

3. Always and never: avoid the use of these words; people e rarely always or never feel something about a statement. It is better to phrase such question using choices such as “most of the time” or “infrequently.”

4. Double-barreled items: This is asking a question that actually measures two things at the same time. Such questions often include the word “and.” Consider this: “Do you like ham and eggs?” How do you answer this if you like one, but not the other. Avoid these.

5. Leading questions: You can consciously or unconsciously allow your own personal biases to creep into your survey questions. If you ask “Do you agree that everyone should undergo drug testing on our campus?” you are leading people in a particular direction and suggesting they should agree with you. You should rewrite this as a statement, “Everyone should undergo drug testing on our campus” and include it is a set of questions that have a range of view points.

This is but a small amount of information on proper question development. Taking these issues into account will help give you richer and more meaningful data when you conduct surveys.

References
1. Nardi PM. Doing survey research: a guide to qualitative methods. Boston, MA: Pearson Education, Inc., 2003

Tuesday, January 17, 2012

Qualitative Research

I return to the conduct of qualitative research. Such research has two main differences from the more well-understood quantitative research: (1) it focuses on social and interpreted, rather than quantifiable, phenomenon, and (2) it aims to discover, describe, and understand, rather than to test and evaluate. It looks at very different questions than does quantitative research. Here is one example: consider a project that wishes to examine changes in pain intensity among a group of patients with low back pain. We can, using quantitative methods, collect baseline pain readings using an instrument such as a Numerical Rating Scale (pain rated on a scale of 0-10, with 0 being no pain and 10 be the worst pain imaginable. We can then collect follow-up ratings 2 weeks after treatment. Each person provides their own self-rated pain measures. Patient #1 might have an initial score of 8 and a follow-up score of 4; patient #2 might report the exact same measures. But, does this mean that their experience and perceptions regarding the pain they experience is exactly the same? We cannot know using quantitative methods. Instead, we might conduct a corollary project in which we interview a select group of patients, so that we can have a better sense and understanding of their lived experience with pain. We would have descriptive information to analyze: words, and text. As a result those who conduct qualitative research rarely discuss validity; instead, they discuss credibility.


There are a series of questions one can use to read and interpret papers presenting qualitative research. According to the Users’ Guides to the Medical Literature (1), questions to ask include:

Is qualitative research relevant? Is my question about social, rather than biomedical, phenomenon? Do I want theoretical or conceptual understanding of the problem?

Are the results credible? Was the choice of participants explicit and comprehensive? Was ethics approval received? Was data collection comprehensive and detailed?

What are the results?

How can I apply the results to patient care? Does the study offer helpful theory? Does it help me understand the context of my practice? Does it help me understand social interactions in clinical care?

These are all good questions through which to view the methods and results of a qualitative study.

References
1. Guyatt G, Rennie D, Meade MO, Cook DJ. Users’ guides to the medical literature, 2nd edition. New York City, NY; McGraw Hill, 2008:341-360

Tuesday, January 10, 2012

Three New Articles

Ritenbaugh C, Nichter M, Kelly KL, et al. Developing a Patient-Centered Outcome Measure for Complementary and Alternative Medicine Therapies I: Defining Content and Format. BMC Complementary and Alternative Medicine 2011, 11:135

ABSTRACT

Background: Patients receiving complementary and alternative medicine (CAM) therapies often report shifts in well-being that go beyond resolution of the original presenting symptoms. We undertook a research program to develop and evaluate a patient-centered outcome measure to assess the multidimensional impacts of CAM therapies, utilizing a novel mixed methods approach that relied upon techniques from the fields of anthropology and psychometrics. This tool would have broad applicability, both for CAM practitioners to measure shifts in patients' states following treatments, and conventional clinical trial researchers needing validated outcome measures. The US Food and Drug Administration has highlighted the importance of valid and reliable measurement of patient-reported outcomes in the evaluation of conventional medical products. Here we describe Phase I of our research program, the iterative process of content identification, item development and refinement, and response format selection. Cognitive interviews and psychometric evaluation are reported separately.
Methods: From a database of patient interviews (n=177) from six diverse CAM studies, 106 interviews were identified for secondary analysis in which individuals spontaneously discussed unexpected changes associated with CAM. Using ATLAS.ti, we identified common themes and language to inform questionnaire item content and wording. Respondents' language was often richly textured, but item development required a stripping down of language to extract essential meaning and minimize potential comprehension barriers across populations. Through an evocative card sort interview process, we identified those items most widely applicable and covering standard psychometric domains. We developed, pilot-tested, and refined the format, yielding a questionnaire for cognitive interviews and psychometric evaluation.
Results: The resulting questionnaire contained 18 items, in visual analog scale format, in which each line was anchored by the positive and negative extremes relevant to the experiential domain. Because of frequent informant allusions to response set shifts from before to after CAM therapies, we chose a retrospective pretest format. Items cover physical, emotional, cognitive, social, spiritual, and whole person domains.
Conclusions: This paper reports the success of a novel approach to the development of outcome instruments, in which items are extracted from patients' words instead of being distilled from pre-existing theory. The resulting instrument, focused on measuring shifts in patients' perceptions of health and well-being along pre-specified axes, is undergoing continued testing, and is available for use by cooperating investigators.

Zhang J, Peterson RF, Ozolins IZ. Student approaches for learning in medicine: What does it tell us about the informal curriculum? BMC Medical Education 2011, 11:87 doi:10.1186/1472-6920-11-87

ABSTRACT
Background: It has long been acknowledged that medical students frequently focus their learning on that which will enable them to pass examinations, and that they use a range of study approaches and resources in preparing for their examinations. A recent qualitative study identified that in addition to the formal curriculum, students are using a range of resources and study strategies which could be attributed to the informal curriculum. What is not clearly established is the extent to which these informal learning resources and strategies are utilized by medical students. The aim of this study was to establish the extent to which students in a graduate-entry medical program use various learning approaches to assist their learning and preparation for examinations, apart from those resources offered as part of the formal curriculum.
Methods: A validated survey instrument was administered to 522 medical students. Factor analysis and internal consistence, descriptive analysis and comparisons with demographic variables were completed. The factor analysis identified eight scales with acceptable levels of internal consistency with an alpha coefficient between 0.72 and 0.96.
Results: Nearly 80% of the students reported that they were overwhelmed by the amount of work that was perceived necessary to complete the formal curriculum, with 74.3% believing that the informal learning approaches helped them pass the examinations. 61.3% believed that they prepared them to be good doctors. A variety of informal learning activities utilized by students included using past student notes (85.8%) and PBL tutor guides (62.7%), and being part of self-organised study groups (62.6%), and peer-led tutorials (60.2%). Almost all students accessed the formal school resources for at least 10% of their study time. Students in the first year of the program were more likely to rely on the formal curriculum resources compared to those of Year 2 (p = 0.008).
Conclusions: Curriculum planners should examine the level of use of informal learning activities in their schools, and investigate whether this is to enhance student progress, a result of perceived weakness in the delivery and effectiveness of formal resources, or to overcome anxiety about the volume of work expected by medical programs.

Tschudi-Madsen H, Kjelsdberg M, Natvig B et al. A strong association between non-musculoskeletal symptoms and musculoskeletal pain symptoms: results from a population study. BMC Musculoskeletal Disorders 2011, 12:285 doi:10.1186/1471-2474-12-285

ABSTRACT

Background: There is a lack of knowledge about the pattern of symptom reporting in the general population as most research focuses on specific diseases or symptoms. The number of musculoskeletal pain sites is a strong predictor for disability pensioning and, hence, is considered to be an important dimension in symptom reporting. The simple method of counting symptoms might also be applicable to non-musculoskeletal symptoms, rendering further dimensions in describing individual and public health. In a general population, we aimed to explore the association between self-reported non-musculoskeletal symptoms and the number of pain sites.
Methods: With a cross-sectional design, the Standardised Nordic Questionnaire and the Subjective Health Complaints Inventory were used to record pain at ten different body sites and 13 non-musculoskeletal symptoms, respectively, among seven age groups in Ullensaker, Norway (n = 3,227).
Results: Results showed a strong, almost linear relationship between the number of non-musculoskeletal symptoms and the number of pain sites (r = 0.55). The number and type of non-musculoskeletal symptoms had an almost equal explanatory power in the number of pain sites reported (27.1% vs. 28.2%).
Conclusion: The linear association between the number of non-musculoskeletal and musculoskeletal symptoms might indicate that the symptoms share common characteristics and even common underlying causal factors. The total burden of symptoms as determined by the number of symptoms reported might be an interesting generic indicator of health and well-being, as well as present and future functioning. Research on symptom reporting might also be an alternative pathway to describe and, possibly, understand the medically unexplained multisymptom conditions.

Tuesday, January 3, 2012

A Short List of Good Books for Teachers

Welcome back and happy New Year! As a first post of the new year, here is a short list of excellent resource texts for teachers.

1. Brookfield S. Teaching for critical thinking. San Francisco, CA; Jossey-Bass, 2011

2. Tierney WG. The impact of culture on organizational decision making: theory and practice in higher education. Sterling, VA; Stylus publishing, LLC, 2008

3. Leamnson R. Thinking about teaching and learning: developing habits of learning with first year college and university students. Sterling, VA; Stylus publishing, LLC, 1999

4. Driscoll A, Wood S. Developing outcomes-based assessment for learner-centered education: a faculty introduction. Sterling, VA; Stylus publishing, LLC, 2007

5. Lauer PA. An education research primer: how to understand, evaluate and use it. San Francisco, CA; Jossey-Bass, 2006

6. Dolence MG, Rowley DJ, Lujan HD. Working toward strategic change: a step-by-step guide to the planning process. San Francisco, CA; Jossey-Bass, 1997

7. Stevens DD, Levi AJ. Introduction to rubrics: an assessment tool to save grading time, convey effective feedback and promote student learning. Sterling, VA; Stylus Publishing, LLC, 2005

8. Michaelson LK, Knight AB, Fink LD. Team-based learning: a transformative use of small groups in college teaching. Sterling, VA; Stylus Publishing, LLC, 2004

9. Allen MJ. Assessing academic programs in higher education. Bolton, MA; Anker Publishing, 2004

10. Gillespie KH, ed. A guide to faculty development: practice advice, examples, and resources. Bolton, MA; Anker Publishing, 2002

Monday, December 19, 2011

Holiday Greetings to You

Dear Folks:

Merry Christmas: http://www.youtube.com/watch?v=wJYZtssgSiw&feature=related

Happy Chanukah: http://www.youtube.com/watch?v=jLqG3BqMC8E&feature=related

Happy Kwanza: http://www.youtube.com/watch?v=LvnQQcyPojs&feature=related

Happy New Year! http://www.youtube.com/watch?v=Z3sXVxqDbFk&feature=related

See you in 2012.

Monday, December 12, 2011

The Five Traditions of Critical Thinking

Continuing with the work of Stephen Brookfield (1), he notes that there are five main traditions regarding critical thinking. These are:

1. Analytic philosophy and logic. He notes that this is the most influential tradition informing how we teach and understand critical thinking. It is a process of constructing and deconstructing arguments, and is a daily part of instruction in US schools. It translates as being able to provide reasons for the opinions, conclusions or statements that a student makes. A classic example would be to show the calculations that led to the correct answer of a math problem, not just the answer itself. Brookfield notes that it is necessary for individuals involved in this form of analysis to be conversant in language, because language can be used to mislead and deceive student. To Brookfield, this smacks of propaganda, and it is necessary for students to be able to perceive language tricks- and we see tricks all the time in our literature, particularly our political literature related to chiropractic. We need to teach our students to see such tricks when they occur, so that they will not be so easily misled.

2. Natural science. This is essentially the hypothetic-deductive model and it is this model which informs the scientific method. Here, we posit a hypothesis, and we then create an experiment to test whether or not we get the expected results our hypothesis generates. We end up either corroborating or falsifying the hypothesis; if the latter, we then generate new hypotheses. Initially, this was espoused by Francis Bacon back in the 1600’s, but in science was expanded upon by Karl Popper, who developed the principle of falsifiability. To be scientific is to be falsifiable. And as a result, in this model the critical thinker is one who is open to having his or her opinions revised as new evidence is available. This is, sadly, not always the case with students, whether in chiropractic or not. Knowledge is provisional and always subject to revision.

3. Pragmatism. Brookfield says this about pragmatism: “Pragmatism emphasizes the importance of continual experimentation to bring about better (in pragmatist terms, more beautiful) social forms.” To build a democratic society, we need to experiment and change constantly in order to make democracy work better. The model here is to experiment, learn from mistakes, and deliberately seek out new information and possibilities. The concept of pragmatism arose from early political philosophers such as Ralph Waldo Emerson and John Dewey, though today it means something radically different from how it applies to critical thinking; to be pragmatic is to take advantage of situations which occur so as to benefit yourself. In education, pragmatism involves a willingness to question dogma and to always be open to revision. This is again not always the case in chiropractic education. Many voices need to be heard so that the learner can question their own assumptions in light of others’ point of view. We might apply a pragmatic approach by offering an activity based on a current situation and allowing student to offer their own approach to resolving the situation.

4. Psychoanalysis. This is the least well understood of the five approaches because it delves into psychoanalysis and psychotherapy. And it is seen in courses that put student experience as the central focus of the curriculum, so it is seen more in social work, nursing and general education (obviously, it is central to clinical psychology). It is based on the concept that each us has a core identity that wants to be realized. In higher education, this forms the basis for what is known as transformative learning, associated with Jack Mezirow. And the approach looks to see how childhood inhibitions interfere with our full development as an adult person.

5. Critical theory. This is an overtly political theory, which grew out of a Marxist approach to education, and which is tied to social justice and uncovering power inequities. It has a core set of assumptions: that western societies are unequal and have inherent discrimination (racial, economic, social), that this is seen as normal and inevitable by dissemination of the dominant ideology, and that critical theory helps to understand this so that change can occur. This may sound esoteric, but consider how chiropractic is positioned in American society compared to the dominant ideology of allopathic medicine. It looks for hegemony (such as is the case for medicine) in order to change it and to effect social action. Our students do recognize this. For example, they are cognizant that the progression of disease and how it is treated may be explained by how pharmaceutical companies and insurance companies structure health care. But they may not recognize it in other areas.

I highly recommend a read through Brookfield’s book, which is fascinating to me and I hope will be to you.

References
1. Brookfield S. Teaching for critical thinking. San Francisco, CA: John Wiley and Sons, 2012:1

Monday, December 5, 2011

Teaching for Critical Thinking

In his excellent book “Teaching for Critical Thinking,” Stephen Brookfield introduces the basic process of critical think as involving 4 components: (1) identifying the assumptions underlying our thinking as it relates to our actions, (2) checking out how much these assumptions are accurate, (3) looking at our ideas and decisions from more than a single perspective, and (4) after doing all of this, taking informed action (1). Brookfield states that “if you can’t think critically your survival is in peril because you risk living a life that- without your being aware of it- hurts you and serves the interests of those who wish you harm.” His approach to critical thinking is certainly more than we might consider to be critical thinking; in our case, we often use the term to denote an ability to critically appraise or assess a journal article in order to glean information from it to use for patient management. Or, conversely, we often think of it as the art of tearing a paper apart. We are not always cognizant of the political margins around our thinking and learning.

To Brookfield, the core process of critical thinking is in hunting assumptions. He defines an assumption as a guide to truth embedded in your mental outlook, and he offers as examples how assumptions occur in daily life: the words people use are assumed to have specific meanings, as do the gestures they use. We make assumptions about political candidates and whether or not they are telling us the truth. All day long assumptions are being made by use, and most are held because our experience tells us to hold them- we might wear a sweater based on an assumption made by reading about the weather in the morning newspaper. Assumptions can occur at far deeper levels. Many are linked to dominant ideologies. We might assume a person wearing worn clothing is poor, that the guy in a suit over there is a college administrator, that everyone we meet is heterosexual because that is assumed to be the norm, etc.

When you think critically, you begin to question the assumptions about how problems are defined. Since we are in a chiropractic college, we make assumptions about the value of chiropractic- and we make assumptions about the value of allopathic medicine. And we act as a result. Here is but one example: when we read a newspaper article lauding a recent chiropractic research finding (or if we read the actual paper on which that news release might be based), we often applaud this finding and take it without delving into its rigor. When we read a paper that casts a more negative light on chiropractic, we often look to see if we can point out all its flaws. We assume the good paper is good and the negative one is bad.

When we think critically, we hunt out our assumptions, check them out to see if they are accurate, try to view these from multiple perspectives and only then are we able to take informed action. And this may play out in education every bit as much as others areas of life.

References
1. Brookfield S. Teaching for critical thinking. San Francisco, CA: John Wiley and Sons, 2012:1