The Digital World Revolution

Let me start with this comment: my step-dad, now 81, does not use a computer and never has. This is all the more amazing when you consider that he was a small business owner, a man who ran 2 army-navy surplus stores with close to 6,000 items in the store. He did inventory by hand, using a paper ledger to track sales of items. My brother now manages the store, and as a 49-year-old, he finally computerized the store, installing a tracking system with SKU numbers on all items. His life is easier. For him. Not for my step-dad, necessarily.

I mention this because we are now teaching a generation of students who grew up with digital technology and cannot comprehend a world in which it did not exist. However, email, for example, has been a present factor in our lives only since around 1998. The internet, as we use it, was born sometime around 1990 as an outgrowth of particle physics; Tim Berner-Lees created the hypertext transfer protocol (HTTP) while working at CERN, which is the world’s largest particle physics laboratory. We now have instant communication, which is both a blessing and a curse. Notice how that during your recent 3-day break you still might receive an email message asking you for a response, as if the break means nothing and you are to be available 24 hours per day, 7 days per week?

I remember when the mail I got came in an envelope. I paid (and pay) my bills by writing a check and mailing that payment back to the company. I am seen as archaic, because I do not use an automatic withdrawal and payment via the internet. Outlook organizes our time down to minute increments. We can go to a conference and find that a speaker is scheduled to speak from 12:12-12:21. Outlook allows us to read messages, organize schedules, and so on.

And this is the world of our students. Think of the technology we now have in place- smart podia, PowerPoint slide sets, wikis, websites and web links, Facebook pages, twitter, podcasts, blogs, and students who walk into class with smartphones, tablets and computers. They learn differently than we did; they don’t buy nor read textbooks, since they can find what they need on the web. They don’t subscribe to journals; it is all free all the time on the web. We are challenged to be creative as a result.

It is worth thinking about. We need to change how we teach; we cannot continue to do what we have always done. Our students do not learn the same way we did and we need to recognize this. We have work to do.

More Thoughts on Sensitivity and Specificity

Recall that the sensitivity and specificity of a diagnostic test refers to its ability to either correctly identify people with a condition of interest (sensitivity) or correctly identify those without the condition of interest (specificity). There are some implications to this. The mnemonic SnOUT means “sensitivity rules out.” Translated, this refers to the fact the greater the sensitivity a test has, the better able it is to correctly identify people with a condition of interest, the more likely a negative test truly rules out that condition. And the mnemonic SpIN means “specificity rules in” In other words, the better able a test can rule out condition of interest (the higher its specificity), the more likely a positive test is truly positive.

Think of it this way. Consider a test where there is a cut-off point. Above that cut-off, everyone is said to be positive, or to have the condition of interest; below it, they do not. For example, we might say that everyone above a temperature of 98.6 degrees is “sick,” and everyone below is “not sick.” But we also know that there will be some people who are sick who do not have an elevated temperature (false negative) and some who are above our cut-off point but are not sick at all (false positives). Now, let’s shift the cut-off point. Let’s move the point where we call people “sick” to 101 degrees. One thing is for sure; everyone with a temperature above 101 degrees is pretty much sick. That is, our test now has a very high sensitivity, because with a cut-off of 101 it can correctly identify sick people very well. But, guess what? The specificity falls now- we have a higher percentage of people with temperatures under out cut-off point who really are sick but our test does not identify them very well.

And the reverse happens when we lower our cut-off point. If we drop the temperature to 97 degrees, virtually everyone under 97 will be correctly identified as “not sick,” increasing the test’s specificity. But we will see a decrease in sensitivity because some people will now be identified as “sick” when they really are not. In any test, as we increase sensitivity we decrease specificity, and vice versa. So, we need to decide whether we want to increase sensitivity at the expense of specificity or specificity at the expense of sensitivity.

So, why would this matter? Tests with high sensitivity may be suitable when the consequence of reporting a false positive finding to the patient is minor. For example, we tell them they have high trigylcerides and they shift to a healthier diet. But, tests with high specificity are better when false positive findings could lead to painful or expensive treatments; i.e. a test that leads to a surgical intervention. In this case, false positives should be minimized.

Sensitivity and Specificity: Examples from the Literature

Here are three examples of how sensitivity and specificity are used in the literature.

Kongsted A, Johannesen E, Leboeuf-Yde C. Feasibility of the STarT back screening tool in chiropractic clinics: a cross-sectional study of patients with low back pain. Chiropr Man Therap.2011 Apr 28;19:10.

ABSTRACT

The STarT back screening tool (SBT) allocates low back pain (LBP) patients into three risk groups and is intended to assist clinicians in their decisions about choice of treatment. The tool consists of domains from larger questionnaires that previously have been shown to be predictive of non-recovery from LBP. This study was performed to describe the distribution of depression, fear avoidance and catastrophising in relation to the SBT risk groups. A total of 475 primary care patients were included from 19 chiropractic clinics. They completed the SBT, the Major Depression Inventory (MDI), the Fear Avoidance Beliefs Questionnaire (FABQ), and the Coping Strategies Questionnaire. Associations between the continuous scores of the psychological questionnaires and the SBT were tested by means of linear regression, and the diagnostic performance of the SBT in relation to the other questionnaires was described in terms of sensitivity, specificity and likelihood ratios.In this cohort 59% were in the SBT low risk, 29% in the medium risk and 11% in high risk group. The SBT risk groups were positively associated with all of the psychological questionnaires. The SBT high risk group had positive likelihood ratios for having a risk profile on the psychological scales ranging from 3.8 (95% CI 2.3 - 6.3) for the MDI to 7.6 (95% CI 4.9 - 11.7) for the FABQ. The SBT questionnaire was feasible to use in chiropractic practice and risk groups were related to the presence of well-established psychological prognostic factors. If the tool proves to predict prognosis in future studies, it would be a relevant alternative in clinical practice to other more comprehensive questionnaires.

Leboeuf-Yde C, Rosenbaum A, Axén I, Lövgren PW, Jørgensen K, Halasz L, Eklund A, Wedderkopp N. The Nordic Subpopulation Research Programme: prediction of treatment outcome in patients with low back pain treated by chiropractors--does the psychological profile matter? Chiropr Osteopat. 2009 Dec 30;17:14

ABSTRACT

Background:  It is clinically important to be able to select patients suitable for treatment and to be able to predict with some certainty the outcome for patients treated for low back pain (LBP). It is not known to what degree outcome among chiropractic patients is affected by psychological factors.

Objectives:  To investigate if some demographic, psychological, and clinical variables can predict outcome with chiropractic care in patients with LBP.

Methods:  A prospective multi-center practice-based study was carried out, in which demographic, clinical and psychological information was collected at base-line. Outcome was established at the 4(th )visit and after three months. The predictive value was studied for all base-line variables, individually and in a multivariable analysis.

Results:  In all, 55 of 99 invited chiropractors collected information on 731 patients. At the 4(th )visit data were available on 626 patients and on 464 patients after 3 months. Fee subsidization (OR 3.2; 95% CI 1.9-5.5), total duration of pain in the past year (OR 1.5; 95% CI 1.0-2.2), and general health (OR 1.2; 95% CI 1.1-1.4) remained in the final model as predictors of treatment outcome at the 4(th )visit. The sensitivity was low (12%), whereas the specificity was high (97%). At the three months follow-up, duration of pain in the past year (OR 2.1; 95% CI 1.4-3.1), and pain in other parts of the spine in the past year (OR1.6; 1.1-2.5) were independently associated with outcome. However, both the sensitivity and specificity were relatively low (60% and 50%). The addition of the psychological variables did not improve the models and none of the psychological variables remained significant in the final analyses. There was a positive gradient in relation to the number of positive predictor variables and outcome, both at the 4(th )visit and after 3 months.

Conclusion:  Psychological factors were not found to be relevant in the prediction of treatment outcome in Swedish chiropractic patients with LBP.

Vanti C, Bonfiglioli R, Calabrese M, Marinelli F, Violante FS, Pillastrini P. Relationship between interpretation and accuracy of the upper limb neurodynamic test 1 in carpal tunnel syndrome. J Manipulative Physiol Ther. 2012 Jan;35(1):54-63. Epub 2011 Oct 27

ABSTRACT

Objective:  This prospective diagnostic test study evaluated the relationship between interpretation criteria and accuracy of the Upper Limb Neurodynamic Test 1 (ULNT1) in the diagnosis of carpal tunnel syndrome.

Methods: A blind comparison with a reference criterion of typical clinical presentation and abnormal median nerve conduction was used. All subjects were first tested with nerve conduction studies and then with ULNT1. Each examiner was blinded to the results collected by the other examiners.

Results: We analyzed 47 subjects and 84 limbs. Considering ULNT1 as positive in the presence of reproduction of symptoms only in the thumb or lateral 2 fingers, we estimated sensitivity to be equal to 40% (95% confidence interval [CI], 0.256-0.564), specificity 79.59% (95% CI, 0.664-0.885), positive predictive value 58.33% (95% CI, 0.388-0.755), negative predictive value 65% (95% CI, 0.524-0.758), positive likelihood ratio 1.96 (95% CI, 1.275-3.012), and negative likelihood ratio 0.7538 (95% CI, 0.490-1.159). Considering the increase of symptoms with contralateral or decrease of symptoms with ipsilateral cervical side bending as mandatory positivity criterion, specificity improved, but sensitivity decreased.

Conclusion: Our investigation suggests that the reproduction of symptoms in the thumb or lateral 2 fingers of the affected arm during ULNT1 has weak diagnostic accuracy for carpal tunnel syndrome.

Sensitivity and Specificity

Sensitivity and specificity are measures used to assess the value of a diagnostic test. For any given test in any given patient, please remember that the disease or condition itself may be present or absent, and the patient may test positive or negative. Given this, we need to know how useful our test really is. We can construct a 2X2 contingency table, where across the top are two columns: disease present, disease absent. We can add along the side test positive, test negative. This creates 4 cells:

• 1.       Disease present, test positive (true positives)
• 2.       Disease absent, test positive (false positives)
• 3.       Disease present, test negative (false negatives)
• 4.       Disease absent, test negative (true negatives)

Keeping this in mind, we can now define sensitivity and specificity and calculate them from our table.

Sensitivity is defined as the ability of a test to correctly identify a patient who truly has the disease in question. It is equal to the number of people with the disease who test positive divided by all the people in which the disease is present: true positives/(true positives + false negatives).

Specificity is the ability of test to properly identify those people who do not have the disease in question. It is equal to the number of people without the disease divided by all the people in which the disease is absent: true negatives/(true negatives + false positives).

Imagine a orthopedic diagnostic test for a herniated intervertebral  disc, tried out on 100 people with acute low back pain. The actual presence absence of herniated disc was diagnosed from MRI and we are comparing the results of our own orthopedic test against that “gold standard.”  The results of our 2x2 table show that our diagnostic test was positive in 20 people in which MRI also demonstrated the presence of a herniated disc; it was negative in 5 people even though there was MRI proof of herniation (false negatives); it was positive in 30 people for which the MRI did not show evidence of herniation (false positives); and it was absent in 45 people when MRI also showed no evidence of herniation (true negatives).  Sensitivity would therefore be 20/(20 + 5) = 20/25 + 0.8, or 80%. If a herniation is present, there is an 80% chance of the test properly identifying it being present.  Specificity would therefore be 45/(30 + 45) =45/75 = 0.6, or 60%. If no herniation is present, there is a 60% chance of the test being negative. However, keep in mind that there is a 40% chance of having a false positive result.

Next week, we will look at some of the implications of sensitivity and specificity.