Selecting Clinically Valuable Assessment Tools

More than 50 years ago, Cambridge professor John Ryle wrote, "The three main tasks of the clinician ... are diagnosis, prognosis and treatment. Of these, diagnosis is by far the most important, for upon it, the success of the other two are dependent."¹

Patients and practitioners increasingly rely on diagnostic/analytical tests for critical information to make smart, health care decisions. In this era of accountability, consumers expect health care providers to base their practices on the best available evidence, and this begins with using the most effective and accurate tests when making a diagnosis/analysis.^2,3

Shortfalls of Medical Testing

Diagnostic testing is an extremely important aspect of care and accounts for approximately one-fourth of all ambulatory care expenditures in America.⁴ Although tests such as MRI and discography have increased the ability to view disc anatomy, they have not improved patient outcomes. Diagnostic costs have soared, but disability expenses have not diminished.⁵ Additionally, the medical profession is recognizing that advances in diagnostic imaging may create risks of overestimating illness prevalence. The identified problems may provoke therapy for clinically irrelevant abnormalities.⁶ Essentially, improved testing reveals a greater number of clinically irrelevant abnormalities, which leads to unnecessary interventions.⁷ Although the risks of the tests themselves may be relatively small, the cascade of subsequent events may quickly spiral out of control.

Assumptions Surrounding Chiropractic Assessment

Doctors of chiropractic perform various static and dynamic physical-examination maneuvers and/or use various instruments in an effort to detect abnormalities indicative of joint dysfunction. Most chiropractors believe their assessment process will locate the specific offending spinal lesion(s).⁸ However, this belief may be based largely on perception, rather than fact.

Chiropractors usually begin the assessment process with a case history and physical examination to rule out serious conditions. Next, we perform specific spinal analysis and locate the offending vertebra(e). Then we deliver our therapy. When the patient improves, we assume that the assessment method located the specific problem. Although this assumption is prevalent, it may be incorrect.

The Assessment Conundrum: What Am I Seeing?

Chiropractors are faced with a challenging paradox: Does the assessment process improve patient outcomes by directing the practitioner to the specific lesion, or is the assessment process trivial and inconsequential to the treatment process? Researchers suggest our spinal manipulations may not be specific.^9,10 If this is the case, specific analytical procedures may fail to provide value to the treatment protocol. The chiropractic therapy might be effective regardless of the specific assessment. However, it is not possible for practitioners to estimate the value of the assessment process to the overall improvement of their patients in a clinical setting. Scientists are needed to examine the value of the specific assessment methods with rigorous testing (e.g., by comparing patients who receive both the assessment and treatment with those who receive the treatment only). Thus, science can become our ally in selecting the most clinically valuable assessments.

Estimating the Value of a Diagnostic/Analytical Test

Faced with more than 100 different assessment methods, how can practitioners select the ones that will add the most value to their treatment regime? The first step is to reference high-quality research studies addressing the three key attributes of the diagnostic or analytical test under consideration: validity, reliability and clinical utility.

Validity is the degree to which a test truly measures what it purports to measure. A valid test is able to detect a true positive and discriminate positive from negative. That is to say, it finds almost all patients who have the condition and hardly any who do not. Sensitivity and specificity are measures commonly used to describe diagnostic accuracy. Sensitivity refers to the proportion of people with a particular disease who are correctly classified as diseased by the test (true positives). Specificity refers to the proportion of people without a particular disease who are correctly classified as disease-free by the test (true negatives). Sensitivity and specificity greater than .80 are reasonable benchmarks.¹¹

Reliability, in diagnostic testing, is the degree to which repeated measures on the same subject agree when made by different observers or by the same observer at different times. If a test is unreliable, test results will change without the patient's condition really changing. However, a reliable test is worthless if it is not concurrently valid. Cohen's kappa coefficient and intra-class correlation coefficient are measures commonly used to describe reliability. Cohen's kappa coefficient greater than .60 or intra-class correlation coefficient greater than .80 are sensible standards.¹¹

The clinical utility or worthiness of a test is determined by its effects on patient outcomes and patient management. A test should improve patient outcomes and direct the practitioner to deliver a different therapy or use a therapy in a different way. If it does not do these things, it lacks clinical utility and should be avoided. Moreover, a test may have good validity and reliability, but there is no reason to perform it if the test does not concurrently have clinical utility.

Doctors who are considering using a particular assessment protocol need data from quality research studies suggesting that the examination possesses good validity, reliability and clinical utility. Additionally, the test should be affordable and the beneficial effects should outweigh any harmful ones (e.g., radiation from X-rays). Practitioners should use the best possible scientific evidence as a basis for every phase of health care decision-making.¹² Otherwise, patient recommendations are based merely upon marketing promises.

Assessment tools that demonstrate good validity, reliability and clinical utility can help practitioners deliver management strategies that yield optimal patient outcomes, whereas tools that do not exhibit those qualities can lead to less favorable results. In order to choose the most beneficial assessment tools, we should take into account not only data from quality research studies, but also clinical circumstances, practitioner judgment and patient preferences. Additionally, we must come to terms with the nature and reality of uncertainty (e.g., having insufficient data or no data).

Selecting the most effective and accurate tests is a fairly sophisticated process. It requires diligence and critical appraisal competency. However, it is the duty of every practitioner to select the most appropriate assessment tools and therapies in order to achieve optimum outcomes. It is our obligation to our patients, our profession and society.

References

1. Ryle J. The Natural History of Disease, 2nd ed. New York: Oxford University Press, 1948.
2. Harrison H. The principles for family-centered neonatal care. Pediatrics, 1993;92:643-50.
3. Irwig L, Tosteson AN, Gatsonis C, et al. Guidelines for meta-analyses evaluating diagnostic tests. Ann Intern Med, 1994;120:667-76.
4. Begg CB. Biases in the assessment of diagnostic tests. Stat Med, 1987;6:411-23.
5. Hazard RG. Occupational low back pain: the critical role of functional goal setting. APS Journal, 1994;3:101-6.
6. Deyo RA, Haselkorn J, Hoffman R, Kent DL. Designing studies of diagnostic tests for low back pain or radiculopathy. Spine, 1994;19(Suppl):2057-65.
7. Mold JW, Stein HF. The cascade effect in the clinical care of patients. N Engl J Med, 1986;314:512-4.
8. Walker BF, Buchbinder R. Most commonly used methods of detecting spinal subluxation and the preferred term for its description: a survey of chiropractors in Victoria, Australia. JMPT, 1997;20:583-9.
9. Beffa R, Mathews R. Does the adjustment cavitate the targeted joint? An investigation into the location of cavitation sounds. JMPT, 2004;27:e2.
10. Ross JK, Bereznick DE, McGill SM. Determining cavitation location during lumbar and thoracic spinal manipulation. Is spinal manipulation accurate and specific? Spine, 2004;29:1452-7.
11. Feise RJ. Evidence-Based Chiropractic: Research Methodology, Biostatistics & Critical Thinking. Fort Collins, CO: Institute of Evidence-Based Chiropractic, 2005.
12. Miké V. Outcomes research and the quality of health care: the beacon of an ethics of evidence. Eval Health Prof, 1999;22:3-32.