Members of the American Medical Association were no doubt chagrined by a recent piece of research in Spine.1 A team of researchers reviewed the measurement recommendations of the AMA's Guides to the Evaluation of Permanent Impairment (AMA Guides) and found them lacking reliability.
The embarrassment is of international scope, as these spinal range-of-motion guides for impairment ratings are used not only in the United States, but in Australia, New Zealand and other countries.
According to the researchers, there are two editions of the AMA Guides that have reliability problems:
"The AMA Guides (2nd edition) recommend measurement of thoracolumbar spine range of motion with a long-arm goniometer (LAG), but fail to cite appropriate evidence of its reliability for use in the spine. In fact, no research was found that investigated the reliability of the LAG for measuring spinal range of motion as recommended in the AMA Guides (2nd edition).
"The current edition of the AMA Guides (4th edition, 1993) recommends the use of a dual inclinometer (DI) to measure lumbar spine range of motion, but several problems are associated with this method of measurement. Although the reliability of the DI has been extensively investigated in both normal subjects and patients with low back pain, broadly variable results have been obtained. Specifically, conflicting findings are evident regarding reliability for the DI in patients with low back pain. High interrater reliability has been reported for total flexion-extension range of motion, and high interrater reliability for flexion but not extension. Moderate to poor intra-rater reliability has been reported for both flexion and extension, with the intra-rater test-retest reliability reportedly better for lumbar flexion than for extension."
The researchers conducted their own measurement tests to see just how substantial the problem was. They used the LAG and DI equipment on 34 subjects with chronic low back or leg pain of at least six-months duration. They tested for interrater and intra-rater reliability.
The testing judged the measurement devices recommended in the AMA Guides as severely lacking:
"This study demonstrated that the methods recommended in both the second and fourth editions of the AMA Guides to measure low back range of motion have poor interrater and intra-rater reliability. Decisions about the amount of compensation entitlement for patients with chronic low back pain based on these measurements are clearly misinformed decisions. It is essential for all parties concerned (employer, employee, insurer) that adjudication of compensation awards be determined in a reproducible way that truly represents the subjects' impairment. The results of the current study show that these two methods of assessment do not achieve this. The implications of these flawed decisions for employers, employees, and insurers are far-reaching and indicate the extent of problems that can occur when unreliable measurements are used in the health sector. "In summary, the findings indicate that a subject measured by two different examiners on the same day may be assessed as having (excluding the contribution from rotation) between 0% and 18% whole-body impairment. This must be considered a conservative estimate of variability because rotation contributes up to 12% of whole-body impairment and thus could further increase this estimate of the range of percentage impairment ratings. Similar findings can be demonstrated for second edition intra-rater reliability as well as fourth edition intra-rater and interrater reliability.
"In conclusion, the system of compensation for patients with chronic low back pain based on the AMA Guides to the Evaluation of Permanent Impairments (2nd edition) is not reliable. Results for the revised fourth edition of the AMA Guides reveal similar, perhaps slightly worse, reliability findings. The implications of these findings are important for the employers, employees, and insurers who use the AMA Guides' system of assessment because they indicate that these measures are not reproducible either between raters or by the same rater. The implications of the findings of the current study for the wider health care arena is that with the increased emphasis on measurable outcomes, it is vital that unreliable measures are not accepted as "good enough" or "near enough" because the information these measures provide may be inaccurate, and thus potentially harmful to all concerned."
Reference - Nitschke JE, Nattrass CL, Disler PB, Chou MC, Ooi KT. Reliability of the American Medical Association Guides' model for measuring spinal range of motion. Spine 1999;24:262-268
