How does the interested clinician decide whether a new technique for assessing visual function is effective (reality) or not (myth)? This article provides some guidelines for making the determination.
SEPARATING MYTH FROM REALITY FOR VISUAL FUNCTION TESTS
Guidelines
An important factor in evaluating the efficacy of a new clinical test procedure pertains to which techniques can provide useful, important, valid clinical findings and which are limited in their overall performance. This is a difficult issue, but I believe that Harper et al22 have provided some useful guidelines. They cite seven standards for the evaluation of diagnostic tests: (1) specification of spectrum composition; (2) analysis of pertinent subgroups; (3) avoidance of work-up or verification bias; (4) avoidance of review bias; (5) presentation of the precision of results for test accuracy; (6) presentation of indeterminate test results; and (7) presentation of test reproducibility. Their article contains details of these criteria and their application to 20 clinical research studies. One of the researchers' rather startling findings was that all 20 clinical studies described in their report demonstrated only limited compliance with the seven aforementioned standards! This revelation clearly indicates that all investigators of clinical diagnostic test procedures should pay closer attention to these issues when conducting future research so that practitioners can receive greater benefit.
I would add several guidelines to those noted earlier. First, I believe that it is important to confirm the findings of a clinical research study, either as part of the initial study or, preferably, by verification from an independent laboratory. The replication of findings usually strengthens their significance, and the robustness of research results enhances their applicability to all clinical situations.
Second, it is useful to determine the variation among research results from many different laboratories. For some reason, a test procedure usually performs better in the hands of the investigator who developed it than for other researchers who are evaluating the technique. A procedure that consistently produces accurate results at different laboratories should generate higher enthusiasm than one that yields variable outcomes.
Finally, longitudinal follow-up evaluations can help reveal the clinical value of a new diagnostic test procedure. With a series of test results obtained over a period of time, it becomes possible to determine the clinical sensitivity, specificity, variability, validity, robustness, predictive value, and other important attributes of the test procedure. Three illustrative examples follow.
Example No. 1
Figure 1 presents the results of SAP and SWAP testing in the right eye of a patient with elevated IOP and other risk factors for developing glaucoma. Visual field locations with sensitivity that is within normal limits (normal) as well as those below the normal 5% (P<.05) and 1% (P<.01) probability limits (abnormal) are given for a 5-year time period for SAP and SWAP. The SWAP results show a superior nasal step in year 1 that progresses to a superior arcuate defect in years 2 through 5. SAP results are within normal limits for the first 3 years of testing, but they begin to show the onset of a superior arcuate defect in years 4 and 5. SWAP deficits occurred several years before the SAP abnormality and were predictive of the development and location of the SAP visual field loss.
Example No. 2
Figure 2 compares FDT perimetry and SAP results for testing performed over a 5-year period on the left eye of a patient with risk factors for developing glaucoma. As in the earlier example, the FDT findings reveal abnormalities approximately 2 years earlier than the SAP results, and they are predictive of the onset and location of subsequent SAP sensitivity losses. FDT results initially show a superior nasal step, which progresses to a superior arcuate defect over 4 years. The deficit remains as a superior nasal step for the 4th year of testing with SAP.
Example No. 3
Figure 3 presents results obtained with rarebit perimetry for several patterns of visual field loss. Briefly, rarebit perimetry incorporates short presentations of very small stimuli consisting of one or two bright spots on a dark background shown on an LCD computer display. The arrangement of the dots accords with a predetermined pattern that is designed to detect early visual field losses in glaucoma and other ocular and neurologic diseases. The patient's task is to indicate whether he detects one or two dots for each presentation. Although the technique is relatively new, the current results for a variety of visual pathway disorders have been clinically useful.20
SUMMARY
Evaluating the clinical impact and significance of a test procedure for visual function assessment in glaucoma remains both a challenge and an enigma. Careful, well-controlled, longitudinal investigations of a large cohort of patients and control subjects are required, but these studies are costly and time-consuming. Following the guidelines set forth herein may help individuals determine which new procedures have distinct clinical benefits and which are of limited value.
Supported in part by a National Eye Institute Research Grant #EY-03424 and the Oregon Lions Sight and Hearing Foundation.
Chris A. Johnson, PhD, is Director of Diagnostic Research, Discoveries in Sight Research Labs, Devers Eye Institute, Portland, Oregon. He is a consultant for Welch Allyn Medical Products and receives research support from Welch Allyn Medical Products and Carl Zeiss Meditec Inc. Dr. Johnson may be reached at (503) 413-5318; cajohnso@discoveriesinsight.org.
1. Johnson CA. Software upgrades for automated perimetry. Glaucoma Today. 2004;2:3:32-34.
2. Sample PA, Bosworth CF, Blumenthal EZ, et al. Visual function-specific perimetry for indirect comparison of different ganglion cell populations in glaucoma. Invest Ophthalmol Vis Sci. 2000;41:1783-1790.
3. Johnson CA, Keltner JL. Principles and techniques of the examination of the visual sensory system. In: Miller NR, Newman NJ, eds. Walsh and Hoyt's Clinical Neuro-Ophthalmology. Baltimore: Lippincott Williams & Wilkins; 1998: 153-235.
4. Johnson CA, Sample PA. Perimetry and visual field testing. In: Adler's Physiology of the Eye. 10th ed. Alm A, Kaufmann P, eds. St. Louis: CV Mosby; 2002: 552-577.
5. Johnson CA. Update on perimetry: new developments. In: Glaucoma in the New Millennium: Proceedings of the 49th Annual Symposium of the New Orleans Academy of Ophthalmology. The Hague: Kugler Publications; 2003: 185-206.
6. Johnson CA. New developments in perimetry. In: Fingeret M, Lewis TL, eds. Primary Care of the Glaucomas. New York: McGraw-Hill; 2000: 271-294.
7. Sample PA, Martinez GA, Weinreb RN. Color visual fields: a 5 year prospective study in eyes with primary open angle glaucoma. In: Mills RP, ed. Perimetry Update 92/93. Amsterdam/New York: Kugler Publications; 1993: 467-473.
8. Sample PA, Weinreb RN. Progressive color visual field loss in glaucoma. Invest Ophthalmol Vis Sci. 1992;33:2068-2071.
9. Johnson CA, Adams AJ, Casson EJ, Brandt JD. Progression of early glaucomatous visual field loss as detected by blue-on-yellow and standard white-on-white automated perimetry. Arch Ophthalmol. 1993;111:651-656.
10. Johnson CA, Adams AJ, Casson EJ. Blue-on-yellow perimetry: a five year overview. In: Mills RP, ed. Perimetry Update 92/93. Amsterdam/New York: Kugler Publications; 1993: 459-465.
11. Casson EJ, Johnson CA, Shapiro LR. Longitudinal comparison of temporal-modulation perimetry with white-on-white and blue-on-yellow perimetry in ocular hypertension and early glaucoma. J Opt Soc Am A. 1993;10:1792-1806.
12. Medieros FA, Sample PA, Weinreb RN. Frequency doubling technology perimetry abnormalities as predictors of glaucomatous visual field loss. Am J Ophthalmol. 2004;137:863-871.
13. Anderson AJ, Johnson CA. Frequency-doubling technology perimetry. Ophthalmol Clin North Am. 2003;16:213-225.
14. Landers JA, Goldberg I, Graham SL. Detection of early visual field loss in glaucoma using frequency-doubling perimetry and short-wavelength automated perimetry. Arch Ophthalmol. 2003;121:1705-1710.
15. Fitzke FW, Poinoosawmy D, Nagasubramanian S, Hitchings, RA. Peripheral displacement thresholds in glaucoma and ocular hypertension. In: Heijl A, ed. Perimetry Update 1988/1989. Amsterdam: Kugler Publishers; 1989: 399-405.
16. Wall M. Jennisch CS. Random dot motion stimuli are more sensitive than light stimuli for detection of visual field loss in ocular hypertension patients. Optom Vis Sci. 1999;76:550-557.
17. Frisen L. High-pass resolution perimetry. A clinical review. Doc Ophthalmol. 1993;83:1-25.
18. Chauhan BC, House PH, McCormick TA, LeBlanc RP. Comparison of conventional and high-pass resolution perimetry in a prospective study of patients with glaucoma and healthy controls. Arch Ophthalmol. 1999;117:24-33.
19. Frisen L. New, sensitive window on abnormal spatial vision: rarebit probing. Vision Res. 2002;42:1931-1939.
20. Martin L, Wanger P. New perimetric techniques: rarebit and frequency doubling technology perimetry in normal subjects and glaucoma patients. J Glaucoma. 2004;13:268-272.
21. Hood DC, Thienprasiddhi P, Greenstein VC, et al. Detecting early to mild glaucomatous damage: a comparison of the multifocal VEP and automated perimetry. Invest Ophthalmol Vis Sci. 2004;45:492-498.
22. Harper R, Reeves B. Compliance with methodological standards when evaluating ophthalmic diagnostic tests. Invest Ophthalmol Vis Sci. 1999;40:1650-1657.
