Experimental Comparisons of Usability Evaluation Methods

1. OVERVIEW -- 2. INTRODUCTION -- 3. THE UNIQUE ROLE AND BURDEN OF EXPERIMENTS IN EMPIRICAL STUDIES OF HUMAN-COMPUTER INTERACTION -- 4. THREATS TO THE VALIDITY OF EXPERIMENTAL STUDIES -- 4.1. Cause-Effect Issues -- Statistical Conclusion Validity -- Internal Validity -- 4.2. Generality Issues -- Construct Validity -- External Validity -- 4.3. Conclusion Validity -- 4.4. Summary: Threats to Validity of Experimental Studies -- 5. THREATS TO THE VALIDITY OF FIVE INFLUENTIAL UEM STUDIES -- 5.1. Jeffries, Miller, Wharton, and Uyeda (1991) -- Overview -- Statistical Conclusion Validity -- Internal Validity -- Construct Validity -- External Validity -- Conclusion Validity -- Summary of the Review of Jeffries et al. (1991) -- 5.2. Karat, Campbell, and Fiegel (1992) -- Overview -- Statistical Conclusion Validity -- Internal Validity -- Construct Validity -- External Validity -- Conclusion Validity -- Summary of the Review of Karat et al. (1992) -- 5.3. Nielsen (1992) -- Overview -- Evaluation -- Statistical Conclusion Validity -- Internal Validity -- Construct Validity -- Conclusion Validity -- Summary of the Review of Nielsen ( 1992) -- 5.4. Desurvire, Kondziela, and Atwood (1992) -- Overview -- Statistical Conclusion Validity: Sample Size -- Internal Validity: Selection -- Construct Validity and External Validity -- Summary of the Review of Desurvire et al. ( 1992) -- 5.5. Nielsen and Phillips (1993) -- Overview -- Statistical Conclusion Validity -- Internal Validity -- Construct Validity and External Validity -- Conclusion Validity -- Summary of the Review of Nielsen and Phillips (1993) -- 5.6. Summary of the Reviews -- 6. OBSERVATIONS AND RECOMMENDATIONS -- 6.1. Predicting and Measuring Usability: Effect Construct Validity -- Hits, False Alarms, Misses, and Correct Rejections -- Tokens, Types, and Categories of Usability Problems -- Convergent Measures -- Effect Construct Validity: Recommendations -- 6.2. Recommendations for Addressing the Four Types of Validity -- Statistical Conclusion Validity -- Internal Validity -- Causal Construct Validity -- External Validity -- Conclusion Validity -- 6.3. Summary of Observations and Recommendations -- 7. CONCLUSIONS -- APPENDIX: VALIDITY OF CLAIMS MADE BY THE STUDIES REVIEWED.