Advancing Natural Language Processing in Educational Assessment

Preface

by Victoria Yaneva and Matthias von Davier

Section I: Automated Scoring

Chapter 1: The Role of Robust Software in Automated Scoring

by Nitin Madnani, Aoife Cahill, and Anastassia Loukina

Chapter 2: Psychometric Considerations when Using Deep Learning for Automated Scoring

by Susan Lottridge, Chris Ormerod, and Amir Jafari

Chapter 3: Speech Analysis in Assessment

by Jared C. Bernstein and Jian Cheng

Chapter 4: Assessment of Clinical Skills: A Case Study in Constructing an NLP-Based Scoring System for Patient Notes

by Polina Harik, Janet Mee, Christopher Runyon, and Brian E. Clauser

Section II: Item Development

Chapter 5: Automatic Generation of Multiple-Choice Test Items from Paragraphs Using Deep Neural Networks

by Ruslan Mitkov, Le An Ha, Halyna Maslak, Tharindu Ranasinghe, and Vilelmini Sosoni

Chapter 6: Training Optimus Prime, M.D.: A Case Study of Automated Item Generation using Artificial Intelligence - From Fine-Tuned GPT2 to GPT3 and Beyond

by Matthias von Davier

Chapter 7: Computational Psychometrics for Digital-first Assessments: A Blend of ML and Psychometrics for Item Generation and Scoring

by Geoff LaFlair, Kevin Yancey, Burr Settles, Alina A von Davier

Section III: Validity and Fairness

Chapter 8: Validity, Fairness, and Technology-based Assessment

by Suzanne Lane

Chapter 9: Evaluating Fairness of Automated Scoring in Educational Measurement

by Matthew S. Johnson and Daniel F. McCaffrey

Section IV: Emerging Technologies

Chapter 10: Extracting Linguistic Signal from Item Text and Its Application to Modeling Item Characteristics

by Victoria Yaneva, Peter Baldwin, Le An Ha, and Christopher Runyon

Chapter 11: Stealth Literacy Assessment: Leveraging Games and NLP in iSTART

by Ying Fang, Laura K. Allen, Rod D. Roscoe, and Danielle S. McNamara

Chapter 12: Measuring Scientific Understanding Across International Samples: The Promise of Machine Translation and NLP-based Machine Learning Technologies

by Minsu Ha and Ross H. Nehm

Chapter 13: Making Sense of College Students' Writing Achievement and Retention with Automated Writing Evaluation

by Jill Burstein, Daniel McCaffrey, Steven Holtzman & Beata Beigman Klebanov

Contributor Biographies