The Development of a Common Framework Scale of Language Proficiency
Published on 23. October 2012
XVIII, 459 pages
978-1-4539-1059-7 (ISBN)
Description
Scales describing language proficiency in a series of levels can provide orientation for educational programmes, criteria for assessment, and reporting to stakeholders. However, in most cases such instruments are produced just by expert opinion. A scale of language proficiency actually implies a descriptive scheme related to theory but usable by practitioners. It also implies a methodology for scaling content to different levels. This book describes the use of both qualitative and quantitative techniques to develop scales for the «Common Reference Levels» in the Common European Framework of Reference for modern languages. Short stand-alone descriptors were (i) developed and classified, (ii) refined and elaborated in workshops, and then (iii) scaled by analyzing the judgments of one hundred teachers on the English language proficiency of the learners in their classes.
More details
Other editions
Additional editions
Book
07/2000
Peter Lang Verlag
€55.90
Article exhausted; check different version
Person
The Author: Brian North is responsible for academic coordination at Eurocentres, a Swiss foundation offering language-learning stays abroad. He has been much involved in the work of the Council of Europe Modern Languages project and is co-author of the Common European Framework of reference for modern languages. His Ph.D. thesis, which won the 1998 TOEFL award for outstanding dissertational research in language testing, concerned the development of language proficiency scales, the subject of this study.
Content
- Intro
- Contents
- List of Figures xi
- List of Tables xiii
- Preface xv
- List of Abbreviations xvii
- Introduction 1
- The Common European Framework 1
- Scaling Proficiency Descriptors 3
- The Context of the Study 4
- 1. Language Proficiency Scales 9
- Definitions 11
- Attractions 11
- Origins 13
- Purposes 17
- Types 21
- Metaphors for Scales 23
- Descriptions of Behaviour and Behavioural Objectives 25
- Difficulties with Common Framework Scales 28
- 2. Description 41
- Definitions of Language Proficiency 41
- Interpretations of Communicative Competence 43
- Competence and Proficiency 47
- The Native Speaker as Criterion 54
- User Perspectives 60
- Categories for Communicative Language Proficiency 62
- Categories for Communicative Activity 98
- A Unitary Competence: Holistic Approaches 115
- Towards Balanced Scale Categories 123
- Summary on Description Issues 128
- 3. Measurement 131
- Criterion-referenced Assessment 131
- The Development of Behaviourally-based Assessment Scales 139
- Uniformity of Scales and Grids 146
- Dimensionality 149
- Types of Measurement Scales 153
- Essentials of a Valid Measurement Scale 156
- Common Methods of Scale Construction 161
- The Rasch Measurement Model 162
- Developing a Framework Scale with the Rasch Model 166
- The Many-faceted Rasch Model 175
- Summary on Measurement Issues 179
- 4. Developing a Descriptor Pool 181
- Provisional Categories 182
- Provisional Levels 183
- Editing 184
- Pre-testing Workshops with Teachers 185
- 5. Data Collection and Correction 193
- Connecting Questionnaires 194
- Rating Conference 198
- Rating Scale 206
- Subjects 207
- Problems with the Analysis 208
- 6. Constructing the Scale 223
- Analysis Specifications and Data Organisation 223
- FACETS Output 225
- Investigating Use of the Rating Scale 230
- Dimensionality: Identifying Problematic Content Strands 233
- Refining the Dimension: Quality Control of Anchor Items 246
- Investigating Variation across Sectors and Regions 255
- Refining the Bank: Quality Control on Individual Descriptors 260
- Establishing an Item Quality Hierarchy 268
- 7. Interpreting the Scale 271
- Setting Cut-offs between Levels 271
- Scale Shrinkage 283
- Content Coherence 285
- Progression in Proficiency 290
- 8. Learner Achievement 311
- Decisions on Data Inclusion 311
- Analysis 316
- Achievement in Educational Sectors 322
- Learner Calibrations from Two Data Sets 329
- Concurrent Validity 331
- 9. Conclusions 335
- Project Results 335
- Replication in Year 2 338
- Conclusions in Relation to the Rasch Model 341
- Conclusions in Relation to Descriptors 343
- Areas for Follow up and Further research 346
- Appendices 351
- Appendix 1: Sample Data Collection Questionnaire 351
- Appendix 2: Sample Rating Conference Mini-questionnaire 357
- Appendix 3: Vertical Scale of Descriptors with Fit and SEM 358
- Appendix 4: Classified Scales of Descriptors with Sources 386
- References 417
