Second Language Research

Contents

Preface

1.

Introduction to Research

1.1

Research Methods

1.2

Different Types of Research

1.3

Ethical Practices

1.3.1

Informed Consent

1.3.1.1

Sufficient Information

1.3.1.2

The Informed Consent Document

1.3.1.3

Participant Comprehension in Informed Consent

1.3.1.4

The Language of Informed Consent

1.3.1.5

Voluntary Participation and Informed Consent

1.3.2

Child Second Language Learners

1.3.3

Issues Involved in Collecting Data Online

1.3.4

WEIRD Populations

1.3.5

Reporting and Anonymity

1.4

Institutional Review of Human Participants Research, Compliance, and Problem-Solving

1.4.1

Purpose of Reviews and IRB Responsibilities

1.4.2

Why Guidelines to Protect Human Subjects Were Developed

1.4.3

Development of Research Codes of Ethics

1.4.4

Preparing a Protocol for the IRB

1.4.5

General Use of Protocols in Research

1.5

Conclusion

Points to Remember

More to Do and More to Think About ...

Sample Responses: Time to Think & Time to Do

Sample Responses: More to Do and More to Think About...

2.

Research Reporting, Research Questions, and Replication

2.1

What Does a Research Report Look Like?

2.1.1

Quantitative Reporting

2.1.1.1

Title Page

2.1.1.2

Abstract

2.1.1.3

Introduction

2.1.1.4

Historical Overview

2.1.1.5

Major Players in this Research Area, Including Questions, Past Findings, and Controversies

2.1.1.6

General Goal of the Paper

2.1.1.7

Research Questions/Hypotheses

2.1.1.8

Methods Section

2.1.1.9

Participants

2.1.1.10

Materials

2.1.1.11

Procedures

2.1.1.12

Analysis

2.1.1.13

Results

2.1.1.14

Discussion/Conclusion

2.1.1.15

Notes

2.1.1.16

References

2.1.1.17

Appendices

2.1.2

Qualitative Reporting

2.1.3

Mixed-Method Reporting

2.2

Identifying Research Questions

2.2.1

Feasibility

2.2.2

Research Questions and Hypotheses

2.3

Replication

2.4

Conclusion

Points to Remember

More to Do and More to Think About ...

Sample Responses: Time to Think & Time to Do

Sample Responses: More to Do and More to Think About...

3.

Common Data Collection Measures

3.1

Introduction

3.2

The Significance of Data Collection Measures

3.3

Researching Formal Models of Language

3.3.1

Acceptability Judgments

3.3.1.1

Materials

3.3.1.2

Procedures

3.3.2

Gradient Judgments: Magnitude Estimation

3.3.3

Elicited Imitation

3.3.4

Interpretation Tasks

3.4

Psycholinguistics-Based Research

3.4.1

Sentence Interpretation

3.4.2

Reaction Time

3.4.3

Self-Paced Reading

3.4.4

Eye-Tracking

3.4.5

Neurolinguistic Elicitation

3.5

Interaction-Based Research

3.5.1

Picture Description Tasks

3.5.2

Spot the Difference

3.5.3

Jigsaw Tasks

3.5.4

Consensus Tasks

3.5.5

Consciousness-Raising Tasks

3.5.6

Computer-Mediated Research

3.6

Strategies and Cognitive Processes

3.6.1

Observations

3.6.2

Introspective Measures

3.6.2.1

Stimulated Recall

3.6.2.2

Think-Alouds or On-Line Tasks

3.6.2.3

Immediate Recalls

3.7

Sociolinguistic/Pragmatics-Based Research

3.7.1

Naturalistic Settings

3.7.2

Elicited Narratives

3.7.2.1

Silent Film

3.7.2.2

Film Strips with Minimal Sound

3.7.2.3

Picture Tasks

3.7.2.4

Narrative Inquiry

3.7.3

Discourse Completion Test (DCT)

3.7.4

Role Plays

3.7.5

Video Playback for Interpretation

3.8

Questionnaires and Surveys

3.9

Existing Databases

3.10

Pilot Testing

3.11

Conclusion

Points to Remember

More to Do and More to Think About ...

Notes

Sample Responses: Time to Think & Time to Do

Sample Responses: More to Do and More to Think About...

4.

Coding

4.1

Preparing Data for Coding

4.1.1

Transcribing Oral Data

4.1.1.1

Transcription Conventions

4.1.1.2

Technology and Transcription

4.2

Data Coding

4.2.1

Scales of Measurement

4.2.2

Coding Nominal Data

4.2.3

Coding Ordinal Data

4.2.4

Coding Interval Data

4.3

Coding Systems

4.3.1

Common Coding Systems and Categories

4.3.1.1

T-Units

4.3.1.2

Suppliance in Obligatory Contexts (SOC)

4.3.1.3

Complexity, Accuracy, and Fluency (CAF)

4.3.1.4

Qualitative Coding Categories

4.3.2

Custom-Made Coding Systems

4.3.2.1

Question Formation

4.3.2.2

Corrective Feedback

4.3.2.3

Classroom Interaction

4.3.2.4

Second Language Writing Research

4.3.3

Coding Interpretive and Qualitative Data

4.4

Inter-Rater Reliability

4.4.1

Calculating Inter-Rater Reliability

4.4.1.1

Simple Percentage Agreement

4.4.1.2

Cohen's Kappa

4.4.1.3

Additional Measures of Reliability

4.4.1.4

Good Practice Guidelines for Inter-Rater Reliability

4.4.1.5

How Data Are Selected for Inter-Rater Reliability Tests

4.4.1.6

When to Carry out Coding Reliability Checks

4.5

The Mechanics of Coding

4.5.1

How Much to Code?

4.5.2

When to Make Coding Decisions

4.6

Software for Coding

4.7

Conclusion

Points to Remember

More to Do and More to Think About ...

Sample Responses: Time to Think & Time to Do

Sample Responses: More to Do and More to Think About...

5.

Research Variables, Validity, and Reliability

5.1

Introduction

5.2

Hypotheses

5.3

Variable Types

5.3.1

Independent and Dependent Variables

5.3.2

Moderator Variables

5.3.3

Intervening Variables

5.3.4

Control Variables

5.4

Operationalization

5.5

Validity

5.5.1

Content Validity

5.5.2

Face Validity

5.5.3

Construct Validity

5.5.4

Criterion-Related Validity

5.5.5

Predictive Validity

5.5.6

Internal Validity

5.5.6.1

Participant Characteristics

5.5.6.2

Language Background

5.5.6.3

Language Learning Experience

5.5.6.4

Proficiency Level

5.5.6.5

Participant Attrition

5.5.6.6

Participant Inattention and Attitude

5.5.6.7

Participant Maturation

5.5.6.8

Data Collection: Location and Collector

5.5.6.9

Instrumentation and Test Effect

5.5.6.10

Equivalence Between Pre- and Posttests

5.5.6.11

Giving the Goal of the Study Away

5.5.6.12

Instructions/Questions

5.5.7

External Validity

5.5.7.1

Sampling

5.5.7.2

Random Sampling

5.5.7.3

Nonrandom Sampling

5.5.7.4

Representativeness and Generalizability

5.5.7.5

Collecting Biodata Information

5.6

Reliability

5.6.1

Rater Reliability

5.6.2

Instrument Reliability

5.6.2.1

Test-Retest

5.6.2.2

Equivalence of Forms

5.6.2.3

Internal Consistency

5.7

Conclusion

Points to Remember

More to Do and More to Think About ...

Notes

Sample Responses: Time to Think & Time to Do

Sample Responses: More to Do and More to Think About...

6.

Designing a Quantitative Study

6.1

Introduction

6.2

Research Materials

6.3

Intact Classes

6.4

Counterbalancing

6.5

Research Design Types

6.5.1

Correlational (Associational) Research

6.5.2

Experimental and Quasi-Experimental Research

6.5.2.1

Comparison Group Design

6.5.2.2

Control Group Design

6.5.3

Measuring the Effect of Treatment

6.5.3.1

Pretest/Posttest Design

6.5.3.2

Posttest Only Design

6.5.4

Repeated Measures Design

6.5.5

Factorial Design

6.5.6

Time-Series Design

6.5.7

One-Shot Designs

6.5.8

Research Syntheses and Meta-Analyses

6.6

Conclusion

Points to Remember

More to Do and More to Think About ...

Note

Sample Responses: Time to Think & Time to Do

Sample Responses: More to Do and More to Think About...

7.

Qualitative and Interpretive Research

7.1

Defining Qualitative and Interpretive Research

7.2

Gathering Qualitative and Interpretive Data

7.2.1

Ethnographies

7.2.1.1

Advantages

7.2.1.2

Caveats

7.2.2

Case Studies

7.2.2.1

Advantages

7.2.2.2

Caveats

7.2.3

Interviews

7.2.3.1

Advantages

7.2.3.2

Caveats

7.2.4

Observations

7.2.4.1

Advantages

7.2.4.2

Caveats

7.2.5

Diaries/Journals

7.2.5.1

Advantages

7.2.5.2

Caveats

7.3

Analyzing Qualitative Data

7.3.1

Credibility, Transferability, Confirmability, and Dependability

7.3.2

Triangulation

7.3.3

The Role of Quantification in Qualitative and Interpretive Research

7.4

Conclusion

Points to Remember

More to Do and More to Think About ...

Note

Sample Responses: Time to Think & Time to Do

Sample Responses: More to Do and More to Think About...

8.

Classroom Research

8.1

Classroom Research Contexts

8.2

Common Techniques for Data Collection in Classroom Research

8.2.1

Observations

8.2.1.1

Conducting Classroom Observations

8.2.1.2

Observation Procedures and Coding Schemes

8.2.1.3

Description of Observation Schemes

8.2.1.4

Caveats to Using or Modifying Existing Observation Schemes

8.3

Introspective Methods in Classroom Research

8.3.1

Uptake Sheets

8.3.2

Stimulated Recall

8.3.3

Diary Research in Classroom Contexts

8.4

Practical Considerations in Classroom Research

8.4.1

Logistical Issues to Consider When Carrying out Classroom Research

8.4.2

Logistics

8.4.2.1

Informed Consent

8.4.2.2

Debriefing Participants and Facilitators

8.4.2.3

Ensuring Confidentiality and Minimizing Disruption

8.4.2.4

Data Segmentation and Coding

8.4.2.5

Considering the Instructional Setting

8.4.2.6

Summary of Logistics

8.5

Purposes and Types of Research Conducted in Classroom Settings

8.5.1

The Relationship between Instruction and Learning in Second Language Classrooms

8.5.2

Action Research

8.5.2.1

Definitions

8.5.2.2

Theory and Background to Action Research

8.5.2.3

Action Research in Practice

8.5.3

Aptitude-Treatment Interaction

8.5.4

Getting Your Footing in the Literature

8.6

Conclusion

Points to Remember

More to Do and More to Think About ...

Sample Responses: Time to Think & Time to Do

Sample Responses: More to Do and More to Think About...

9.

Mixed Methods

9.1

Introduction

9.2

What Are Mixed Methods?

9.3

Why Use Mixed Methods?

9.4

Types of Mixed-Methods Studies

9.4.1 Concurrent Mixed-Methods

9.4.2. Sequential Mixed-Methods

9.5

Conducting a Mixed-Methods Study

9.5.1

Deciding Whether or Not to Conduct a Mixed-Methods Study

9.5.2

Planning Phase

9.6

Analyzing Data from a Mixed-Methods Study

9.7

Software for Analysis

9.8

Conclusion

Points to Remember

More to Do and More to Think About ...

Sample Responses: Time to Think & Time to Do

Sample Responses: More to Do and More to Think About...

10.

Concluding and Reporting Research

10.1

The Importance of Reporting Research

10.2

The Final Stages in Reporting Quantitative Research

10.2.1

The Discussion

10.2.2

Limitations, Future Research, and Conclusion Sections

10.3

The Final Stages in Reporting Qualitative Research

10.4

Reporting Mixed-Methods Research

10.5

Checklist for Completing Reports of Research

10.5.1

The Research Problem and Questions

10.5.2

The Research Hypotheses

10.5.3

The Audience

10.5.4

The Abstract

10.5.5

The Literature Review

10.5.6

The Design of the Study

10.5.7

Logistics

10.5.8

Participants

10.5.9

Data Gathering

10.5.10

Data Analysis

10.5.11

Conclusions

10.5.12

References

10.5.13

Footnotes, Endnotes, Figures, and Tables

10.5.13.1

Footnotes and Endnotes

10.5.13.2

Figures

10.5.13.3

Tables

10.5.14

Author's Note/Acknowledgments

10.5.15

Post-Research Concerns

10.5.16

Final Touches and Formatting

10.6

Open Science

10.7

Conclusion

Points to Remember

More to Do and More to Think About ...

Sample Responses: Time to Think & Time to Do

Sample Responses: More to Do and More to Think About...

Glossary

References

Subject Index