1

The purpose of this chapter is to provide administrators and faculty with an understanding of learning analytics and its relationship to existing roles and functions so better institutional decisions can be made about investments and activities related to these technologies.

An Overview of Learning Analytics

John Zilvinskis, James Willis, III, Victor M. H. Borden

Higher education administrators and leaders have a sense that there is some magic going on in other industries and sectors, where people appear to be taking better advantage of the seemingly endless and growing amount of data available to improve sales and target customer support. There is a sense that higher education is far behind in this endeavor and so they are willing to invest significantly in new, expensive technologies. Campus leaders hear about some cases where institutions are using these methods and technologies to gain competitive advantage; they are impressed and do not want to be left behind. No one has figured out the magic formula, and even very well-known examples have not been sustained. For example, in 2012 The Chronicle of Higher Education (Parry, 2011) featured Rio Salado College for its ability to use online student behavior to predict student class performance; however, 1-year retention rates between fall 2014 and fall 2015 are only 38% for full-time students and 26% for part-time students (National Center for Education Statistics, 2017). These rates are among the lowest 2% and lowest 9%, respectively, among public, 2-year colleges, not surprising for an open access institution with large online programs, but certainly not industry leading.

The truth is, as documented in this volume, implementing and sustaining learning analytics initiatives are just not that simple, nor do they necessarily result in dramatic improvements. It's not that these new technologies and methods are unhelpful, but rather it's that they don't address the more complex aspects of higher education, including the incredible diversity and complexity of learning outcomes across the curriculum and the complex organizational arrangements. Even for institutions committed to "data-driven decision making," stakeholders often make the mistake of thinking the data will tell them what to do, as opposed to realizing that data themselves do not drive decision making; it is the interpretation of the data that creates change. At the center of these issues are three important questions: (1) What are learning analytics? (2) Who should be involved in these projects? (3) What are some important, broad principles institutions should consider when developing learning analytics? The purpose of this chapter and indeed the remainder of this volume is to provide some answers to these important questions.

Learning Analytics Defined

Though the use of data in higher education for operational and decision support is certainly not new, the computational processes involved in modelling it for prediction, intervention, and tracking have expanded exponentially in recent years. Consequently, there is a lack of maturity and consistency in the naming terms, conventions, and standards related to learning analytics. It is not uncommon to attend a higher education conference panel or sit in on a campus meeting where attendees use "analysis" and "analytics" interchangeably. Certainly, formal definitions and distinctions have been asserted (for example, between educational data mining and learning analytics), but there are notable differences in the formal literature and the usage of terms in practice is much looser and inconsistent. In their article, "Academic Analytics: A New Tool for a New Era," Campbell, DeBlois, and Oblinger (2007) wrote, "Analytics marries large datasets, statistical techniques, and predictive modeling" (p. 42). Learning analytics uses both traditional data (from student records, surveys, and so on) along with new types of data emanating from transactional systems like learning management systems, online course platforms, social networks, and so on. However, there are finer nuances in defining various types of analytics as one begins to consider the domain and types of learning analytics projects. For the purposes of this chapter, we define learning analytics as the process of using live data collected to predict student success, promote intervention or support based on those predictions, and monitor the influence of that action.

Distinguishing Learning Analytics Projects by Level of Analysis

There are two separate but interrelated domains for this work: the work that engages faculty in improving student learning within individual classes and programmatic curriculum (learning analytics), and the more holistic student support applications that do not focus directly on learning but more on student progress, persistence, and completion (academic analytics, according to Long and Siemens, 2011). In their conceptual framework of analytics in higher education, van Barneveld, Arnold, and Campbell (2012) add that predictive analytics can be used in both of these domains to draw upon historical data to predict future outcomes that can guide intervention. Later in this volume, Pistilli and Wagner describe learner analytics as using historical data from student records to predict the outcomes of current students, with the aim of intervening for students who are predicted to have a low likelihood of success. Already in this paragraph, we've drawn distinctions between learning analytics, academic analytics, predictive analytics, and learner analytics, relating to the level of analysis (student vs. class vs. curriculum), chronological characteristics of predictors (historical vs. contemporaneous), and type of outcomes (learning/behavior/development vs. retention/graduation); therefore, it is easy to see why newcomers find themselves uncertain when trying to specify the type of analytic project they wish to implement.

Distinguishing Learning Analytics Projects by Intended User

Learning analytics projects are often distinguished by the intended user or recipient of information. Previous work in analytics has led to the creation of tools to assist faculty with examining data from individual classes (Campbell et al., 2007). Chapters 2 and 5 in this volume have this focus. Other learning analytics projects primarily inform the work of academic advisors and other support staff with student guidance and coaching (Aguilar, Lonn, & Teasley, 2014; Barber & Sharkey, 2012). Chapters 3 and 4 fall into this category. Still other projects provide data directly to learners (Baker, 2007). Chapter 6 considers this target audience, at least in part. Analytics are also used to provide senior managers with management information related to teaching, learning, and student success (Buerck, 2014), which is also noted in Chapters 3 and 8. Regardless of the intended user, the chapters of this volume demonstrate that numerous campus partners must collaborate to implement a successful learning analytics project.

Relationship to Existing Roles and Functions

Implementation of learning analytics projects requires not only an understanding of the domain or the type of analytic project, but also an understanding of the amount of work and types of expertise needed. It is not feasible for even the most dedicated educators to create their own analytic systems. In addition to involving faculty as domain experts, information providers, and end-users, analytic project development often relies on collaboration among staff from several support units, such as centers for teaching and learning, student support services, institutional research, and information technology. Indeed, the development of learning analytics projects can change how these units routinely operate by providing opportunities for collaboration with new partners and new end-users. This may also lead to changes in the types of skillsets required for staff in these units.

Centers for Teaching and Learning

Staff within a center for teaching and learning (CTL) or similarly named unit typically provide faculty development programs related to curriculum and course design and assessment. These units promote communities of practice around specific pedagogies (for example, service learning) and provide support for using new educational tools and technologies. CTL staff provide required expertise in instructional and pedagogical design for developing new capacity for learning analytics among faculty (Borden, Guan, & Zilvinskis, 2014).

Student Support Services

Educators serving in academic advising, student affairs, and supplemental instruction use traditional data sources (for example, advising records, student needs assessments, registrar information) to determine the most effective interventions to recommend to individual students, such as counseling, tutoring, and peer mentoring. These staff members typically adopt a holistic view of student life and development across the curriculum, cocurriculum, and extracurriculum. Student advising and support providers often have the most extensive experience in dealing with students directly as they formulate academic and life goals (Drake, 2011). When working in learning analytics, the expertise of these educators provides perspective regarding the complex relationship between in-class and out-of-class demands on student life, while also supplying an understanding of how learning analytics...