Evidence-Based Learning: Futures Using learning design and learning analytics to empower teachers to meet students’ diverse needs

With the introduction of learning design in early 2000 and learning analytics in 2012, the OU has led the way in how teachers make complex decisions to design interactive courses, and how students can maximise their learning potential. The next obvious steps would be to include AI, personalisation, and student-led learning analytics to provide learning opportunities that meet the unique needs of each learner, but whether this would be technically feasible and pedagogically desirable will be discussed. In this chapter we will look at recent and future developments concerning the “holy trinity” of learning design, learning analytics, and how teachers can help institutions like the OU to ensure that our current and future students’ needs are met. Furthermore, we will reflect on the affordances and limitations of learning design and learning analytics to help teachers to adapt their teaching and learning practices to meet learners’ needs


Introduction
The Open University (OU) has been at the forefront of innovation in teaching and learning since its inception in 1969. As highlighted in the previous chapter, even when most people did not have access to a computer, let alone a smart phone, the OU was actively experimenting and rolling out innovative ICT systems and applications to help support teachers to deliver exciting and relevant approaches to help students meet their needs. In this chapter, we will primarily reflect on major developments of innovative teaching practice since 2010 that have shaped the OU and wider environment, and vice versa. In particular, this chapter will focus on the holy trinity between learning design, learning analytics, and teachers.
Like many other institutions across the globe, as highlighted in Chapter 6 the OU continuously explores the opportunities information technology affords to provide a better, more consistent, and ideally more personalised service to its learners, teachers, and wider stakeholders (Herodotou et al., 2017;Hidalgo, 2018;Rienties & Toetenel, 2016;Tait, 2018). Globally (Dalziel, 2016;Hernández-Leo et al., 2018;Lockyer & Dawson, 2012;Mangaroska & Giannakos, 2018) as well as within the OU Rienties et al., 2018a;Rienties & Toetenel, 2016;Toetenel & Rienties, 2016a; van Ameijde et al., 2018) there is an increased recognition that learning design is an essential driver for learning, as well as empowering teachers to meet students' needs. For example, using concepts originally developed by Conole (2012) the Open University Learning Design Initiative (OULDI) has been implemented on a large-scale within the OU (Cross et al., 2012;van Ameijde et al., 2018). An excellent example of this large-scale implementation comes from a review of 157 learning designs of OU modules, whereby Toetenel and Rienties (2016a) found a wide tapestry of interactive and unique learning designs, from more traditional constructivist designs to more socio-constructivist designs.
However, learning design by itself is just a useful approach to depict how teachers design a particular learning activity or a complete course. Only when learning design is combined with how students are actually engaging with these learning designs do we start to make real progress. One way to empower learning design is to use learning analytics data of students. As argued by a range of researcher and practitioners (Calvert, 2014;Ferguson et al., 2016;Hlosta et al., 2015;Toetenel & Rienties, 2016a;Wolff et al., 2013) learning analytics may empower distance learning institutions like the OU to provide near real-time actionable feedback to students and teachers about what the "best" next step in their learning journeys might be. For example, the OU uses learning analytics dashboards displaying learner and learning behaviour to our academic staff and associate lecturers (ALs) in order to provide more real-time, or just-intime support for students. (Herodotou et al., 2017Hlosta et al., 2015). Furthermore, some institutions like Universiteit van Amsterdam (Berg et al., 2016), University of Keele (de Quincey et al., 2019), and Maastricht University (Tempelaar et al., 2018b) have successfully experimented with providing learning analytics data directly to students in order to support their learning processes and self-regulation.
As also highlighted in Chapter 3, the role of teachers in making sense of these dynamic and complex systems is vital. In fact, how teachers are making sense of the teaching and learning practice, its students, and data arising from the complex interactions of students with learning resources, peers, and teachers, has become even more important in the last 5-10 years (Herodotou et al., 2017Hidalgo, 2018;Rienties et al., 2016a;, 2019Tait, 2018). As demonstrated by a range of projects within the OU as well as outside the OU (Guri-Rosenblit, 2018;Lawless & Pellegrino, 2007), the teacher is the key success factor in making pedagogy and technology work. As highlighted elsewhere in this book in Chapter 2, the OU has a relatively unique approach to teaching and learning, whereby typically central academic staff supported by TEL professionals design and produce high-quality online courses (Jones & Issroff, 2005;Jones et al., 1996). The actual implementation and "teaching" of these modules (i.e., courses) is done by a combined team of module academics and ALs, who typically would support around 20 students per group (Herodotou et al., 2017;Toetenel & Rienties, 2016a;van Ameijde et al., 2018). With current movements towards co-creation and integration of ALs and (former) students into module production and presentation, in this chapter we use the broader notion of a "teacher" to refer to a person working together with other experts to effectively design, implement, and/or evaluate the teaching and learning practices to meet students' needs (Olney et al., 2018;Rienties et al., 2013Rienties et al., , 2019. Using the Beyond Prototypes framework developed by Scanlon et al. (2013), which is described in Chapters 1 and 2, we will aim to illustrate how the holy trinity of learning design, learning analytics, and teachers can help institutions like the OU to ensure that our current and future students' needs are met. The Beyond Prototypes case studies that led to the development of the framework indicate that Technology Enhanced Learning (TEL) needs to be understood as a 'complex' . This 'complex' is made up of a series of elements that need to be addressed together, as reproduced in Figure 1.1 of Chapter 1 of this book.
The second to outer level of Figure 1.1 shows the different communities that are all involved in the TEL complex: the student community, pedagogic research community, teacher community and technical communities. These communities are all necessarily involved in our learning analytics work (Ferguson et al., 2016;Herodotou et al., 2019;Rienties et al., 2019). This work is being undertaken by a group of researchers within the Computers and Learning research group (CALRG). One aspect of that work is what we call data wrangling' (Ullmann et al., 2018) and includes iterative conversations with academics in the university who are responsible for developing our modules. Essentially the data wranglers team interpret the student data and then have conversations with academics about how changes might be made to the modules to improve student learning.
As illustrated in Figure 7.1, there may be inherent tensions between the three base layers of learning analytics, learning design, and teachers. Depending on how effectively organisations are able to balance these three "forces", the more we can meet the unique and individualised students' needs (i.e., the higher or flatter the pyramid will become).

Learning Design
As highlighted by a systematic review of 43 studies on learning design by Mangaroska and Giannakos (2018) few institutions have implemented learning design on such a large scale as the OU. Conole (2012) started experimenting with mapping learning design processes, whereby they "developed an approach to using learning design as a methodology to guide design and foster creativity in concert with good practice in the creation of learning activities". Building on this initial work, the OU's learning design taxonomy was established as a result of the Jisc-sponsored OU Learning Design Initiative (OULDI) (Cross et al., 2012;Rienties et al., 2017), and was developed over five years in consultation with eight other Higher Education institutions. In contrast to instructional design, learning design is process based (Conole, 2012): following a collaborative design approach in which OU module teams, curriculum managers and other stakeholders make informed design decisions with a pedagogical focus, by using representations in order to build a shared vision. For a detailed description of the OULDI approach, we refer to work published elsewhere Rienties & Toetenel, 2016;van Ameijde et al., 2018).
In one of the first studies to visualise the complex decisions that OU teachers make when designing courses, Toetenel and Rienties (2016a) used the OULDI approach to classify 157 modules at the OU. As illustrated in Figure 7.2, Figure 7.1: Balancing learning design, learning analytics, and support by teachers to meet students' needs. substantial depth and breadth of learning designs is present at the OU, perhaps reflecting the unique and diverse nature of the disciplines and the creative people that work at the OU. A considerable number of OU modules had a relatively high focus on assimilative activities, as well as assessment. At the same time, some OU modules used a perhaps more innovative pedagogical design, whereby for example Module 94 had nearly 60% of productive activities (i.e., creating, building, making, doing) for students to work with, while nearly 40% of activities in Module 56 were experiential (i.e., practice, apply, mimic, experience). Perhaps surprisingly for an online distance institution, less than 5% of learning activities on an average of modules mapped in 2016 were devoted towards communication activities (i.e., student to student, staff to student, student to staff).  Retrieved from Toetenel and Rienties (2016b).
In follow-up work of 148 learning designs by Toetenel and Rienties (2016b), the introduction of a systematic learning design initiative, consisting of visualisation of initial learning design and workshops, helped OU teachers to focus on the development of a range of skills and more "balanced" learning designs. As illustrated in Figure 7.3, when OU teachers were given visualisations of their initial learning design activities (i.e., orange) compared to teachers who were not given these visualisation (i.e., blue), they adjusted their designs towards more student-active activities, such as communication and finding information, while reducing the emphasis on assimilative activities.

Learning analytics
Although these above visualisations of learning design decisions made by teachers are an important advancement in terms of understanding our design practice, a next logical step would be to explore how these learning design decisions influence students' affect, behaviour, and cognition. One way to do this is to use learning analytics, which is commonly defined as "the measurement, collection, analysis and reporting of data about learners and their contexts, for purposes of understanding and optimizing learning and the environments in which it occurs" (Ferguson, 2012, p.307). As noted by Scanlon et. al. (2013, p.37) "learning analytics can provide actionable intelligence".
A considerable literature from the OU has emerged around both conceptual development (Clow, 2013;Ferguson, 2012;Ferguson & Buckingham Shum, 2012), how to evidence that learning analytics works (Ferguson et al., 2016;Ferguson & Clow, 2017;Rienties et al., 2016b), and how to design appropriate predictive learning analytics to effectively support different groups of OU students (Calvert, 2014;Herodotou et al., 2017;Rienties et al., 2019;Wolff et al., 2013). In fact, a recent bibliometric review of learning analytics has found that the OU is the most prolific institution in publishing about learning analytics (Adeniji, 2019).
With the arrival of fine-grained log-data and the emergence of learning analytics as a research field there are potentially more, and perhaps new, opportunities to map how students with different affective, behavioural, and cognitive learning needs want to engage with the OU Rienties et al., 2019;Rogaten et al., 2019). This is part of a commitment to investigate students' practices: part of our efforts to understand our learning ecology, an important layer of the TEL complex captured in Figure 1.1 in Chapter 1. As noted in Chapter 6, the CALRG's early work included a focus on affect. This was unusual at the time but this emphasis has continued. For example, with trace data on students' affect, the OU is currently exploring how emotional expression could be identified in written text, such as chat, discussion forums, or feedback from students (Aznar et al, 2016;Chua et al., 2017;Hillaire et al., Submitted;Ullmann et al., 2018). For example, Hillaire et al. (Submitted) showed that effective sentiment analyses approaches could be developed to identify positive, negative, and mixed emotions when 500+ students collaborated online in an interactive chat environment. Similarly, Ullmann et al. (2018) found, when using sentiment analyses of 51,000 student evaluation comments from 23 large OU modules, that substantial differences in lived and affective experiences could be identified.
Currently, experiments using techniques like eye gaze investigate how students are making sense of complex and simple texts (Rets, 2018). Furthermore, a range of studies within the OU have combined self-reported dispositions with how students are engaging with tasks over time (Tempelaar et al., 2012(Tempelaar et al., , 2015(Tempelaar et al., , 2018a. These affective data could be useful in providing more personalised feedback to students, such as giving automated hints to a "surface" learner with math anxiety that, say, engaging with a worked example on task 15 would help him to better understand this math problem and reduce his math anxiety, while for a "deep" but disengaged learner for the same task 15 providing a hint to read the theoretical modelling narrative could prevent her from being bored. In terms of students' behaviour, substantial progress has been made over the last five years in terms of identifying and predicting effective behaviour (e.g., engagement, time on task, clicks). For example, our state-of-the-art predictive learning analytics system called OU Analyse has been providing effective support to hundreds of teachers across dozens of modules where students might need some additional support (Herodotou et al., 2017;Hlosta et al., 2015;Wolff et al., 2013). OU Analyse uses a combination of machine learning and artificial intelligence approaches to predict which students are doing well, and who might be at risk not submitting the next assignment. One remaining challenge for learning analytics research is to deliver "actionable feedback", which might be achieved by taking into account the context in which learners, teachers, and the respective learning data is situated (Chua et al., 2017;Herodotou et al., 2017;Hidalgo, 2018;Rienties & Toetenel, 2016).
Finally, in terms of students' cognition some substantial progress has been made in the OU to signpost students about what they could do next, and what might fit better with their learning needs. For example, several module teams have been experimenting with asking for real-time feedback from students. Similarly, several module teams have implemented Computer-Based Assessments (CBA), which give automatic feedback to students . Preliminary analyses across 74 modules seemed to indicate that these CBA have a positive impact on engagement of students, and on higher pass and retention rates .
At the same time, as argued by Rienties et al. (2019) in a recent review held during an interactive workshop of leading experts and users of learning analytics at the OU, many of the 42 participants indicated a strong need to further develop learning analytics approaches to allow for effective communication and personalisation with students, while at the same time providing the learning analytics tools as part of an integrated design that is based upon a solid evidence-base.

Linking learning analytics with learning design
In terms of linking learning design with learning analytics approaches, several substantial steps have been made by CALRG researchers in the last five years (Mangaroska & Giannakos, 2018;Rienties et al., 2017). For example, Rienties and Toetenel (2016) linked 151 modules taught in 2012-2015 at the OU followed by 111,256 students with students' behaviour using multiple regression models and found that learning designs strongly predicted Virtual Learning Environment (VLE) behaviour and performance of students, as illustrated in Figure 7.4. Findings indicated that the primary predictor of academic retention was how teachers designed their modules, in particular the relative amount of so-called "communication activities" (i.e., student to student, teacher to student, student to teacher).
In contrast, student satisfaction was negatively predicted by these communication activities, whereby students in particular preferred to work in modules following more traditional distance learning designs, such as constructivist learning designs. This may be an important finding as in particular in online learning there tends to be a focus on designing for individual cognition rather than social learning activities (Arbaugh, 2014;Koedinger et al., 2013), while recently several researchers have encouraged teachers and researchers to focus on the social elements of learning (Arbaugh, 2014;Ferguson & Buckingham Shum, 2012) Building on this initial work, Quan Nguyen has made substantial steps towards more dynamic, temporal conceptualisations and empirical analyses linking learning design from a day-week-module perspective with how students are actually engaging  For example, a large-scale empirical study by Nguyen et al. (2017) on learning designs of 74 modules over 30 weeks revealed that the way teachers designed their Figure 7.4: Learning design strongly influences student behaviour, satisfaction and performance (Adjusted from Rienties and Toetenel (2016).
learning designs could explain up to 69% of the variance in VLE behaviours. For example, the weekly workload of the seven learning design activities of one module is illustrated in Figure 7.5. As highlighted from this visualisation, there were substantial fluctuations in expected workloads on a weekly basis, whereby there were specific weeks with a relatively high workload, primarily links to assessment points. As indicated by the red line in Figure 7.5, the average VLE engagement of students on a weekly basis also fluctuated, and primarily peaked when assessments were due. In follow-up work looking at when and what students are engaging with in one fully online module, ; found that students made conscious decisions not to follow the course schedule, by either studying well in advance, or catching up after the course schedule.

Role of teachers in using learning analytics and learning design
Irrespective of the specific learning design and the learning analytics approaches used, teachers will always play an essential role in online and distance learning (Guri-Rosenblit, 2018;Lawless & Pellegrino, 2007;van Leeuwen et al., 2015). Several authors (Herodotou et al., 2017;Rienties et al., 2013Rienties et al., , 2018a have indicated that beyond designing learning activities and managing the learning process teachers have a social, personal counselling role, whereby teachers provide pedagogical support and evaluate learning progression and outcomes. With the advancements of learning design and learning analytics it is anticipated that teachers will increasingly receive unprecedented amounts of Retrieved from Nguyen et al. (2017).
information, insight, and knowledge about their learners and their diverging needs. Learning analytics dashboards may provide teachers with opportunities to support learner progression, and perhaps personalised, rich learning (FitzGerald et al., 2018;Rienties et al., 2016b;Tempelaar et al., 2015). Indeed, two recent systematic reviews of 26 and 55 learning analytics dashboards studies (Jivet et al., 2018;Schwendimann et al., 2017) indicated that teachers and students will be able to obtain (almost) real-time information about how, where, and when to study. Beyond providing just-in-time support (Daley et al., 2016;Herodotou et al., 2017), learning analytics may help teachers to fine-tune the learning design if large numbers of students are struggling with the same task (Hidalgo, 2018;Rienties et al., 2016a;Rienties & Toetenel, 2016). In line with the Beyond Prototypes framework, paying attention to the ecology of practices, one layer of the policy context is a key element of our approach to learning analytics, so teacher and learning practices and perceptions are investigated. Regarding teachers, a recent large-scale study by Rienties et al. (2018a) amongst 95 experienced teaching staff at the OU indicated that many teachers were sceptical about the perceived ease of use of learning analytics tools. Most teachers indicated a need for additional training and follow-up support for working with learning analytics tools.
These findings resonate with a recent study by Herodotou et al. (2017), who compared how 240 teachers made use of learning analytics predictions and visualisations in OU Analyse at the OU (Hlosta et al., 2015;Wolff et al., 2013). Herodotou et al. (2017) found that most teachers struggled to turn learning analytics predictions and recommendations into concrete actions for their students-at-risk. Follow-up qualitative interviews with five teachers who used OU Analyse indicated that they preferred to learn a new learning analytics system by experimenting and testing the various functionalities of learning analytics dashboards by trial-and-error (Herodotou et al., 2017;Herodotou et al., 2019). However, at this moment the OU does not actively track how teachers are making interventions, and what the best way could be to provide effective feedback for different groups of students.

Conclusion and future directions
In the last ten years universities and distance learning institutions like the OU have experienced unprecedented change. Beyond the "neo-liberalist waves" running through many universities, the affordances and limitations of technology to transform universities as exciting and relevant places of learning and teaching have fundamentally impacted the way universities are run, as indicated elsewhere in this book in Chapter 2 and Chapter 3.
The central message of this chapter is that learning design, learning analytics, and teachers together can support student success. With the emergence of learning design combined with learning analytics, there is an increased narrative developing that teachers should start to pro-actively think, reflect, and act upon data. While there is widespread evidence that learning analytics tools and predictive engines could accurately identify which students might need some additional support, there is mixed evidence (Ferguson et al., 2016;Ferguson & Clow, 2017;Herodotou et al., 2017;Rienties et al., 2018a) as to whether universities and teachers in particular are ready to engage with these tools and approaches.
As emphasised in the Beyond Prototypes framework: "In the TEL complex, practices include explicit aspects of teachers' practices…. " (Scanlon, 2013, p.29), and it is increasingly evident that without an appropriate understanding of the context in which learners and teachers are learning, learning analytics may not be as effective as hoped (Ferguson et al., 2016). At the same time, our ground-breaking research Rienties et al., 2018b;Rienties & Toetenel, 2016) linking learning design (i.e., what do teachers design) and what, how, and when students are actually engaging with these learning activities could have a transformative impact on how we teach at the OU, and perhaps more importantly how we can develop, test, and implement new educational theories of effective learning design. Given the tremendous impact of learning design on what students do on a daily and weekly basis ;), we need a much better understanding of why teachers are designing particular learning activities, and how these learning activities relate to learners' needs.
Indeed recent research has highlighted that on a more micro-level learners at the OU have substantially different learning needs and ambitions (Law, 2015;Li et al., 2017), depending on a complex interplay of affective (Hillaire et al., Submitted;Tempelaar et al., 2018b), behavioural (Chua et al., 2017;Rets, 2018;Rizvi et al., 2018), and cognitive factors, as well as socio-economic and demographic factors (Richardson, 2015). Therefore, in the remainder of this chapter we will primarily focus on how to provide more personalised and individualised support for learning in the next 2-5 years.

Moving forwards
With the renewed and increased interest in Artifi cial Intelligence (Holmes et al., 2019;Luckin et al., 2016;Rizvi et al., 2018) there is an emerging narrative developing that universities should start to embrace some of the affordances of AI. In particular for some of the more mundane tasks that students and staff need to complete on a frequent basis (e.g., registering for a course, asking for an exception, replying to standard emails), providing automated responses using AI could provide some quick efficiency savings. Similarly, in providing automatic responses to standard or frequently asked questions, chat bots can learn to effectively support learners.
Personalisation (FitzGerald et al., 2018) and student-led analytics Prinsloo & Slade, 2017) are two specific themes that are emerging from the literature that could start to play an important role for distance learning providers in the near future. Although distance learning theoretically can provide flexible options to learners depending on their needs, we continue to see many distance learning providers offering "one-size-fits-all" courses starting on say the 1 st of October and finishing in June/July. Given that many learners do not necessarily want to follow courses on these dates, some may want to start earlier or later, and others might want to move faster or slower (FitzGerald et al., 2018), it remains interesting why most providers of education are still focussed on one-size-fits all solutions. Obviously, economic efficiency arguments are provided, like economies of scale, and logistical and administrative processes need to be adjusted to accommodate multiple variations of a course, but with the support of learning analytics and a student-led analytics approach, distance learning organisations could vary their provision to different groups of learners, with specific learning needs. Finally, student-led analytics, whereby students themselves determine what they want to share and see in terms of their own data Prinsloo & Slade, 2017), will become an emergent issue that distance learning institutions need to plan for. Again the Beyond Prototypes framework could be useful to help distance learning organisations to visualise the complex and changing relations.