Sander Latour recently wrote a brief article in which he reflected upon his experience at EDUCAUSE 2013 and, in particular, on the ways in which learning analytics was represented at the conference. In this article, he argues that the ways in which predictive analytics are currently employed in higher ed share several characteristics with fast food: cheep and readily digestible, but perhaps of little long term nutritive value. While I agree with Latour’s conclusion that “Learning Analytics should be about learning, and learning is what we should aim for,” I would like to both temper and strengthen his argument by clarifying the distinction between learning and predictive analytics.
I have several significant reservations about the definition of learning analytics posited by SoLAR, but it is frequently cited as authoritative and will do for the purpose of this discussion:
“SoLAR defines learning analytics as the measurement, collection, analysis and reporting of data about learners and their contexts, for the purposes of understanding and optimizing learning and the environments in which it occurs” 
Learning analytics, then, is a general term referring to any use of data that seeks to facilitate learning (I disapprove of the language of ‘optimization,’ here, which presumes a very narrow conception of learning as something that can be evaluated with respect to a known standard. Although this may be possible in the case of learning conceived in terms of information transfer, or perhaps skill mastery, it fails to appreciate that there are other types of learning that are not so easily measured or evaluated . . . but I digress). Predictive analytics, on the other hand, makes use of inferential statistics and/or machine learning methods in order to make probabilistic determinations about future performance on the basis of past behavior and other dispositional characteristics. Because of the generality of the term ‘learning analytics,’ there will definitely be some situations in which predictive and learning analytics overlap. If we conceive of learning as skill mastery (perhaps operationalized as something like having answered a specific number of questions correctly on a set number of consecutive quizzes), then it may be possible to strongly correlate something like LMS activity to learning rate. There are, however, applications of predictive analytics that are not, strictly speaking, concerned with learning, but rather are better described as academic analytics (i.e. business intelligence specifically applied to educational contexts). My favorite example of predictive analytics comes from Georgia State University, where Timothy M. Renick discovered that high performing students with even small unpaid bills the day before the payment deadline were at an increased risk of attrition. Consequently, GSU offered nearly 200 mini ‘Panther Grants’ to students who were dropped for nonpayment. The grants successfully nudged these students over the payment hump, thereby increasing the school’s retention rates and generating more than $660,000 in tuition and fee revenue that would otherwise have been lost. In speaking with Dr. Renick in person about the project, I was pleased to learn that their predictive analytics initiative also included giving every academic advisor a second monitor, so that at-risk students had an opportunity to review their behaviors, and collaborate with advisors in determining a plan of action that would increase their chances of success at the institution (i.e. retention, minimum grade point achievement, vocational aspirations, and satisfaction).
Predictive analytics, then, amounts to arriving at probabilistic expectations about future performance on the basis of past behavior (this, of course, raises the question of whether learning is a behavior, or whether behaviors merely function as an indirect way of quantifying learning, which is intractable). Is Purdue’s Course Signals (and other similar analytics products, like Blackboard’s new Retention Center, for example) a predictive analytics product? Latour uses Course Signals as exemplary in his criticism of predictive analytics. This is not, however, an accurate characterization, despite the fact that it and other similar products are frequently treated as if they had some kind of predictive power (such products, in fact, encourage this kind of mis-characterization through the use of labels like ‘on track’ and ‘at risk’). These kinds of dashboard are not predictive, because they are not probabilistic. They neither employ inferential statistics nor machine learning methods. Rather, they are dumb indicators that merely report interaction frequencies (login attempts, access to materials, grade performance, etc) and produce alerts if a particular student’s behavior deviates beyond a pre-determined (and usually arbitrary) percentage from the class average (note, that these dashboards do not even employ measures of dispersion, to check to see if a particular student’s performance differs significantly from the mean). These products are easy to produce, and so relatively inexpensive, but I agree with Latour that they are not particularly interesting. I would add, however, that they are also potentially dangerous, since they make implicit claims to predictive power that are illegitimate, but may nevertheless be taken seriously by instructors and students alike.
In a recent meeting with representatives of a large LMS firm, it was mentioned that a review of data from one institution revealed that there was no significant observable difference in performance (grade point or retention) between students with high grades on the basis of engagement. In other words, among students entering a course with high grades, their level of engagement within the online learning environment had no significant impact on their final performance in the course. With ‘fast-food’ analytics products, high performing students with low levels of engagement may quite possibly be flagged as at risk, at the same time as their levels of engagement lower the class average in a way that makes unengaged low-performers more difficult to detect. A truly predictive model (predictive analytics as ‘haute cuisine,’ so to speak) would easily deal with these differences).
Predictive analytics are not all ‘fast food.’ In order for our predictive analytics to be valuable, however, we need to ensure that claims are actually predictive (rather than description masquerading in predictive clothing), and that we are clear about what exactly is being predicted (i.e. a measurable behavior). In the absence of these considerations, however, our analytics will be fast food, indeed.