attrition and the year of the MOOC

Like tens of thousands of others, I signed up to take a few of the online course offerings made available through udacity and coursera. And, like tens of thousands of others, I completed nary a one of them. I knew this was going to happen. In fact, I preregistered and registered for the classes with almost no intent on finishing them. Really, I just wanted to see what the offerings were like, across a series of computer science and social science areas I have interests in. I also imagined I would pick one of them, and well and truly do the class. I didn’t. I’m fairly self-motivated for things I find stimulating. (I did, after all, actually write a dissertation.) But, the demands of family, commuting, teaching three classes, serving as a book review editor, and all the rest squeezed out my ability to really concentrate on any of these classes. In fact, for the most part, I didn’t manage to get past watching a few videos.

And, it turns out, neither did droves of other people.

One of the classes I was most excited to follow was Natural Language Processing, offered by a pair of Stanford professors. The course just recently concluded after its 8 week run. Interestingly, in a final email, the professors included statistics about course participation:

  • Preregistered: ~70,000
  • Registered: 42,223
  • Watched at least 1 video: 24,287 This group included me!
  • Watched all videos: 4,030
  • Got a statement of achievement: 1,466
  • Total video downloads: 860,854

There were eight programming assignments. The stats on submission of those assignments:

  • PS1 8,425
  • PS2 4,527
  • PS3 3,721
  • PS4 2,994
  • PS5 2,529
  • PS6 2,005
  • PS7 1,911
  • PS8 1,654

These numbers represent quite astounding levels of attrition, and bring to the fore, I think, a number of questions for the efficacy of this particular form of MOOC. First, I wonder what the difference in participation rates is between a course like this on coursera or udacity, and Jim Groom’s Digital Story Telling? Granted, the two courses offer very different experiences and to very different ends. But, those differences point directly to tensions in the move to offer open, online courses. On the one hand, Groom and his merry band of pranksters are engaging in acts of creativity that have next to nothing to do with credentialing. The CS and other courses offered in this Spring’s explosion of MOOCs feel, to me, much more pragmatic, much more credential-oriented. It’s like the difference between genuine DIY/edupunk learning and Kamenentz’s retail version, analyzed so well in Stephen Downes’ review of Kamenentz’s The Edupunk’s Guide.

Certainly, many people in the cast of thousands who dropped out of the NLP class did so because the content was hard. Hard in a technical way that’s different from the ‘hard’ of a course on digital storytelling. And, there is no denying that for the 1400 people who finished the course successfully, the NLP MOOC was a success. Heck, I know more about NLP (and command line utilities, for that matter) just from the handful of videos I watched. I’m sure that many others learned some subset of the course’s objectives as well. But, the attrition statistics point to some real problems with the online model still proffered by ventures like udacity, coursera, and the other elite university ventures. I know that if I had attrition rates of 97% in my traditional face-to-face courses, I would have intervention from my department, from my college. Is it fair to compare the two settings? I think it is, only to the extent that advocates for online education trumpet the internets as a source for revolutionary change in the higher ed model. Or, it could just be that the NLP class is an outlier, and the material is hard enough that only 3% of those who were motivated enough to enroll themselves in the class were capable of getting it. I doubt that’s the case.

More likely, the current model of user-generated forums, videos with embedded multiple-choice questions, robo-graded assignments, and credential-oriented outcomes really don’t generate the kind of engagement needed to keep the 97% involved for just 8 weeks. Is this a glass-half-full/glass-half-empty type of question? I wonder. Groom and ds106 raised $12,643 from kickstarter in a few weeks, in part because they lacked institutional support of a Harvard, Stanford, Michigan, MIT, or Penn or the ‘startup’ feel and Silicon Valley ties of udacity’s Sebastian Thrun. And yet, as an outside observer of this process (and small time kickstarter supporter of ds106), the educator in my finds one of these two models much more exciting than the other.

Did any of you finish one of the coursera or udacity courses?

About these ads
About

Associate Professor of Early Latin America Department of History University of Tennessee-Knoxville

Tagged with: , ,
Posted in Uncategorized
13 comments on “attrition and the year of the MOOC
  1. tedunderwood says:

    My experience was very like yours. I wanted to see what was on offer, but doubted from the beginning that I had the time to work through their syllabus (or that it was necessary, given how much motivation I already have to learn specific topics … as they become necessary to my agenda).

    I’m not sure people like you & me should count as “attrition,” because this was a very planned and deliberate sort of attrition. And there might have been a fair number of people among the registrants who had similar spectatorial motivations from the beginning. However, I certainly do agree that motivation and time management are issues for the MOOC model.

  2. I watched all of the Machine Learning videos and did 90% of the quizzes when it was offered in Fall 2011 (at that time, they had 2 official tracks) and I watched about 60% of the NLP courses, but didn’t do the quizzes or home-works. I really liked the ML class and got a lot out of it because it was essentially a lite-intro to the math behind learning algorithms. I feel I *get* learning algorithms at a fundamental level, even if I’m not in a position to do anything with them professional.

    In all, my experience was positive.

    However, I suspect there’s something else at play here too. I almost finished a PhD in linguistics and took several NLP and machine learning related courses a few years ago (I found the Coursera NLP course a bit too simplistic and I got bored). But I know a fair number of natural language processing professionals (i.e., trained engineers) who also took the courses and tweeted and blogged about them. They are the ones who finished, I suspect.

    I think that’s where these MOOCs stand right now. As good professional development. There is a parallel in the IT industry with companies like Learning Tree. If you’re a working IT pro who needs to brush up on network security, you can pay to take a quick course. These classes are very expensive though (in the thousands of dollars).

    MOOC offers a platform to bring high quality specialty topics to working professionals to augment their skill set. Can it serve as a first-course? That I doubt, at least not with current technology. Had I never taken graduate level coursework in the topics already, I’m not sure I would have found them as satisfying.

  3. Chad,
    I think you hit the nail on the head, the bottom line is ds106 is really not about credentialing for the open online students, it is more about community, honing certain skills, and deciding to push one’s creativity. As far ads the drop-out rate for registered students it is very, very low, maybe 1-5% (but that is roughly 50-200 students across all registered classes). As for open, online students, that is far more difficult because we don;t offer anything in terms of a credential, we simply acknowledge, unlike most MOOCs you listed, that you do what you do and move on.

    In all this though I think the real difference is the idea of a community behind a class, and ds106 has been fortunate enough to create one that has been sustained over time. Why? Well because for many it is both fun work, practical for sharpening skills, and professionally useful for creating deeper connections. It’s a network of activity born of loosely connected creative interests.

    Another thing that Brad Kozlek brought up that I think is important to a ds106 model, is the “Why wasn’t I consulted” logic of the web (http://bradkozlek.com/2012/05/why-wasnt-i-consulted/), in other words how does a class allow those taking it to integrate people’s concerns, ideas, and creativity. That for me is the real power of the ds106 model, and the idea of courses that abdicate enough power to let those taking it help build the experience is truly rare currently, but I think it is increasingly going to be the norm.

  4. Matt says:

    I think this points to a failure to rethink the classroom model as education moves to the web. Though Coursera and MITx/EDx and others are doing an impressive job of expanding classes offered at elite institutions to new audiences, they’re doing little to rethink the ways in which online learning (especially in such massive, distributed environments) differs from in-person classes. Participating in these classes mostly involves watching videos of lectures and taking quizzes — hardly a forward-thinking vision of online education.

    One of the things that’s really notable and impressive about Jim’s work on DS106 is the way the project handles assignments. Not only can students submit assignments and have other students take them (thus reworking the top-down model of the traditional classroom), but the course also encourages loose, flexible, and open encounters with course materials that have the effect of making it a less structured learning experience than what we’re seeing on Coursera and the like. As Jim notes, it’s about a larger community experience of participatory learning, but it’s also about leaving room for participants to float in and out of the experience throughout the semester (and beyond).

    All of this is making me rethink my somewhat negative reaction to the Mozilla badges project; maybe what’s needed in these MOOCs is the opportunity for students to have more limited, modular, and open engagements with course materials that would provide them with some kind of acknowledgement of learning/doing the work, but that would not necessitate participation in every element of the course all the way through the semester. And that, perhaps, would allow us to rethink what we mean by “attrition.”

  5. I think one of the most important things we need to consider, is whether ‘courses’ are even a valid way to learn. 42,000 people, with different prior knowledge and experience, expected to follow the exact same path to get from their own ‘here,’ to an expert’s idea of ‘there,’ seems a bit absurd.

  6. Mike Cunha (@almostMike) says:

    I’m one of the 1466 that hung around for the full NLP course and completed all the work. I also completed 2 other Coursera classes this Spring. Here’s my take:

    As tedunderwood stated above, counting registrants like Ted and Chad in an “attrition rate” is misleading. I don’t think it’s really fair to generalize about the success/failure of the course based on the few summary statistics cited in the article. These seem like a poor proxy to participants true motivations and behavior.

    It’s definitely not fair to compare to face-to-face in-classroom rates. Students in that setting have much more at stake when making the decision of whether or not to stick with it (namely, tuition money). Coursera registrants on the other hand have nothing at stake, and that’s great! In my opinion, that’s the best part about all the open-classroom platforms popping up this year; anyone is free to get their feet wet and investigate just what it would take to really understand a topic that interests them.

    For many, the amount of time required to complete all the assignments and finish the course will probably outweigh their interest in completing the course. For others, the course may only present a couple of topics that the registrant actually wants to learn about. I’d be more worried about a course that didn’t have 10′s of thousands of registrations (read interest).

    As far as my experience went:

    I was impressed with the community that developed in the forums for the NLP class despite this online course being so structured and similar to an in-classroom syllabus. I don’t have any experience with Udacity or DS106 so I can’t compare. Now that the class is over I don’t think the forums would have been nearly as productive or lively without the structure of the class.

    Contrary to what some others have said, I liked the structure of the class… I liked that there were deadlines. I would not have learned the material covered at such a quick pace otherwise, I’m too busy. Everyone’s busy and there’s always an excuse to do it later. With deadlines, everyone in the forums was generally at the same point in the class, working on, and discussing the same topics. That said, they clearly underestimated the need for more TA’s active participation in the forums.

    For me, the class difficulty was quite good. There was definitely a lot of room for improvement, but considering who was teaching the class and the free price-point, I can’t complain.

    As far as addressing user-suggestions and adapting as the course progressed I felt like they were incredibly responsive considering the professor’s other teaching commitments. Expecting the syllabus to change half-way through an 8-week course in it’s pilot run is a little unreasonable in my opinion.

    The forum space did seem to evolve fairly quickly though, in other Coursera classes like Design and Analysis of Algorithms students created assignments and had active recitation-style discussions in the forums and the professor posted additional thoery questions… you could dive in as deep as you wanted.

    Overall, I think Coursera has taken a pretty good first step in adapting some very exclusive in-classroom courses to the free web and I plan on taking more of their offerings.

  7. [...] been working through CS101: Building a Search Engine on udacity. I’ve written critical things about the new wave of massive online courses before, in part because I’ve never finished one. [...]

  8. [...] Attrition and the Year of the MOOC – This blog post discusses the attrition rate of MOOCs (which can be as high as 97%). Since [...]

  9. [...] on parezcoydigo.wordpress.com Like this:LikeBe the first to like this. Bookmark the [...]

  10. [...] engage students? That might genuinely be a significant step forward, provided it can overcome the 97% attrition rate some early Udacity attempts have seen. But engaging interactivity remains a potential, not [...]

  11. Meg McCabe says:

    I am currently taking a Coursera course along with 170,000 other people. At least that many signed up. My experience mirrors what has been posted here. The first week was a flurry of excitement. I signed on to a student-led study group with 18 others who all agreed to take responsibility to lead the online discussion for one of the 12 weeks of the course. We are now in week 8 and all but two of us have dropped out. I am determined to stick with it as part of my research on online teaching models. Here’s my take-away thus far:

    1. The unidirectional dissemination of content is informative, but not engaging.
    2. I consider the Coursera model more of a LEARNING RESOURCE than a course because it does not provide interaction or feedback from the professor or any instructor. This seems to me one of the features that distinguish courses from book groups or study groups that have a shared focus and materials, but work on their own. There is nothing inherently wrong with this, it’s just not a course.
    3. Planning a course — online and face-to-face — requires one to consider how learners interact with the material. Most educators recognize that this is a subjective process that involves students questioning and expressing ideas as they are formed. The teacher (as someone with presumably more experience with the material and an eye on the course goals) has a responsibility to participate with the students in the learning process. If not, they might as well just write a book, make a video and call it what it is — A Learning Resource.

  12. skipper44 says:

    Simple fact: Thank to Udacity CS101, I’ve picked up a skill that I did not have earlier. Even if I haven’t made it past Unit 6, I am ‘good to go’ on reading a problem that can be solved using code.

    Sure we pick up skills throughout our working life, but not in such quantum chunks. There’s no doubt about it, I can pick up a problem and be reasonably confident I can write code to solve/address it. My solution may not be the most efficient. When was the last time I could boast of such achievement?

    . . . . and yes, I could never have progressed through this learning process without the MOOC platform, unevolved as we may think – more akin to a WIP (work in process), something ongoing as we considered Joyce’s towering example of literary work. I think back to occasions during my post college life when I thought I’d learn from programming books, and run through a lot of code with paper and pencil. How unrealistic and frustrating that would’ve been – perhaps putting me off learning this for life.

  13. […] tweets, meet-ups, etc.) We must leave factory and assembly-line models of education behind. The super high attrition rate of MOOCs (80-90% for many courses) is not a weakness, but a strength. It means many people are finding it […]

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

parecer:
parecer:

Hacer juicio ú dictamen acerca de alguna cosa... significando que el objeto excita el juicio ú dictamen en la persona que le hace.

Deducir ante el Juez la accion ú derecho que se tiene, ó las excepciones que excluyen la accion contrária.

RAE 1737 Academia autoridades
Buy my book!



Chad Black

About:
I, your humble contributor, am Chad Black. You can also find me on the web here.
Follow

Get every new post delivered to your Inbox.

Join 50 other followers

%d bloggers like this: