How to Measure the Value of a MOOC
"Just ask students," writes Medill's Owen YoungmanDecember 9, 2013
This article originally appeared in the Chronicle of Higher Education on Dec. 9, 2013.
By Owen Youngman
What's the value of my university's name on the "statement of accomplishment" that students just earned for passing my first massive open online course? Is it at all enhanced by the robo-signature of their new favorite MOOC professor? Or, as Samuel Goldwyn probably didn't say about verbal contracts, is it just not worth the paper it's written on? (And might it be of higher value if the students print it out using a laser printer instead of an inkjet?)
My facetiousness shouldn't mask the issue lurking behind the questions. When a student's educational investment is measured mostly in time, attention, and effort rather than in dollars, euros, or bitcoin, what sort of value will be ascribed to the credential she ultimately earns?
I discussed that with David L. Parkyn, president of my alma mater, North Park University, in Chicago, over breakfast recently as I was getting ready to teach my first MOOC, for Northwestern University's Medill School of Journalism, Media, Integrated Marketing Communications. After batting around some of the typical bullet points on size and international reach, we moved quickly toward the college model, which links two things that have not always been quite so intertwined: credentialing and educating.
"Credentialing" helps to pay some of the bills for "educating." Tuition dollars from credential seekers (and their parents and loan officers) allow a college to pursue its broad public purpose and permit faculty members to pursue their own research, passions, and self-improvement while sharing their perspectives with the seekers.
So, what MOOCs may force us to consider is whether a particular credential—a diploma—is necessary to demonstrate learning, as well as who gets to decide the relative value of the possible alternatives.
In my MOOC, "Understanding Media by Understanding Google," which was held on Coursera, students were assigned to read a few dozen excerpts from six recent books and watch my interviews with three of their authors. They were also asked to read or view more than 80 other pieces of background information, from newspaper and magazine articles to blog posts and YouTube videos. In addition, they were to take half a dozen quizzes and write five brief essays, with citations, in response to homework questions. On top of that, they were asked to interact with one another in discussion forums.
A total of 1,196 students in 87 countries did well enough to earn a "statement of accomplishment," scoring at least 70 points out of 100. Of the 19,000 who ever came to class out of the 55,000 who registered (and the 2,400 who turned in homework), they're the ones who "passed." What will they do with that printout? What does their effort mean? Not surprisingly, among the more than 25,000 posts in our discussion forums, several hundred tackled that question head-on.
- "Work is scarce and every little bit helps, so that's the main reason I'm focusing on getting a certificate. The efforts are the same, but the proof of them will help me along."
- "I hope anyone who reads these [certificates] also appreciate and give credit to the discipline and diligence it took to ace the quizzes and complete these courses."
- "As an employer, I would not recruit anyone based on MOOC certificates, but I would accept that someone who has them has a genuine interest in further education."
- "At the end of the day people aren't going to challenge you on the certificate, they are going to challenge you on your ideas, which you can develop throughout the course."
- "I post all courses [on LinkedIn] because I think it shows I'm committed to continuing education. The topics I choose to study say something about my interests."
To go with such qualitative feedback, I also gathered quantitative data through a 20-question post-course survey of the 1,196 certificate earners, more than 800 of whom responded. When asked for their level of agreement with the statement "I plan on listing my participation in a résumé, CV, or online profile," 73 percent of the 755 who answered that question said they "completely" or "mostly" agreed. And why not, given that 89 percent of the sample agreed just as strongly that "taking this course has made me feel smarter, better educated, and/or better informed"?
So those students will largely have no compunction about attempting to boost their prospects by highlighting the "discipline and diligence" it took them to succeed in the course. Perhaps that's the first half of assigning value. The as-yet-determined half is, Will bosses and hiring managers buy it? My early view is that many will.
I am tempted to draw a comparison between how college-admissions offices look at the applications of hundreds of equally qualified prospective students today and how they did so when the only things I offered coming out of high school were my GPA and test scores. When browsing a résumé at high speed, will managers who see university names like Northwestern, Stanford, and Penn dismiss their inclusion with a quick "Oh, never mind, that was free"?
In my survey, some 90 percent of my "graduates" said that they had already earned at least a four-year degree. So it's not about replacing their existing credential—just adding a few new ones in order to get ahead or stay ahead. Or, based on the survey question above, to add to the confidence they need to continue to better themselves. Perhaps that sentiment is best evidenced by this quote from a student: "It doesn't matter [about the certificate], I'm a winner anyway. I won over myself, my laziness, my lack of time, all other possible excuses. I didn't have anybody or anything that would push me toward it, like I have at school. It was just me and my motivation."
That's a balance of "credentialing" and "educating" that I am happy to embrace.
- Owen Youngman is a professor of journalism at Northwestern University.