The Babson survey itself noted that, even as more students are taking online courses, the growth rate is slowing and some university leaders are becoming more skeptical about how much students are really learning. For example, the percent of academic leaders who say that online education is the same or superior to face-to-face instruction dipped to 74 percent from 77 percent. That was the first reversal in a decade-long trend of rising optimism.
Opinions are one thing, but what do we actually know about online learning outcomes? Unfortunately, there have been thousands of studies, but precious few that meet scientific rigor. One big problem is that not many studies have randomly assigned some students to online courses and others to traditional classes. Without random assignment, it’s quite possible that the kinds of students who choose to take online courses are different from the kinds of students who enroll in traditional courses. Perhaps, for example, students who sign up for online classes are more likely to be motivated autodidacts who would learn more in any environment. Researchers can’t conclude if online learning is better or worse if the student characteristics are different in online courses at the onset.
On June 3, 2014, an important study, “Interactive learning online at public universities: Evidence from a six-campus randomized trial,” published earlier in Journal of Policy Analysis and Management, won the highest rating for scientific rigor from What Works Clearinghouse, a division inside the Department of Education that seeks evidence for which teaching practices work. The study was written by a team of education researchers led by economist William G. Bowen, former president of Princeton University and the Andrew Mellon Foundation, and Matthew M. Chingos of the Brookings Institution.
The online study looked at more than 600 students who were randomly assigned to introductory statistics classes at six different university campuses. Half took an online stats class developed by Carnegie Mellon University, supplemented by one hour a week of face-to-face time with an instructor. The other half took traditional stats classes that met for three hours a week with an instructor. The authors of the study found that learning outcomes were essentially the same. About 80 percent of the online students passed the course, compared with 76 percent of the face-to-face students, which, allowing for statistical measurement errors, is virtually identical.
The university campuses that participated in the fall 2011 trial were the University at Albany (SUNY), SUNY Institute of Technology, University of Maryland Baltimore County, Towson University (University of Maryland), Baruch College (CUNY) and City College (CUNY).
It’s worth emphasizing that this was not a pure online class, but a hybrid one. One hour a week of face-to-face time with an instructor amounts to a third of the face-to-face time in the control group. Also, the Carnegie Mellon course not only required students to crunch data themselves using a statistical software package, but it also embedded interactive assessments into each instructional activity. The software gives feedback to both the student and the instructor to let them know how well the student understood each concept. However, the Carnegie Mellon software didn’t tailor instruction for each student. For example, if students answered questions incorrectly, the software didn’t direct them to extra practice problems or reading.
