Higher Ed Analytics


I was reading Karine Joly's post on the CollegeWebEditor blog about what the "next big thing" might be for higher education analytics. Big and small data has been a hot topic outside education for longer than it has been hot in education, so we have some catching up to do.

Karine got 11 speakers from the 2nd edition of the online Higher Ed Analytics Conference (Feb 5, 2014) who are analytics experts in higher ed to make some predictions.

Here are 6 that I think are good ideas to consider for your own institution's analytics roadmap.

Universal Analytics – Shelby Thayer from Penn State University "I’m hoping it can be to find a way to tie together the entire web presence and experience. Look for Universal Analytics to play a major role over the next year or two. I’m excited to see what the impact will be. Although, for higher education, I think the 'next big thing' is in our hands – something we can overcome ourselves with more resources and better collaboration."

Google Tag Manager – Colleen Clark from Ithaca College "I think higher ed institutions are starting to become more comfortable with the idea of integrating analytics with online marketing. As new digital campaigns are launched, multiple tags may need to be placed on the website. Tag Management solutions such as Google Tag Manager will begin to become more widespread in 2014 by replacing multiple tag requests with a single unified code on the website for all tagging needs."

Predictive Analytics – Michelle Tarby from Le Moyne College " I’d like to move into using Google Analytics to do predictive modeling. Can we use the behavior of our visitors to predict someone is more likely to apply, visit or request more information about us? What does that look like? How does the source of the visit relate to meeting a goal? Are other metrics a better predictor (number of visits, particular pages they end up on, number of pages viewed)? Once those models are built, how do we build content targeting? What increases the likelihood of conversion based on our models?"

Multi-Channel Integration – Stephanie Hatch Leishman from MIT "While this isn’t the 'next big thing' in the for-profit world, it’s something I believe we’re still aiming for in higher ed. In 2014, I see more universities focusing on analytics throughout all their communications, including email, social, web, print, etc. For many institutions, content creation still may not be fully integrated, and still in silos, so our analytics follow suit."

Real-Time Social Media Analytics – Tim Nekritz from SUNY Oswego "I expect better, more robust reporting of real-time analytics will really allow us to pivot and change content and delivery even faster. The main hitch in Google and social media analytics is the delay in receiving the most valuable quantitative and qualitative information. Once somebody figures out how to streamline that so that I quickly know, say, whether something I posted on the homepage that we think is important is or isn't getting any play, we can think about whether modifying its placement, related visual or phrasing can help it perform better."

Finally, here is one thing that doesn't require a lot of technology, but might be even harder than implementing analytics software on a campus.

Data Sharing Among Institutions – C.Daniel Chase from The University of Tennessee at Chattanooga  "The next big thing that I would like to see is, cross-institutional sharing of data. The idea being that universities could voluntarily agree to participate and compete a short form describing their institution (public/private, 2/4 year, undergraduate & graduate enrollment) and add a short piece of javascript to their pages that would submit their data to a central account—perhaps using Google Analytics—that could then be used as a resource for site comparison to your peers. With this kind of data, everyone would have a better perspective on how well they were doing individually, and they would be able to review those that were doing well for good ideas! The ‘friendly competition’ would all help us develop better websites!"

The Educational Core and Standards

measureAs we were headed into the new school year last fall, an ASCD poll on the "Most Attention Getting Topics for the Year in K-12 Education" had caught my attention. 

Included were some hot and buzz-worthy topics like:
STEM/STEAM with a rating of 7.40%
Data analysis and decision-making  5.67%
The changing role of educators  3.31%
Personalized learning/adaptive learning  2.99%
Early-childhood education  1.57%
Online and blended instruction/MOOCs  1.42%

But for the K-12 world of education, the topic of the Common Core Standards and their orientation, implementation and assessment got the overwhelming vote at a whopping 77.64%.

Unfortunately, in higher education there is very little recognition or interest in these Common Core State Standards. That's unfortunate because they will have a real impact on the kind of students we see in the years to come.

The Common Core Standards are an effort to provide a consistent, clear understanding of what students are expected to learn, so teachers and parents know what they need to do to help them.

The standards were designed to be robust and relevant to the real world and hopefully reflect the knowledge and skills that young people need for success in college and careers. Since this is a national effort, there is much talk about American students being prepared for the future and being able to "compete successfully in the global economy."

It's hard to not to agree with those intentions. But the Standards were met with lots of resistance from teachers, schools and even from states. I think that is to be expected with any big program that tries to set English and math standards and is pushed by Washington, D.C. education trade organizations and the current administration.

Some states have put on hold or even de-funded implementation of the standards. Some have pulled out of the consortia developing tests tied to them.

Federal programs like Race to the Top grants and No Child Left Behind waivers are often tied to the adoption of Common Core standards and assessments. But the Race to the Top money is spent and so states are taking a different view at Common Core.

It is an over-simplification to explain the standards in just a few sentences, but what got a lot of press was that Common Core reduces the amount of classic literature, poetry and drama taught in English classes by 60 percent in favor of reading non-fiction - the prose of work.

In the math area, it delays the progression to Algebra I (seen as the gateway course to all higher math) by two years.

Media coverage, like this NPR report, like to point out the extremes and inconsistencies. The public can easily see that if a fourth-grader in Arkansas gets a “proficient” on his state test but would have been given a "failing" score on it he lived in Massachusetts, something is wrong.

That is why many supporters of having clear and high standards for every child in the United States by grade level see it as a practical and effective solution for the idea that now some students learn less than peers in other states.

I am not enough of an expert to praise or condemn the Standards, but I do think that ALL educators, especially those who are post-secondary, need to become better informed.

The Pop Culture MOOC Experience

65,000 people signed up for a MOOC offered on the Canvas Network by the University of California, Irvine that was based on AMC's very popular TV show The Walking Dead.

It makes an interesting test case. Though the "course" (not really) had interdisciplinary objectives, it sounds like it could be fun for the viewer who wants to get more into it.

It was called "Society, Science, Survival: Lessons from AMC’s 'The Walking Dead.'" It was free - as a MOOC should be - and it ran for 8 weeks. It was offered on Instructure’s Canvas Network MOOC platform (I taught a MOOC there last year.) The teachers/facilitators were four UC Irvine professors from different disciplines: Zuzana Bic, public health; Joanne Christopherson, social sciences; Michael Dennin physics; and Sarah Eichhorn, mathematics.

Their goal was to use the show as a way to do case studies related to concepts from post-disaster nutrition, the foundations of human survival and stereotypes in a Darwinian environment. Sounds like a course.

Now that it has ended, I have seen a few stories online that focused on the fact that just 2,203 of the 65,000 people who enrolled in the course completed it. Completion horror stories have, unfortunately, become the big story in MOOCs the past six months. The folks at UC Irvine say the low completion rate doesn't bother them. They did get 80% of participants to spend at least an hour working on the class.

Actually, the course was designed to allow students to drop in and out of the modules. I did the same with my course. If you passed a quiz at the end of a lesson, students would earn a badge and getting all eight badges meant a certificate of completion.



But the story here might be more about the idea of seeing whether or not offering a "pop culture" course would attract a new audience or mean greater engagement and completion. Their completion rate is less than the usual 10-15% that is usually attached to MOOCs - and no one is happy with those numbers in academia.

90% of the students said they had never taken a MOOC before. (They got survey responses from 12,000 participants.) It might be more significant that 59% had never tried an online class at all.

I am sure you could get even more engagement and "completion" if you dropped some of the school elements of a course, increased the pop culture elements ("Let's learn about the actors and watch clips!") and the gamification elements ("Correct answers help you kill zombies!) and offered some swag or prizes ("Meet the cast!").

Of course, that's not what academic MOOCs are supposed to be all about.

We are still learning about how MOOCs work and how they might help enhance learning online and offline. I have never viewed completion as the the mark of success in MOOCs and don't see all non-engaged students as "lurkers" because I know some of them are "auditors" interested in only a portion of the course content.

We have things to learn from non-academic MOOCs too.


MOOCs and the Trough of Disillusionment

The MOOC honeymoon seems to be over. 2012 was its year of stardom and then in 2013, it was the time to bash them. And 2014 is the year to...?

If you accept Gartner's methodology of "Hype Cycles" of how a technology evolves over time, then before 2012 (2011 or earlier) was the "Technology Trigger." MOOCs got started and we heard the occasional story of proof-of-concept example via a few articles. There were no products or platforms or business models.

In 2012 (called "The Year of the MOOC" by the NY Times) things began to shift. We started to hear about things like Coursera and more and more universities began to launch courses using vendors or on their own.

In late 2012, and then more so in 2013, we probably entered the "Peak of Inflated Expectations." All that press for the success stories brought on the inevitable stories of failures. Sometimes it was a single course, sometimes the whole program at a university and sometimes an entire provider. Udacity is an example of the latter, as its founder basically threw in the towel and said that MOOCs might be better suited to corporate training than college education.

So, perhaps we are this year in the "Trough of Disillusionment."  I have been working on a chapter for a book on MOOCs and one of the most difficult parts of the writing has been how things have changed since I first started writing only six months ago. Interest in the MOOC is definitely waning rather than waxing right now. The implementations that failed to deliver get more attention than the more successful experiments. I suspect that the amount of time and money to be invested in MOOCs this year may be less than in 2013.

Am I disappointed?  Not really. Surprised? No.

I accept the basic premise of those tech cycles. And if there is some validity to them, then we are going to work our way out of the slough this year. (A slough, by the way, is literally a swamp or side channel only sporadically filled with water. Figuratively, it means a situation characterized by lack of progress or activity.)

Hopefully, by the end of this year we will be climbing the "Slope of Enlightenment." We will need to be able to showcase some MOOC examples that consistently benefit students and institutions. The next generation providers and platforms will need to appear and some new enterprise investments will need to exist.

It may take a few years for us to reach any type of "Plateau of Productivity" where there is wide and mainstream adoption of MOOC learning. We don't even have clear criteria for assessing these offerings right now - but it is coming. The disruption has occurred. That was good. Now we need to see what can be built from there.

Here is a presentation on MOOC progress by Una Daly, who I have worked with in the past on efforts to bring open educational resources (particularly open textbooks) up that Slope of Enlightenment.

Corporate Doctorates

I read a number of posts last fall about JPMorgan Chase's plan to give $17 million to start a doctoral program at the University of Delaware. The companyplans to renovate a building to house the program, put up money to pay program faculty and pay a full ride for students seeking a degree. There were suggestions that there may even be JPMorgan employees on dissertation committees and they may have input on which faculty members will teach in the program. (A university spokeswoman emphasized JPMorgan will have an advisory role, but not a decision-making role.)

Is this a trend?  Will we see donors having more of an interest and input in how money is being used? Will universities build programs and degrees suited to the employee needs of a company? At least 40 percent of the budget for the UDel program would come from
JPMorgan, which is set to become the largest single corporate
contributor to the University of Delaware in its history.

It's not hard to imagine Google and Stanford forming a partnership that develops the kind of doctoral students that Google wants to hire. Of course, that type of student would be attractive to other companies too, but "designed" in a Google way.

Companies frequently donate money and resources to short term educational projects. JPMorgan's Code for Good Challenges, for example, is a two-day event that brings together college students studying technology to develop solutions to challenges faced by nonprofits.

The "JPMorgan Ph.D." (as I saw it described in one article) would be be in "financial services analytics." That kind of "big data" focus would be appealing to many financial companies.

Bruce Weber, the dean of Delaware's business school, said in an insidehighered.com article that working closely with industry will help academics prepare students for the real world, something some business school deans feel is not happening enough.

“I think it’s a great way for business schools to really enhance the education they provide to their students by reaching out to the industry,” he said. JPMorgan, which has a strong presence in Delaware, and the university have been working closely together since 2009. Weber, who joined the university three years ago, said officials had by then realized university programs weren't hitting the mark in terms of what employers wanted. So, JPMorgan helped the university create and pay for its Institute for Financial Analytics and Delaware created a minor in global enterprise technology. Now, JPMorgan has a fully staffed and functioning office -- using student interns -- on the first floor of the university's Purnell Hall.
Of course, not all faculty are pleased with the plan. Some seem to feel that this type of Ph.D. is "a non-academic Ph.D." Might a university's close ties with one financial institution hurt its relationship with others when it comes to internships, recruiting and donations?

The bigger issue here is that at a time when colleges rely on donors to make up for declines in state support, these types of partnerships make sense. Colleges have always had corporate donors. You might name a lab or even a building for a donor. But generally, the donor had very little control over how programs were run.

Is this a sign of a new kind of corporate/university partnership?