Measuring MOOCs

measuringMOOC enrollment surpassed 35 million in 2015 and though the blush is off the MOOC rose, they are clearly a new learning context for many people.

Are they transformative? I believe so, in the ways that they have created new platforms for online learning, reached new audiences and started conversations about alternative forms of learning and even alternatives within traditional education around credits and degrees.

Many of the current offerings labeled as MOOC are more moOC - that is, the numbers are less massive and the courses and content are not really "open" in the original intent and definition of that word.

Certificate-granting MOOCs and ones for college credit and professional development are what I see as the current trends of interest.

But the topic of quality in MOOCs, and online learning in general, has never gone away. Two reports recently examine that consideration.

The Babson annual “Online Report Card” is in its thirteenth (and I read, its final) year. The report's introduction caught my attention because it says that we are at a stage when “distance education is clearly becoming mainstream” and the divide between "online learning" and simply learning is less evident. The report doesn't spend much time on quality and seems to put MOOCs in the same category as other online learning.

The report titled  “In search of quality: Using Quality Matters to analyze the quality of Massive, Open, Online Courses (MOOCs)”  applied the Quality Matters™ higher education rubric (not the Continuing and Professional Development version) to six MOOCs, offered by three providers, Coursera, edX and Udacity.

Was that a fair test?

Critics of MOOCs will point to the result that all six MOOCs failed to meet QM’s passing grade of 85%. The QM rubric standards are grouped into eight dimensions and the MOOCs performed especially poorly at learner interaction and engagement, and learner support.

When my university first started to have students evaluate their online courses at the end of the last century, it used the same criteria and survey that was used for regular classes. That made some sense at first because they wanted to measure one against the other. Online offerings always did well in "the use of technology and media" category, but not very well in some of the face-to-face items such as lectures and teacher engagement.  After a few years, it was clear that a new survey made specifically for online courses was needed. But our online student survey would not be fair to use for a MOOC, especially one that is truly Massive and Open. A well designed course with many thousands of students, using OER, possibly no textbook, and taken for no credit or fees is just not going to be able to be measured against a good online course with a small number of students motivated by tuition, a grade, credits and a degree to be completed.

Efforts the past few years to evaluate MOOCs and establish standards of quality are important, but we have quite a ways to go.

Backwards Design


I brought up the topic of "backward design" last fall in one of my classes at NJIT and the students had never heard the term. One of them, not surprisingly, took his phone and did a search on Wikipedia, and we were both surprised that there was no entry. Not on Wikipedia? Then it kind of doesn't exist.

So, I put in a request for a page to be created, and recently I was notified that one has been created and we are adding and revising it.

Backward design is a method of designing educational curriculum by setting goals before choosing instructional methods and forms of assessment. That seems so logical, but we very often go the other way. "Teach to the test" would be a very obvious example.

The 3 stages are usually described as:

1. Identify the results desired (What are the big ideas and skills you want students to take away from the learning? What should the students know, understand, and be able to do?

2. Determine acceptable levels of evidence that support that the desired results have occurred (How will you assess those results? What will you accept as evidence that student understanding took place - observations, tests, presentations, projects?

3. Finally, design learning events that can make desired results happen (For example, what knowledge, skills and even tools will students need to achieve those desired results?

Backward design challenges common methods of curriculum planning, such as starting with a textbook as the content.

A common analogy used in explaining backward design is to think of it as using a "road map" where we start with the destination and then look for the best way to get there.  By that analogy, traditional design would be using a road map with no destination in mind. 

My NJIT students, some of whom are engineers, thought this sounded like "reverse engineering." Reverse engineering, also called back engineering, is the processes of extracting design information from anything man-made and re-producing it - a process that often involves disassembling something (a mechanical device, electronic component, computer program, or biological, chemical, or organic matter) and analyzing its components and workings in detail.

Backward design is not so hands-on, but it wasn't a bad place to start with the students. Actually, backward design is closer to the components of the ADDIE model which is commonly used in instructional design. These days instructional design is most often associated with technology and platforms for the development of learning experiences, especially online, and, unfortunately, less often associated with face-to-face teaching and pedagogy.

Historically, Ralph W. Tyler is given credit for the idea of "backward design" (though he didn't use that term in 1949). He was referring to a statement of objectives used to indicate the kinds of changes in the student to be brought about so that instructional activities can be planned and developed in a way likely to attain these objectives.

In curriculum design in the late 1980s, Jay McTighe and Grant Wiggins launched Understanding by Design (UbD)which promotes a backward approach to design that delays the planning of classroom activities until goals have been clarified and assessments designed. They promote this as a way to avoid the problems of "textbook coverage" and "activity-oriented" teaching, in which no clear priorities and purposes are apparent. UbD also promotes the use of standards and regular reviews of results (achievement data and student work) followed by targeted adjustments to curriculum and instruction.

UbD has been widely used in K-12 education and Wiggins' and McTighe's "WHERE" approach is also used for the assessment stage of the process. W stands for students knowing where they are heading, why they are heading there, what they know, where they might go wrong in the process, and what is required of them. The H stands for hooking the students on the topic of study. E stands for students exploring and experiencing ideas and being equipped with the necessary understanding to master the standard/outcome being taught. R stands for providing opportunities for students to rehearse, revise, and refine their work, and the E stands for student evaluation.

As the new Wikipedia entry notes, criticisms of this backward design approach do exist. For example, this approach ties teacher effectiveness more on the success of the students based on formulated assessments rather than ability to connect knowledge and skills to the needs and interests of students. Some critics feel that this approach shows a lack of concern with social and cultural differences within the classroom.

Teach Online, Even If Your School Doesn't Offer a Platform

If you have never had the opportunity to teach online and have wondered what it's like, here's a chance to find out. Canvas offers you a chance to try out their learning management system (LMS) for free. They offer two options: Take Canvas for a test drive with a free, two-week trial account that is pre-loaded with course content so that you can explore without having to build from scratch. But, even better, is the offer to actually teach your existing class on Canvas for free, forever. "You bring the content and students. We’ll provide the awesome platform, " says Canvas.

Sure, this is an offer meant to help market the platform and entice you to recommend it at your institution, but take advantage of it. That is especially true if you have never taught online and want to give it a try. Perhaps your school doesn't even offer the option to supplement your face-to-face class with an online section. Though I am more involved in how any LMS including Canvas is used in higher education, this is probably even more applicable to pre-college. (Look at how the platform is being used in K-12 education.)  

I have designed online learning and taught in a number of learning management systems over the years - WebBoard, WebCT, Blackboard, eCollege, Sakai, Moodle and Canvas. My first experience with Canvas was when I taught a MOOC in the Canvas Network back in 2013. That was a meta-MOOC called "Academia and the MOOC" and was intended to attract teachers as well as others in academic roles (instructional designer, support staff, administration and student).

I found Canvas easy to use, but it seemed like a work-in-progress at the time. It lacked many of the tools I was used to having built-in (equation editor, white board, blog, wiki and journal features etc.). But here are some interesting things that came out of that experience.

Teaching that MOOC led me to connect with many other online instructors. Some had take my "course" (which was more of a large conversation) in order to try out Canvas as much as to learn about MOOCs.

dip your toe inWhile I was facilitating the MOOC, I was contacted by two other New Jersey colleges that were considering moving to Canvas. The instructional designers at both schools separately reported the same phenomena at their colleges. The instructional design staff felt as I had when I encountered Canvas - it seemed "underpowered." But, their faculty really liked it for pretty much the same reason: it was clean and simple and didn't have all those "tools we never use." Both colleges now use Canvas.

I think that anyone currently teaching at any level should have experienced being a student and being a teacher in an online setting. There is just no getting around the fact that it is and will continue to be a part of what learning has become and how it is offered.

Dip your foot into the online water - or just jump in with your whole course. It's not as scary as it looks.

How Do You Define Personalized Learning?

Higher education leaders share how they define personalized learning.

Including: Randy Bass, Vice Provost for Education, Georgetown University; Tahnja Wilson, Sr. Mgr. of Strategic Design Initiatives, Arizona State University; David Wiley, Chief Academic Officer, Lumen Learning; Bryan Alexander, Founder, Bryan Alexander Consulting LLC; Shannon McCarty, Dean of Instruction & Academic Affairs, Rio Salado College; Michael Crow, President, Arizona State University;  Tristan Denley, Vice Chancellor for Academic Affairs ,Tennessee Board of Regents



Coding as a (second) Language


There is global interest in teaching programming in schools. Initiatives that come from outside education, like, which is backed by Mark Zuckerberg and Bill Gates, are trying to get more students learning a second (or third) language, but it's not one that is spoken. But I also see a backlash of those who say that writing code is a terrible way for humans to instruct computers and that newer technology may render programming languages "about as useful as Latin."

I support some middle ground. Teaching some coding as part of regular language study in English and world language classes.

This week I am giving a presentation at the NJEDge.Net Faculty Best Practices Showcase that I titled "Code as a (second) Language." It's not about becoming a programmer. Learning about code, like learning about grammar, is about understanding how a system of communication works below the surface.

There are several "computer science, meets humanities" programs. One is at Stanford University, which offers a new major there called CS+X  which is a middle ground between computer science and any of 14 disciplines in the humanities, including history, art, and classics. 

What are the cognitive advantages to learning a second language? Learning any system of signs, symbols and rules used to communicate improves thinking by challenging the brain to: recognize & negotiate meaning, work within structures and rules, and master different language patterns.

As a longtime language teacher - and shorter term coder - I know that code-switching (and that is the term) occurs when a speaker alternates between two or more languages, or language varieties, in the context of a single conversation. That can be done between English and French, but also between English and Java. 

Whether you are working in a traditional language class or a programming class, memorizing rules and learning new vocabulary strengthens overall memory. Multilingual people are better at remembering lists or sequences. Language study & coding forces a focus on knowing important information & excluding extraneous information. We have all heard and read beautiful” and elegant language, such as in a Shakespeare play or great poem, but programmers and mathematicians also talk about beautiful and elegant code and equations.

logoThe conference this week is about STEAM -- STEM plus the arts, including language arts.

Engineering and other STEM subjects are appealing to students in part because they often include hands-on, real-world applications. Many students also feel that these majors lead to better job prospects. Of course, learning to think like an engineer could be useful no matter what students decide to pursue. An increasing number of high schools offer introduction to engineering courses that are project-based, an inquiry-centered. 

There is a Code as a Second Language National Initiative that brings tech professionals and software engineers into schools to introduce students to coding in classes, but also in after-school sessions and events like coding jams. 

This is all great, but my interest here is bring the coding found in STEM courses into languages classes. 

How is a programming language comparable to a spoken language?

My idea is not without precedents. Natural language processing looks at syntax, semantics and models of language analysis, interpretation & generation. Human language technology continues to grow. On a large scale, products like Google and other search tools and Apple's Siri and speech drive commercial uses. The field of computational linguistics is one that grew out of early machine translation efforts and generated mechanized linguistic theories.

There are many programming languages we might use, depending on the grade level and applications. Although JAVA is the most popular programming language, and the AP computer science exam uses a Java subset, it is more than many students will have time to learn. There are coding options that I have written about here for using simpler languages (such as SCRATCH) and tools to aid in writing programs

Although Java might not be the coolest language to use these days, you can do many things with it - including tapping into the current interest by young people for Minecraft. Using mods for Minecraft makes Java more beginner-friendly.  

Language teachers can work with STEM teachers, especially in K-12 schools, to show students the connections between concepts like syntax and help bridge student knowledge of the two fields and also understand commonalties in communications.

The 2016 NJEDge.Net Faculty Best Practices Showcase is a venue to showcase faculty work, work-in-progress or posters to the New Jersey Higher Ed and K-12 communities. Registration and Information on the presentations at

View the "Coding as a (second) Language" slides via Slideshare by Kenneth Ronkowitz