Journey into OER

 This is the first in a series of posts about my journey into OER.  If you are not familiar with Open Educational Resources (OER), you should be.  OER is course materials: syllabi, assignments, lecture notes, and even texts, which are available for adaptation and use for free.  There is almost certainly some OER in your field. If you are like the majority of faculty that I know, you care about your students’ learning and you want them to be successful. One of the major impediments to college graduation is the cost, which has grown faster over the last few decades than nearly any other component of inflation.  Textbook costs are a major part of the cost of higher education.

The strength and the weakness of OER is that it is free. One of the first principles of economics is that there is no such thing as a free lunch—everything has an opportunity cost. If the user doesn’t pay it, someone else has to or else the good or service will not get provided. Much of existing OER is craft work, interesting and useful things created as more or less one-off projects. This is not a bad thing—indeed, it describes pretty much everything I’ve done in my professional life. Until recently.

Two years ago, I helped create a free, open source principles of economics text for OpenStaxCollege. OpenStax has introductory text books in a variety of fields. You should check to see if yours is represented. The books are free in a variety of formats, but OpenStax has been challenged with the question of how to keep their textbooks current and available. Someone has to pay the bills to run the servers. Someone has to pay for updates and revisions. Or these things won’t happen. OpenStax’s strategy has been three pronged: One approach is to ask for donations. I don’t know how successful that has been, but I suspect donations alone will not be enough to pay the bills. Another approach is to work with commercial firms to provide after-market products that are not free, though they may be effective at enhancing learning and they are not very expensive compared to traditional commercial textbooks and their accoutrements. In return, the commercial firms provide a modest revenue stream to OpenStax, enough perhaps to maintain the existing collection of books. The third approach is to solicit grant funding for major projects, either creating new books or making major revisions of existing ones. Granting agencies support innovative new ideas, but they tend to be less supportive of maintenance operations, including revisions. The Gates Foundation has been this way.

One of the goals of the Gates Foundation’s Next Generation Courseware Challenge has been to develop a business model that allows for operation at scale, but that doesn’t require continued grant funding. Enter Lumen Learning. Full disclosure: I am working for Lumen on the project I’m about to describe.   This means I am not an objective observer, which means I am going to tell you what I really think: Go figure!

Lumen is a for-profit company. How can a for-profit company provide OER? I had a hard time wrapping my head around this, but I may have figured it out now.  Lumen is giving away its content for free.  After all, that’s the way OER is supposed to be.  Here is the principles of macroeconomics text.  Here the the principles of micro text.  Then Lumen used this OER to create digital courseware that will replace the textbooks students use in (at present) four of their courses. Lumen will sell this product to colleges and universities at a significantly lower price than students would pay for a text book (approximately 10%), something comparable to what OER after market companies are charging for their products. It will be up to schools to determine how they are going to pay for the Lumen product, perhaps by charging a course fee, but once the school opts in, the students will have immediate access to the text from day 1. Research has shown that when students have access to the textbook on the first day of class, they are far more likely to finish and succeed in the course. In short, the product is not free, but neither is it very expensive. The purpose is to allow the product to be used by a large number of schools and students, but in a way that is economically sustainable and won’t require constant searches for money to support.

(To be continued.)


Image Credit: oer_logo_EN_1 by Breno Trautwein via twitter.





Posted in Uncategorized | Leave a comment

Kudos to @YouBreakIFix

This week is final exam week at UMW.  I’ve been grading essays and now exams for what seems like forever, certainly more than a week now.  On Tuesday evening, I sat down at my kitchen table to start grading intermediate theory exams.  I set up my MacBookAir and then poured myself a large (16pz) glass of water, which I set down on the coaster next to my laptop.  As I got up to get something else, I knocked the water glass over, spilling the contents on the keyboard of my laptop.  Judging from all the water I mopped up from the table and the floor, it didn’t *all* go into the computer, but still…  The laptop immediately went dark.

A lot of things went through my mind, only some of them repeatable.  I thought about how expensive it would be to replace my MacBook.  I wondered how/if I could recover my data. But most of all I wondered how I was going to reconstruct my grades for the entire semester for all my courses.  Before grades are due, early next week.

I immediately DM’d my go-to computer guy, Obi Wan Andy Rush, aka @rushaw.  He sent me to a very helpful {and hopeful) webpage.  As I did a little more research, I learned a few things:

  • Several people wrote that if I went to the Apple Store, the Apple folks would immediately want to replace the logic board, for $1200.
  • If I went to Best Buy, they would charge me $50 to tell me they couldn’t fix the laptop. That didn’t sound palatable to me.
  • I found several positive references to  I called them and they confirmed that if they could resurrect my MacBook, the charge would be $200 + possibly more for parts, if necessary (e.g. a new battery).  But it they couldn’t fix the laptop, there would be no charge.  And they would do it within four days.

I took the laptop over yesterday and today they called me with the good news.  My MacBook was fixed and the charge was $204, including the sales tax.  YouBreakIFix was extremely professional and I highly recommend them for any computer repairs you need. They also fix cell phones.

The first thing I did was to email myself a copy of my grades spreadsheet.  The second thing was to order the external hard drive Andy recommended I get nearly a year ago.  I can’t believe how well this has worked out.

Posted in Uncategorized | Leave a comment

Response to “Colonizers and Edupunks”

This post began as a comment on Robert Barrow’s “Colonizers and Edupunks(&C.): Two Cultures in OER” Thanks, Rob for prompting my thinking on this.

The question of open texts vs. radical OER is an interesting one. My training as an economist colors my view on this. One lesson of economics is don’t let the perfect be the enemy of the good. An innovation doesn’t have to be perfect; it just has to be better.

While I haven’t thought this through yet, it feels to me like those in the radical OER camp are changing underlying assumptions without perhaps being explicit about that. Or they could be speaking from a different context. More on this below.

I accept Rob’s observation that customizing a open text invalidates (or may invalidate) the efficacy conclusions. That doesn’t prove the efficacy suffers in a particular context. Of course, it depends on the type and degree of customization. One reason for customizing is to make a text work better in a specific context. I know the context in which I teach.  I’m willing to take the chance of improvement (or the risk of worsening) from customization that makes sense to me. In short, statistical inference isn’t everything.

Replacing a commercial text with OER may not leave the teaching approach unchanged. Look at the Lumen Waymaker platform. Full disclosure: I helped create the Lumen platform.  We don’t know yet, nor will we for some time, but the mastery/personalization/analytics approach it embodies may enhance student learning, especially in lower level courses that emphasize content and conceptual knowledge.

2814710002_711e3b2d82_mSuppose one is teaching a introductory course to 100 or more first year students. This may not be the ideal situation for learning, but it is very common in higher ed, at least in the U.S. Moving from a lecture-based course to the type of course described by advocates of open ed, where students create their own knowledge base, is simply not feasible with class sizes that large, just as teaching a composition course that large (for example) wouldn’t work either.

Could one do this in a small seminar ? Certainly.  I’ve done that myself in upper level seminars.  But that’s not the question here. Do students learn better using a mastery approach with no arbitrary time limits on how long a student should take to learn a specific lesson? The research indicates yes. Do students learn better using a one-on-one tutorial approach? Same answer. For someone teaching 100 or more students, the question is what can I do to enhance the learning of the students in the context I am facing? An open textbook, especially the more immersive types like Lumen’s Waymaker platform may have the potential to do just that.

Images courtesy of:

  • Teddy-Rised via flickr That Huge Lecture Theatre!
  • Michael Summers via flickr That’s One Lucky Fella Continue reading
Posted in Uncategorized | 1 Comment

Mastery Learning as Intrinsic Motivation?


At the 2015 Educause Annual Meeting, the opening keynote speaker was Daniel Pink, who drew from several of his recent books, including Drive, which I blogged about here. Pink’s thesis is that success comes from three factors: mastery, autonomy & purpose. Pink summarized the research on how pay affects performance, observing that for tasks that have any cognitive dimension, once you reach a certain level of salary, pay for performance after that is counter productive. It actually lowers people’s productivity.

As I listened to Pink talk, I wondered if that could be applied to grades and learning.

There is a belief that grades motivate students in their courses: that the reward of a higher grade (or the fear of a lower grade) will bring forth more student effort. As an economist, the logic makes sense. But I wonder if there is any evidence about that, especially in light of the evidence that Pink cites. If we draw a parallel between Pink’s argument and education, perhaps that belief doesn’t hold up however much people believe it.


Consider mastery learning. Mastery learning is an approach where students study a topic until they learn enough to achieve mastery. Students work at their own pace, but they don’t move on to the next topic until they have learned the current one at a fairly high level of expertise. Additionally, the expectation is that students won’t necessarily achieve mastery on the first try. So failure is expected, but it’s not the end result, rather, it’s one step towards mastery. Kind of like real life, in my experience.

In a sense, mastery learning is a pass-fail system, but the passing level is higher than a typical C grade. Actually, mastery is a different, more subtle notion of learning than that assumed by a standard grading system. In my mind, mastery conveys an ability more than knowledge of some content. It’s the ability to use the content in an appropriate way. It’s the difference between hearing about something and actually practicing it—being able to do it. This is really what’s implicit in the seminar classes I teach, though sometimes it’s more successful than others.

What I tell my seminar students is: “This isn’t going to be a typical class. If you take this course seriously, if put in the effort to genuinely engage with the material and genuinely engage with the class, you will earn an acceptable grade.” But what’s an acceptable grade? The answer varies by student, but in my mind acceptable translates into a B or an A. For this to work, students have to trust that if they do their part, I will follow through. Some students find it very difficult to not know to the percent what their grade is at any point in time. I understand that, but that’s not what my seminars are about.

My thinking here is that a student’s preoccupation with grades actually gets in the way of the learning for all of reasons Pink’s discusses. Jesse Stommel made this point in a tweeted response to my previous post. In Pink’s language, mastery corresponds to that certain, acceptable level of compensation. Once that is (almost) guaranteed, once we have grades off the table, students can focus their attention on learning.

* Images courtesy of Denise Krebs & Celestine Turner via twitter

Posted in Uncategorized | 2 Comments

Do Grades Indicate Student Learning? Should They?

89447510_238bf11554_mIn my view, grades as summative assessments should indicate student learning. Of course, it’s not that simple in practice. This may be anathema to say publicly, but grading is as much art as science. Think about it. Teachers can’t observe learning directly so they try to interpret signs of content knowledge, skills and habits of the mind (e.g. approaching a problem the way an economist would). In other words, grading is an attempt to read the minds of one’s students, based on observations and evidence they provide. Does a piece of student writing show understanding of the required concepts? Does it correctly analyze the situation posed or answer the prompt? Does the student’s language ability distort what the student knows?

“Objective” assessments have their own pitfalls. Does the student correctly interpret what is being asked? Is the terminology used in multiple choice questions the same as in the text, or the same as that used by the lecturer? Will the student’s mathematical ability compromise his ability to show what she knows about economics or geology or whatever subject is being assessed? Did the student blacken the oval on the answer sheet that matches the answer they selected on the test document?

Grades can be affected by numerous things that are not student learning. Was a student ill the day of the exam? Was the student upset by something going on in their lives that adversely impacted their performance on a given assignment/assessment. Had they gotten enough sleep? I had a first year student, who despite my advice, studied for the 14 hours immediately prior to an 8am final exam. About 15 minutes into the exam, he blanked and was unable to complete it. Of course, the outcome was the student’s fault, but my point is simply that his performance didn’t correctly reflect his learning in the course.

I had a colleague once who had a policy that late papers received a 50% deduction. He spelled this out explicitly on the course syllabus, so from the administrative perspective the policy was aboveboard and acceptable. The purpose of the late penalty was to induce students to submit assignments on time. One student wrote an excellent paper (the colleague told me so), but turned the paper in late (by 15 minutes). The reason for the lateness isn’t important here. The colleague awarded the student 100 points on the paper less the 50 percent deduction for a net grade of 50, an F. The grade was awarded based on the stated policy so the grade was correct in a procedural sense. But did the grade accurately affect student learning?

I had another colleague who declined to grade a student paper because the student had deposited the paper in his campus mailbox, rather than submitting it in class. The student violated the instructor’s rules; therefore, she paid the penalty.

Most of us can agree that these are extreme examples, but most of us can also probably think of less egregious examples of where failure to follow directions (stated or unstated) or other deviations from expectations resulted in grade deductions. My question is not are these deductions fair, but do they bias the grade away from measuring student learning. I think the answer is yes.

Grades are never completely objective. As teachers, we need to own that. I think the best we can do is provide our best professional judgment of how much they reflect student learning. But at the same time we should be humble about our ability to always get it right.

Posted in Uncategorized | 2 Comments

Call Me Don


I believe profoundly in what I am about to write. I am not naive about the difficulties involved, but neither do I think we can assume the problem away, as I believe is common in higher education today.

The primary purpose of (most) colleges and universities is, or should be, to educate their students.And yet, few schools have any clear evidence of the extent to which that learning occurs. We have outcomes assessment structures. But at least in many programs, that says little about how well and how much students are learning.

Few students truly master their studies because that’s not their goal. Students have been trained for twelve or more years to play and win the “game of school.” For most, the grade is the only thing that matters. For some, mere passing is the goal.

Few faculty expect their students to achieve mastery, either. We assume that not all students are cut out for biology, or economics, or [insert your favorite discipline]. After all, our discipline is rigorous! We use certain courses to “weed out” the students who can’t make the grade (pun intended). We also assume that some students aren’t cut out for college at all (though that doesn’t stop us from accepting those students’ tuition and fees).

But what if we are wrong about what students are able to accomplish?

What if (virtually) all our students were capable of substantially mastering what we teach, if only we taught them in a different way. Crazy talk? Would faculty be willing to make the change? I doubt it. Faculty are incredibly resistant to change. Look how difficult it is for faculty to switch to a new textbook? They have a course full of lecture notes built around the old book. They would have to go through those notes and make changes where necessary, at a minimum changing the terminology to match the new text. This is reason enough to stick with the latest edition of a $300 textbook, rather than try a free, open source text. But I digress.

Why would faculty want to make the change? For one thing, the incentives are all wrong. Tenured faculty have a comfortable existence. We can teach our favorite areas of expertise, with little or no accountability. Change adds to our workload, and might worsen student learning. There’s no guarantee it would lead to an improvement.

At a school like mine (a regional public university which claims to value teaching effectiveness) in order to earn tenure, faculty need to be able to provide evidence that their teaching is not terrible. (Friends have told me that’s not even required at research universities.) Neither departments nor institutions do much to assess teaching effectiveness, though. To be sure, the school administers course evaluations to students. In some departments, senior faculty sit on a lecture to see how their junior colleagues are doing. Department chairs are supposed to mentor weak teachers. But as far as I know (after 30+ years of experience), there is no direct measurement of student learning. And there is little intervention in to help junior colleagues improve their teaching. “Not terrible” is good enough. Not terrible doesn’t warrant the intervention of the department chair, the dean or the provost.

I’m not suggesting that faculty aren’t conscientious teachers. In my experience (limited to my regional public institution), most faculty care about their teaching. We enjoy seeing the lightbulb go off in our students’ eyes. We just don’t know how effective our teaching is from semester to semester. Course evaluations can’t tell us that. Rare visits by colleagues can’t tell us that. If it ain’t broke, don’t fix it.

Faculty earn promotion and merit pay (when there is any) from their scholarly activity. That is also the ticket to a position at another university, if desired. I’ve never heard of a faculty member being recruited to another school because of above average teaching ability, but there is a plethora of evidence that instructors lose their positions despite or even because of their strong teaching ability. It is not that unusual to hear of faculty earning teaching awards, who were subsequently turned down for tenure.

I am not opposed to research and other forms of scholarly activity. Research informs my teaching and teaching informs my research. I just don’t think it should trump teaching effectiveness at (most) 4-year schools.

I would be surprised if my administration did not object to this characterization. Look at the weight assigned to teaching in faculty evaluation. Look at how much money has been allocated to (thrown at?) teaching development; look at our well resourced university teaching center. I don’t deny that opportunities are there for individual faculty members who want to take advantage of them. But who has the time, given all the other (more important?) things we are asked to do? There is admissions and retention work, faculty advising, participating in our first year experience, recruitment of new faculty, faculty governance and scholarly activity, just to name a few of the tasks I spend my time on. All of these things are important, but are they more important than teaching?

Is there an ongoing, central conversation among a critical mass of faculty about teaching at our institutions? How often do faculty discuss teaching effectiveness? How often is student learning discussed at department meetings, chair’s meetings, provost’s meetings or faculty governance meetings?

Regardless of the “weight” given to teaching in faculty evaluation, tenure and promotion, the bar is too low. The evidence is not there.

Perhaps I am expecting too much from faculty. Most Ph.D. students, after all, are not trained to teach. New Ph.D.s are assumed to be able to pick up teaching on the fly. They are bright people with years of experience as students. They can surely teach the way they were taught.

In the last several decades, researchers have discovered a tremendous amount about how students learn. But few faculty outside of cognitive science and education are trained in these findings. Why would they be since a Ph.D. is supposed to convey research ability and content expertise in their one’s field.

Paul Bruno, citing a recent report from Deans of Impact, states:

[W]e believe the art of teaching should also be informed by a robust understanding of the learning sciences so that teachers can align their decisions with our profession’s best understanding of how students learn.

I agree.

What if universities chose to build a reputation based on evidence of how well their students learn? Wouldn’t students want to go to those schools? Wouldn’t parents or others paying the bills want that?

What if faculty were at least minimally trained in how students learn?

What if incentives for faculty were aligned with the goals of teaching and learning effectiveness?

It would start with the assumption that all our students can learn, if properly instructed.

It would follow with the assumption that all faculty can be trained to teach effectively.

It would require higher standards and accountability.

It would require committed leadership.

And it would require the willingness to change.

Posted in Uncategorized | Leave a comment

Kudos to United Airlines (& American Airlines)

I have complained many times about poor airline experiences so it seems only right that I call attention to an unusually good experience.  I missed my flight home due to my mistake.  An amazingly helpful gate agent in Indianapolis bailed me out.  (Note to United: I think her name was Wanda!)  She found the last seat on an American flight and rebooked me.  The new flight was direct to my final destination instead of making a connection.   I will end up getting home about the same time as I was originally scheduled.  Thank you United Airlines, and thanks to American as well.

Posted in Uncategorized | Leave a comment

Quantitative Analysis of OpenStax’ Free, Open-Source Principles of Macroeconomics text

Last fall, I adopted OpenStax’ Principles of Macro text for my course of the same name. I have blogged earlier about the creation of that text. After the semester, I conducted a statistical analysis of student learning in the course as compared to the previous year’s sections when I used a well-known commercial textbook.

I used the same methodology as my earlier analysis comparing my online and face-to-face courses, but added in one more semester for a total of 4 sections and 89 observations.

The model used raw (uncurved) final exam scores as the dependent variable, and the following as explanatory variables:

  • Student GPA (less their grade in my course) — this was to capture some measure of how bright each student was, and to a certain extent, how hard they work.
  • Credit Hours Earned — this was to control for how much experience each student had in university coursework.
  • Gender — the literature says that women do less well in economics than men.
  • Whether the Sections was Honors or Regular—last year I taught both section as face-to-face courses, but one was an honors section that followed a more abstract treatment of the material.
  • Whether the Section was Online or Face-to-Face—last year I taught no online sections, so the Online variable might not have had adequate controls.
  • OpenStax—the treatment variable for the text used in both sections last year.

The results were quite interesting and differed a bit from my earlier study. The most significant determinant of final exam scores remained Cumulative GPA. Credit hours earned was not statistically significant this time. Gender was statistically positive meaning that women scored higher on the final exam. Honors students (or more precisely, students in the honors section) scored significantly higher. This variable had the largest effect size. Online students scored statistically lower. Since this differs from my earlier analysis, I plan to explore this more in the future, but it’s not critical for this analysis. Finally, the OpenStax variable was not statistically different from zero. That is, students using the OpenStax text scored no differently than those using the commercial text.

I’m happy to share the details of the statistical results with anyone who is interested.



Posted in Uncategorized | Leave a comment

The Lumen Project, v. 1

I have a really long draft of a post on the project I’ve been working on with Lumen Learning for much of the last year, but it’s not polished yet <jk> so let me just make a small point about it.  The project involves developing digital learning environments for two courses: Principles of Macro & Principles of Micro.  The content is divided into very small chunks.  At the end of each chunk is a short formative assessment using “multiple select” questions, which require students to select ALL the responses that are true. After each question students are asked how confident they feel about their answer.  The idea is to try to teach students metacognition as they learn economics.

I am pilot testing the courseware in one of my sections of Principles of Macro this semester.  A student in that section who was a serious student in my more traditional class last semester emailed me to ask, “Are there going to be any of these multiple select questions on the course exams, because if there are, I’m seriously thinking about dropping the course.”  When I shared this with David Wiley (@opencontent), he replied that that was really interesting and he wondered why the student responded that way.  I told him I thought it was probably two things:  First, multiple select is something new, something different than my students are used to.  And second, multiple select requires more thought.  I continued to think about this until I realized that this additional thought was completely consistent with the underlying theme in the courseware of metacognition.  I think that if students take that extra thinking seriously, it will likely over time lead to deeper learning.  I don’t think this was what we originally planned, but that’s a hypothesis anyway!

Posted in Uncategorized | Leave a comment

Teaching in Parallel

Over the last year, my approach to blogging has been to refine my thinking until I get it just right before posting.  You should see all the drafts I have!  I’m going to try something different here that some brighter than I call, wait for it…blogging.  I have an insight worth sharing so I’m just going to put it out there, and worry about getting it right later.

6683425837_1b064066c5_mAs I have done before, this term I’m teaching two sections of Principles of Macro, one online and one face-to-face.  There’s a lot more I should say about the former, but let me just make one point:  I find a terrific benefit to teach online and face-to-face in parallel.  I learn things in each that improves the teaching in the other.  Case in point, Friday I moderated a discussion of the role of economics in the liberal arts and sciences.  (Is there a role?)  I do this to help students situate my class in our schools curriculum.  It’s always a fun discussion and students seem to get something from it.

I often put a question on the first exam that originates from the discussion.  Which raised a potential problem in my head: The online section had no such discussion.  I use the same exams in both sections, so asking about a topic we didn’t discuss would make the online group appear to learn less.  I pondered that problem in the back of my head yesterday, and this morning woke up with an answer.  Since I used twitter to communicate with my online course daily, I posted the same two prompts that I used in class Friday.  While responding to some of the student responses, I ran across this article from today’s Washington Post, which provided some richness that neither the students nor I brought up online.  And then it occurred to me that I should post the same article on the course website for my face-to-face section.  I think this turned out to be a win, win for all of my students.

Image courtesy of Martin Weller via flickr

Posted in Uncategorized | Leave a comment