Mastery Learning as Intrinsic Motivation?


At the 2015 Educause Annual Meeting, the opening keynote speaker was Daniel Pink, who drew from several of his recent books, including Drive, which I blogged about here. Pink’s thesis is that success comes from three factors: mastery, autonomy & purpose. Pink summarized the research on how pay affects performance, observing that for tasks that have any cognitive dimension, once you reach a certain level of salary, pay for performance after that is counter productive. It actually lowers people’s productivity.

As I listened to Pink talk, I wondered if that could be applied to grades and learning.

There is a belief that grades motivate students in their courses: that the reward of a higher grade (or the fear of a lower grade) will bring forth more student effort. As an economist, the logic makes sense. But I wonder if there is any evidence about that, especially in light of the evidence that Pink cites. If we draw a parallel between Pink’s argument and education, perhaps that belief doesn’t hold up however much people believe it.


Consider mastery learning. Mastery learning is an approach where students study a topic until they learn enough to achieve mastery. Students work at their own pace, but they don’t move on to the next topic until they have learned the current one at a fairly high level of expertise. Additionally, the expectation is that students won’t necessarily achieve mastery on the first try. So failure is expected, but it’s not the end result, rather, it’s one step towards mastery. Kind of like real life, in my experience.

In a sense, mastery learning is a pass-fail system, but the passing level is higher than a typical C grade. Actually, mastery is a different, more subtle notion of learning than that assumed by a standard grading system. In my mind, mastery conveys an ability more than knowledge of some content. It’s the ability to use the content in an appropriate way. It’s the difference between hearing about something and actually practicing it—being able to do it. This is really what’s implicit in the seminar classes I teach, though sometimes it’s more successful than others.

What I tell my seminar students is: “This isn’t going to be a typical class. If you take this course seriously, if put in the effort to genuinely engage with the material and genuinely engage with the class, you will earn an acceptable grade.” But what’s an acceptable grade? The answer varies by student, but in my mind acceptable translates into a B or an A. For this to work, students have to trust that if they do their part, I will follow through. Some students find it very difficult to not know to the percent what their grade is at any point in time. I understand that, but that’s not what my seminars are about.

My thinking here is that a student’s preoccupation with grades actually gets in the way of the learning for all of reasons Pink’s discusses. Jesse Stommel made this point in a tweeted response to my previous post. In Pink’s language, mastery corresponds to that certain, acceptable level of compensation. Once that is (almost) guaranteed, once we have grades off the table, students can focus their attention on learning.

* Images courtesy of Denise Krebs & Celestine Turner via twitter

Posted in Uncategorized | 1 Comment

Do Grades Indicate Student Learning? Should They?

89447510_238bf11554_mIn my view, grades as summative assessments should indicate student learning. Of course, it’s not that simple in practice. This may be anathema to say publicly, but grading is as much art as science. Think about it. Teachers can’t observe learning directly so they try to interpret signs of content knowledge, skills and habits of the mind (e.g. approaching a problem the way an economist would). In other words, grading is an attempt to read the minds of one’s students, based on observations and evidence they provide. Does a piece of student writing show understanding of the required concepts? Does it correctly analyze the situation posed or answer the prompt? Does the student’s language ability distort what the student knows?

“Objective” assessments have their own pitfalls. Does the student correctly interpret what is being asked? Is the terminology used in multiple choice questions the same as in the text, or the same as that used by the lecturer? Will the student’s mathematical ability compromise his ability to show what she knows about economics or geology or whatever subject is being assessed? Did the student blacken the oval on the answer sheet that matches the answer they selected on the test document?

Grades can be affected by numerous things that are not student learning. Was a student ill the day of the exam? Was the student upset by something going on in their lives that adversely impacted their performance on a given assignment/assessment. Had they gotten enough sleep? I had a first year student, who despite my advice, studied for the 14 hours immediately prior to an 8am final exam. About 15 minutes into the exam, he blanked and was unable to complete it. Of course, the outcome was the student’s fault, but my point is simply that his performance didn’t correctly reflect his learning in the course.

I had a colleague once who had a policy that late papers received a 50% deduction. He spelled this out explicitly on the course syllabus, so from the administrative perspective the policy was aboveboard and acceptable. The purpose of the late penalty was to induce students to submit assignments on time. One student wrote an excellent paper (the colleague told me so), but turned the paper in late (by 15 minutes). The reason for the lateness isn’t important here. The colleague awarded the student 100 points on the paper less the 50 percent deduction for a net grade of 50, an F. The grade was awarded based on the stated policy so the grade was correct in a procedural sense. But did the grade accurately affect student learning?

I had another colleague who declined to grade a student paper because the student had deposited the paper in his campus mailbox, rather than submitting it in class. The student violated the instructor’s rules; therefore, she paid the penalty.

Most of us can agree that these are extreme examples, but most of us can also probably think of less egregious examples of where failure to follow directions (stated or unstated) or other deviations from expectations resulted in grade deductions. My question is not are these deductions fair, but do they bias the grade away from measuring student learning. I think the answer is yes.

Grades are never completely objective. As teachers, we need to own that. I think the best we can do is provide our best professional judgment of how much they reflect student learning. But at the same time we should be humble about our ability to always get it right.

Posted in Uncategorized | 2 Comments

Call Me Don


I believe profoundly in what I am about to write. I am not naive about the difficulties involved, but neither do I think we can assume the problem away, as I believe is common in higher education today.

The primary purpose of (most) colleges and universities is, or should be, to educate their students.And yet, few schools have any clear evidence of the extent to which that learning occurs. We have outcomes assessment structures. But at least in many programs, that says little about how well and how much students are learning.

Few students truly master their studies because that’s not their goal. Students have been trained for twelve or more years to play and win the “game of school.” For most, the grade is the only thing that matters. For some, mere passing is the goal.

Few faculty expect their students to achieve mastery, either. We assume that not all students are cut out for biology, or economics, or [insert your favorite discipline]. After all, our discipline is rigorous! We use certain courses to “weed out” the students who can’t make the grade (pun intended). We also assume that some students aren’t cut out for college at all (though that doesn’t stop us from accepting those students’ tuition and fees).

But what if we are wrong about what students are able to accomplish?

What if (virtually) all our students were capable of substantially mastering what we teach, if only we taught them in a different way. Crazy talk? Would faculty be willing to make the change? I doubt it. Faculty are incredibly resistant to change. Look how difficult it is for faculty to switch to a new textbook? They have a course full of lecture notes built around the old book. They would have to go through those notes and make changes where necessary, at a minimum changing the terminology to match the new text. This is reason enough to stick with the latest edition of a $300 textbook, rather than try a free, open source text. But I digress.

Why would faculty want to make the change? For one thing, the incentives are all wrong. Tenured faculty have a comfortable existence. We can teach our favorite areas of expertise, with little or no accountability. Change adds to our workload, and might worsen student learning. There’s no guarantee it would lead to an improvement.

At a school like mine (a regional public university which claims to value teaching effectiveness) in order to earn tenure, faculty need to be able to provide evidence that their teaching is not terrible. (Friends have told me that’s not even required at research universities.) Neither departments nor institutions do much to assess teaching effectiveness, though. To be sure, the school administers course evaluations to students. In some departments, senior faculty sit on a lecture to see how their junior colleagues are doing. Department chairs are supposed to mentor weak teachers. But as far as I know (after 30+ years of experience), there is no direct measurement of student learning. And there is little intervention in to help junior colleagues improve their teaching. “Not terrible” is good enough. Not terrible doesn’t warrant the intervention of the department chair, the dean or the provost.

I’m not suggesting that faculty aren’t conscientious teachers. In my experience (limited to my regional public institution), most faculty care about their teaching. We enjoy seeing the lightbulb go off in our students’ eyes. We just don’t know how effective our teaching is from semester to semester. Course evaluations can’t tell us that. Rare visits by colleagues can’t tell us that. If it ain’t broke, don’t fix it.

Faculty earn promotion and merit pay (when there is any) from their scholarly activity. That is also the ticket to a position at another university, if desired. I’ve never heard of a faculty member being recruited to another school because of above average teaching ability, but there is a plethora of evidence that instructors lose their positions despite or even because of their strong teaching ability. It is not that unusual to hear of faculty earning teaching awards, who were subsequently turned down for tenure.

I am not opposed to research and other forms of scholarly activity. Research informs my teaching and teaching informs my research. I just don’t think it should trump teaching effectiveness at (most) 4-year schools.

I would be surprised if my administration did not object to this characterization. Look at the weight assigned to teaching in faculty evaluation. Look at how much money has been allocated to (thrown at?) teaching development; look at our well resourced university teaching center. I don’t deny that opportunities are there for individual faculty members who want to take advantage of them. But who has the time, given all the other (more important?) things we are asked to do? There is admissions and retention work, faculty advising, participating in our first year experience, recruitment of new faculty, faculty governance and scholarly activity, just to name a few of the tasks I spend my time on. All of these things are important, but are they more important than teaching?

Is there an ongoing, central conversation among a critical mass of faculty about teaching at our institutions? How often do faculty discuss teaching effectiveness? How often is student learning discussed at department meetings, chair’s meetings, provost’s meetings or faculty governance meetings?

Regardless of the “weight” given to teaching in faculty evaluation, tenure and promotion, the bar is too low. The evidence is not there.

Perhaps I am expecting too much from faculty. Most Ph.D. students, after all, are not trained to teach. New Ph.D.s are assumed to be able to pick up teaching on the fly. They are bright people with years of experience as students. They can surely teach the way they were taught.

In the last several decades, researchers have discovered a tremendous amount about how students learn. But few faculty outside of cognitive science and education are trained in these findings. Why would they be since a Ph.D. is supposed to convey research ability and content expertise in their one’s field.

Paul Bruno, citing a recent report from Deans of Impact, states:

[W]e believe the art of teaching should also be informed by a robust understanding of the learning sciences so that teachers can align their decisions with our profession’s best understanding of how students learn.

I agree.

What if universities chose to build a reputation based on evidence of how well their students learn? Wouldn’t students want to go to those schools? Wouldn’t parents or others paying the bills want that?

What if faculty were at least minimally trained in how students learn?

What if incentives for faculty were aligned with the goals of teaching and learning effectiveness?

It would start with the assumption that all our students can learn, if properly instructed.

It would follow with the assumption that all faculty can be trained to teach effectively.

It would require higher standards and accountability.

It would require committed leadership.

And it would require the willingness to change.

Posted in Uncategorized | Leave a comment

Kudos to United Airlines (& American Airlines)

I have complained many times about poor airline experiences so it seems only right that I call attention to an unusually good experience.  I missed my flight home due to my mistake.  An amazingly helpful gate agent in Indianapolis bailed me out.  (Note to United: I think her name was Wanda!)  She found the last seat on an American flight and rebooked me.  The new flight was direct to my final destination instead of making a connection.   I will end up getting home about the same time as I was originally scheduled.  Thank you United Airlines, and thanks to American as well.

Posted in Uncategorized | Leave a comment

Quantitative Analysis of OpenStax’ Free, Open-Source Principles of Macroeconomics text

Last fall, I adopted OpenStax’ Principles of Macro text for my course of the same name. I have blogged earlier about the creation of that text. After the semester, I conducted a statistical analysis of student learning in the course as compared to the previous year’s sections when I used a well-known commercial textbook.

I used the same methodology as my earlier analysis comparing my online and face-to-face courses, but added in one more semester for a total of 4 sections and 89 observations.

The model used raw (uncurved) final exam scores as the dependent variable, and the following as explanatory variables:

  • Student GPA (less their grade in my course) — this was to capture some measure of how bright each student was, and to a certain extent, how hard they work.
  • Credit Hours Earned — this was to control for how much experience each student had in university coursework.
  • Gender — the literature says that women do less well in economics than men.
  • Whether the Sections was Honors or Regular—last year I taught both section as face-to-face courses, but one was an honors section that followed a more abstract treatment of the material.
  • Whether the Section was Online or Face-to-Face—last year I taught no online sections, so the Online variable might not have had adequate controls.
  • OpenStax—the treatment variable for the text used in both sections last year.

The results were quite interesting and differed a bit from my earlier study. The most significant determinant of final exam scores remained Cumulative GPA. Credit hours earned was not statistically significant this time. Gender was statistically positive meaning that women scored higher on the final exam. Honors students (or more precisely, students in the honors section) scored significantly higher. This variable had the largest effect size. Online students scored statistically lower. Since this differs from my earlier analysis, I plan to explore this more in the future, but it’s not critical for this analysis. Finally, the OpenStax variable was not statistically different from zero. That is, students using the OpenStax text scored no differently than those using the commercial text.

I’m happy to share the details of the statistical results with anyone who is interested.



Posted in Uncategorized | Leave a comment

The Lumen Project, v. 1

I have a really long draft of a post on the project I’ve been working on with Lumen Learning for much of the last year, but it’s not polished yet <jk> so let me just make a small point about it.  The project involves developing digital learning environments for two courses: Principles of Macro & Principles of Micro.  The content is divided into very small chunks.  At the end of each chunk is a short formative assessment using “multiple select” questions, which require students to select ALL the responses that are true. After each question students are asked how confident they feel about their answer.  The idea is to try to teach students metacognition as they learn economics.

I am pilot testing the courseware in one of my sections of Principles of Macro this semester.  A student in that section who was a serious student in my more traditional class last semester emailed me to ask, “Are there going to be any of these multiple select questions on the course exams, because if there are, I’m seriously thinking about dropping the course.”  When I shared this with David Wiley (@opencontent), he replied that that was really interesting and he wondered why the student responded that way.  I told him I thought it was probably two things:  First, multiple select is something new, something different than my students are used to.  And second, multiple select requires more thought.  I continued to think about this until I realized that this additional thought was completely consistent with the underlying theme in the courseware of metacognition.  I think that if students take that extra thinking seriously, it will likely over time lead to deeper learning.  I don’t think this was what we originally planned, but that’s a hypothesis anyway!

Posted in Uncategorized | Leave a comment

Teaching in Parallel

Over the last year, my approach to blogging has been to refine my thinking until I get it just right before posting.  You should see all the drafts I have!  I’m going to try something different here that some brighter than I call, wait for it…blogging.  I have an insight worth sharing so I’m just going to put it out there, and worry about getting it right later.

6683425837_1b064066c5_mAs I have done before, this term I’m teaching two sections of Principles of Macro, one online and one face-to-face.  There’s a lot more I should say about the former, but let me just make one point:  I find a terrific benefit to teach online and face-to-face in parallel.  I learn things in each that improves the teaching in the other.  Case in point, Friday I moderated a discussion of the role of economics in the liberal arts and sciences.  (Is there a role?)  I do this to help students situate my class in our schools curriculum.  It’s always a fun discussion and students seem to get something from it.

I often put a question on the first exam that originates from the discussion.  Which raised a potential problem in my head: The online section had no such discussion.  I use the same exams in both sections, so asking about a topic we didn’t discuss would make the online group appear to learn less.  I pondered that problem in the back of my head yesterday, and this morning woke up with an answer.  Since I used twitter to communicate with my online course daily, I posted the same two prompts that I used in class Friday.  While responding to some of the student responses, I ran across this article from today’s Washington Post, which provided some richness that neither the students nor I brought up online.  And then it occurred to me that I should post the same article on the course website for my face-to-face section.  I think this turned out to be a win, win for all of my students.

Image courtesy of Martin Weller via flickr

Posted in Uncategorized | Leave a comment

Don’t start down that slippery slope!

The seminar has finished a first attempt at fleshing out the research question we will investigate.  We then chose books for small groups to acquire and read prior to reporting out to the class.  For the last week or so, I’ve been presenting some of the tools of international finance (that would be the course content in a more traditional course), tools that students will need as they make progress in our research.  During this time, I’ve gotten the sense that some students are not really engaged, that they are not reading the books assigned because they think they have plenty of time for that later.  This shows up in the lack of comments in class and also the lack of blogging by some.

This behavior makes sense from the perspective of a student, but not from the perspective of a research associate.  What to do about this?

It would very easy to react by threatening a quiz or otherwise imposing penalties; indeed, that was the first thing that came to my mind.  I know it would work, because I have enough teaching experience to create a set of incentives so that students will do what I want.  But I had to tell myself, DON’T GO DOWN THAT ROAD, since it would compromise the research team ethos I’m trying to build for this course.  A research team leader wouldn’t say “I’m going to dock you 10 points for not reading!”  What would he or she do?

Part of this is my fault.  I implicitly assumed that the students would be doing what I would do.  But I am the one with the most experience doing research.  I haven’t communicated well enough to them, or provided enough structure for them to see what they should be doing.

To that end, today I will revisit our research question matrix.  I will create three groups of students:  One to investigate the financial aspects of the Euro problem; one to investigate the economic aspects and one to investigate the political aspects.  I organized the groups so that most of the financial group is business students; most of the economic group is economics majors; and most of the political group is international affairs.  But each group also has one member from each of the other majors.  The other thing I incorporated is that each group has someone who read each of the assigned books.  I will charge each group with identifying their respective aspects and reporting back in two weeks.  I will also give them time to organize groups today.

And I will encourage them to blog what they’re finding, and any questions they find.


Image courtesy of Paul Graham Raven (via flickr)

Posted in Uncategorized | Leave a comment

What’s the definition of insanity?

Last summer I attended the second annual UMW Game Camp, at which interested faculty 10241646_c42aeb5d71_mwere introduced to the variety of ways that gaming culture and practices could be incorporated into one’s teaching.  Game Camp was jointly led by Mary Kayler, Director of our Center for Teaching Excellence & Innovation (@saptiva), and Andi Smith, Associate Professor of Historic Preservation (@smithpres).  The main takeaway for me was a way of letting students choose between alternative assignments to earn points towards their final grade in my online principles of economics course.  This was a positive change in that it allowed students to choose assessments which showcased their particular learning styles.  But that’s not what this post is about.

Rather, last fall I tried something else that was inspired by Game Camp.  For a number of years, I have urged my introductory students during the first week of class to read the course website/syllabus carefully.  Then I have given them a quiz about it on the first Friday.  Year in and year out, the vast majority of students failed the one question quiz, even though I tried to make the questions easy for anyone who read carefully.

After game camp, I had a different idea:  I replaced the stick with a carrot.  Instead of a one question quiz, I created a scavenger hunt with half a dozen questions.  The result last semester was near 100% success.  The students loved it and seemed to think I was giving them free points.  Perhaps more importantly,  they learned what was on the syllabus, which after all was the point.  Thanks Mary & Andi!


Image courtesy of Abby Chicken via flickr

Posted in Uncategorized | Leave a comment

Teaching as Modelling Disciplinary Practice

I believe that one of the highest forms of teaching occurs through modelling disciplinary practice, ideally when students forget that you are teaching.  This is the first week for our Spring semester and I am teaching my senior seminar in international finance.  The seminar began life as a more traditional seminar covering the traditional topics in the field, but over the last decade (I only get to teach this every three years), it has morphed into something more interesting, both for the students and me.

The something is a collaborative research project in which I model doing research in economics.  The project isn’t part of the course—it is the course.  We start with a question:  This year it will be “Is the European Monetary System (e.g. the Euro) sustainable in its current configuration?”  Then I ask: “What do we need to know to answer that question?”  The result is a list of questions.  E.g. What is the Euro?  What do we mean by sustainable?  What is the Eurozone?  What are the pros and cons of membership? What are the political dimensions involved?

We work recursively backwards developing a nested set of questions, which will form the syllabus for the course.   For each set of questions, we divide up the work, go out and research each part, reporting back online before discussing what we’ve found in class.  Through the process, we develop an understanding of the issues which hopefully leads us to an answer to the initial research question.

We will spend this first week, developing an initial plan for the research.  I don’t intend to mention grades until the students bring them up.  We will collaboratively determine how grades will be determined, though I have some ideas in mind for part of what I would like to see.  For example, I want each student to write their own answer to the research question at least several weeks before the end of term; then we will collaboratively develop a group response to publish.

This is pretty much the most “unschooled” course I teach.  The success or failure of a course like this depends on several things:  The question needs to be one for which there is no established answer.  Students need to believe that I don’t have the answer, and that it’s up to all of us to develop one.  For a project like this to work, the group of students and their commitment to the process is critical.  Ideally, every student needs to buy in to the premise.  In practice I’ve found that it only takes a few students who take it as a regular course for credit and a grade, to ruin the seminar.  More than any other course I teach, I actively recruit for this one, ideally one third economics majors, one third international affairs majors and one third business administration majors.  For the first day, I asked students to introduce themselves and discuss what they bring to the group that could help us in our research.

The most successful incarnation of the seminar was Spring 2009 when we analyzed the Global Financial Crisis as it was occurring.  The course outcome was this website.  I think I may have as good a group this year.  Wish me luck!

Posted in Uncategorized | Leave a comment