Quantitative Analysis of OpenStax’ Free, Open-Source Principles of Macroeconomics text

Last fall, I adopted OpenStax’ Principles of Macro text for my course of the same name. I have blogged earlier about the creation of that text. After the semester, I conducted a statistical analysis of student learning in the course as compared to the previous year’s sections when I used a well-known commercial textbook.

I used the same methodology as my earlier analysis comparing my online and face-to-face courses, but added in one more semester for a total of 4 sections and 89 observations.

The model used raw (uncurved) final exam scores as the dependent variable, and the following as explanatory variables:

  • Student GPA (less their grade in my course) — this was to capture some measure of how bright each student was, and to a certain extent, how hard they work.
  • Credit Hours Earned — this was to control for how much experience each student had in university coursework.
  • Gender — the literature says that women do less well in economics than men.
  • Whether the Sections was Honors or Regular—last year I taught both section as face-to-face courses, but one was an honors section that followed a more abstract treatment of the material.
  • Whether the Section was Online or Face-to-Face—last year I taught no online sections, so the Online variable might not have had adequate controls.
  • OpenStax—the treatment variable for the text used in both sections last year.

The results were quite interesting and differed a bit from my earlier study. The most significant determinant of final exam scores remained Cumulative GPA. Credit hours earned was not statistically significant this time. Gender was statistically positive meaning that women scored higher on the final exam. Honors students (or more precisely, students in the honors section) scored significantly higher. This variable had the largest effect size. Online students scored statistically lower. Since this differs from my earlier analysis, I plan to explore this more in the future, but it’s not critical for this analysis. Finally, the OpenStax variable was not statistically different from zero. That is, students using the OpenStax text scored no differently than those using the commercial text.

I’m happy to share the details of the statistical results with anyone who is interested.



Posted in Uncategorized | Leave a comment

The Lumen Project, v. 1

I have a really long draft of a post on the project I’ve been working on with Lumen Learning for much of the last year, but it’s not polished yet <jk> so let me just make a small point about it.  The project involves developing digital learning environments for two courses: Principles of Macro & Principles of Micro.  The content is divided into very small chunks.  At the end of each chunk is a short formative assessment using “multiple select” questions, which require students to select ALL the responses that are true. After each question students are asked how confident they feel about their answer.  The idea is to try to teach students metacognition as they learn economics.

I am pilot testing the courseware in one of my sections of Principles of Macro this semester.  A student in that section who was a serious student in my more traditional class last semester emailed me to ask, “Are there going to be any of these multiple select questions on the course exams, because if there are, I’m seriously thinking about dropping the course.”  When I shared this with David Wiley (@opencontent), he replied that that was really interesting and he wondered why the student responded that way.  I told him I thought it was probably two things:  First, multiple select is something new, something different than my students are used to.  And second, multiple select requires more thought.  I continued to think about this until I realized that this additional thought was completely consistent with the underlying theme in the courseware of metacognition.  I think that if students take that extra thinking seriously, it will likely over time lead to deeper learning.  I don’t think this was what we originally planned, but that’s a hypothesis anyway!

Posted in Uncategorized | Leave a comment

Teaching in Parallel

Over the last year, my approach to blogging has been to refine my thinking until I get it just right before posting.  You should see all the drafts I have!  I’m going to try something different here that some brighter than I call, wait for it…blogging.  I have an insight worth sharing so I’m just going to put it out there, and worry about getting it right later.

6683425837_1b064066c5_mAs I have done before, this term I’m teaching two sections of Principles of Macro, one online and one face-to-face.  There’s a lot more I should say about the former, but let me just make one point:  I find a terrific benefit to teach online and face-to-face in parallel.  I learn things in each that improves the teaching in the other.  Case in point, Friday I moderated a discussion of the role of economics in the liberal arts and sciences.  (Is there a role?)  I do this to help students situate my class in our schools curriculum.  It’s always a fun discussion and students seem to get something from it.

I often put a question on the first exam that originates from the discussion.  Which raised a potential problem in my head: The online section had no such discussion.  I use the same exams in both sections, so asking about a topic we didn’t discuss would make the online group appear to learn less.  I pondered that problem in the back of my head yesterday, and this morning woke up with an answer.  Since I used twitter to communicate with my online course daily, I posted the same two prompts that I used in class Friday.  While responding to some of the student responses, I ran across this article from today’s Washington Post, which provided some richness that neither the students nor I brought up online.  And then it occurred to me that I should post the same article on the course website for my face-to-face section.  I think this turned out to be a win, win for all of my students.

Image courtesy of Martin Weller via flickr

Posted in Uncategorized | Leave a comment

Don’t start down that slippery slope!

The seminar has finished a first attempt at fleshing out the research question we will investigate.  We then chose books for small groups to acquire and read prior to reporting out to the class.  For the last week or so, I’ve been presenting some of the tools of international finance (that would be the course content in a more traditional course), tools that students will need as they make progress in our research.  During this time, I’ve gotten the sense that some students are not really engaged, that they are not reading the books assigned because they think they have plenty of time for that later.  This shows up in the lack of comments in class and also the lack of blogging by some.

This behavior makes sense from the perspective of a student, but not from the perspective of a research associate.  What to do about this?

It would very easy to react by threatening a quiz or otherwise imposing penalties; indeed, that was the first thing that came to my mind.  I know it would work, because I have enough teaching experience to create a set of incentives so that students will do what I want.  But I had to tell myself, DON’T GO DOWN THAT ROAD, since it would compromise the research team ethos I’m trying to build for this course.  A research team leader wouldn’t say “I’m going to dock you 10 points for not reading!”  What would he or she do?

Part of this is my fault.  I implicitly assumed that the students would be doing what I would do.  But I am the one with the most experience doing research.  I haven’t communicated well enough to them, or provided enough structure for them to see what they should be doing.

To that end, today I will revisit our research question matrix.  I will create three groups of students:  One to investigate the financial aspects of the Euro problem; one to investigate the economic aspects and one to investigate the political aspects.  I organized the groups so that most of the financial group is business students; most of the economic group is economics majors; and most of the political group is international affairs.  But each group also has one member from each of the other majors.  The other thing I incorporated is that each group has someone who read each of the assigned books.  I will charge each group with identifying their respective aspects and reporting back in two weeks.  I will also give them time to organize groups today.

And I will encourage them to blog what they’re finding, and any questions they find.


Image courtesy of Paul Graham Raven (via flickr)

Posted in Uncategorized | Leave a comment

What’s the definition of insanity?

Last summer I attended the second annual UMW Game Camp, at which interested faculty 10241646_c42aeb5d71_mwere introduced to the variety of ways that gaming culture and practices could be incorporated into one’s teaching.  Game Camp was jointly led by Mary Kayler, Director of our Center for Teaching Excellence & Innovation (@saptiva), and Andi Smith, Associate Professor of Historic Preservation (@smithpres).  The main takeaway for me was a way of letting students choose between alternative assignments to earn points towards their final grade in my online principles of economics course.  This was a positive change in that it allowed students to choose assessments which showcased their particular learning styles.  But that’s not what this post is about.

Rather, last fall I tried something else that was inspired by Game Camp.  For a number of years, I have urged my introductory students during the first week of class to read the course website/syllabus carefully.  Then I have given them a quiz about it on the first Friday.  Year in and year out, the vast majority of students failed the one question quiz, even though I tried to make the questions easy for anyone who read carefully.

After game camp, I had a different idea:  I replaced the stick with a carrot.  Instead of a one question quiz, I created a scavenger hunt with half a dozen questions.  The result last semester was near 100% success.  The students loved it and seemed to think I was giving them free points.  Perhaps more importantly,  they learned what was on the syllabus, which after all was the point.  Thanks Mary & Andi!


Image courtesy of Abby Chicken via flickr

Posted in Uncategorized | Leave a comment

Teaching as Modelling Disciplinary Practice

I believe that one of the highest forms of teaching occurs through modelling disciplinary practice, ideally when students forget that you are teaching.  This is the first week for our Spring semester and I am teaching my senior seminar in international finance.  The seminar began life as a more traditional seminar covering the traditional topics in the field, but over the last decade (I only get to teach this every three years), it has morphed into something more interesting, both for the students and me.

The something is a collaborative research project in which I model doing research in economics.  The project isn’t part of the course—it is the course.  We start with a question:  This year it will be “Is the European Monetary System (e.g. the Euro) sustainable in its current configuration?”  Then I ask: “What do we need to know to answer that question?”  The result is a list of questions.  E.g. What is the Euro?  What do we mean by sustainable?  What is the Eurozone?  What are the pros and cons of membership? What are the political dimensions involved?

We work recursively backwards developing a nested set of questions, which will form the syllabus for the course.   For each set of questions, we divide up the work, go out and research each part, reporting back online before discussing what we’ve found in class.  Through the process, we develop an understanding of the issues which hopefully leads us to an answer to the initial research question.

We will spend this first week, developing an initial plan for the research.  I don’t intend to mention grades until the students bring them up.  We will collaboratively determine how grades will be determined, though I have some ideas in mind for part of what I would like to see.  For example, I want each student to write their own answer to the research question at least several weeks before the end of term; then we will collaboratively develop a group response to publish.

This is pretty much the most “unschooled” course I teach.  The success or failure of a course like this depends on several things:  The question needs to be one for which there is no established answer.  Students need to believe that I don’t have the answer, and that it’s up to all of us to develop one.  For a project like this to work, the group of students and their commitment to the process is critical.  Ideally, every student needs to buy in to the premise.  In practice I’ve found that it only takes a few students who take it as a regular course for credit and a grade, to ruin the seminar.  More than any other course I teach, I actively recruit for this one, ideally one third economics majors, one third international affairs majors and one third business administration majors.  For the first day, I asked students to introduce themselves and discuss what they bring to the group that could help us in our research.

The most successful incarnation of the seminar was Spring 2009 when we analyzed the Global Financial Crisis as it was occurring.  The course outcome was this website.  I think I may have as good a group this year.  Wish me luck!

Posted in Uncategorized | Leave a comment

test 1

test 1, 2, 3

Posted in Econ482 | Leave a comment

Bryan Alexander, Economist

Last semester I co-taught an Honors First Year seminar with my colleague Bob Rycroft.  Our FSEMs are designed to give FY students the kind of intimate academic experience that is more commonly associated with upper level seminars.  The FSEMS are specifically supposed to introduce our students to college-level writing, research and oral communication.  Honors FSEMs are supposed to provide a more rigorous experience.

I could probably write a book on what I learned from the FSEM, but suffice it to say that teaching FY students is different from teaching upper class students.  When you haven’t taught a class full of FYs in a while it’s easy to forget that.

In this post I want to discuss just one episode that occurred in the course.  The subject of 15513506426_ef0ab5d13f_mthe seminar was Economic Inequality.  As you may know, Thomas Piketty’s Capital in the 21st Century was one of the most publicized books of 2014.  But at 600+ pages, it wasn’t going to work to assign this to FY students to read, especially since Piketty’s work was only one part of the course.  Instead, we assigned one of the many good summaries available.  Anyway, I had a crazy idea that I was able to persuade my co-teacher to let us try.

The idea involved bringing my good friend Bryan Alexander into the seminar via Google Hangout.  We told the class that 4587900301_c9c17d2917_qBryan was a colleague of Thomas Piketty, and that Bryan would present Piketty’s argument to us.  For those who don’t know, Bryan is not an economist, though economics is an interest of his and he did read Piketty’s book.  Rather, Bryan holds a Ph.D. in English, and is currently self-employed as a futurist and consultant.  I wrote a script for Bryan summarizing Piketty’s argument.  After presenting the argument, Bob and I would each give our perspective on Piketty’s argument, and then we would open the floor to questions.  I told Bryan that if he were asked any questions he couldn’t handle, he should refer them to me.  On the day in question, we set up the Google Hangout with Bryan and his performance exceeded our expectations.  The students took notes and asked appropriate questions.

At the beginning of the next class period, I asked the students, “What did you think of Dr. Alexander?”  They replied effusively saying how much they enjoyed and appreciated the presentation.  As one student put it, “It was really great to hear from an expert!”  I responded by asking what about Bob & I, and she said, “well you know, a real expert!”  We thanked the students for their feedback and went on with the scheduled lesson.  The following class period was Friday.  I waited until the last five minutes of class, and then announced, “Remember Dr. Alexander?  Everything he said was a fraud.”  The students were stunned.  I continued, “The information he presented was accurate.  I know because I wrote his script.”  The students showed confusion, denial and in some cases, anger.  Then as the clock ticked down, I announced “For the weekend, I’d like you to think about why we did this. … See you Monday!”

On Monday, we began class by repeating the question:  Why did we go through this charade with you students?   Several students immediately proposed the answer—That we wanted them to think critically about the information they received in our class, even when they received it from the instructors.  The lesson was learned.

For the rest of the semester, at various times students asked, “How do we know you’re telling us the truth?”  “You don’t,” we replied.

Images courtesy of:

  • Tony Roberts via flickr
  • David Porter via flickr
Posted in Uncategorized | Leave a comment

On “The Limits to Open”

Ryan Brazell recently blogged on his experience at OpenEd 2014 in which he said,

While I can see the potential short-term gains, I’m deeply concerned about the questions we’re not asking surrounding OERs and open education more generally. I agree that people are more important than assets … but which people are left out of the above narrative? “Value” is not a neutral concept. When we create it for students, who gets to decide what that looks like? “Value” doesn’t come out of thin air. When we create value for students, from whom are we extracting it? “Open” sounds, on the surface, like a noble goal. When we makes things open — whether classrooms or party invites or educational resources — who are we unintentionally excluding? In the long term, are we actually fixing higher education, or further entrenching existing inequity?

I want to respond to Ryan’s post from an economist’s perspective.  I recognize that I may be answering a different question from the one he is asking.  But I hope some of what I say helps people’s understanding.

From an economist’s perspective, value is not a zero sum game.  Successful businesses are those that create goods or services that people value, that is, successful businesses create value that wasn’t there before.

My son went to school recently with a small bag of Lay’s Potato Chips, which he likes okay.  At lunch, one of his classmates produced a bag of Flaming Hot Cheetos, which are my son’s favorite and which the classmate didn’t care for.  They traded and each party was better off as a result.  Value was created, and no one lost out.

Of course, not all value creation is painless for everyone.  When Apple created the iPod, Apple created a great deal of value on balance for consumers, but producers of substitute products (e.g. Sony Walkman) lost out.   That’s pretty characteristic of how market economies work; Joseph Schumpeter described this as “creative destruction”.  But the thing is, more value gets created than destroyed so on balance consumers are better off.

When we create value for students, we aren’t extracting it from anyone (on net).   My first book (with a traditional publisher) was written for a course on a subject (introductory research methodology in economics) for which no textbook existed.  The book was of great value to the students who used it.  I know this because many students, both my own and from other institutions, have told me so over the years.  In this case since there was no previous text, the value I created, at a list price of $44, was a clear win.

With open source materials, we are privileging students with access to the internet, and clearly some students—those without internet access–are left out.  But the real question (again speaking as an economist) is, are we reaching students who were left out before?  The answer is clearly yes.  We know that between 1/3 and ½ of undergraduates don’t buy the texts for their classes.  Open source materials are either free (online) or inexpensive, allowing more students to be able to afford them.  For example, my OpenStax Principles of Economics book is available free at http://openstax.com in a variety of formats.  If one wants to buy the affiliated aftermarket product from Sapling Learning, the price for text & software is $40.

So for class resources like textbooks, the value to the student is in what the resource helps them learn.  The author writes the book, and the instructor decides to adopt it, so both have a role in defining the value.   Can you learn without a text?  Sure.  Can you learn more with the text?  Every author hopes so.

As long as the resource is free (or modestly priced), the value is almost certainly worth the price.

Posted in Uncategorized | Leave a comment

How Good is a Free, Open Source Text: Part 2?

This post is where I go off the reservation.  It is the third in a series of posts about the process of creating a free, open source textbook.  I ended the previous post with a question:  Is the book I helped create for OpenStaxCollege the best principles of economics book on the market? I would like to suggest that this is a more complicated question than it seems.


Beauty Contest

If you asked a Ph.D. economist whether Greg Mankiw’s Principles text is better than the OpenStax principles text, most economists would probably say yes.  Mankiw’s book is a good one.  I’m singling it out only because it is the market leader, the prototype for the current crop of commercial text books.  [On the other hand, I know of no assessment evidence that evaluates Mankiw’s book vis a vis other intro texts, but that’s another issue.]

But the real question is not how good Ph.D.s think a textbook is, but rather how much students learn from it.  In a way, this is a variation on John Maynard Keynes’ beauty contest metaphor for the stock market. Keynes observed that the way to make money in the stock market is not to invest in the best companies, but rather to invest in what the majority of other people think are the best companies.

If students neither buy nor read the text because it is expensive, they aren’t learning anything from it.  Thus, a “great” book that isn’t read by students is not better than a good book that students actually read!

So where does that leave us?  I believe that introductory textbooks these days are 3084239579_f1813df8cf_zcommodities.  They may not be identical, but they are close enough substitutes that any experienced faculty member can teach their intro course with pretty much any textbook on the subject.  Publishers don’t want you to believe this, but I think it’s true.  Think about how little time instructors devote to textbook selection.  They certainly act as if the choice doesn’t matter.

Half of undergraduates do not buy the text.  They make this decision for one of two related reasons: either they don’t see the value given the price, or they don’t think they can afford it.

What is the value-added in an intro textbook?  Students often see textbooks as substitutes for class lectures.  If they go to class, why do they need the book?  I don’t agree with this logic, but many students think his way.  To defeat this logic, we need to raise the value/price calculation in students’ eyes.  If you accept that intro texts are commodities, it’s going to be very difficult to raise the value.  The much more promising approach is to lower the price.  And free is a very attractive price.

In my view, while textbooks have value, the real value-added is in the ancilliary products, specifically online problems, quizzes and other activities that allow students to work with the course content until they master it.  This is not just my idea.  The latest issue of Campus Technology has an article subtitled, “Technology is moving the digital textbook from print look-alike to next-generation learning platform.”  Subjects like economics really benefit from students regularly working with the content in thoughtful, interactive, adaptive and low-stakes ways.  At present, the quality of these exercises lags behind the technology with which students can work them, but that’s an issue for another post. Suffice it to say we need to get beyond “drill & kill.”  Before digital, such practice was limited by the instructor’s (or teaching assistants’) ability to grade and return student work quickly (tasks which are ideally suited to computers).   Which is to say, such practice was very limited.  Note that the real benefit in this ancilliary software is self-directed learning along with self-assessment of mastery.  It is not primarily course management and grades.

Think about how commercial textbooks are priced.   Commercial publishers put a high price on the textbooks, but then a relatively low price on the ancilliary products, which have been perceived, after all, as ancilliary.  In my view, this is exactly backwards since the value to students is no longer in the course content, but in the digital learning environment around the content.  This is the reason why OpenStaxCollege has teamed up with commercial aftermarket producers, including Sapling Learning, and WebAssign which have at least some of the features I’m describing, at really modest prices, currently between $30 and $40, including the text. Disclaimer: I have no financial interest in these companies, but I think they offer some really interesting products.   If you genuinely care about your students learning, I urge you to take a look at these products. Faculty at nearly 100 schools to date have adopted OpenStax’ Principles of Economics.  While commercial publishers are also moving in these directions, they can’t give away the content for free, like OpenStaxCollege does.

Photo credits:

  • JanWillemsen via flickr
  • Ben.Gallagher via flickr


Posted in Uncategorized | Leave a comment