Call Me Don

7227337422_97b62ef492_m

I believe profoundly in what I am about to write. I am not naive about the difficulties involved, but neither do I think we can assume the problem away, as I believe is common in higher education today.

The primary purpose of (most) colleges and universities is, or should be, to educate their students.And yet, few schools have any clear evidence of the extent to which that learning occurs. We have outcomes assessment structures. But at least in many programs, that says little about how well and how much students are learning.

Few students truly master their studies because that’s not their goal. Students have been trained for twelve or more years to play and win the “game of school.” For most, the grade is the only thing that matters. For some, mere passing is the goal.

Few faculty expect their students to achieve mastery, either. We assume that not all students are cut out for biology, or economics, or [insert your favorite discipline]. After all, our discipline is rigorous! We use certain courses to “weed out” the students who can’t make the grade (pun intended). We also assume that some students aren’t cut out for college at all (though that doesn’t stop us from accepting those students’ tuition and fees).

But what if we are wrong about what students are able to accomplish?

What if (virtually) all our students were capable of substantially mastering what we teach, if only we taught them in a different way. Crazy talk? Would faculty be willing to make the change? I doubt it. Faculty are incredibly resistant to change. Look how difficult it is for faculty to switch to a new textbook? They have a course full of lecture notes built around the old book. They would have to go through those notes and make changes where necessary, at a minimum changing the terminology to match the new text. This is reason enough to stick with the latest edition of a $300 textbook, rather than try a free, open source text. But I digress.

Why would faculty want to make the change? For one thing, the incentives are all wrong. Tenured faculty have a comfortable existence. We can teach our favorite areas of expertise, with little or no accountability. Change adds to our workload, and might worsen student learning. There’s no guarantee it would lead to an improvement.

At a school like mine (a regional public university which claims to value teaching effectiveness) in order to earn tenure, faculty need to be able to provide evidence that their teaching is not terrible. (Friends have told me that’s not even required at research universities.) Neither departments nor institutions do much to assess teaching effectiveness, though. To be sure, the school administers course evaluations to students. In some departments, senior faculty sit on a lecture to see how their junior colleagues are doing. Department chairs are supposed to mentor weak teachers. But as far as I know (after 30+ years of experience), there is no direct measurement of student learning. And there is little intervention in to help junior colleagues improve their teaching. “Not terrible” is good enough. Not terrible doesn’t warrant the intervention of the department chair, the dean or the provost.

I’m not suggesting that faculty aren’t conscientious teachers. In my experience (limited to my regional public institution), most faculty care about their teaching. We enjoy seeing the lightbulb go off in our students’ eyes. We just don’t know how effective our teaching is from semester to semester. Course evaluations can’t tell us that. Rare visits by colleagues can’t tell us that. If it ain’t broke, don’t fix it.

Faculty earn promotion and merit pay (when there is any) from their scholarly activity. That is also the ticket to a position at another university, if desired. I’ve never heard of a faculty member being recruited to another school because of above average teaching ability, but there is a plethora of evidence that instructors lose their positions despite or even because of their strong teaching ability. It is not that unusual to hear of faculty earning teaching awards, who were subsequently turned down for tenure.

I am not opposed to research and other forms of scholarly activity. Research informs my teaching and teaching informs my research. I just don’t think it should trump teaching effectiveness at (most) 4-year schools.

I would be surprised if my administration did not object to this characterization. Look at the weight assigned to teaching in faculty evaluation. Look at how much money has been allocated to (thrown at?) teaching development; look at our well resourced university teaching center. I don’t deny that opportunities are there for individual faculty members who want to take advantage of them. But who has the time, given all the other (more important?) things we are asked to do? There is admissions and retention work, faculty advising, participating in our first year experience, recruitment of new faculty, faculty governance and scholarly activity, just to name a few of the tasks I spend my time on. All of these things are important, but are they more important than teaching?

Is there an ongoing, central conversation among a critical mass of faculty about teaching at our institutions? How often do faculty discuss teaching effectiveness? How often is student learning discussed at department meetings, chair’s meetings, provost’s meetings or faculty governance meetings?

Regardless of the “weight” given to teaching in faculty evaluation, tenure and promotion, the bar is too low. The evidence is not there.

Perhaps I am expecting too much from faculty. Most Ph.D. students, after all, are not trained to teach. New Ph.D.s are assumed to be able to pick up teaching on the fly. They are bright people with years of experience as students. They can surely teach the way they were taught.

In the last several decades, researchers have discovered a tremendous amount about how students learn. But few faculty outside of cognitive science and education are trained in these findings. Why would they be since a Ph.D. is supposed to convey research ability and content expertise in their one’s field.

Paul Bruno, citing a recent report from Deans of Impact, states:

[W]e believe the art of teaching should also be informed by a robust understanding of the learning sciences so that teachers can align their decisions with our profession’s best understanding of how students learn.

I agree.

What if universities chose to build a reputation based on evidence of how well their students learn? Wouldn’t students want to go to those schools? Wouldn’t parents or others paying the bills want that?

What if faculty were at least minimally trained in how students learn?

What if incentives for faculty were aligned with the goals of teaching and learning effectiveness?

It would start with the assumption that all our students can learn, if properly instructed.

It would follow with the assumption that all faculty can be trained to teach effectively.

It would require higher standards and accountability.

It would require committed leadership.

And it would require the willingness to change.

Posted in Uncategorized | Leave a comment

Kudos to United Airlines (& American Airlines)

I have complained many times about poor airline experiences so it seems only right that I call attention to an unusually good experience.  I missed my flight home due to my mistake.  An amazingly helpful gate agent in Indianapolis bailed me out.  (Note to United: I think her name was Wanda!)  She found the last seat on an American flight and rebooked me.  The new flight was direct to my final destination instead of making a connection.   I will end up getting home about the same time as I was originally scheduled.  Thank you United Airlines, and thanks to American as well.

Posted in Uncategorized | Leave a comment

Quantitative Analysis of OpenStax’ Free, Open-Source Principles of Macroeconomics text

Last fall, I adopted OpenStax’ Principles of Macro text for my course of the same name. I have blogged earlier about the creation of that text. After the semester, I conducted a statistical analysis of student learning in the course as compared to the previous year’s sections when I used a well-known commercial textbook.

I used the same methodology as my earlier analysis comparing my online and face-to-face courses, but added in one more semester for a total of 4 sections and 89 observations.

The model used raw (uncurved) final exam scores as the dependent variable, and the following as explanatory variables:

  • Student GPA (less their grade in my course) — this was to capture some measure of how bright each student was, and to a certain extent, how hard they work.
  • Credit Hours Earned — this was to control for how much experience each student had in university coursework.
  • Gender — the literature says that women do less well in economics than men.
  • Whether the Sections was Honors or Regular—last year I taught both section as face-to-face courses, but one was an honors section that followed a more abstract treatment of the material.
  • Whether the Section was Online or Face-to-Face—last year I taught no online sections, so the Online variable might not have had adequate controls.
  • OpenStax—the treatment variable for the text used in both sections last year.

The results were quite interesting and differed a bit from my earlier study. The most significant determinant of final exam scores remained Cumulative GPA. Credit hours earned was not statistically significant this time. Gender was statistically positive meaning that women scored higher on the final exam. Honors students (or more precisely, students in the honors section) scored significantly higher. This variable had the largest effect size. Online students scored statistically lower. Since this differs from my earlier analysis, I plan to explore this more in the future, but it’s not critical for this analysis. Finally, the OpenStax variable was not statistically different from zero. That is, students using the OpenStax text scored no differently than those using the commercial text.

I’m happy to share the details of the statistical results with anyone who is interested.

 

 

Posted in Uncategorized | Leave a comment

The Lumen Project, v. 1

I have a really long draft of a post on the project I’ve been working on with Lumen Learning for much of the last year, but it’s not polished yet <jk> so let me just make a small point about it.  The project involves developing digital learning environments for two courses: Principles of Macro & Principles of Micro.  The content is divided into very small chunks.  At the end of each chunk is a short formative assessment using “multiple select” questions, which require students to select ALL the responses that are true. After each question students are asked how confident they feel about their answer.  The idea is to try to teach students metacognition as they learn economics.

I am pilot testing the courseware in one of my sections of Principles of Macro this semester.  A student in that section who was a serious student in my more traditional class last semester emailed me to ask, “Are there going to be any of these multiple select questions on the course exams, because if there are, I’m seriously thinking about dropping the course.”  When I shared this with David Wiley (@opencontent), he replied that that was really interesting and he wondered why the student responded that way.  I told him I thought it was probably two things:  First, multiple select is something new, something different than my students are used to.  And second, multiple select requires more thought.  I continued to think about this until I realized that this additional thought was completely consistent with the underlying theme in the courseware of metacognition.  I think that if students take that extra thinking seriously, it will likely over time lead to deeper learning.  I don’t think this was what we originally planned, but that’s a hypothesis anyway!

Posted in Uncategorized | Leave a comment

Teaching in Parallel

Over the last year, my approach to blogging has been to refine my thinking until I get it just right before posting.  You should see all the drafts I have!  I’m going to try something different here that some brighter than I call, wait for it…blogging.  I have an insight worth sharing so I’m just going to put it out there, and worry about getting it right later.

6683425837_1b064066c5_mAs I have done before, this term I’m teaching two sections of Principles of Macro, one online and one face-to-face.  There’s a lot more I should say about the former, but let me just make one point:  I find a terrific benefit to teach online and face-to-face in parallel.  I learn things in each that improves the teaching in the other.  Case in point, Friday I moderated a discussion of the role of economics in the liberal arts and sciences.  (Is there a role?)  I do this to help students situate my class in our schools curriculum.  It’s always a fun discussion and students seem to get something from it.

I often put a question on the first exam that originates from the discussion.  Which raised a potential problem in my head: The online section had no such discussion.  I use the same exams in both sections, so asking about a topic we didn’t discuss would make the online group appear to learn less.  I pondered that problem in the back of my head yesterday, and this morning woke up with an answer.  Since I used twitter to communicate with my online course daily, I posted the same two prompts that I used in class Friday.  While responding to some of the student responses, I ran across this article from today’s Washington Post, which provided some richness that neither the students nor I brought up online.  And then it occurred to me that I should post the same article on the course website for my face-to-face section.  I think this turned out to be a win, win for all of my students.

Image courtesy of Martin Weller via flickr

Posted in Uncategorized | Leave a comment

Don’t start down that slippery slope!

The seminar has finished a first attempt at fleshing out the research question we will investigate.  We then chose books for small groups to acquire and read prior to reporting out to the class.  For the last week or so, I’ve been presenting some of the tools of international finance (that would be the course content in a more traditional course), tools that students will need as they make progress in our research.  During this time, I’ve gotten the sense that some students are not really engaged, that they are not reading the books assigned because they think they have plenty of time for that later.  This shows up in the lack of comments in class and also the lack of blogging by some.

This behavior makes sense from the perspective of a student, but not from the perspective of a research associate.  What to do about this?

It would very easy to react by threatening a quiz or otherwise imposing penalties; indeed, that was the first thing that came to my mind.  I know it would work, because I have enough teaching experience to create a set of incentives so that students will do what I want.  But I had to tell myself, DON’T GO DOWN THAT ROAD, since it would compromise the research team ethos I’m trying to build for this course.  A research team leader wouldn’t say “I’m going to dock you 10 points for not reading!”  What would he or she do?

Part of this is my fault.  I implicitly assumed that the students would be doing what I would do.  But I am the one with the most experience doing research.  I haven’t communicated well enough to them, or provided enough structure for them to see what they should be doing.

To that end, today I will revisit our research question matrix.  I will create three groups of students:  One to investigate the financial aspects of the Euro problem; one to investigate the economic aspects and one to investigate the political aspects.  I organized the groups so that most of the financial group is business students; most of the economic group is economics majors; and most of the political group is international affairs.  But each group also has one member from each of the other majors.  The other thing I incorporated is that each group has someone who read each of the assigned books.  I will charge each group with identifying their respective aspects and reporting back in two weeks.  I will also give them time to organize groups today.

And I will encourage them to blog what they’re finding, and any questions they find.

 

Image courtesy of Paul Graham Raven (via flickr)

Posted in Uncategorized | Leave a comment

What’s the definition of insanity?

Last summer I attended the second annual UMW Game Camp, at which interested faculty 10241646_c42aeb5d71_mwere introduced to the variety of ways that gaming culture and practices could be incorporated into one’s teaching.  Game Camp was jointly led by Mary Kayler, Director of our Center for Teaching Excellence & Innovation (@saptiva), and Andi Smith, Associate Professor of Historic Preservation (@smithpres).  The main takeaway for me was a way of letting students choose between alternative assignments to earn points towards their final grade in my online principles of economics course.  This was a positive change in that it allowed students to choose assessments which showcased their particular learning styles.  But that’s not what this post is about.

Rather, last fall I tried something else that was inspired by Game Camp.  For a number of years, I have urged my introductory students during the first week of class to read the course website/syllabus carefully.  Then I have given them a quiz about it on the first Friday.  Year in and year out, the vast majority of students failed the one question quiz, even though I tried to make the questions easy for anyone who read carefully.

After game camp, I had a different idea:  I replaced the stick with a carrot.  Instead of a one question quiz, I created a scavenger hunt with half a dozen questions.  The result last semester was near 100% success.  The students loved it and seemed to think I was giving them free points.  Perhaps more importantly,  they learned what was on the syllabus, which after all was the point.  Thanks Mary & Andi!

 

Image courtesy of Abby Chicken via flickr

Posted in Uncategorized | Leave a comment

Teaching as Modelling Disciplinary Practice

I believe that one of the highest forms of teaching occurs through modelling disciplinary practice, ideally when students forget that you are teaching.  This is the first week for our Spring semester and I am teaching my senior seminar in international finance.  The seminar began life as a more traditional seminar covering the traditional topics in the field, but over the last decade (I only get to teach this every three years), it has morphed into something more interesting, both for the students and me.

The something is a collaborative research project in which I model doing research in economics.  The project isn’t part of the course—it is the course.  We start with a question:  This year it will be “Is the European Monetary System (e.g. the Euro) sustainable in its current configuration?”  Then I ask: “What do we need to know to answer that question?”  The result is a list of questions.  E.g. What is the Euro?  What do we mean by sustainable?  What is the Eurozone?  What are the pros and cons of membership? What are the political dimensions involved?

We work recursively backwards developing a nested set of questions, which will form the syllabus for the course.   For each set of questions, we divide up the work, go out and research each part, reporting back online before discussing what we’ve found in class.  Through the process, we develop an understanding of the issues which hopefully leads us to an answer to the initial research question.

We will spend this first week, developing an initial plan for the research.  I don’t intend to mention grades until the students bring them up.  We will collaboratively determine how grades will be determined, though I have some ideas in mind for part of what I would like to see.  For example, I want each student to write their own answer to the research question at least several weeks before the end of term; then we will collaboratively develop a group response to publish.

This is pretty much the most “unschooled” course I teach.  The success or failure of a course like this depends on several things:  The question needs to be one for which there is no established answer.  Students need to believe that I don’t have the answer, and that it’s up to all of us to develop one.  For a project like this to work, the group of students and their commitment to the process is critical.  Ideally, every student needs to buy in to the premise.  In practice I’ve found that it only takes a few students who take it as a regular course for credit and a grade, to ruin the seminar.  More than any other course I teach, I actively recruit for this one, ideally one third economics majors, one third international affairs majors and one third business administration majors.  For the first day, I asked students to introduce themselves and discuss what they bring to the group that could help us in our research.

The most successful incarnation of the seminar was Spring 2009 when we analyzed the Global Financial Crisis as it was occurring.  The course outcome was this website.  I think I may have as good a group this year.  Wish me luck!

Posted in Uncategorized | Leave a comment

test 1

test 1, 2, 3

Posted in Econ482 | Leave a comment

Bryan Alexander, Economist

Last semester I co-taught an Honors First Year seminar with my colleague Bob Rycroft.  Our FSEMs are designed to give FY students the kind of intimate academic experience that is more commonly associated with upper level seminars.  The FSEMS are specifically supposed to introduce our students to college-level writing, research and oral communication.  Honors FSEMs are supposed to provide a more rigorous experience.

I could probably write a book on what I learned from the FSEM, but suffice it to say that teaching FY students is different from teaching upper class students.  When you haven’t taught a class full of FYs in a while it’s easy to forget that.

In this post I want to discuss just one episode that occurred in the course.  The subject of 15513506426_ef0ab5d13f_mthe seminar was Economic Inequality.  As you may know, Thomas Piketty’s Capital in the 21st Century was one of the most publicized books of 2014.  But at 600+ pages, it wasn’t going to work to assign this to FY students to read, especially since Piketty’s work was only one part of the course.  Instead, we assigned one of the many good summaries available.  Anyway, I had a crazy idea that I was able to persuade my co-teacher to let us try.

The idea involved bringing my good friend Bryan Alexander into the seminar via Google Hangout.  We told the class that 4587900301_c9c17d2917_qBryan was a colleague of Thomas Piketty, and that Bryan would present Piketty’s argument to us.  For those who don’t know, Bryan is not an economist, though economics is an interest of his and he did read Piketty’s book.  Rather, Bryan holds a Ph.D. in English, and is currently self-employed as a futurist and consultant.  I wrote a script for Bryan summarizing Piketty’s argument.  After presenting the argument, Bob and I would each give our perspective on Piketty’s argument, and then we would open the floor to questions.  I told Bryan that if he were asked any questions he couldn’t handle, he should refer them to me.  On the day in question, we set up the Google Hangout with Bryan and his performance exceeded our expectations.  The students took notes and asked appropriate questions.

At the beginning of the next class period, I asked the students, “What did you think of Dr. Alexander?”  They replied effusively saying how much they enjoyed and appreciated the presentation.  As one student put it, “It was really great to hear from an expert!”  I responded by asking what about Bob & I, and she said, “well you know, a real expert!”  We thanked the students for their feedback and went on with the scheduled lesson.  The following class period was Friday.  I waited until the last five minutes of class, and then announced, “Remember Dr. Alexander?  Everything he said was a fraud.”  The students were stunned.  I continued, “The information he presented was accurate.  I know because I wrote his script.”  The students showed confusion, denial and in some cases, anger.  Then as the clock ticked down, I announced “For the weekend, I’d like you to think about why we did this. … See you Monday!”

On Monday, we began class by repeating the question:  Why did we go through this charade with you students?   Several students immediately proposed the answer—That we wanted them to think critically about the information they received in our class, even when they received it from the instructors.  The lesson was learned.

For the rest of the semester, at various times students asked, “How do we know you’re telling us the truth?”  “You don’t,” we replied.

Images courtesy of:

  • Tony Roberts via flickr
  • David Porter via flickr
Posted in Uncategorized | Leave a comment